I'm not trying to deny that various composers impugn certain characters to certain keys. Especially before the widespread use of equal temperament in the late 1700's, different keys were actually made up of different intervals, and so really did sound different. The association of different pieces with certain keys by later composers clearly affected their own compositions. There's also the psychological anchoring of our pitch universe around C major, which generally causes us to think of sharp keys as "brighter" than flat keys. We make this association because modulation within a piece of music to a sharper key area sounds "bright," but the C major anchor is completely arbitrary. Thus, Beethoven once remarked that he could tell whether a composer had penned a piece in g-flat major or f-sharp major, without looking at the score, simply by the character of the piece, even though the keys are enharmonically equivalent. It all depended on whether the composer was thinking of the key of the piece as six fifths on the flat side of c major, or six fifths to the sharp side.
In addition, the physical characteristics of different instruments clearly affect the way music sounds in different keys. Pieces written for b-flat clarinet will sound different in other keys because of the different fingerings and other stuff about clarinet I don't understand; similarly for piano pieces that include more black notes than white, etc.
Nevertheless, the rather widespread notion that scales on different pitch levels have intrinsic characters of their own can't possibly be true. In an equal-tempered system of tuning, all the intervals in every major scale are identical, and all the intervals in every minor scale are identical. The only intrinsic difference between scales starting on different pitches is, of course, the pitch level.
I hear people remark all the time about why a composer would write a piece in a given key, but rarely does anyone ever mention the most obvious reason given above: it sounds "right" at that pitch level. Any piece you know would probably sound really wrong if displaced by an octave. Most pieces would sound weird displaced by a fifth in either direction too. Displacing something by a half-step or a whole step is a much smaller change, but still affects our perceptions. [This is, incidentally, why composers often include a lot of octave jumps in the second section of the recapitulations of sonata movements, but rarely in the expositions. They composed the theme for the dominant key in the exposition; in order to make it sound "right" in the tonic in the recap, they often include it in different octaves as a sort of balance.]