Recently I've taken to starting recitals by playing popular jingles from TV or radio shows. It's fun to see who else in the audience listens to Serial or NPR, or is familiar with the Champion's League theme (come on people, it may be the most widely-recognized music on the planet). Admittedly, it's also to win over the audience and impress them right off the bat, because rattling off something by ear is unusual and seldom taught in the world of (so-called) classical music and traditional music education (a topic for another time).
At the intermission of one show, someone asked me: can you play anything by ear, instantly? The answer is a definite and emphatic no, but the question reveals a lot about people's misunderstanding not just of music, but of human cognition and the limitations of memory.
People assume that playing music by ear is difficult because amateur listeners (and even amateur musicians, see: educational issues above) are not used to identifying pitches or the distances between them. Sadly for us musicians, identifying notes is only the first and easiest hurdle to clear when trying to learn music by ear. Remembering and rattling off a piece, armed only with the ability to identify pitches, would be like memorizing a poem by remembering the sequence of letters. The real problem is not identification but information reduction, taking a whole slew of sound and unconsciously reducing it to larger abstract musical components.
Think about the process you would go through in memorizing some lines of text to write out later. First, let's keep it simple, and imagine you just had to learn a single sentence, like, "What is your favorite piece of music and why do you like it?" Piece of cake, right?
But what if the line were in a foreign language that you didn't know ("Cila është pjesa juaj e muzikës dhe pse nuk ju pëlqen?" **from google translate, possibly not accurate)? In that case, your best bet would be remembering the sequence of letters that make up the spelling of the phrase. Think how difficult that might be for even a few words, how much more work, and how prone you would be to mistakes. Even worse, if you misremembered even just a couple of letters, it might distort the meaning or render the phrase completely meaningless. You couldn't remember it accurately with a single glance, even though remembering any of the individual component letters or words on their own might not be difficult at all. (In a language with different symbols, things would be an order of magnitude more difficult, though in principle the same: remembering any single letter in Arabic might not be too difficult, but good luck with a string of them).
The difference is, in your native language you don't have to remember the individual letters—or even words—that make up the phrase, but only a single concept (or two) of favorite-music-ness. If you mis-remembered the exact wording, you would still easily reproduce something that preserved the meaning ("What's your favorite piece of music and why do you love it so much?").
I was reading this morning (pretty quickly, but not that quickly) about speed-reading in the New York Times, and the problem is the same (though one level of abstraction higher). I used to think if my eyes could just move faster through text, I could read faster. But the limiting factor in reading isn't the speed of cramming letters and words into your brain through your eyes, it's remembering, keeping track of, and making sense of what your eyes are sending to your brain. Turns out, most people can only keep track of 5-9 "concepts" at a time, and so reading is really an exercise in grouping words into concepts, absorbing them, and then moving on. People who can "chunk" the most words into a single abstract idea or concept, then, will tend to be faster readers.
The go-to example in psychology textbooks is memory for chess positions. People used to think master chess players had great memories because they could remember an entire chessboard much more accurately than amateur players, but it turns out expert players are no better (or only slightly better) at remembering a randomly arranged board than amateurs. They're better with boards from real games because rather than store the pieces and their positions one by one, they store the board in relation to other "types" of positions they've seen before, and so only have to remember a few ideas, rather than all 32 pieces' positions.
I know very little about chess, so I can't really explain what those "types" are, but what I do know a lot about is music! And the process for learning something by ear is very similar, only the "types" of things you have to remember are music-specific instead of chess-specific. Identifying this pitch or that pitch from something you hear is essential, but it's only as helpful as remembering that the word "what's" starts with a w, or that white bishop is in position a6. In other words, it's a total red herring and completely misses the point. What's helpful in reproducing a musical phrase is reducing it to something close to something you've heard before and stored in memory already from hearing and categorizing thousands of times already, to a series of harmonies and rhythms and patterns that you already know.
One final story to clear up a musical misconception about absolute, or perfect, pitch. Absolute pitch is the ability to identify the pitch class of a sounded note (a, b, c, etc., basically, where it is on the piano). People sometimes think of absolute pitch as a great musical gift, and it doesn't hinder anything, but it's also, again, a red herring in terms of musical cognition. Example: a classmate who had absolute pitch in one of my music classes in high school. We were learning to identify different chord types (major, minor, etc.) by sound and by sight (hearing them or seeing them written down on the staff). Just as a point of reference, most of my beginner piano students can learn to do the sound identification task without too much difficulty. But rather than listen to the quality of the chord overall in the sound training, my fellow student would try to hear all the pitches individually, write them down, and then determine what type of chord was played by sight. In other words, instead of reducing the information, she was multiplying it! Like a great speller declaring, "I'll remember the phrase 'what's your favorite piece of music and why' by spelling it out each time, rather than storing its meaning." Moral: absolute pitch plays little to no role in musical memory since it generally doesn't help reduce information at all.
And even relative pitch (the ability to identify relationships between two pitches), while important and more valuable than perfect pitch, is only the first small step in learning to play by ear, since it, too, only aids the identification of pitches, but not necessarily the ability to store and remember those relationships.