When I was studying advanced Spanish in Mexico several years ago, I was asked, "Which do you find the hardest: reading, speaking, or listening?" I'd never really thought about it before, and I was surprised to find that my answer was "listening". How many times have you heard a person say of a foreign language, "I can understand it, I just can't speak it!" Indeed, I feel that way about some of my other languages, but with Spanish, listening to and comprehending spoken speech is always my biggest challenge.
One can define four categories of language ability: the three mentioned above, plus writing. The ease and difficulty of each is a function of the qualities associated with that particular ability, as well as the context in which that ability is required. For example, reading a language is a passive, visual activity, whereas speaking is an active, auditory activity. In general, too, the visual categories (reading and writing) do not have the kind of urgency associated with the auditory categories (speaking and listening), although there are of course exceptions, such as when chatting online. So we can define the following matrix:
One might think that the passive categories would be easier. A person's recognition vocabulary (words he understands) is typically much larger than his active vocabulary (words he would actually use). And how many times have you thought you understood something, only to become confused when trying to explain it to someone else? But there is a tradeoff; when using passive language ability, you do not have control. You could be trying to understand someone who uses street slang and speaks at lightning speed, or trying to read Don Quixote in the original. Active language use is more difficult for the beginner who doesn't have a large enough foundation of vocabulary or grammar, but an advanced speaker can usually make himself understood in some way or another.
Similarly, one might also think that the delayed/visual categories would be easier, since one has the advantage of time. In fact, many people can read and write in a language far better than they can speak it. But the register of language that is used for reading and writing is usually much higher; think not only of great literature, but also of legal documents, formal letters, and the like. Moreover, one is held to higher standards of grammar when dealing with the written word. Again, one can come up with instances of high-register spoken language, such as a philosophical lecture, but the vast majority of spoken language use is for conversation, which uses relatively simple vocabulary and grammar (no 200-word sentences here, even in German!).
So for Spanish, I find that listening is the most difficult, particularly when I'm asked to interpret for people who speak rapidly and with pronunciation and vocabulary that is very different from what's taught in the classroom. Reading is the second most difficult, as I am painfully reminded every time I try to slog through another chapter of Gabriel García Marquez's Cien años de soledad. It's the passive activities that get me, whereas when I'm speaking or writing I can generally get my thought across somehow, even though my grammar suffers when I try to speak quickly.
For my other languages, in which I'm considerably less fluent, the active language use is more difficult; there are many instances where I simply can't come up with a basic word I need, but I can understand what people say to me. This is because I operate in these languages at a much more basic level. Were my German at the level where I was attempting to read Goethe or understand a technical presentation, I'm sure those would be far greater challenges.
It should be obvious from all this that using a single word to qualify one's language ability is woefully inadequate, as much so as describing a person's political leanings as "conservative" or "liberal". With respect to what? In what context? But if you're seeking to improve your facility with a foreign language, thinking about your proficiency in these four categories will help identify your weaknesses. As for myself, I'll continue listening to Spanish radio and interpreting at Rotacare for uninsured Spanish speakers, and I hope the day will come when it will all be easy.