Another day, another person trying to solve (and monetize) the online language learning conundrum. This time it’s Luis von Ahn, MacArthur Grant winner and ReCAPTCHA founder, who has been all over the Internet talking about his latest project Duolingo.

Luis, who has made a fortune wielding a giant hammer called “CROWDSOURCING!”, has unsurprisingly come to see language learning as a giant nail ripe for striking. He figures that he can develop some online language courses, encourage people to practice their skills by translating content from the web, and offer the courses for free by selling the translations that users generate in the process. (I have now helped him with this endeavor by translating two news articles on the violence in the Central African Republic).

By making everything free instead of, say, $500, Duolingo is aimed squarely at undoing the tyranny of Rosetta Stone, which I can only applaud. Democratizing language learning is a worthy goal. But it’s worth remembering the old adage that if a web service is free, you are the product being sold.

When Duolingo’s Portuguese course came out of beta a few weeks ago, I signed up. Besides looking at the Portuguese course, I also worked through the German course so I could experience the platform as a true beginner.

Duolingo has done a few things right, and one of them is that it’s taken language gamification to new heights. It’s about time someone leveraged the addictive power of games to help you do something productive, and Duolingo delivers, giving you a slick eye-candy interface, lots of points and trophies and badges to earn, “level-up” moments, and motivational emails when you don’t log in for a few days. There’s even a world map of sorts that lets you chart your own path through the courses. The mobile app looks just as good and allows you to use your downtime during the day for a quick study session.

There is real depth to the courses, too — the Portuguese course that I tried out takes you well into high-intermediate territory (though I never found out how far, exactly, due to the irritating requirement that you take a quiz to demonstrate mastery before skipping lessons). I liked that every single exercise linked to its own discussion thread where there were often knowledgeable native speakers on hand to answer questions. And the integrated memory schedule flashcard app that gets automatically populated with the words you’ve learned is a nice touch.

Every language system has its guiding metaphor (Pimsleur: the Conversation; Rosetta: the Multiple-Choice Test; Memrise: the Flashcard). The metaphor in Duolingo is Translation. And this means that almost everything in Duolingo happens in the context of a sentence, which seems at first like a good idea. You are either translating sentences from English to Portuguese, or from Portuguese to English, and learning new vocabulary and grammar incidentally along the way via Rosetta Stone-style multiple choice activities. All the sentences have good audio pronunciation, and some activities challenge you to transcribe a sentence from the audio alone, which is a very good exercise.

But the way the system harvests its sentences from web content leads to a dry learning experience. Some of the sentences appear to have been created by stringing random words together, so you will find yourself translating awkward things like “The teachers have water” or “We watched tons of coffee” or “That’s not the way people are treated” instead of, I don’t know, “How are you? Which way to the beach? Is there a library around here where I could check out a Pimsleur course?” The sentences are so far removed from anything that you might actually want to use in conversation that I doubt how much value there is in rote translation. Many sentences are flat out wrong. I eventually tired of sending so many error reports.

Duolingo requires that you unlock earlier lessons to get to later ones, but it gives you some choice in the order in which you work through them

Another problem is the basic challenge of having a computer validate a human translation: there are so many possible correct translations for a given sentence that a very complex algorithm is needed to determine whether what you have entered is acceptable or not. They have obviously spent considerable time developing this algorithm, and it is indeed impressive, but impressive is just not good enough here. I often entered perfectly valid translations that the system judged as wrong, and in some cases I had to just guess at the particular phrasing that would count as a correct answer. This is no way to learn a language as fluid and nuanced in its sentence structure as Portuguese (or German for that matter).

In forcing you to give only ‘correct’ responses, Duolingo deceives you into thinking that only certain translations are possible — “algumas moedas” but not “umas moedas”, for example, or “seu vestido” but not “o vestido dela”. This might not be so bad, but after three incorrect answers you are forced to restart the lesson at the beginning, leading to frustration and, ultimately, me giving up on Duolingo.

I did learn a few German pronouns and conjugations. It was addictive, for a few days. But very quickly it became boring — there’s little joy in translating awkward sentences plucked at random from the Internet, completely devoid of any interesting context. Die Lehrer haben Wasser. The Portuguese course was the same — different sentences, but no life, no brasilidade.

Yet, the biggest problem with Duolingo is that there is no provision for gaining an oral fluency with the language. This is important because most people learn a language because they want to be able to have conversations in that language. And for that you need to practice listening, speaking and pronunciation above all.

In terms of pronunciation, Duolingo leaves you to glean what you can from the recordings (there is a 1/2 speed option that helps with this). But this is backwards! Oral fluency has to come first, or else you haven’t really acquired the language (in the Stephen Krashen sense of the word), you’ve simply learned to manipulate visual symbols. Certainly, I do think grammar ought to be taught, and it is possible to be conversant in a language by symbolic manipulation alone, but how much more delightful it is when you have truly internalized that grammar by its sound, and that can only happen — in the beginning stages at least — by copious listening and speaking. Reading, I think, is the least useful mode for learning because there is no sound input, and sound is the basis for acquisition (though the LingQ folks will surely disagree). Speech comes prior to writing in language, but it comes last in Duolingo. [If you need an “ah-ha!” moment on what genuine acquisition and ‘comprehensible input’ feel like, check out Krashen’s two mini-lessons on German — granted, it is much easier to make the input comprehensible with a closely related language like German, but the point stands]

This preference for reading over listening and verbal production is a fundamental problem with most online language courses, so I don’t mean to pick on Duolingo specifically. Pimsleur is the only solo course I’ve found that comes close to replicating the experience of acquiring language through listening and speaking in conversation. And Pimsleur could be much better than it is. Yet no one has set about making a better Pimsleur because, my goodness, what a lot of work to script and record all those hours of conversations, make them progressive and engaging, incorporate the memory schedules…! Much easier to just build a flashcard app, or a multiple choice quiz, or a crowdsourcing engine. And so instead we get Duolingo.

I’m sure some people will like Duolingo a lot, and I’m all for whatever keeps you motivated and coming back — small, consistent, daily practice is the only way to really learn. The best thing about Duolingo may be its ability to get people interested enough in the language that they seek out other resources — first and foremost, a good teacher.