Google Translate fails on simple sentences

Note: Systran is here.
Bing is here.
DeepL is here

Updated March 2019.

I was not able to find Microsoft as distinguished from Bing, so that is not included on this page.

This collection should under no circumstances be taken as any kind of serious benchmark. Not only is it very small, but, as the title indicates, it was originally formed as a collection where Google Translate, specifically, failed; hence, it is inherently unfair to Google Translate. It is just an unsystematic collection of translation problems that seem like they should be easy, but trip up Google Translate and other machine translation programs as of fall 2016. I check these intermittently to see what progress has been made. Microsoft Translate hedges its bets by producing two translations (sometimes the same).

Earlier versions:
September - December 2016
December 2016
May 2017

Note: I don't actually speak French, so I am grateful to Pascal Amsili for his corrections on a number of items.

I wasn't able to find Microsoft Translate (separate from Bing) at the time of this test, so it is not included.

Updated 8/13/2018.

Fr: Nous sommes lundi. [It's an odd idiom.]
En: Today is Monday.
GT. We are Monday.
Bing. It's Monday. [Correct]
Systran. We are on Monday
DeepL: It's Monday. [Correct]
En: The electrician whom we called to fix the telephone works on Sundays.
Fr: L'électricien que nous avons appelé pour réparer le téléphone travaille le dimanche.
GT, Bing: L'électricien que nous avons appelé pour réparer le téléphone fonctionne le dimanche.
GT, Bing: L'électricien que nous avons appelé pour réparer le téléphone le dimanche. [sic; the word for "works" is omitted]
DeepL: L'électricien que nous avons appelé pour réparer le téléphone travaille le dimanche. [Correct]

Note: On the same sentence, GT makes the same mistake translating into German and Spanish, but it gets Italian right. However, if you change it to the negative ("does not work") then it gets Italian wrong as well. If you add commas and change the restrictive clause to the non-restrictive clause, then it works correctly. DeepL gets the negative sentence wrong in Italian, but right in French, German, and Spanish, and the positive sentence right in all four languages.
En: The clock stopped working.
Ge: Die Uhr funktioniert nicht mehr.
GT: Die Uhr hat aufgehört zu arbeiten.
Bing, Systran: Die Uhr hörte auf zu arbeiten.
DeepL: Die Uhr funktioniert nicht mehr.

Hannah Best approves the translations "Die Uhr funktioniert nicht mehr" or "Die Uhr geht nicht mehr". Christina Behme suggests "Die Uhr ist stehengeblieben". The translations given by GT, Bing, and Systran are characterized as understandable, but no one would ever say them.
Fr: L'horloge a cessé de marcher.
En: The clock has stopped runnng.
GT: The clock has stopped walking.
Bing, Systran: The clock stopped walking.
DeepL: The clock stopped working. [Correct]
(Contributed by Léon Bottou).
Fr: Je ne peux pas nager.
En: I cannot swim. I can't swim.
GT: I can not swim. A small mistake, but an odd one for so simple a sentence.
Bing, Systran, DeepL: I can't swim. [Correct]
Fr: Elles
Sp: Ellas
GT, Systran. Ellas. [Correct]
Bing. Que.
DeepL. Ellos.
En: Pierre called to Marie and Estelle but they didn't hear him.
Fr: Pierre a appelé Marie et Estelle mais elles ne l'ont pas entendu.
GT, Bing, Systran, DeepL: Pierre a appelé Marie et Estelle mais ils ne l'ont pas entendu.
Fr. Les clés sont à ma mère.
En. The keys belong to my mother.
GT, Systran, The keys are to my mother.
Bing: The keys are my mother's [Correct].
DeepL: The keys belong to my mother. [Correct]
En. Pierre's parents miss him.
Fr. Pierre manque à ses parents.
GT, Bing, Systran. Les parents de Pierre lui manquent.
Bing Fr. Les parents de Pierre le manquent.
DeepL: Il manque aux parents de Pierre. [OK]
En. The dog is pregnant.
Fr. La chienne est enceinte.
GT, Bing, Systran, DeepL: Le chien est enceinte.
Adapted from a suggestion of Richard Socher.
En. Can you close the door, darling?
Fr. Peux-tu fermer la porte, chéri?
GT. Pouvez vous fermer la porte, chéri?
Bing, Systran. Peux-tu fermer la porte, chéri? [Correct]
DeepL: Tu peux fermer la porte, chéri? [Correct]

If this seems unfair, note that if you turn the sentence around, "Darling, can you close the door", GT correctly give "Chéri, peux-tu fermer la porte?". So GT is capable of detecting the relation between the endearment and the familiar pronoun one way, but not the other.
En. Marie said, "I will always be your friend."
Fr. Marie a dit: "Je serai toujours votre amie."
GT. Marie dit: "Je serai toujours votre amie [Correct]."
Systran. Marie a dit, "Je serais toujours ton ami."
Bing. Marie a dit, "je serai toujours votre ami."
DeepL. Marie a dit, "Je serais toujours ton amie." [Correct]
En. Pierre chatted with his neighbor and sent regards to her father.
Fr. Pierre a discuté avec son voisine et a envoyé des salutations à son père.
Google: Pierre a bavardé avec sa voisine et a envoyé ses salutations à son père. [Correct]
Systran: Pierre bavardait avec son voisin et envoya le regard à son père.
Bing. Pierre bavarda avec son voisin et envoya des salutations à son père.
DeepL. Pierre discuta avec son voisin et envoya des salutations à son père.
En. Pierre chatted with his neighbor and he sent regards to her father.
Fr. Pierre a discuté avec son voisine et il a envoyé des salutations à son père.
Google: Pierre a discuté avec sa voisin et il a envoyé des salutations à son père.
Systran: Pierre bavardait avec son voisin et envoya le regard à son père.
Bing. Pierre a bavardé avec son voisin et il a envoyé des salutations à son père.
DeepL. Pierre a discuté avec son voisine et a envoyé des salutations à son père. [Correct]
Example 13 and 14 differ only in that 14 has "he" in the second clause. The key point for the correctness of translation is catching that the neighbor must be female, since there is a reference to "her" father (obviously not Pierre's). In the first version, Google is the only one to catch this; in the second, DeepL is the only one. It is certainly striking that the six translations are all over the place, both in their choice of tense (perfect, imperfect, or simple past --- I have no idea myself which is best) and in their choice of "bavarder" or discuter for "chat".
En. My neighbor is a woman.
Fr. Ma voisine est une femme.
Bing, Systran.. Mon voisin est une femme.
GT, DeepL. Ma voisine est une femme. [Correct].
En. I always tell them that they're ugly, so they hit me.
Fr. Je leur dis toujours qu'ils sont laids, alors ils me frappent.
GT, Bing. Je leur dis toujours qu'ils sont laids, alors ils m'ont frappé.
Systran. Je leur dis toujours qu'ils sont moches, donc ils m'ont frappé.
DeepL. Je leur dis toujours qu'ils sont laids, alors ils me frappent. [Correct]
Fr. Je découpe un avocat pour le déjeuner.
En. I cut up an avocado for lunch.
GT. I cut out a lawyer for lunch.
Bing, Systran. I cut a lawyer for lunch.
DeepL: I'm cutting up a lawyer for lunch.
Thanks to Houda Bouamor for the suggestion.
Fr: On peut maintenant transférer des sous par Messenger en France.
En: One can now transfer money by Messenger in France.
GT: We can now transfer sub by Messenger in France.
Bing: We can now transfer sub by Messenger to France.
Systran: We can now transfer them by Messenger to France.
DeepL: You can now transfer money through Messenger in France. [Correct]
Yann LeCun posted the original French on Facebook 11/7/17.
German: Na bitte, geht doch. Herzlichen Glückwunsch! (Colloquial idiom, used when someone finally accomplishes something. 2/24/18)
En: See, you can do it! Congratulations!
GT: Oh look, it works. Congratulations!
Systran: Well, please, go on. Thank you very much.
Bing: Come on, please. Congratulations!
DeepL: There you go, there you go. Congratulations! [Pretty good]
Contributed by Christina Behme.
Fr: Je t'aime.
DeepL English: I love you. I love you.
DeepL Spanish: Te amo. Te amo.
Similarly doubled in German, Dutch, Polish, and Italian.
Very strange.
En: The soup is hot because it contains jalapenos.
Sp: La sopa está picante porque contiene jalapeños.
GT, Systran, Bing, DeepL: La sopa está caliente porque contiene jalapeños. (i.e. hot in temperature)
(Contributed by Robert Krovetz).
(This example is from 3/16/19).
En: Stars in couture scarves dance.
Admittedly this is something a little strange about this sentence, but it seems to me the meaning is perfectly clear: Some movie stars wearing couture scarves are dancing.
This is a modification of a garden-path sentence found by Leora Morgenstern in an NY Times op-ed: "Stars in jewels and couture scarf In-N-Out burgers on their way home from the Oscars."
Fr: Les stars en foulards couture dansent.
Ger: Stars in Couture-Schals tanzen.
Sp: Estrellas en bufandas de alta costura bailan.
It: Le stelle ballano in sciarpe couture. (Thanks to Francesca Rossi for all the information about Italian.)
GT:
Fr: Les stars des foulards couture dansent. (That is, the stars of couture scarves dance.)
Ger: Stars in Couture-Schals tanzen. (Correct.)
Sp: Baile de estrellas en bufandas de alta costura. (That is, the dance of stars in couture scarves -- a noun phrase rather than a sentence.)
It: Le stelle in sciarpe couture ballano. (Verb is in wrong place.)
Bing:
Fr: Étoiles dans la danse de foulards de couture. (That is, Stars in the dance of couture scarves -- a possible reading of the English, but not a natural one.)
Ger: Stars in Couture-Schals tanzen. [Correct]
Sp: Estrellas en el baile bufandas de alta costura. (Not sure what this would mean, but clearly not right, since "dance" is a noun.)
It: Stelle in sciarpe Couture danza. (Singular verb; should be plural).
Systran:
Fr: Les étoiles dans les foulards de couture dansent. [Correct].
Ger: Sterne in Couture-Tüchern tanzen. [Correct]
Sp: Las estrellas en pañuelos de couture bailan. [Correct]
It: Le stelle nelle sciarpe di moda ballano. (Verb is in the wrong place.)
DeepL:
Fr: Les stars des foulards de couture dansent. (Again, the stars of couture scarves)
Ger: Stars in der Couture tanzen Schals. ("Scarves" has become the object of "dance").
Sp: Las estrellas de las bufandas de alta costura bailan. (Again, this is the stars of couture scarves.)
It: Danzano stelle in sciarpe couture. (Verb in the wrong place).
Incidentally, the Stanford Parser gives the wrong parse for this sentence:
```
(ROOT
   (S
     (NP
       (NP (NNP Stars))
       (PP (IN in)
           (NP (NN couture))))
     (VP (VBZ scarves)
         (NP (NN dance)))
     (. .)))
```

Machine translation fails from Chinese found by Yuling Gu, June 20, 2018.