Google Translate fails on simple sentences: Eleventh version April 2022
Ernie Davis
Note: Google Translate is
here.
Systran is
here.
Bing is
here.
DeepL is
here
This collection should under no circumstances be taken as any kind of
serious benchmark. Not only is it very small, but, as the title indicates,
it was
originally formed in 2016 as a collection where Google Translate, specifically,
failed; hence, those versions of this page were
inherently unfair to Google Translate. In more recent
pass, I have particularly aimed at DeepL, which is the acknowledged leader of
the pack for the languages it covers; hence, it is systematically unfair to
DeepL.
It is just
an unsystematic collection of translation problems that seem like they
should be easy, but trip up Google Translate, DeepL, and the other translation
programs, during the time period I have been doing this.
I check these intermittently to see
what progress has been made, and update the page accordingly. As you
can see by comparing with the earlier versions, the state of the art has
improved steadily, but there are still plenty of "easy" sentences that
trip up the best publicly available systems.
Earlier versions:
September - December 2016
December 2016
May 2017
August 2018
March 2019
June 2018
December 2019
April 2020
December 2020
July 2021
Note: I don't actually speak French, so I am grateful to Pascal Amsili for
his corrections on a number of items.
Updated 4/8/22
In this iteration, my rule has been to include examples if either (a) DeepL gets
it wrong (DeepL is the probably the strongest of the translation programs, though
Google Translate seems to be gaining ground lately, and is not easy to break)
(b) two or more of the others get it wrong;
or (c) the sentence is particularly simple, and one of the programs gets it wrong.
Examples
-
The mechanic who we called to fix the clock doesn't work any longer,
unfortunately.
Fr: Le mécanicien que nous avons appelé pour réparer l'horloge ne
travaille plus, malheureusement.
Systran, Bing: Le mécanicien que nous avons appelé pour réparer l'horloge ne
fonctionne plus, malheureusement.
Google, DeepL: Le mécanicien que nous avons appelé pour réparer l'horloge ne
travaille plus, malheureusement. [Correct]
The point here is that "works" in English can mean either "labors" or
"functions correctly".
Many other languages, including French, Spanish, Italian,
German have separate words for these.
The programs have certainly improved
over time in making this distinction (see the previous versions of this page)
but can still be fooled.
GT gets it right in French, Spanish, German but gets it wrong in Italian.
Systran and Bing get it wrong in all four languages.
DeepL gets it right in all four languages.
DeepL gets the much simpler sentence "The carpenter doesn't work any longer"
wrong translating into Portuguese. The other programs all get it right.
-
En: The dishwasher I got from the plumber works.
Ge: Der Geschirrspüler, den ich vom Klempner bekommen habe, funktioniert.
GT: Die Spülmaschine habe ich vom Klempnerwerk bekommen.
Bing: Die Spülmaschine, die ich vom Klempnerwerk bekommen habe.
Systran: Die Spülmaschine, die ich vom Klempner bekam, funktioniert. [Correct]
DeepL: Der Geschirrspüler, den ich vom Klempner bekommen habe, funktioniert.
[Correct]
-
Fr: Je ne peux pas nager.
En: I can't swim.
GT: I can not swim. [A small mistake, but an odd one for so simple a
sentence]
Bing, Systran, DeepL: I can't swim. [Correct]
-
Fr: Elles
Sp: Ellas
GT, Systran: Ellas [Correct]
Bing, DeepL: Ellos. DeepL offers as alternatives "En", "Se" and "Ellos." with a
a period, none of which is at all right.
-
En: Pierre called to Julia and her sister but they didn't hear him.
Fr: Pierre a appelé Julia et sa soeur mais elles ne l'ont pas entendu.
Systran, Bing:
Pierre a appelé Julia et sa soeur mais ils ne l'ont pas entendu.
GT, DeepL: Pierre a appelé Julia et sa soeur mais elles ne l'ont
pas entendu.
[Correct]
With a more subtle indicator of gender, all four programs get it wrong,
even though the sentence structure is simpler.
En: The doctor and her sister shouted to Jacques, but they weren't loud enough.
Fr:
Le docteur et sa sœur ont crié à Jacques, mais elles n'étaient pas assez forts.
GT, Bing,
Le médecin et sa sœur ont crié à Jacques, mais ils n'étaient pas assez forts.
Systran:
Le docteur et sa soeur crièrent à Jacques, mais ils n'étaient pas assez forts.
DeepL:
Le docteur et sa sœur ont crié à Jacques, mais ils n'étaient pas assez forts.
-
En. Pierre's parents miss him.
Fr. Pierre manque à ses parents.
GT. Les parents de Pierre lui manquent.
Bing Systran, DeepL: Il manque aux parents de Pierre. [OK]
-
En. They gave birth.
Fr. Elles ont acouché.
GT, Bing. Ils ont accouché.
DeepL. Ils ont donné naissance.
Systran. Elles ont acouché. [Correct]
Contributed by Francois Charton, 3/14/21.
-
En. They will be going to a school for girls.
Fr. Elles iront dans une école pour filles.
GT, Systran, DeepL. Ils iront dans une école pour filles.
Bing, DeepL. Elles iront dans une école pour filles. (Correct)
Contributed by Francois Charton, 3/14/21.
-
En. The dog that I gave to my sister is pregnant.
Fr: La chienne que j'ai donné à ma soeur est enceinte.
GT, Systran, Bing, DeepL: Le chien que j'ai donné à ma soeur est enceinte.
Adapted from a suggestion of Richard Socher.
In July 2021 GT and Bing got this correct.
-
En. Pierre's sister said, "I am your neighbor."
Fr: La soeur de Pierre a dit : "Je suis votre voisine."
Bing, DeepL: La soeur de Pierre a dit : "Je suis ton voisin."
GT, Systran, : La soeur de Pierre a dit : "Je suis ta voisine." [Correct]
In July 2021, Bing got this right.
-
En. My neighbor is a woman.
Fr. Ma voisine est une femme.
GT, Systran, Bing: Ma voisine est une femme. [Correct].
DeepL : Mon voisin est une femme. (DeepL got this right in June 2019,
but not in December 2019 through April 2022.)
-
En: I always tell my brothers that they fell on their head when they were
babies, so they hit me.
Fr: Je dis toujours à mes frères qu'ils sont tombés sur la tête
quand ils étaient bébés, alors ils me frappent.
GT, Systran, Bing, DeepL:
Je dis toujours à mes frères qu'ils sont tombés sur la tête
quand ils étaient bébés, alors ils m'ont frappé.
-
Fr. Je suis allé à l'épicerie et j'ai choisi un avocat.
En. I went to the grocery store, and I picked out an avocado.
GT, Bing: I went to the grocery store and chose a lawyer.
Systran: I went to the grocery store and picked a lawyer.
DeepL. I went to the grocery store, and chose an avocado. [Correct]
Thanks to Houda Bouamor for the suggestion.
-
En: The soup is hot because it contains jalapenos.
Sp: La sopa está picante porque contiene jalapeños.
GT, Systran, Bing: La sopa está caliente porque contiene jalapeños.
(i.e. hot in temperature)
DeepL: La sopa está picante porque contiene jalapeños. [Correct]
(Contributed by Robert Krovetz).
-
En: The florist says that Sam often comes into her shop but he
never buys her flowers.
Fr: Le fleuriste dit que Sam vient souvent dans son magasin mais qu'il n'achète
jamais ses fleurs.
GT, Bing, DeepL: Le fleuriste dit que Sam vient souvent dans sa
boutique mais qu'il ne lui achète jamais de fleurs.
Systran:
Le fleuriste dit que Sam vient souvent dans son magasin mais qu'il n'achète
jamais ses fleurs. [Correct]
Adapted from a cartoon pointed out by Christina Behme, which goes the other way:
Marriage counsellor: Your wife says you never buy her flowers.
Clueless husband: To be honest, I never knew she sold flowers.
--- from "Sad and Useless Humor".
-
Fr: J'ai acheté des fleurs à Mary car elle n'en avait pas vendu de toute la
journée.
En: I bought flowers from Mary because she hadn't sold any all day.
GT: I bought flowers for Mary because she hadn't sold any all day.
Systran: I bought Mary flowers because she hadn't sold any all day.
Bing, DeepL: I bought flowers from Mary because she hadn't sold any all day.
[Correct]
Note "J'ai acheté des fleurs à Mary" can mean either "I bought flowers from
Mary" or "I bought flowers for Mary".
-
Fr: Emily a acheté des fleurs à Pierre, pas à Jacques, parce que Jacques
a déjà reçu des fleurs de Richard.
En: Emily bought flowers for Pierre, not for Jacques, because Jacques already
got flowers from Richard.
GT: Emily bought flowers for Pierre, not Jacques, because Jacques has
already received flowers from Richard. [Correct]
Systran: Emily bought flowers from Pierre, not from Jacques, because Jacques has
already received flowers from Richard.
Bing, DeepL: Emily bought flowers from Peter, not James, because James has
already received flowers from Richard.
-
Ger: Johann bekam zwei Beagles im Tierheim, aber sie griffen seine Tochter an,
also brachte er sie zurück.
En: Johann got two beagles at the pound, but they attacked his
daughter, so he brought them back.
GT,
Systran, Bing:
Johann got two beagles at the shelter, but they attacked his daughter,
so he brought her back.
Bing: John got two beagles in the shelter, but they attacked his daughter,
so he brought her back.
DeepL: Johann got two beagles at the pound, but they attacked his
daughter, so he brought them back. [Correct]
-
En: The workers at the canning factory wash fish.
Fr: Les ouvriers de la conserverie lavent des poissons.
GT, Systran, Bing, DeepL: Les ouvriers de la conserverie lavent le poisson.
In 12/20, DeepL got this right, but as of 7/21 and 4/22 DeepL gets it wrong.
Machine translation fails from Chinese
found by Yuling Gu, June 20, 2018.