Google has added to its translation app this new AR feature that can let you translate in real time some text into another language. At the moment not all the languages are supported and some have a better accuracy than others.
I’ve tried to translate a couple of things from Italian to English and I didn’t have the results I’ve expected. I tried a couple of pictures from a Pokemon manga and I’ve noticed that it has big problem in translating almost everything. If there are only few words it can guess a couple of them, but with the increasing number of words it’s really bad. Moreover, it completely can’t recognize a word if it’s split into two lines. It has also problems in translating words in particular fonts (Immagine Pokemon).
Finally I tried with a card box with only few words and the translation was really good.
I think that the service is not that bad, but Google needs to improve accuracy, especially when there are many words, when the font is not standard and when words are split. The main problem is that if the translation is not accurate it’s useless because you can’t even see the text. So, I would like to add the possibility to split the screen in two: on one side, the phone can show the normal text and on the other side the translated text. In this way, someone can mix the translation (even not perfect and with some missing words) with its own knowledge of the language.
Imagining a future in which we can use this AR translation on a particular device such as AR glasses, I think there would happen two possible scenarios. If the application will be more accurate translating almost all in the best way it would be an amazing tool (probably all the human translators don’t think the same).
Conversely, if the system is as inaccurate as it is now, it could be a problem because if in the smartphone you can see the real text with your eyes, in this case you should put away the glasses to control the real text, then put them on again, etc. and that’s not a comfortable thing to do. Moreover, in this way the app translates all the words with its own algorithm. Imagine that at Google works some bad people that want to change the meaning of something with a specific (wrong) translation, they could do it and you won’t recognize it because the real text is hidden. So even in this case a good solution could be to add a translation, but at the same way leave the user free to choose if hiding completely the real text or just. In this way we can take the pro to have a live translation in loco and we don’t have cons such as possible unrecognizable and ill-intentioned mistakes.
In conclusion, I think that live translations in AR are the future and I'm sure that this feature will be improved a lot in the next years. At the moment is not so realiable, but in some situations such as translating few words from a language that the user don't know at all (in my case it could be russian/chinese street signals) could be very useful. Hiding the real text is a double-edged sword: if the translation is correct it could be awesome, but if it's not it can be a mess.