You might think that advanced AI such as ChatGPT can easily handle conversations in any language but well… things aren't always so straightforward. We work with international businesses every day and we’ve seen some weird and funny stuff that sometimes may be harmful to your business.
In this article, I’ll talk about the limitations of using tools like OpenAI’s ChatGPT in terms of multilingual capabilities, why things are the way they are, and what can you do about it.
If you prefer to watch the video version, head straight to YouTube:
VIDEO
Using multilingual AI assistants in business Let’s imagine you’re running an e-commerce business. It’s 2023 and using AI chatbots and assistants, for example, for automating customer support, is getting more and more common. You know, these little chats in the bottom right of a website that are sometimes actually useful? Yeah, they’re probably using artificial intelligence (or…they hired a mechanical Turk).
The reasons why ChatGPT speaks worse in non-English languages Availability of training data That's mainly because models behind tools like ChatGPT were trained on a ton of internet data, which has way more content in English compared to other languages.
Response speed Let’s now move on to another aspect — the speed of responses. The need for speed. Everyone wants that, right? But if we want to speak about speed, we must understand how Large Language Models interpret and process text and what role tokenization plays in it.
What is tokenization and how it influences speed LLMs, to accurately understand language, use tokenization, which is the process of breaking down text into sub-word segments, so-called ‘tokens’, for processing. To illustrate how that works, let’s take the word 'tokenization'. It can be split into 'token-', '-iz-', and '-ation'. The words are split this way, so they can become building blocks that can be reused to build other words. We can re-use the building blocks of tokenization to build e.g. 'token -s', 'real-iz -e', and 'n-ation '.this article ).
What it means for business use cases You may still argue that conversational AI systems such as ChatGPT are doing fine and you’ll probably be right. For a personal user engaging in basic conversations with the AI system in their native language, the quality and speed are often sufficient for continued use. Even when mistakes arise, e.g. the agent makes a grammatical error, they can be easily overlooked because nothing of significance depends on it.
Potential solutions — make your AI Assistant fluent in non-English languages So what can you do about it? Seasoned Googlers know it’s usually better to search in English and then just translate whatever you need to your native language.
LLMs' struggle with brand names Let me use an example. Let’s take the heavy industry and English-to-Polish translation. Polish energy experts use the term "kocioł" to refer to what in English is a "burner" - a critical component in their energy systems. However, a direct translation might mistakenly render "kocioł" as "oven" in English, which is a common translation, but incorrect in this specific context, and it will make you look like a newbie.Custom Translations to give you control over this aspect.
Conclusion While LLMs are a whiz at many tasks, they aren’t always polyglots. especially in the business world. You need to account there for the varying speed and quality of responses when using these AI Systems in different languages. Low reliability, longer response time and higher cost (due to the increased numbers of tokens used in less popular languages) of multilingual handling may get your company in trouble. You need to be sure that the AI tools you use are correct every time and don’t discourage or — in the best case — confuse your customers.
.jpg)
.jpg)
.jpeg)
