An Overview of Amazon Translate

In previous Medium articles, I presented Amazon Polly for text to speech production, and Amazon Transcribe for speech to text production. In this article, I will present another Amazon AI/ML service: Amazon Translate.

Whether we want to translate the text for our website, provide localized text for a software application, or real-time translation, Amazon Translate is the solution. The service accepts text in one language and converts it to text in a different language. As mentioned in the documentation, the goal is not to provide a “perfect” translation, but good enough for the translated text to be meaningful to the reader.

Amazon Translate is part of the AI/ML family of services available in Amazon Web Services. It relies upon neural machine translation engines using a collection of evolving datasets that provide the basis for Amazon Translate’s continuous learning and ability to provide increasingly accurate translations.

Before the release of Amazon Translate in 2017, software developers had to use translation services relying upon a static heuristic model, and which often required including significant third party code into their application. With Amazon Translate, both real-time and batch translation services are available using an API call from the AWS Software Development Kit (SDK) for your specific programming language. Using an existing API simplifies the ability to localize a web site or provide multi-lingual data by pre-processing the data or translating it on-demand.

Not every word will translate accurately into another language. Company names, product brand names, characters, and other terms unique to your industry, company, or culture may not have a corresponding translation into another language. Amazon Translate solves this problem by providing a Custom Terminology capability allowing you to define how the specific term is translated into a specific language. This capability should decrease the number of translations required from professional translators, meaning lower costs for your organization and software development project.

Whether it’s just a few words or large volumes of text, Amazon Translate scales easily as your translation needs increase. The service provides consistently fast and reliable translations, regardless of the volume of translation requests you make.

The Amazon Translate Overview Page, highlights several uses for Amazon Translate include analyzing social media content to understand consumer viewpoints related to your product, regardless of the language the comments are written in. This solution uses both Amazon Translate and Amazon Comprehend for the Natural Language Processing analysis.

Another example is the real-time translation of both dynamic and static information. Continuous translation of real-time information is difficult to keep up with because the volume of information and the number of translators you can afford to hire are inversely aligned. For example, web sites, gaming content, audio, and video can all benefit from real-time translation.

One example is real-time translation for support and help-desk services. Users are not limited by their knowledge of the language your corporation uses. A Spanish speaking user (for example) can type in Spanish, the text is translated to English using Amazon Translate and displayed to the support agent. The support agent can then respond in English and the text is then translated to Spanish. Any text communication could be processed using Amazon Translate before sending it to the desired audience.

Amazon Translate is not limited to translations between one language and English. It can translate from and to any supported language. The available languages are listed in the Amazon Translate documentation. When associating a source language and a target language, you get a language pair.

As of June 27, 2020, Amazon Translate supports 55 languages.

Amazon Translate uses a neural translation engine as opposed to statistical and rule-based translation engines. We should briefly understand the differences between these approaches.

Rule-based translation relies upon linguistic rules and massive dictionaries. There is a linguistic rule set and a bilingual dictionary for each language pair. Additionally, the rule-based engine requires “extensive lexicons with morphological, syntactic, and semantic information, and large sets of rules”. This means there is a large up-front investment in creating a translation for a specific language pair, and maintaining the linguistic rules and dictionaries.

Statistical translation uses statistical analysis models derived from monolingual and bilingual dictionaries. Building the translation model can be quick, but the quality of the dictionaries and the number of words in those dictionaries impact the translation and its accuracy. While the statistical model can determine the semantics and context, we can more aptly compare statistical translation to a paper bilingual dictionary, like English-Spanish/Spanish-English.

The challenge with the rule-based and statistical models is the effort to create the initial linguistic rules and dictionaries, followed by the ongoing maintenance, followed by the limited contextual analysis before the translation is completed. This makes the initial development and ongoing maintenance costs considerable.

Neural Translation engines like Amazon Translate can produce more accurate translations by using deep learning techniques implemented through a neural network. The neural network considers the entire context of the sentence to translate instead of a few words either before or after the word being translated.

To perform a translation, which we will see in the next section, we must have a UTF-8 encoded text file, which is known as the source text. The result of the translation is UTF-8 encoded output text, which could have a different number of characters depending upon the specific language pair.

The translation engine reads the source text one sentence at a time and analyzes each word to construct a semantic representation. This work is performed by the encoder. Once the semantic representation is created, the decoder uses it to translate one word at a time.

With Amazon Translate we can choose to not specify a specific source language which is beneficial in certain contexts where you may not know what language the user is communicating in. For example, a help desk chat application might have to support all languages to English. To specify automatic language detection, use auto as the source language.

Using automatic language detection results in additional charges as Amazon Comprehend’s Natural Language Professor (NLP) is used to detect the language.

Julien Simon’s 2017 Medium article A quick look at Machine Translation with Amazon Translate is a little dated, but he introduces the use of Amazon SageMaker in the implementation of Amazon Translate.

Amazon Translate is available in the AWS Management Console, through the AWS Command Line Interface and the AWS Software Development Kit. Let’s look at using Amazon Translate in the Management Console and the Software Development Kit (SDK).

This article does not explain how to set up an AWS account, the CLI, or the SDK.

Before looking at using the SDK to submit a translation request, we are going to see how to submit a translation request using the console. Once you have logged in to the AWS Management Console, navigate to Amazon Translate and, expand the sidebar to see the available options. Otherwise, the overview page allows you to go immediately to real-time translation.

Amazon Translate Overview Page (by author)
Amazon Translate Overview Page (by author)

Clicking on the real-time translation menu item displays a view where you can select the source and target language pair, and provide the source text.

Real-time translation view (by author)
Real-time translation view (by author)

As you type the source text, the translated text appears in the other text box. Here is an example of translating English into French.

Real-time translation of English to French (by author)
Real-time translation of English to French (by author)

This image shows the option to use the Custom Terminology option, where you can insert unique words for your specific case and define how they will be translated into other languages. This option is disabled by default.

The real-time translation page additionally shows the JSON which would be needed for real-time translation of the source text using the CLI or SDK. If no text is provided in the source box, the JSON includes the configuration for the language pair. This allows you to use variable text in your application when performing the translation.

JSON Application Integration (by author)
JSON Application Integration (by author)

The real-time translation as illustrated in the AWS Management Console is synchronous — as you type the text, it is immediately translated. However, this approach has a limit of 5,000 characters. In many cases, this is enough for many of the use cases presented thus far.

Asynchronous or batch translation is used for larger amounts of text or any situation where you don’t need near-instantaneous translation. Batch translation uses an S3 bucket to retrieve the source file and store the translated text. Each source file can be up to 5 GB in size.

Using batch translation involves

  • uploading the documents for translation to an S3 bucket;
  • specifying the S3 bucket where the translated results should be stored; and,
  • providing an IAM role with read access to the source and read/write access to the destination S3 buckets.

If you make use of prefixes (folders), then you could use a single bucket with different prefixes. With the source text copied to the specified S3 bucket, launch the batch translation request and wait for the results.

During the research for this article, I observed the batch translation times can be 15 minutes for even small amounts of text.

The obvious caveat applies here — all the files placed into the S3 bucket must have the same original language and be translated to the same target language. The target language is specified when the batch job is submitted, allowing you to translate the source text into many different languages. As we will see later, the results are stored in an S3 bucket/prefix which is specific to that translation job.

Using the AWS Software Development Kit (SDK) to interact with Amazon Translate is similar to other AWS services. This part of the article will look at using the SDK to submit translation jobs and get status on batch translation jobs.

A simplistic example for submitting a translation request looks like:

client = boto3.client('translate')
response = client.translate_text(
Text=text, SourceLanguageCode=source, TargetLanguageCode=target)

Building a more complex solution could execute like this:

$ ./xlate submit
Enter text to translate: Today is Saturday
Enter source language: en
Enter target language: fr
Aujourd'hui c'est samedi

This sample program illustrates the requirements for the translate_text SDK call including the language code for the source and target language, along with the text to translate. This highlights an important aspect of the SDK, which is the use of a language code, which is a pre-defined 2-5 character code. Knowing what language code to use involves checking the list of supported languages.

If not provided on the command line, the illustrated sample program (implemented in Python using Click) prompts for the required information to complete the request. The response is the translated text.

Before diving into batch translation mode, we need to discuss the IAM requirements for the batch translation task. This is because when the batch translation job is submitted, you have to specify a data access role providing Amazon Translate with the appropriate permissions.

Here is a sample policy statement that can be applied to the data access role. These are the minimum permissions needed for this role for the translation job to succeed.

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"translate:*",
"comprehend:DetectDominantLanguage",
"cloudwatch:GetMetricStatistics",
"cloudwatch:ListMetrics"
],
"Resource": "*"
}
]
}

Submitting a batch translation job through the SDK involves providing the information shown here.

response = client.start_text_translation_job(
JobName=name,
InputDataConfig={
'S3Uri': s3_source,
'ContentType': s3_source_type
},
OutputDataConfig={
'S3Uri': s3_output
},
DataAccessRoleArn=role,
SourceLanguageCode=source_language,
TargetLanguageCodes=[
target_language
]
)

Using a sample program (implemented in Python using Click) to illustrate using the SDK, this is what the command line looks like (split onto multiple lines for readability):

$ ./xlate submit-batch 
--s3source=s3://xlate-media/in/
--s3output=s3://xlate-media/out/
--s3source-type=plain
--fromlanguage=en
--tolanguage=fr
--role="arn:aws:iam::ACCOUNT_ID:role/translate-access-role"
--name=test17
--region=us-east-1

The s3Source and s3Output values use prefixes to define where the source files are located and where the translated files should be placed.

The S3 URI values must end with a ‘/’ or an error is produced by the SDK when the translation job is submitted.

The s3source-type value defines if the file is plain text or HTML. My sample program converts these values to text/plain or text/html. This simplifies entering this long command line. The fromlanguage and tolanguage values define the source and destination language codes. It is not possible to use the language name like "Spanish"; you must use the appropriate language code, 'es' in the SDK API call (although if you are building an interface to the SDK, you could implement the language names and translate them to the language codes.

The role defines the IAM role which Amazon Translate will use when performing the batch translation. If the role is not provided, an error is provided when the job is submitted. The name value provides a name for the job, The name is used to retrieve the job status after the translation is submitted. We can see the status of the job by using the list_jobs SDK function call (sample from my demonstration code):

id: 1232f451846b2b7506f4d52910bbe418 name: test18 status: COMPLETED 
submitted: 2020-07-22 22:44:27.928000-05:00 end: 2020-07-22 22:54:42.731000-05:00
output: s3://xlate-media/out/548985610555-TranslateText-1232f451846b2b7506f4d52910bbe418/
Your job has completed successfully.

Getting the results of the batch translation job involves retrieving them from the S3 URI specified when the job was submitted. But, it is a little more complicated, because each translation job is itself saved in a folder below the S3 URI.

If there is more than one file in the S3 input bucket, each file is started as an individual batch translation job, each with a unique JobId, although the job name is applied to each job.

In this example, our job results are found in

s3://xlate-media/out/548985610555-TranslateText-1232f451846b2b7506f4d52910bbe418/

If we use the AWS CLI, we can see the contents of the bucket using the command

aws s3 ls s3://xlate-media/out/548985610555-TranslateText-1232f451846b2b7506f4d52910bbe418/

The results show the file with our translation, and a folder named details, which contains a JSON file with additional data on the job.

The source text is

This is a test file, which is written in English.  This file is used for Amazon Translate to test the batch translation capability.

The output text reads (in French),

Ceci est un fichier de test, qui est écrit en anglais. Ce fichier est utilisé pour Amazon Translate pour tester le
capacité de traduction par lots.

There are a lot more capabilities in Amazon Translate which I haven’t touched on in this article. If you are building applications needing language localizations, you could process the text through Amazon Translate. Using interactive services like chats, or to translate text on-demand, it is possible to provide an improved customer experience.

The obvious question for any translation service is “How accurate is it?”. To answer this question, I reached out to my LinkedIn network for people fluent in English-French and English-Spanish language pairs. They were provided with the translated text and asked to provide the English translation. I asked them to create a paragraph in either French or Spanish and the corresponding English translation. I then took the French or Spanish text and translated it to English and compared it with their provided translation.

The results of the tests are documented in this article and consolidated here for brevity. If you would like to read the detailed comparison check out that related article.

The Amazon Translate results were good when compared to the original English text or the English translation provided by the translator. As you look at the translated text and compare it with the original, it is evident that while the words used may be different, e.g. company instead of corporation, the meaning of the text is evident.

One caveat is content which has multiple formats, like programming documentation where there is both explanatory text and code samples.  Amazon Translate can't distinguish between content which should and should not be translated, meaning that the code sample is also translated which can have untended consequences.

Amazon Translate follows the pay-as-you-go pricing model like many AWS services. Free-Tier usage is limited to 2 million characters per month for 12 months. The 12 month free tier period starts the first day you use the service. The 2 million character limit includes whitespace. Once your Free-Tier is over, or you exceed 2 million characters ina month, the cost is USD 15.00 per million characters.

Any content which you create and need converting to a different language for your audience can be prepared in advance and published. If you don’t know what languages you want to prepare in advance, then you could select to perform a real-time translation when the consumer requests it.

That being said, using Amazon Translate for any specific project should depend on your unique situation. These factors should be considered in your decision:

  1. When you need near real-time translation, it is appropriate to use Amazon Translate.
  2. When 100% accuracy is less important than ”getting the point across”, Amazon Translate makes sense.
  3. For long, complex documents where accuracy is more important, the use of a professional translator is more appropriate.
  4. Text involving cultural specific language, events, or history, may be more appropriate for a professional translator.
  5. Text which is ”short-lived”, such as a chat session or e-mail, is suitable for Amazon Translate. It is important to consider that issues could arise because people may not always type what they mean.
  6. Text which is intended for storage in an archival system or used for reference by others may benefit from a professional translator.
  7. Be careful of redundant phrases. Be clear and concise.
  8. Validate the translation of specific terms like “help-desk”, can translate to another language, but may not be as effective as using the term appropriate for that language.

The service is easy to use and as outlined in the additional article, the translation results are effective.

Thank you to Chanelle Dupuis, for her assistance with the English-French part of this evaluation. Ms. Dupuis is currently working toward her Ph.D. in French Studies at Brown University and is fluent in French, Spanish, Portuguese, and English.

Thank you to Victoria Martinez Adalid, for her assistance with the English-Spanish part of the evaluation. Ms. Adalid is a Professional English-Spanish Translator.

And finally, thank you to my wife Sandii Hare for painstakingly reading every article and listening to my ongoing ramblings about what I learn in the research process.

A Deep Dive into Amazon Polly

A Five Overview of AWS Transcribe

Amazon Comprehend

Amazon Polly: Bringing Audio to my Medium Articles

Amazon SageMaker

Amazon Translate Overview

A Quick look at Machine Translation with Amazon Translate

AWS Software Development Kit and Programming Tools

Introduction to Amazon Translate Digital Training

What is Machine Translation? Rule Based Machine Translation vs. Statistical Machine Translation

Chris is a highly-skilled Information Technology, AWS Cloud, Training and Security Professional bringing cloud, security, training, and process engineering leadership to simplify and deliver high-quality products. He is the co-author of seven books and author of more than 70 articles and book chapters in technical, management, and information security publications. His extensive technology, information security, and training experience make him a key resource who can help companies through technical challenges.

This article is Copyright © 2020, Chris Hare.

Written by

Chris is the co-author of seven books and author of more than 70 articles and book chapters in technical, management, and information security publications.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store