Phoenix I18n

In my previous articles I covered the various aspects of Elixir—a modern functional programming language. Today, however, I would like to step aside from the language itself and discuss a very fast and reliable MVC framework called Phoenix that is written in Elixir.

This framework emerged nearly five years ago and has received some traction since then. Of course, it is not as popular as Rails or Django yet, but it does have great potential and I really like it.

In this article we are going to see how to introduce I18n in Phoenix applications. What is I18n, you ask? Well, it is a numeronym that means "internationalization", as there are exactly 18 characters between the first letter "i" and the last "n". Probably, you have also met an L10n numeronym which means "localization". Developers these days are so lazy they can't even write a couple of extra characters, eh?

Internationalization is a very important process, especially if you foresee the application being used by people from all around the world. After all, not everyone knows English well, and having the app translated into a user's native language gives a good impression.

It appears that the process of translating Phoenix applications is somewhat different from, say, translating Rails apps (but quite similar to the same process in Django). To translate Phoenix applications, we use quite a popular solution called Gettext, which has been around for more than 25 years already. Gettext works with special types of files, namely PO and POT, and supports features like scoping, pluralization, and other goodies. 

So in this post I am going to explain to you what Gettext is, how PO differs from POT, how to localize messages in Phoenix, and where to store translations. Also we are going to see how to switch the application's locale and how to work with pluralization rules and domains.

Shall we start?

Internationalization With Gettext

Gettext is a battle-tested open-source internationalization tool initially introduced by Sun Microsystems in 1990. In 1995, GNU released its own version of Gettext, which is now considered to be the most popular out there (the latest version was 0.19.8 at the time of writing this article). Gettext may be used to create multilingual systems of any size and type, from web apps to operational systems. This solution is quite complex, and we are not going to discuss all its features, of course. The full Gettext documentation can be found at gnu.org.

Gettext provides you all the necessary tools to perform localization and presents some requirements on how translation files should be named and organized. Two file types are used to host translations: PO and MO.

PO (Portable Object) files store translations for given strings as well as pluralization rules and metadata. These files have quite a simple structure and can be easily edited by a human, so in this article we will stick to them. Each PO file contains translations (or part of the translations) for a single language and should be stored in a
directory named after this language: en, fr, de,
etc.

MO (Machine Object) files contain binary data not meant to be edited directly by a human. They are harder to work with, and discussing them is out of the scope of this article.

To make things more complex, there are also POT (Portable Object Template) files. They host only strings of data to translate, but not the translations themselves. Basically, POT files are used only as blueprints to create PO files for various locales.

Sample Phoenix Application

Okay, so now let's proceed to practice! If you'd like to follow along, make sure you have installed the following:

• 
  OTP (version 18 or higher)


• 
  Elixir (1.4+)


• 
  Phoenix framework (I'm going to be using version 1.3)

Create a new sample application without a database by running:

mix phx.new i18ndemo --no-ecto

--no-ecto says that the database should not be utilized by the app (Ecto is a tool to communicate with the DB itself). Note that the generator might require a couple of minutes to prepare everything.

Now use cd to go to the newly created i18ndemo folder and run the following command to boot the server:

mix phx.server

Next, open the browser and navigate to https://localhost:4000, where you should see a "Welcome to Phoenix!" message.

Hello, Gettext!

What's interesting about our Phoenix app and, specifically, the welcoming message is that Gettext is already being used by default. Go ahead and open the demo/lib/demo_web/templates/page/index.html.eex file which acts as a default starting page. Remove everything except for this code:

<div class="jumbotron">
  <h2><%= gettext "Welcome to %{name}!", name: "Phoenix" %></h2>
</div>

This welcoming message utilizes a gettext function which accepts a string to translate as the first argument. This string can be considered as a translation key, though it is somewhat different from the keys used in Rails I18n and some other frameworks. In Rails we would have used a key like page.welcome, whereas here the translated string is a key itself. So, if the translation cannot be found, we can display this string directly. Even a user who knows English poorly can get at least a basic sense of what's going on.

This approach is quite handy actually—stop for a second and think about it. You have an application where all messages are in English. If you'd like to internationalize it, in the simplest case all you have to do is wrap your messages with the gettext function and provide translations for them (later we will see that the process of extracting the keys can be easily automated, which speeds things up even more).

Okay, let's return to our small code snippet and take a look at the second argument passed to gettext: name: "Phoenix". This is a so-called binding—a parameter wrapped with %{} that we'd like to interpolate into the given translation. In this example, there is only one parameter called name.

We can also add one more message to this page for demonstration purposes: 

<div class="jumbotron">
  <h2><%= gettext "Welcome to %{name}!", name: "Phoenix" %></h2>
  <h3><%= gettext "We are using version %{version}",
  version: "1.3" %></h3>
</div>

Adding a New Translation

Now that we have two messages on the root page, where should we add translations for them? It appears that all translations are stored under the priv/gettext folder, which has a predefined structure. Let's take a moment to discuss how Gettext files should be organized (this applies not only to Phoenix but to any app using Gettext).

First of all, we should create a folder named after the locale it is going to store translations for. Inside, there should be a folder called LC_MESSAGES containing one or multiple .po files with the actual translations. In the simplest case, you'd have one default.po file per locale. default here is the domain's (or scope's) name. Domains are used to divide translations into various groups: for example, you might have domains named admin, wysiwig, cart, and other. This is convenient when you have a large application with hundreds of messages. For smaller apps, however, having a sole default domain is enough. 

So our file structure might look like this:

• en
  • LC_MESSAGES
    
    
    • default.po
    
    
    • admin.po
    
    
    
    
  
  


• ru
  • LC_MESSAGES
    
    
    • default.po
    
    
    • admin.po
    
    
    
    
  
  

To starting creating PO files, we first need the corresponding template (POT). We can create it manually, but I'm too lazy to do it this way. Let's run the following command instead:

mix gettext.extract

It is a very handy tool that scans the project's files and checks whether Gettext is used anywhere. After the script finishes its job, a new priv/gettext/default.pot file containing strings to translate will be created.

As we've already learned, POT files are templates, so they store only the keys themselves, not the translations, so do not modify such files manually. Open a newly created file and take a look at its contents:

## This file is a PO Template file.
##
## `msgid`s here are often extracted from source code.
## Add new translations manually only if they're dynamic
## translations that can't be statically extracted.
##
## Run `mix gettext.extract` to bring this file up to
## date. Leave `msgstr`s empty as changing them here as no
## effect: edit them in PO (`.po`) files instead.
msgid ""
msgstr ""

#: lib/demo_web/templates/page/index.html.eex:3
msgid "We are using version %{version}"
msgstr ""

#: lib/demo_web/templates/page/index.html.eex:2
msgid "Welcome to %{name}!"
msgstr ""

Convenient, isn't it? All our messages were inserted automatically, and we can easily see exactly where they are located. msgid, as you've probably guessed, is the key, whereas msgstr is going to contain a translation.

The next step is, of course, generating a PO file. Run:

mix gettext.merge priv/gettext

This script is going to utilize the default.pot template and create a default.po file in the priv/gettext/en/LC_MESSAGES folder.  For now, we have only an English locale, but support for another language will be added in the next section as well.

By the way, it is possible to create or update the POT template and all PO files in one go by using the following command:

mix gettext.extract --merge

Now let's open the priv/gettext/en/LC_MESSAGES/default.po file, which has the following contents:

## `msgid`s in this file come from POT (.pot) files.
##
## Do not add, change, or remove `msgid`s manually here as
## they're tied to the ones in the corresponding POT file
## (with the same domain).
##
## Use `mix gettext.extract --merge` or `mix gettext.merge`
## to merge POT files into PO files.
msgid ""
msgstr ""
"Language: en\n"

#: lib/demo_web/templates/page/index.html.eex:3
msgid "We are using version %{version}"
msgstr ""

#: lib/demo_web/templates/page/index.html.eex:2
msgid "Welcome to %{name}!"
msgstr ""

This is the file where we should perform the actual translation. Of course, it makes little sense to do so because the messages are already in English, so let's proceed to the next section and add support for a second language.

Multiple Locales

Naturally, the default locale for Phoenix applications is English, but this setting can be changed easily by tweaking the config/config.exs file. For example, let's set the default locale to Russian (feel free to stick with any other language of your choice):

config :demo, I18ndemoWeb.Gettext, default_locale: "ru"

It is also a good idea to specify the full list of all supported locales:

config :demo, I18ndemoWeb.Gettext, default_locale: "ru", locales: ~w(en ru)

Now what we need to do is generate a new PO file containing translations for the Russian locale. It can be done by running the gettext.merge script again, but with a --locale switch:

mix gettext.merge priv/gettext --locale ru

Obviously, a priv/gettext/ru/LC_MESSAGES folder with the .po files inside will be generated. Note, by the way, that apart from the default.po file, we also have errors.po. This is a default place to translate error messages, but in this article we are going to ignore it.

Now tweak the priv/gettext/ru/LC_MESSAGES/default.po by adding some translations:

#: lib/demo_web/templates/page/index.html.eex:3
msgid "We are using version %{version}"
msgstr "Используется версия %{version}"

#: lib/demo_web/templates/page/index.html.eex:2
msgid "Welcome to %{name}!"
msgstr "Добро пожаловать в приложение %{name}!"

Now, depending on the chosen locale, Phoenix will render either English or Russian translations. But hold on! How can we actually switch between locales in our application? Let's proceed to the next section and find out!

Switching Between Locales

Now that some translations are present, we need to enable our users to switch between locales. It appears that there is a third-party plug for that called set_locale. It works by extracting the chosen locale from the URL or Accept-Language HTTP header. So, to specify a locale in the URL, you would type https://localhost:4000/en/some_path. If the locale is not specified (or if an unsupported language was requested), one of two things will happen:

• If the request contains an Accept-Language HTTP header and this locale is supported, the user will be redirected to a page with the corresponding locale.


• Otherwise, the user will be automatically redirected to a URL that contains the code of the default locale.

Open the  mix.exs file and drop in set_locale to the deps function:

  defp deps do
    [
      # ...
      {:set_locale, "~> 0.2.1"}
    ]
  end

We must also add it to the application function:

  def application do
    [
      mod: {Demo.Application, []},
      extra_applications: [:logger, :runtime_tools, :set_locale]
    ]
  end

Next, install everything:

mix deps.get

Our router located at lib/demo_web/router.ex requires some changes as well. Specifically, we need to add a new plug to the :browser pipeline:

  pipeline :browser do
    # ...
    plug SetLocale, gettext: DemoWeb.Gettext, default_locale: "ru"
  end

Also, create a new scope:

  scope "/:locale", DemoWeb do
    pipe_through :browser

    get "/", PageController, :index
  end

And that's it! You can boot the server and navigate to https://localhost:4000/ru and https://localhost:4000/en. Note that the messages are translated properly, which is exactly what we need!

Alternatively, you may code a similar feature yourself by utilizing a Module plug. A small example can be found in the official Phoenix guide.

One last thing to mention is that in some cases you might need to enforce a specific locale. To do that, simply utilize a with_locale function:

Gettext.with_locale I18ndemoWeb.Gettext, "en", fn ->
  MyApp.I18ndemoWeb.gettext("test")
end

Pluralization

We have learned the fundamentals of using Gettext with Phoenix, so the time has come to discuss slightly more complex things. Pluralization is one of them. Basically, working with plural and singular forms is a very common though potentially complex task. Things are more or less obvious in English as you have "1 apple", "2 apples", "9000 apples" etc (though "1 ox", "2 oxen"!).

Unfortunately, in some other languages like Russian or Polish, the rules are more complex. For example, in the case of apples, you'd say "1 яблоко", "2 яблока", "9000 яблок". But luckily for us, Phoenix has a Gettext.Plural behavior (you may see the behavior in action in one of my previous articles) that supports many different languages. Therefore all we have to do is take advantage of the ngettext function.

This function accepts three required arguments: a string in singular form, a string in plural form, and count. The fourth argument is optional and can contain bindings that should be interpolated into the translation.

Let's see ngettext in action by saying how much money the user has by modifying the demo/lib/demo_web/templates/page/index.html.eex file:

<p>
  <%= ngettext "You have one buck. Ow :(", "You have %{count} bucks", 540 %>
</p>

%{count} is an interpolation that will be replaced with a number (540 in this case). Don't forget to update the template and all PO files after adding the above string:

mix gettext.extract --merge

You will see that a new block was added to both default.po files:

msgid "You have one buck. Ow :("
msgid_plural "You have %{count} bucks"
msgstr[0] ""
msgstr[1] ""

We have not one but two keys here at once: in singular and in plural forms. msgstr[0] is going to contain some text to display when there is only one message. msgstr[1], of course, contains the text to show when there are multiple messages. This is okay for English, but not enough for Russian where we need to introduce a third case: 

msgid "You have one buck. Ow :("
msgid_plural "You have %{count} bucks"
msgstr[0] "У 1 доллар. Маловато будет!"
msgstr[1] "У вас %{count} доллара"
msgstr[2] "У вас %{count} долларов"

Case 0 is used for 1 buck, and case 1 for zero or few bucks. Case 2 is used otherwise.

Scoping Translations With Domains

Another topic that I wanted to discuss in this article is devoted to domains. As we already know, domains are used to scope translations, mainly in large applications. Basically, they act like namespaces.

After all, you may end up in a situation when the same key is used in multiple places, but should be translated a bit differently. Or when you have way too many translations in a single default.po file and would like to split them somehow. That's when domains can come in really handy. 

Gettext supports multiple domains out of the box. All you have to do is utilize the dgettext function, which works nearly the same as gettext. The only difference is that it accepts the domain name as the first argument. For instance, let's introduce a notification domain to, well, display notifications. Add three more lines of code to the demo/lib/demo_web/templates/page/index.html.eex file:

<p>
  <%= dgettext "notifications", "Heads up: %{msg}", msg: "something has happened!" %>
</p>

Now we need to create new POT and PO files:

mix gettext.extract --merge

After the script finishes doing its job, notifications.pot as well as two notifications.po files will be created. Note once again that they are named after the domain. All you have to do now is add translation for the Russian language by modifying the priv/ru/LC_MESSAGES/notifications.po file:

msgid "Heads up: %{msg}}"
msgstr "Внимание: %{msg}"

What if you would like to pluralize a message stored under a given domain? This is as simple as utilizing a dngettext function. It works just like ngettext but also accepts a domain's name as the first argument:

dgettext "domain", "Singular string %{msg}", "Plural string %{msg}", 10, msg: "demo"

Conclusion

In this article, we have seen how to introduce internationalization in a Phoenix application with the help of Gettext. You have learned what Gettext is and what type of files it works with. We have this solution in action, have worked with PO and POT files, and utilized various Gettext functions.

Also we've seen a way to add support for multiple locales and added a way to easily switch between them. Lastly, we have seen how to employ pluralization rules and how to scope translations with the help of domains.

Hopefully, this article was useful to you! If you'd like to learn more about Gettext in the Phoenix framework, you may refer to the official guide, which provides useful examples and API reference for all the available functions.

I thank you for staying with me and see you soon!