By Craig Silverstein

Khan Academy started as a collection of videos, but now has over 100,000 pieces of written content, from exercises to articles to programming challenges. All of these are now available in multiple languages. But the Khan Academy codebase was originally written to be English-only. We had to retrofit the codebase to support internationalization (i18n) and localization (l10n) of written content after a lot of infrastructure was already in place. As most guides to i18n and l10n will tell you, life is much happier if you design for them before the fact. This was not an option for us.

The task was made more difficult by the variety of technologies we’ve used over the years. We use 5 different HTML-rendering technologies:

1. jinja2 (for our Python server code)
2. react (for our modern JavaScript code)
3. raw JavaScript (for older JavaScript code)
4. handlebars (for HTML that has to be rendered via both Python and JavaScript)
5. Python (for our very old Python server code, which just wrote HTML directly from source)

and all of them needed to be converted to add i18n markup.

i18nize-templates

There are plenty of tools out there to handle the actual translation of strings; we use Babel and Jed.

And there are plenty of services out there to manage the actual translation of strings; we use Crowdin.

What is lacking is a tool that will mark up all the natural language text in your code and templates; this is the process that determines what text to show to translators. For this task, we developed i18nize-templates, a tool for finding natural language text in a variety of templating languages, and automatically munging it to be i18n-aware.

::: {.label} sample input (jinja2): :::

“` {.html}

Badges

“` ::: {.label} **sample output**: ::: “` {.html}

{{ _(“Badges”) }}

“` ::: {.label} **sample input (handlebars)**: ::: “` {.html} From the author: {{{ discussionFormControls “Post feedback” }}} “` ::: {.label} **sample output**: ::: “` {.html} {{#_}}From the author:{{/_}} {{#_}}{{{ discussionFormControls “Post feedback” }}}{{/_}} “` i18nize-templates isn’t magic: it can’t convert `item{#if n != 1#}s{#endif}` to the proper ngettext call. But it can reduce the time needed to annotate templates by over 90{8f4f32d1c7b49be91940769544f9246738fa263bda282b485c2c255cc064ffb6}. i18nize-templates can convert raw HTML, jinja2 templates, and handlebars templates. (Due to similarities between jinja2 and django, it may also support django templates, though this is untested.) It can also convert text files written using jinja2 or handlebars. Using i18nize-templates ======================= We are pleased to announce i18nize-templates as an open source Python module. You can install i18nize-templates via “` {.sh} $ pip install i18nize-templates “` Rewriting templates ——————- “` {.sh} $ pip install i18nize-templates $ echo “Hello {{world}}!” | i18nize-templates i18nizing – {{ _(“Hello {8f4f32d1c7b49be91940769544f9246738fa263bda282b485c2c255cc064ffb6}(world)s!”, world=world) }} $ echo “Hello {{world}}!” | i18nize-templates –handlebars i18nizing – {{#_}}Hello {{world}}!{{/_}} “` Extracting natural language text ——————————– You can also just use i18nize-templates as a Python library to easily extract runs of natural language text from HTML and templated-HTML (or templated-text) documents. Here’s a Python snippet we use to fake-translate our website into our testing language, called box-language (): “` {.python} import re import i18nize_templates def translate_to_boxes(jinja2_file_contents): def parser_callback(s, segment_separates_nltext): if s is None: return ” # called at end-of-parse elif (segment_separates_nltext or (s.startswith(‘{{‘) and s.endswith(‘}}’)) or (s.startswith(‘<') and s.endswith('>‘))): return s # do not translate else: return re.sub(r’w’, u’u25a1′, s) # alnum -> box parser = i18nize_templates.Jinja2HtmlLexer(parser_callback) return parser.parse(jinaj2_file_contents) “` Extracting JavaScript ——————— Sometimes, i18nize-templates is useful just because it knows how to parse templated HTML. For instance, for some of our code, we need to extract JavaScript (inside `