Getting localized in a wagtail way

A bit of history

So I have some kind of website based on Django since 2017 domain name I got few years earlier. For a first years there were no Wagtail and almost all pages were static and localization was managed by django generated .po files and for dynamic content such as description of jobs at Resume was made via django-modeltranslation.

In September 2020, I suddenly had a strong desire to be able to manage content via some kind of Admin panel without spending too much time on the development of that functionality. A stranger from the internet suggested using Wagtail for managing content. So, I tried it and chose to move forward with it. However, there was an issue since it was Wagtail 2.10, it doesn't provided specific way of support multi-language content. There were mentioned a wagtail-modeltranslation package. Since it's based on django-modeltranslation I didn't see anything bad in using it. Everything about that functionality works perfectly, with the caveat that I didn't update content on the website very often.

Motivation

Four years flew by. Year 2024.

And while I'm on the job hunt and have some spare time. I decided to update everything on this site and write some posts. That is, to fulfill a promise I made seven years ago about there will be a blog.

I didn't want to write several versions of posts for each language at once. But I would like to be able to somehow link different versions of posts between them. That the user could change the language and get to the post with his chosen language. And I wanted to be able to make the content more dynamic, i.e. start using StreamField. And I don't want to write my own code, it's better that this code was already written by someone and I didn't have to invent my own bicycles. I think it's easier to fix bugs in an open source project than to write own code first and then look for bugs in it and fix them by yourself mainly because no one else using that code.

But ran into a few complications:

After adding Page even that shouldn't be translated it required to be registered with within modeltranslation. Without this it will be always appears error after attempt to save updated Page. I think it's a minor issue.
After attempt to create localization fields for StreamField and run makemigrations command I got an error: "StreamField is not supported by modeltranslation." Seems it could be fixed but anyway looks strange. And if it's something wrong with my configuration I will require dive deep into that issue and make some changes in code. It will be a medium issue.
About managing content so I though that all posts will be written on the single language and then will be translated to other languages. So to link those Blog Post it will require from me make my own implementation of this mechanic. (medium)
It appears that since wagtail 2.11 just after a few weeks after I integrated it on my website. There's a recommended solution for content localization. It's not a problem, but sometimes it's better to take the beaten path so there are fewer problems on the road. Or only problems that are well explored.

So I choose to move forward with migrations to wagtail-localize. Yeah wagtail right now have some kind integrated functionality but I didn't find there some functionality that could be useful for me in the future like "Synchronize content from another locale". That I planning to use so the version on Russian language will have all pages that exists in English tree even if it not localized.

How localization works

To change something from one approach to another, it is often useful to know how it works internally.

modeltranslation way

So basically it require developer to register each model with decorator and supply list of fields that should be translated. After that with DB migrations fields for each locale appears in the DB. For example for wagtail Page model with my configuration with two languages en-English and ru-Russian field that model get additionally to "title" fields "title_en" and "title_ru". And wagtail-modeltranslation make some monkey-patching so only those specific fields for each locale (in my case it's "title_en" and "title_ru") appears in the Admin panel.

wagtail (or wagtail-localization) way

Wagtail creates model Locale where could be stored all locales that could be enabled in the settings file of the project. And each instance of Page model contain foreign key to that Locale model and "translation_key" with UUID so it's possible to select all versions of the Page for all locales by making query with that UUID. And there's a multiple root pages and each locale have it's own tree so Administrator could gradually give permission only a specific language branches to Editor. Edit pages independently for each locale independently is also possible.

Planning of migration process

First, I went in search of some ready-made instruction. And found a tutorial how to migrate from wagtail-modeltranslation to wagtail-localize as PR at GitHub . In short, it offers to remove one package first, and put another one. But in this case we will lose information stored in specialized localized fields of the model. And will require manually copy localized content to newly created locale page tree.

So I'm thinking of going the other way:

I'm gonna put in two packages.
Copy pages tree. So each locale will have the same tree. So it will be two copy of the same Page with the same information each field title, title_en, title_ru.
Copy information from localized field to base field. So for English locale and I will copy title_en to title and for Russian title_ru to title.
De-register translation for those models and migrate DB. So title_en and title_ru will be removed from Pages.

Probably those process could be made without downtime and users wont notices the changes.

Migration to Wagtail Localize

I made backup and prepare locale environment where I will check that all stages works well. And then repeat the same process on the production.

Followed official documentation of installation of wagtail-localize I added

WAGTAIL_I18N_ENABLED = True

near

USE_I18N = True

Also I set

WAGTAIL_LOCALIZE_DEFAULT_TRANSLATION_MODE = "simple"

because I believe that depending on language content could be changed. So if I would like to translation of the page will be synced then I can manually set that via interface. And content on the pages aren't synced by default. It's also required to keep those changes from localized fields to base fields otherwise that content will be invisible and user will required to disable synchronization of the page manually. Well maybe in the future it will work some other way and there won't be such problems. Or maybe it's just me who didn't see something and ran into this problem.

And made setting WAGTAIL_CONTENT_LANGUAGES equal to LANGUAGES

WAGTAIL_CONTENT_LANGUAGES = LANGUAGES = [
    ("en", _("English")),
    ("ru", _("Russian")),
]

Then in Wagtail admin interface going to Settings -> Locales can see that just default Locale for the project(in my case it's English) -> Add Locale -> Choosing secondary language(Russian in my case) and enable synchronization from default locale(English). From that moment each page should have entities for each locale.

It's important after locale added submit those pages from that locale to translation.

Choose root page for created locale
...(triple point) -> Edit
Translate this page
Enable checkbox Include subtree
Submit

Now time to copy value from localized fields to original. I found manage command update_translation_fields it's filling translation fields with original values. So I looking on that manage command made another manage command that will make same updates but in reverse order. It should find all pages that have translated fields and copy value from it to original field based on FK locale. I made file located at -=root_project_package=-/management/commands/copy_localized_text_to_base_field.py with following content:

from django.core.management.base import BaseCommand
from django.db.models import F

from modeltranslation.translator import translator
from modeltranslation.utils import build_localized_fieldname
from wagtail.models import Locale


class Command(BaseCommand):
    help = "Copy content of localized via modeltranslation fields to base field."

    def handle(self, *args, **options):
        # get all models excluding proxy- and not managed models
        models = translator.get_registered_models(abstract=False)
        models = [m for m in models if not m._meta.proxy and m._meta.managed]
        # get only linked with locale models (should be all models inherited from Page)
        models = [m for m in models if hasattr(m, "locale")]

        locale_id_to_code = dict(Locale.objects.values_list("id", "language_code"))
        for model in models:
            self.stdout.write(f"Processing model {model.get_verbose_name()}")
            opts = translator.get_options_for_model(model)
            for locale_id, lang in locale_id_to_code.items():
                for field_name in opts.all_fields.keys():
                    def_lang_fieldname = build_localized_fieldname(field_name, lang)
                    result = (
                        model._base_manager.rewrite(False)
                        .filter(
                            **{
                                "locale": locale_id,
                                f"{def_lang_fieldname}__isnull": False,
                            }
                        )
                        .exclude(**{field_name: F(def_lang_fieldname)})
                        .update(**{field_name: F(def_lang_fieldname)})
                    )
                    self.stdout.write(
                        f"Updated {result} records for {field_name} field in {lang} locale"
                    )

        self.stdout.write(
            self.style.SUCCESS("Successfully copied localized text to base fields.")
        )

Important note: It tested with django-modeltranslation==v0.19.3 on different versions it could required some changes. Even with a different project configuration, some tweaking may be required. In this case it's one time command and of course it's better to have unit tests that will check all possible cases but I don't think this extra work will ever pay off.

Now necessary choose root page for newly created locale and choose translate it and all child pages.

Then run a manage command

python3 manage.py copy_localized_text_to_base_field

On that stage there should be multiple copy of the same Page in the DB one for each locale and at fields such title should contain information equal to related locale ie page linked with en locale will contain at title value from title_en.

Updating the language change functionality

I had used set language redirect form. But something went wrong on that step. So instead of deal with it I just copied and slightly modify example of changing languages code from wagtail-locale.

{# not forget to load tags on top of the template #}
{% load ... wagtailcore_tags %}

{% if page %}
  <i class="fa-solid fa-language fa-2xl" aria-hidden="true"></i>
  {% for translation in page.get_translations.live %}
    {% get_language_info for translation.locale.language_code as lang %}
    <a href="{% pageurl translation %}" rel="alternate" hreflang="{{ lang.code }}">
      {{ lang.name_local }}
    </a>
  {% empty %}
    No translations available
  {% endfor %}
{% endif %}

Better deploy those changes after second Locale already created via settings of wagtail web interface. Otherwise user will unable to change locale up until moment when it created.

If redirect doesn't work correctly from the first attempt after adding locales it could be useful to restart web-server(recreate Kubernetes pod in my case).

Updating localization of menu buttons

I registered a tag that fetch menu buttons for the navbar. But after those changes it always show text on the English. So I slightly update it.

from django import template
from wagtail.models import Site

register = template.Library()


@register.simple_tag(takes_context=True)
def navigation_menu(context):
    root_page = Site.find_for_request(context['request']).root_page.localized
    menu_items = root_page.get_children().live().in_menu()
    return [root_page] + list(menu_items)

I specified root_page.localized to find localized root_page and from it I will get all pages that should appears in the navbar.

Removing wagtail-modeltranslation

IMPORTANT: On that step we have to deploy and migrate all data in the way described on previous stages on all instances. Because on that stage will be created some migrations that will purge recurring information out of the DB. But if there's missed Pages for some locale and\or information hasn't copied to base field then that information could be missed on that stage. Better to create a backup of the DB if unsure that everything done right.

I will continue use django-modeltranslation for some entities not related to wagtail. But I will use only wagtail-locale for edit Pages. So I completely removed wagtail-modeltranslation

poetry remove wagtail-modeltranslation

Then removed

"wagtail_modeltranslation",
    "wagtail_modeltranslation.makemigrations",
    "wagtail_modeltranslation.migrate",

from INSTALLED_APPS

My pages for localization was registered with decorator modeltranslation.decorators.register at translation.py so I cleanup that too.

I also unregistered FormField that was inherited AbstractFormField since there will be multiple form with different fields for each specific language.

At that moment those changes could be deployed and checked on the server without DB migrations. But to edit content it may required to create migrations and migrate DB.

Once everything is verified, you can run the database migrations and remove the prefixed fields.

Results

Overall, the process proved to be straightforward. But I had a few errors a few times, either because the pages didn't submit for translation, or because I had nuances in the data that was stored in the database. So out of four attempts I got the first and the fourth. That's why I strongly recommend practicing locally and then trying it on production. It is also possible that in the future there will be some more nuances, but perhaps my experience will be useful to someone.

Posted on June 5, 2024

With tags: django wagtail localization python