Add multi-language support to Windows installer

[Dimi20cen]

Summary

This PR restores multi-language support to the Windows installer, addressing a regression from our legacy version. It implements a modern, industry-standard translation workflow using .po files.

All user-facing strings have been externalized from kolibri.iss into language files. Additionally, all installer related files (the main script, translations, and dependencies) have been moved into a new installer/ directory for better organization.

Changes

• Externalized Strings: All user-facing text has been moved from kolibri.iss into separate .isl language files, enabling a language selection dialog on startup.
• PO/ISL Conversion Scripts: Added isl_to_po.py and po_to_isl.py to convert between Inno Setup's .isl format and the universal .po format.
• Automated Build Process: The build-installer-windows command now automatically compiles all .po files into the required .isl format, ensuring the installer is always built with the latest translations.
• New Makefile Targets: Added translations-export, translations-compile, and new-language to streamline the entire localization lifecycle.

References

Fixes #181

Translation Workflow

Prerequisites

1. Install dependencies again, (make dependencies).
2. Run

make get-whl whl="https://github.com/learningequality/kolibri/releases/download/v0.18.1/kolibri-0.18.1-py2.py3-none-any.whl"

3. Run make pyinstaller to build the application.
4. Have Inno Setup installed.
5. Have polib installed, pip install polib.

1. Adding a New Language (Developer Task)

This is performed once to scaffold the necessary files for a new language.

1. Create the Language File:
   Use the new-language Makefile target to create a new .isl file. You need to provide the language name in English (e.g., 'Spanish', 'French'). This name will be used for the filename.

     # Example for adding Spanish
     make new-language LANG="Spanish"

   This command runs the create_new_language.py script, which creates installer/translations/Spanish.isl. The file will be pre-filled with standard Inno Setup translations if available, otherwise it will be a copy of the English template with empty values for translation.

2. Enable the Language in the Installer Script:
   Open installer/kolibri.iss and add a new entry in the [Languages] section. The Name is a short language code, and the MessagesFile points to the file you just created.

   [Languages]
   Name: "en"; MessagesFile: "translations\English.isl"
   Name: "de"; MessagesFile: "translations\German.isl"
   ; Add the new line for Spanish
   Name: "es"; MessagesFile: "translations\Spanish.isl"
   

3. Commit the New Language File:
   Add the newly created Spanish.isl file and the changes to kolibri.iss to Git and commit them. This establishes the new language in the codebase.

2. Exporting Strings for Translation (Developer Task)

Now, you need to generate a .po file, which is the format that will be used in Crowdin.

1. Run the Export Script:
   Use the translations-export target. You must provide both the LANG (the filename basis, e.g., "Spanish") and the LANG_CODE (the standard locale code, e.g., "es_ES").

   make translations-export LANG="Spanish" LANG_CODE="es_ES"

   This runs the isl_to_po.py script, which reads the English source text from English.isl and creates a new installer/translations/Spanish.po file. The msgid fields will contain the English strings, and the msgstr fields will be empty, ready for translation.

2. Upload the .po File to Crowdin:

Warning

Don't know about this part

3. Translating the Content (Translator Task)

Warning

Don't know about this part either

4. Integrating Completed Translations (Developer Task)

Once translations are complete or have been updated in Crowdin, they need to be brought back into the repository.

1. Download the Translated .po File from Crowdin:

2. Update the Local .po File and commit changes:

   • Replace the existing installer/translations/Spanish.po file with the newly translated version.
   • Commit the changes to the Spanish.po file.

5. Building the Installer with the New Language (Developer Task)

This is the final step where the translated .po files are automatically compiled and included in the installer.

1. Build the Installer:
   Simply run the standard build command for the Windows installer.

   make build-installer-windows

2. Automatic Compilation:
   The build-installer-windows target automatically runs the translations-compile dependency first. This process:

   • Finds all .po files in the installer/translations/ directory (including the Spanish.po you just updated).
   • Runs the po_to_isl.py script for each one.
   • This regenerates the corresponding .isl file (e.g., Spanish.isl) with the latest translations from the .po file.

3. Test the Final Installer:

   • Navigate to the dist-installer/ directory and run the setup .exe.
   • A language selection dialog will appear at the start.
   • Select "Spanish" from the dropdown menu.
   • Proceed with the installation and verify that all dialogs, buttons, and messages now appear in Spanish.

[rtibbles]

I think this has most of the right pieces in place! Just a couple of clarifications that came to mind - and I'm not entirely sure we need all of the code here with the right workflow in place.

[rtibbles]

I don't think we need this script. The workflow we would most likely use is to take the English isl file, convert it to a po file then upload it to Crowdin (the translation platform that we use).

We would then download the translated po files for each language, and convert those to isl files for each language. I think your two conversion scripts should handle all that is needed in that case?

[Dimi20cen]

take the English isl file, convert it to a po file then upload it to Crowdin

If we don't use create_new_language.py, then the translator would have to re-translate even the standard strings that InnoSetup already provides (for languages like spanish).

Instead if we use create_new_language.py, then only the custom strings would need to be translated.

Do I miss something?

[rtibbles]

Crowdin lets you upload existing translations for a source as well - so if we have the original inno translations, we can convert them to a po file for upload.

My basic thought is that I'd like us to specify which languages we want to support, and then conditionally upload already existing translations for those languages for which translations already exist. It is also possible that I'm just misreading the workflow you're intending.

[rtibbles]

Might be easier to use the locale code as the filename en.isl rather than the language name? Especially as we have the language name encoded in here anyway!

[Dimi20cen]

Currently, create_new_language.py uses the language name (e.g. Spanish) to find the packaged translation in C:\Program Files (x86)\Inno Setup 6\Languages.

We could modify the create_new_language.py script to take a locale code as input, and then use an internal dictionary to look up the Inno Setup Filename it needs to find.

Something like that

LOCALE_TO_INNO_NAME = {
    "de": "German",
    "es": "Spanish",
    "fr": "French",
    "it": "Italian",
    # ...
}

[rtibbles]

Yeah, that works for me - we end up doing this kind of mapping quite a lot for translations, because every translation system uses something different. We use the locale (e.g. es_ES) canonically, because it is the most specific, whereas just saying "Spanish" is very ambiguous - is that Spanish from Spain, Spanish from Argentina, Spanish from Latin America?

So, essentially for each locale code that we want to support, we would also make a note of which Inno Setup language name we would pull translations from.

[rtibbles]

Where do we find out these values?

[Dimi20cen]

They can be found here: https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-lcid/a9eac961-e77d-41a6-90a5-ce1a8b0cdb9c

I was thinking of adding this to the readme, when this PR is complete

[rtibbles]

That would be great - I suppose we could even add this information to the locale lookup object that we're defining above:

LOCALE_INFO = {
    "de": {
        "inno_setup_name": "German",
        "ms_languageid": "0x0007",
    },
    # ...
}

If we wanted to get really fancy, we could fetch the page and parse the HTML table - but that seems very unneccessary!

[rtibbles]

What does languagecodepage mean?

[Dimi20cen]

It specifies the Windows ANSI code page that the language file text is encoded in. It tells Inno Setup how to correctly read and display the localized strings if the file isn’t in Unicode (UTF-8).

However how that the files are UTF-8, its not needed.
We can either remove it when creating a new language or maybe we can leave it as is

[rtibbles]

I think if we can be sure to encode in UTF-8 that seems like the best solution!

[rtibbles]

I think the main distinction to keep in mind is that we treat English as the "source", and all other languages as the "translations".

So, the workflow is, we upload every message we need in English in a po file - but then for some languages we will essentially have 'pre-filled' translations based on what InnoSetup already provides (either from its built in translations or community supported translations).