xous-core/locales/README.md

# Localization

The `lib.rs` file in this crate is auto-generated by `xtask`. It must be manually generated by
calling `xtask generate-locales` every time changes are made to the localization strings.

## Where are the Strings?
Locale files can be added to any server by making a `locales` directory within the server, and
then placing a `.json` file in there with a format like this:

```json
{
    "mainmenu.sleep": {
        "en": "Sleep now",
        "zh": "睡眠模式",
        "en-tts": "Sleep now"
    },
    "mainmenu.backlighton": {
        "en": "Backlight on",
        "zh": "背光开启",
        "en-tts": "🔇"
    },
    "mainmenu.backlightoff": {
        "en": "Backlight off",
        "zh": "背光关闭",
        "en-tts": "🔇"
    },
    "mainmenu.closemenu": {
        "en": "Close menu",
        "zh": "关闭功能表",
        "en-tts": "Close menu"
    }
}
```

Note that "en-tts" is a locale for english/vision-impaired. `🔇` means that an item does not exist
for that locale (can be used in any locale, not just speech-to-text locales). Omitting a locale
entirely without using the 🔇 character will lead to a panic.

## How to Code with Locales
The JSON file is parsed by `xtask` into a lib.rs file which is referred to within a server by
the following idiom:

```rust
// insert `locales = {path = "../../locales"}` inside the Cargo.toml of the server
use locales::t;

name: String::<64>::from_str(t!("mainmenu.backlighton", xous::LANG)),
```

The `t!(string_reference, language)` takes `string_reference` which is a programmer-readable
string that refers to the localized string, and `language` is the language code used inside
the localization file.

## How to Change the Display Language
A global `LANG` variable is provide inside `xous-rs/src/locale.rs` so that the language
may be set by changing a single file.

Strings are statically mapped into the kernel for a given locale, so changing languages requires
recompiling the OS with the appropriate code selected. This method has a few prominent pros
and cons, as such:

Cons:
- Requires a recompilation and update to change the language

Pros:
- Smallest binary size
- Least memory & code execution overhead
- Simple

The recompilation issue can be handled with some CI tooling to output versions of the OS
for each language with each automatic compilation pass. The updating issue is more annoying.
However, any dynamic solution requires quite a bit of tooling; and given that the kernel
is `no_std`, it's not possible to pull in full-fledged localization systems like `fluent`.
Furthermore, we don't have a filesystem yet to store alternate language files, and the OS
needs to have some prompts for the user in order to unlock the filesystem in the first place.

Nothing prevents a more sophisticated application-level server later on that operates
in `std` from pulling in a more featureful, dynamic localization framework; but it's an
explicit goal to keep the kernel small, simple, and fast.

## Internationalization Helper

In the tools directory you find `i18n_helper.py` that can be
very useful for managing and adding localizations:

```
$ ./tools/i18n_helper.py -h
usage: i18n_helper.py [-h] [-v] [-l] [-i] [-m] [-o] [-n NEW_LANG] [-f FROM_LANG]

Xous i18n Helper

options:
  -h, --help            show this help message and exit
  -v, --verbose         Prints details of each action
  -l, --list-languages  Lists current translations
  -i, --list-i18n-files
                        Lists i18n files
  -m, --missing         Shows missing translations
  -o, --show-ok         Shows OK translations
  -n NEW_LANG, --new-lang NEW_LANG
                        Add support for a new lang
  -f FROM_LANG, --from-lang FROM_LANG
                        Copy this existing lang for the new lang
$
```

The `--list-languages` function will show the currently supported langs:

```
$ ./tools/i18n_helper.py --list-languages
en
ja
zh
en-tts
$
```

The `--missing` function will show the disposition of the translations
(add `--show-ok` to see the complete translation status). The output
is "FILE tab JQ-LIKE-PATH tab STATUS" which makes it easy to spot
situations like completely absent translations that might cause a panic:

```
$ ./tools/i18n_helper.py --missing | grep ABSENT
services/status/locales/i18n.json	rtc.set_time_modal.ja	ABSENT
services/status/locales/i18n.json	rtc.set_time_modal.zh	ABSENT
services/status/locales/i18n.json	rtc.set_time_modal.en-tts	ABSENT
$
```
Possible statuses include:
* **OK** (if `--show-ok` flag is turned on)
* **MISSING** the entry has the special unicode character: 🔇
* **MACHINE_TRANSLATION** the entry ends with ` *MT*` to show that it was done by machine translation and requires human review.
* **TEMPORARY** a temporary solution, borrowed from another language like English (e.g. ends in `  *EN*`)
* **ABSENT** this translation key is _not present_ and could cause a panic!


The `--new-lang` function will add a new lang by copying an
existing lang and adding the suffix `" *EN*"` to the new translation
(example if the `--from-lang` is `en`):


```
$ ./tools/i18n_helper.py --verbose --from-lang en --new-lang fr
verbose mode
-- get languages --
adding new lang "fr" from "en" by appending " *EN*"

$
```

Each file will be backed up with the suffix `.orig` appended.
Once you are satisfied with the translations you can remove these
backup files like this:

```
$ find . -name '*.json.orig' | xargs rm
```

If you need to re-run the helper and an `.orig` version of the file
is present it will be treated as the original version.

_NOTE:_ You may want to add the new lang to the other tools
that know about available languages:

* https://github.com/betrusted-io/xous-core/blob/main/tools/updater/precursorupdater/precursorupdater.py#L154
* https://github.com/betrusted-io/xous-core/blob/main/tools/backup.py#L420
* https://github.com/betrusted-io/xous-core/blob/main/tools/restore.py#L541
* https://github.com/betrusted-io/xous-core/blob/main/tools/legacy/update_ci.sh#L14
* https://github.com/betrusted-io/xous-core/blob/main/tools/legacy/factory_reset.sh#L14