Generalize translations and allow translating app-independent repositories

[trev]

Background

With adblockplusui we have the first repository with localization that isn't tied to a specific platform. Currently we have no good way to translate files in this repository.

What to change

The way I see it, we currently deduce the following parameters from the platform:

• Locales directory path (_locales vs. chrome/locale)
• Format of the locale identifiers (ISO-15897 with variations like es_419 vs. BCP-47)
• Format of the localization files (Chrome-like JSON vs. DTD)
• Platform-specific list of available locales
• Default locale

For a more generic localization approach we need to introduce these parameters in the metadata file:

[locales]
base_path =_locales
name_format = BCP-47
file_format=chrome-json
target_platforms=chrome firefox
default_locale = en-US

All these parameters should be optional if they can be deduced from the platform. However, adblockplusui repository should get a metadata.generic file and they should be mandatory there.

[trev]

This will involve lots of refactoring so I will try to break the changes down into small chunks. The first chunk is under review, more will come.

[trev]

First chunk landed: ​https://hg.adblockplus.org/buildtools/rev/ef7cafc88f90

[kzar]

Two questions:

• Does anyone know which format Chrome and Firefox use for locale identifiers? (From the description it implies that Chrome uses ISO-3166 and Firefox uses BCP 47, but isn't ISO-3166 for countries not languages?)
• What is the function of the possible_locales field? (I'm guessing the example in the issue description is a typo and instead of having a value like chrome firefox it should instead have a list of locales like en_US and de?)

[sebastian]

Replying to kzar:

• Does anyone know which format Chrome and Firefox use for locale identifiers? (From the description it implies that Chrome uses ISO-3166 and Firefox uses BCP 47, but isn't ISO-3166 for countries not languages?)

Yes, that isn't accurate. Chrome uses lang[_region] where lang is an ISO 639-1 code. And region, if given, is an ISO 3166-1 alpha 2 code, with the exception of es-419.

• What is the function of the possible_locales field? (I'm guessing the example in the issue description is a typo and instead of having a value like chrome firefox it should instead have a list of locales like en_US and de?)

I'm guessing that possible_locales=chrome firefox is a shorthand for all locales supported by either Chrome and Firefox. But I'm not entirely sure either, and yes that should probably have been specified more precisely in the issue description.

[kzar]

Replying to sebastian:

Yes, that isn't accurate. Chrome uses lang[_region] where lang is an ISO 639-1 code. And region, if given, is an ISO 3166-1 alpha 2 code, with the exception of es-419.

Any objection to just using the strings gecko or chrome to describe the locale name format instead? I don't think something like iso-639-1+iso3166-1ish is very catchy.

I'm guessing that possible_locales=chrome firefox is a shorthand for all locales supported by either Chrome and Firefox. But I'm not entirely sure either, and yes that should probably have been specified more precisely in the issue description.

Yea, I'm still not clear on that one either. I guess for now I'll make the logic as follows:

1. If the field is specified, use the locales given.
2. If the field is not specified and type is gecko, call buildtools.packagerGecko.getLocales
3. Otherwise just use a directory listing of the specified locales directory.

[trev]

I actually meant names to be a more objective description of the format than "whatever Google chose to use right now." You are right, ISO-3166 isn't the correct term - it's rather ISO-15897, or maybe just POSIX.

possible_locales on the other hand is the list of available locales and this one is specific to the particular platform. Our locale tools currently know how to calculate the list of locales for Firefox and Chrome. This isn't really important for packaging, rather for setting up Crowdin projects - we have to know which languages we need.

[kzar]

​https://hg.adblockplus.org/buildtools/rev/9f83ac702260