Upload to Crowdin is not working (presumably due to API change)

[tlucas]

How to reproduce

1. Change the target language of the test project to German only.
2. Go to the repo "adblockpluschrome".
3. Change "basename" in "metadata.chrome" to this-is-very-much-a-test-proje.
4. Try to update the target translations for the project:

   ./build.py -t chrome setuptrans <project-api-key>
   

Observed behaviour

The call succeeds, when visiting the temporarily configured project and checking the target languages, nothing has changed.

Expected behaviour

The call should succeed and the Crowdin project should have every currently available language in adblockpluschrome configured as a target translation. On failure an error should be displayed.

Note

• This should not affect the Crowdin integration in the CMS repository.
• Uploading new files is also failing, but with a HTTP 400 error instead of silently.
• Requests to the languages_list endpoint are broken too now since they've moved the languages into the data key.

Hints for testers

The patch affects all four crowdin related build.py commands, hence all of them should be verified:

./build.py -t chrome setuptrans <project-api-key>

should configure the target languages of the crowdin project

./build.py -t chrome translate <project-api-key>

should update the crowdin master files i.e. update existing source files, add new source files, delete obsolete source files

./build.py -t chrome uploadtrans <project-api-key>

should upload locally available translations (fails for files which are not present in the crowdin master)

./build.py -t chrome gettranslations <project-api-key>

should download all available translations from crowdin

[kzar]

Tristan and I were chatting about this in IRC, it looks like Crowdin have changed their API. Firstly the language list is now inside the data key, ​this change fixed that problem. Secondly the add-file endpoint is returning 400 when I try to add files using the translate command. It's more than possible there were other breaking changes too :(, I didn't have any more time to look into it.

It looks like we'll have to make changes to both buildtools and cms to get our translation integrations working again.

[tlucas]

Please note - from what i currently know, the cms should not be affected at all - opposing the original assumption that both interfaces were broken.

We still might want to create one shared Crowdin interface, which could then be used by both buildtools and cms

[sebastian]

I am not sure whether it will be worth to share code for the CrowdIn integration between buildtools and the cms, or whether this will create more problems than it solves. Both systems implement a different workflow. While with our release process and therefore buildtools import and upload of translations are separate steps, this is one combined step with the cms. Also buildtools only uploads source strings and invalidates any translation if the source string has changed, which differs from the cms as well. So what remains to be shared wouldn't be much, and probably just results in additional boilerplate just for the matter of making some trivial code reusable, not to mention the problems of dependency management.

[abpbot]

A commit referencing this issue has landed:
​Issue 5777 - Update crowdin interface

[kvas]

Replying to sebastian:

I am not sure whether it will be worth to share code for the CrowdIn integration between buildtools and the cms, or whether this will create more problems than it solves. Both systems implement a different workflow...

I was thinking more of a low level primitives sharing (like add language, upload translation strings, etc.), rather than sharing at the level of workflow. And it does create some notable advantages, for example in terms of testability (mock the library in application code, test the code that talks to Crowdin once in the library). You might still be right about the problems vs. benefits balance though.

[tlucas]

A commit referencing this issue has landed:
​Issue 5777 Refactor crowdin interface pt. 2