Opened 2 years ago

Closed 2 years ago

#5815 closed defect (fixed)

Upload to Crowdin with non-ASCII characters in source translation fails

Reported by: sebastian Assignee: sebastian
Priority: P1 Milestone:
Module: Automation Keywords:
Cc: tlucas Blocked By:
Blocking: Platform: Unknown / Cross platform
Ready: yes Confidential: no
Tester: Unknown Verified working: no
Review URL(s):

https://codereview.adblockplus.org/29560569

Description

How to reproduce

Run ./build.py -t chrome translate <API_key> from adblockpluschrome.

Observed behaviour

No translations are submitted to Crowdin. The following error is logged to stderr:

Traceback (most recent call last):
  File "./build.py", line 18, in <module>
    buildtools.build.processArgs(BASE_DIR, sys.argv)
  File "/home/sebastian/src/adblockpluschrome/buildtools/build.py", line 607, in processArgs
    commands[command](baseDir, scriptName, opts, args, type)
  File "/home/sebastian/src/adblockpluschrome/buildtools/build.py", line 55, in __call__
    return self._handler(baseDir, scriptName, opts, args, type)
  File "/home/sebastian/src/adblockpluschrome/buildtools/build.py", line 328, in updateTranslationMaster
    basename, key)
  File "/home/sebastian/src/adblockpluschrome/buildtools/localeTools.py", line 404, in updateTranslationMaster
    data, headers = crowdin_prepare_upload(update)
  File "/home/sebastian/src/adblockpluschrome/buildtools/localeTools.py", line 356, in crowdin_prepare_upload
    ).format(boundary=boundary, name=name, data=data, mimetype=mimetype)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2026' in position 4772: ordinal not in range(128)

This seems to be a regression introduced with #5777. This bug is only being triggered when the source translation contains any non-ASCII characters.

Expected behaviour

The source translations on Crowdin should be updated with the content from _locales/en_US. No error should occur.

Change History (3)

comment:1 Changed 2 years ago by sebastian

  • Review URL(s) modified (diff)
  • Status changed from new to reviewing

comment:2 Changed 2 years ago by abpbot

A commit referencing this issue has landed:
Issue 5815 - Fixed Crowdin upload with non-ASCII characters

comment:3 Changed 2 years ago by sebastian

  • Resolution set to fixed
  • Status changed from reviewing to closed
Note: See TracTickets for help on using tickets.