Opened 3 years ago

Closed 3 years ago

#5392 closed change (fixed)

Map Facebook & Crowdin locale codes

Reported by: juliandoucette Assignee: juliandoucette
Priority: P2 Milestone:
Module: Websites Keywords:
Cc: kvas, jsonesen, wspee, ire, saroyanm, matze, ferris, rraceanu Blocked By: #5402
Blocking: Platform: Unknown / Cross platform
Ready: no Confidential: no
Tester: Unknown Verified working: no
Review URL(s):

https://codereview.adblockplus.org/29480675/

Description (last modified by juliandoucette)

Background

We can use og:locale and og:locale:alternate meta fields in our web pages to specify the current page locale and available locales for each page (See example).

However, it seems the crodwin language codes are not the same format or value as facebook language codes.

What to change

Create a global function that provides og:locale codes based on website locale codes provided in settings.ini.

Use this function to set og:locales on all websites.

Change History (17)

comment:1 Changed 3 years ago by scheer

  • Cc rraceanu added; scheer removed

comment:2 Changed 3 years ago by juliandoucette

  • Blocked By 5402 added

comment:3 Changed 3 years ago by juliandoucette

  • Description modified (diff)
  • Summary changed from Standardize locale codes and formats to Map Facebook & Crowdin locale codes

Note: I had previously guessed that Facebook's scraper would be smart enough to guess the correct locale based on a two letter code or a code with a - instead of a _. However, I performed the test outlined in #5402 using these incorrect formats and fb_locale was not passed as a result.

comment:4 Changed 3 years ago by matze

What you call "Facebook compatible locale codes" is an actual locale or locale code, though without any .codeset or @modifier. The Open Graph protocol explicitly requires locales to be given in this notation.

Now these "crowdin language codes" are part of their API documentation. They are not the same as a locale. So, where's the connection? Why, where and when do you want to translate from a language code within a vendor's API to a locale compliant with Open Graph?

comment:5 Changed 3 years ago by juliandoucette

What you call "Facebook compatible locale codes" is an actual ​locale or locale code, though without any .codeset or @modifier. The ​Open Graph protocol explicitly requires locales to be given in this notation.

Yes.

I called them "Facebook compatible locale codes" and not "locale codes" because there are two exceptions that Facebook makes to the ISO standard which are documented here.

Now these "crowdin language codes" are part of their API documentation. They are not the same as a locale. So, where's the connection? Why, where and when do you want to translate from a language code within a vendor's API to a locale compliant with Open Graph?

I want to write Jinja like this:

 <meta property="og:locale" content="{{ locale }}">
 {% for alternate_locale in locales %}
   {% if alternate_locale != locale %}
     <meta property="og:locale:alternate" content="{{ alternate_locale }}">
   {% endif %}
 {% endfor %}

To set "Facebook compatible" og:locale and og:locale:alternate meta field values based on the locales of our web pages.

I was assuming that the locales provided to our CMS via our websites settings.ini were "Crowdin language codes" because many of them were only two chars. However, it seems there is a little bit of conversion going on under the hood here already. Nonetheless, the locale codes that are already being used in our websites' [settings.ini, paths, and Crowdin] are not the same as those supported by Facebook via Open Graph tags.

Last edited 3 years ago by juliandoucette (previous) (diff)

comment:6 follow-up: Changed 3 years ago by matze

Thank you for the explanation! Quick follow-up question: Can't you define a set of /locale_path/ specific variables within our CMS already? Hence perform that very mapping / computation for alternate_locale in the repository?

comment:7 in reply to: ↑ 6 Changed 3 years ago by juliandoucette

Replying to matze:

Thank you for the explanation! Quick follow-up question: Can't you define a set of /locale_path/ specific variables within our CMS already?

I believe the locale_paths you are referring to come from language codes defined in settings.ini per repository.

Hence perform that very mapping / computation for alternate_locale in the repository?

Yes. But I'd prefer not to add this function to every website repository - hence this issue (to add such a function to CMS).

I'm guessing that it would be easiest to use facebook codes in settings.ini and simplify them for crowdin (and potentially paths/urls) in CMS. But I trust Sitescripts to come up with the most appropriate solution considering the potential side effects (changing urls, changing locale codes in crodwin)

comment:8 Changed 3 years ago by ire

  • Cc ire added; iaderinokun removed

comment:10 Changed 3 years ago by juliandoucette

  • Blocking 1274 added

comment:11 Changed 3 years ago by juliandoucette

  • Component changed from Sitescripts to Websites
  • Description modified (diff)
  • Priority changed from Unknown to P2

Moving this to websites module based on a conversation with kvas on IRC.

Mapping (more specific) ~Facebook codes to (less specific) ~Crowdin codes can be ambiguous. We think it's better idea to implement this as a shared global in website-defaults (given that we are also prioritizing the implementation of shared globals/includes/filters/etc).

comment:12 Changed 3 years ago by juliandoucette

  • Blocking 5329 added

comment:13 Changed 3 years ago by juliandoucette

  • Blocked By 5329 added
  • Blocking 5329 removed
  • Priority changed from P2 to P3
  • Review URL(s) modified (diff)

This has been implemented as a filter in #5329 review. However, I would like to refactor this implementation after #5447 goes live.

comment:14 Changed 3 years ago by juliandoucette

  • Blocking 1274 removed

comment:15 Changed 3 years ago by juliandoucette

  • Priority changed from P3 to P2

Bumping this up in priority because the existing implementation is fragile (because it's hard coded with no fallback for missing keys).

comment:16 Changed 3 years ago by juliandoucette

  • Owner set to juliandoucette

comment:17 Changed 3 years ago by juliandoucette

  • Blocked By 5329 removed
  • Resolution set to fixed
  • Status changed from new to closed

Fixed by:

---

Facebook will scrape all available locales and display them according to your Facebook account locale. Unfortunately there is a bug in Facebook preview that causes all share previews to display in English. But the final share will display in the appropriate Facebook account locale.

Note: See TracTickets for help on using tickets.