[CMS] Animation XML files not generated for non English languages

[SMed79]

How to reproduce

1. Go to ​https://adblockplus.org/fr/faq_basics
2. Click Me montrer comment faire link to play the animation.

Observed behaviour

No animation is played because the background request to ​https://adblockplus.org/fr/animations/blockbanner.xml returns 404.

Expected behaviour

The animation should play and the XML file should be returned successfully.

(For an example of what it should look like start the CMS test server locally and then browse to ​http://localhost:5000/fr/faq_basics and ​http://localhost:5000/fr/animations/blockbanner.xml )

Notes

• While the test server works properly, cms.bin.generate_static_pages doesn't seem to generate the animation XML files for other languages at all.
• See the related forum post ​https://adblockplus.org/forum/viewtopic.php?f=9&t=40258

[greiner]

The issue appears to be that the XML is located under a language-specific URL (e.g. /en/animations/blockbanner.xml) but that there are no versions of those in other languages. Therefore there should be a fallback to english if there is no translated version of a resource, similar to how texts fall back to their base text if they're not translated.

Note that this also applies to other similar XML files.

We should also find out whether there have been translated versions of those before migrating from Anwiki and recover them.

[kzar]

I think this might actually be an infrastructure issue. When using the CMS test server everything works properly, but I can also reproduce the problem as described when playing animations on the live site.

What do you think Matze, possible problem with the Nginx config for web.adblockplus.org?

I expect it to behave as follows:

• ​https://adblockplus.org/ko/animations/blockbanner.xml to return the /en/animations/blockbanner.xml file as /ko/animations/blockbanner.xml doesn't exist.
• ​https://adblockplus.org/fr/animations/blockbanner.xml to return /fr/animations/blockbanner.xml as it actually does exist.

But currently both return 404 instead.

[juliandoucette]

I agree that it would be better to solve this problem with infrastructure.

[matze]

While looking into this issue I found another one that may create some confusion in this context, so I set up a distinct ticket (#3750). However, the issue described in this very ticket here seems to be no defect in infrastructure:

There is no fr version generated from blockbanner.xml.tmpl in the current state, en only. I am not even sure if the currently intended approach (creating localized content from a template) is even properly supported by the CMS software at this point.

Also, the /xx/foo to /en/foo fallback is explicitly not included in the requested feature set for the httpd configuration, only the canonical /foo is language-sensitive and behaves in that manner. Both ​https://testpages.adblockplus.org/ as well as ​https://adblockplus.org/ are consistent in their behavior, just as intended.

As soon as you've clarified this issue, feel free to request updates or the introduction of additional behavior when you believe they are necessary.

[saroyanm]

I also don't think this is related to infrastructure, I'm currently investigating the issue, but when I generate content locally I also miss animation in the localization, could be that it's also related to CMS will keep updated.

[kzar]

You're both right, those files aren't actually being generated by cms.bin.generate_static_pages at all. Appears to be a CMS issue instead of a problem with the Nginx config.

[saroyanm]

Replying to kzar:

You're both right, those files aren't actually being generated by cms.bin.generate_static_pages at all. Appears to be a CMS issue instead of a problem with the Nginx config.

Yes, because seems like currently we only generate the content when there is a corresponding locale, somehow we need to be sure that test script and CMS behave in similar way, also not sure if it's a good idea to generate pages for locales that are not exist, we need to be sure that linkify function will still return original version for the translations that do not exist yet.

[kzar]

Yes you're right. While the translation_ratio for a page is considered 1 for pages with no strings we were accidentally omitting them from the page list anyway.

[abpbot]

A commit referencing this issue has landed:
​https://hg.adblockplus.org/cms/rev/f7494bddefec