Opened 4 years ago

Last modified 18 months ago

#3349 new change

[CMS] Replace the markdown package with a CommonMark compliant one

Reported by: saroyanm Assignee:
Priority: Unknown Milestone:
Module: Sitescripts Keywords: cms, markdown
Cc: kvas, sebastian, juliandoucette Blocked By:
Blocking: Platform: Unknown / Cross platform
Ready: no Confidential: no
Tester: Unknown Verified working: no
Review URL(s):

Description (last modified by kzar)


Our CMS currently uses the markdown package for rendering Markdown pages. While it works, we have encountered some difficulties in how certain things are rendered. As the format was never specified too well, different implementations often handle edge cases in completely different ways and the markdown package is no exception.

More recently the CommonMark initiative has come along, they are working at specifying the Markdown format clearly, so that all implementations can render the format consistently.

What to change

Investigate our options for a different, CommonMark compatible Markdown rendering module. Test for any differences in how any of our websites that use the CMS are rendered.


Our problems with the current markdown package

To be continued...

Change History (9)

comment:1 Changed 4 years ago by saroyanm

  • Blocking 3257 added
  • Description modified (diff)

comment:2 Changed 4 years ago by trev

  • Cc trev added

comment:3 Changed 4 years ago by kzar

  • Cc sebastian added
  • Description modified (diff)
  • Keywords cms markdown added
  • Owner set to kzar
  • Summary changed from Use CommonMark Instead of Markdown to [CMS] Replace the markdown package with a CommonMark compliant one

comment:4 Changed 4 years ago by sebastian

The rationale for this change, as outlined in the issue description at the moment, seem a little weak to me. The only limitation so far with markdown outlined in the issue description, isn't really a limitation but more like a configuration/usage issue. Note that markdown comes with a built-in extension that you can enable to allow markdown inside HTML elements.

Also upstream CommonMark only provides JS and C libraries. However, there is an inofficial and unmaintained Python port and a more recent fork of it. But the latter isn't even on PyPI yet (that means it needs to be installed manually from source). In fact they didn't even released a version yet, and don't seem to be a mature project. While on the other hand, the markdown module we currently use has been defacto-standard in the Python universe for years.

Also note that replacing the markdown rendering module, would require to re-test all pages showing content rendered from markdown.

Last edited 4 years ago by sebastian (previous) (diff)

comment:5 Changed 4 years ago by trev

Ok, I was unaware of this extension. If it works for us then enabling it sounds like a simple solution for the immediate issue.

Generally, I would prefer CommonMark given that a clean specification and a test suite exist for it, and that specification is actually good enough for us without any non-standard extensions. However, with the CommonMark module changing maintainers right now we might be better off waiting with a decision until the new maintainers actually upload their version to PyPI. So this is a more long term project.

Supposedly, performance of the CommonMark module is better. This isn't something we need to believe however, we can test ourselves whether this improves performance in our case.

comment:6 Changed 4 years ago by kzar

  • Blocking 3257 removed
  • Description modified (diff)
  • Owner kzar deleted

Once CommonMark-py's switch of maintainer has completed and the project has matured we can come back to this again. We all agree that it will be nice to use a CommonMark compliant Markdown library in the future.

For now we're going to just enable the markdown.extensions.extra extension, see #3354.

comment:7 Changed 20 months ago by kvas

  • Cc juliandoucette added

CommonMark-py development has moved to Read the Docs organization, which seems like an appropriate and sustainable place to be. I think the maintainer issue can be considered resolved.

There's also another Python implementation called paka.cmark that uses CFFI around the reference implementation in C. We can see which one we prefer.

The spec is not final yet, but the authors are planning to release version 1.0 real soon nowthis year.

Given all this, it might make sense to think about reviving this ticket, if it's still relevant. What do you think, @saroyanm, @juliandoucette?

comment:8 Changed 20 months ago by kzar

  • Cc kvas added; kzar removed

comment:9 Changed 18 months ago by fhd

  • Cc trev removed
Note: See TracTickets for help on using tickets.