luce

Luce

Luce is a CommonMark compliant parser and renderer which supports a few common extensions.

Luce is a port of the Dart markdown package.

Installation

1. Add the dependency to your shard.yml:

  dependencies:
    luce:
      git: https://codeberg.org/supercell/luce
      version: ~> 0.4.0

1. Run shards install

Usage

require "luce"

puts Luce.to_html("Hello *Markdown*") # => <p>Hello <em>Markdown</em></p>

Syntax extensions

A few Markdown extensions, beyond what was specified in the orignal Perl Markdown implementation, are supported. By default, the ones supported in CommonMark are enabled. Any individual extension can be enabled by specifying an Array of extension syntaxes in the block_syntaxes or inline_syntaxes argument of Luce.to_html.

The currently supported inline extension syntaxes are:

• InlineHTMLSyntax.new() - approximately CommonMark's definition of "Raw HTML".

The currently supported block extension syntaxes are:

• FencedCodeBlockSyntax - Code blocks familiar to Pandoc and PHP Markdown Extra users.
• HeaderWithIdSyntax - ATX-style headers have generated IDs, for link anchors (akin to Pandoc's auto_identifiers).
• SetextHeaderWithIdSyntax - Setext-style headers have generated IDs for link anchors (akin to Pandoc's auto_identifiers).
• TableSyntax - Table syntax familiar to GitHub, PHP Markdown Extra, and Pandoc users.

For example:

html = Luce.to_html(%(Hello <span class="green">Markdown</span>),
    inline_syntaxes: [Luce::InlineHTMLSyntax.new])

puts html # => <p>Hello <span class="green">Markdown</span></p>\n

Extension Sets

To make extension management easy, you can also just specify an extension set. Both Luce.to_html and Document.new accept an extension_set named parameter. Currently, there are four pre-defined extension sets.

• Luce::ExtensionSet::NONE includes no extensions. With no extensions, Markdown documents will be parsed with a default set of block and inline syntax parsers that closely match how the document might be parsed by the original Perl Markdown implementation.

• Luce::ExtensionSet::COMMON_MARK includes two extensions in addition to the default parsers to bring the parsed output closer to the CommonMark specification:

  • Block Syntax Parser

  • FencedCodeBlockSyntax

  • Inline Syntax Parser

  • InlineHTMLSyntax

• Luce::ExtensionSet::GITHUB_FLAVOURED includes five extensions in addition to the default parsers to bring the parsed output cose to the GitHub Flavoured Markdown specification:

  • Block Syntax Parser

  • FencedCodeBlockSyntax

  • TableSyntax

  • Inline Syntax Parser

  • InlineHTMLSyntax

  • StrikethroughSyntax

  • AutolinkExtensionSyntax

• Luce::ExtensionSet::GITHUB_WEB includes eight extensions. The same set of parsers used int he GITHUB_FLAVOURED extension set with the addition of the block syntax parsers, HeaderWithIdSyntax and SetextHeaderWithIdSyntax, which add id attributes to headers and inline syntax parser, EmojiSyntax, for parsing GitHub style emoji characters:

  • Block Syntax Parser

  • FencedCodeBlockSyntax

  • HeaderWithIdSyntax, which adds id attributes to ATX-style headers, for easy intra-document linking.

  • SetextHeaderWithIdSyntax, which adds id attributes to Setext-style headers, for easy intra-document linking.

  • TableSyntax

  • Inline Syntax Parser

  • InlineHTMLSyntax

  • StrikethroguhSyntax

  • EmojiSyntax

  • AutolinkExtension

Custom syntax extensions

You can create and use your own syntaxes.

require "luce"

syntaxes = [Luce::TextSyntax.new("nyan", sub: "~=[,,_,,]:3")]
puts Luce.to_html("nyan", inline_syntaxes: syntaxes)
# => <p>~=[,,_,,]:3</p>

HTML sanitization

This shard offers no features in the way of HTML sanitization. Read Estevão Soares dos Santos' great article, "Markdown's XSS Vulnerability (and how to mitigate it)", to learn more.

The authors recommend that you perform any necessary sanitization on the resulting HTML, for example via the sanitize shard.

Development

Currently matches version 7.0.2 of the Dart markdown package. Work continues on updating to match newer releases. That said, until we've matched the latest version of Dart markdown (7.1.0 as of writing), Luce will stay pre-1.0, since there will be some breaking changes.

Contributing

1. Fork it (https://codeberg.org/repo/fork/21123)
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

CommonMark compliance

This shard contains a number of files in the tools directory for tracking compliance with CommonMark.

Updating the CommonMark stats when changing the implementation

1. Update the shard and test code, making sure that tests still pass.
2. Run crystal tools/stats.cr --update-files to update the per-test results tools/common_mark_stats.json and the test summary tools/common_mark_stats.txt.
3. Verify that more tests now pass - or at least, no more tests fail.
4. Make sure you include the updated stats files in your commit.

Updating the CommonMark test file for a spec update

1. Check out the CommonMark source. Make sure you checkout a major release.
2. Dump the test output overwriting the existing tests file.

   > cd /path/to/commonmark-spec
   > python3 test/spec_tests.py --dump-tests > \
     /path/to/luce/tools/common_mark_tests.json

1. Update the stats files as described above. Note any changes in the results.
2. Update any references to the existing spec by searching for https://spec.commonmark.org/0.30 in the repository. (Including this one.) Verify the updated links are still valid.
3. Commit changes, including a corresponding note in CHANGELOG.md.

Contributors

• supercell - creator and maintainer