Testing

The commonmark spec.txt file is an excellent format which provides the narrative description, source and generated HTML. It is a format against which to run parser compliance tests.

The format of this file was modified to add AST output to allow testing of the generated AST which is crucial for using this parser for syntax highlighting. All test util classes were modified to handle the original format and the extended format.

To allow testing of parser options and extensions, the format was extended to specify which options should be used in running the test. The test subclass must provide a mapping from the option string to a test specific options set. This allows a single test spec file to be used to test more than one parser configuration.

In all cases the original format is supported and the original spec.txt file is used to validate parser compliance.

FullSpecTestCase class was added which regenerates the spec text with the expected HTML and AST replaced by the parser generated results then asserting that this is equal to the original. This allows comparing compliance to full spec in one place and in the case of running the tests in JetBrains IDEA, makes it easy to copy generated results to the expected text. This makes creating new tests much easier.

If the spec file does not have an AST section then the expected AST will not be generated or validated, nor will it be present in the generated full file result.

The markdown headers in the spec are used to mark sections. Each test case has the following format:

• The last markdown header is used as the Section value of SpecExample instance

• all text in the start line of an example, between example and end of line is ignored

• if the start example line end in: options(....) then the text between () is used as the option set identifier, leading and trailing blanks of the identifier are ignored. If the resulting identifier is empty then default parser configuration will be used.

• FullSpecTestCase will add Section, example number and any options.

Sample spec examples with and without the options() clause

## Reference Repository Keep First tests

Test repository KEEP_FIRST behavior, meaning the first reference def is used

```````````````````````````````` example Reference Repository Keep First tests: 1
[ref]

[ref]: /url1
[ref]: /url2
[ref]: /url3
.
<p><a href="/url1">ref</a></p>
.
Document[0, 46]
  Paragraph[0, 6]
    LinkRef[0, 5] textOpen:[0, 0] text:[0, 0] textClose:[0, 0] referenceOpen:[0, 1, "["] reference:[1, 4, "ref"] referenceClose:[4, 5, "]"]
      Text[1, 4] chars:[1, 4, "ref"]
  Reference[7, 19] refOpen:[7, 8, "["] ref:[8, 11, "ref"] refClose:[11, 13, "]:"] urlOpen:[0, 0] url:[14, 19, "/url1"] urlClose:[0, 0] titleOpen:[0, 0] title:[0, 0] titleClose:[0, 0]
  Reference[20, 32] refOpen:[20, 21, "["] ref:[21, 24, "ref"] refClose:[24, 26, "]:"] urlOpen:[0, 0] url:[27, 32, "/url2"] urlClose:[0, 0] titleOpen:[0, 0] title:[0, 0] titleClose:[0, 0]
  Reference[33, 45] refOpen:[33, 34, "["] ref:[34, 37, "ref"] refClose:[37, 39, "]:"] urlOpen:[0, 0] url:[40, 45, "/url3"] urlClose:[0, 0] titleOpen:[0, 0] title:[0, 0] titleClose:[0, 0]
````````````````````````````````

## Reference Repository Keep Last tests

Test repository KEEP_LAST behavior, meaning the last reference def is used

```````````````````````````````` example(Reference Repository Keep Last tests: 1) options(keep-last)
[ref]

[ref]: /url1
[ref]: /url2
[ref]: /url3
.
<p><a href="/url3">ref</a></p>
.
Document[0, 46]
  Paragraph[0, 6]
    LinkRef[0, 5] textOpen:[0, 0] text:[0, 0] textClose:[0, 0] referenceOpen:[0, 1, "["] reference:[1, 4, "ref"] referenceClose:[4, 5, "]"]
      Text[1, 4] chars:[1, 4, "ref"]
  Reference[7, 19] refOpen:[7, 8, "["] ref:[8, 11, "ref"] refClose:[11, 13, "]:"] urlOpen:[0, 0] url:[14, 19, "/url1"] urlClose:[0, 0] titleOpen:[0, 0] title:[0, 0] titleClose:[0, 0]
  Reference[20, 32] refOpen:[20, 21, "["] ref:[21, 24, "ref"] refClose:[24, 26, "]:"] urlOpen:[0, 0] url:[27, 32, "/url2"] urlClose:[0, 0] titleOpen:[0, 0] title:[0, 0] titleClose:[0, 0]
  Reference[33, 45] refOpen:[33, 34, "["] ref:[34, 37, "ref"] refClose:[37, 39, "]:"] urlOpen:[0, 0] url:[40, 45, "/url3"] urlClose:[0, 0] titleOpen:[0, 0] title:[0, 0] titleClose:[0, 0]
````````````````````````````````

The first part is the markdown source, the expected HTML is separated by single . on the line. Expected AST is added as a third part to the original spec.txt, also separated by a single . on the line.

The best way to create a test for an extension is to start with a copy of an existing one and modify the markdown source for the extension, deleting the expected HTML and AST text but leaving the . separator lines. Running the FullSpecTestCase derived test will create a full spec file with all section filled in with actual results. These should be validated then copied to the spec file.