-
Notifications
You must be signed in to change notification settings - Fork 286
Testing
The commonmark spec.txt file is an excellent format which provides the narrative description, source and generated HTML. It is a format against which to run parser compliance tests.
The format of this file was modified to add AST output to allow testing of the generated AST which is crucial for using this parser for syntax highlighting. All test util classes were modified to handle the original format and the extended format.
To allow testing of parser options and extensions, the format was extended to specify which options should be used in running the test. The test subclass must provide a mapping from the option string to a test specific options set. This allows a single test spec file to be used to test more than one parser configuration.
In all cases the original format is supported and the original spec.txt file is used to validate parser compliance.
FullSpecTestCase
class was added which regenerates the spec text with the expected HTML and
AST replaced by the parser generated results then asserting that this is equal to the original.
This allows comparing compliance to full spec in one place and in the case of running the tests
in JetBrains IDEA, makes it easy to copy generated results to the expected text. This makes
creating new tests much easier.
If the spec file does not have an AST section then the expected AST will not be generated or validated, nor will it be present in the generated full file result.
The markdown headers in the spec are used to mark sections. Each test case has the following format:
-
The last markdown header is used as the
Section
value ofSpecExample
instance -
all text in the start line of an example, between
example
and end of line is ignored -
if the start example line end in:
options(....)
then the text between()
is used as the option set identifier, leading and trailing blanks of the identifier are ignored. If the resulting identifier is empty then default parser configuration will be used. -
FullSpecTestCase
will addSection
, example number and any options.
Sample spec examples with and without the options()
clause
## Reference Repository Keep First tests
Test repository KEEP_FIRST behavior, meaning the first reference def is used
```````````````````````````````` example Reference Repository Keep First tests: 1
[ref]
[ref]: /url1
[ref]: /url2
[ref]: /url3
.
<p><a href="/url1">ref</a></p>
.
Document[0, 46]
Paragraph[0, 6]
LinkRef[0, 5] textOpen:[0, 0] text:[0, 0] textClose:[0, 0] referenceOpen:[0, 1, "["] reference:[1, 4, "ref"] referenceClose:[4, 5, "]"]
Text[1, 4] chars:[1, 4, "ref"]
Reference[7, 19] refOpen:[7, 8, "["] ref:[8, 11, "ref"] refClose:[11, 13, "]:"] urlOpen:[0, 0] url:[14, 19, "/url1"] urlClose:[0, 0] titleOpen:[0, 0] title:[0, 0] titleClose:[0, 0]
Reference[20, 32] refOpen:[20, 21, "["] ref:[21, 24, "ref"] refClose:[24, 26, "]:"] urlOpen:[0, 0] url:[27, 32, "/url2"] urlClose:[0, 0] titleOpen:[0, 0] title:[0, 0] titleClose:[0, 0]
Reference[33, 45] refOpen:[33, 34, "["] ref:[34, 37, "ref"] refClose:[37, 39, "]:"] urlOpen:[0, 0] url:[40, 45, "/url3"] urlClose:[0, 0] titleOpen:[0, 0] title:[0, 0] titleClose:[0, 0]
````````````````````````````````
## Reference Repository Keep Last tests
Test repository KEEP_LAST behavior, meaning the last reference def is used
```````````````````````````````` example(Reference Repository Keep Last tests: 1) options(keep-last)
[ref]
[ref]: /url1
[ref]: /url2
[ref]: /url3
.
<p><a href="/url3">ref</a></p>
.
Document[0, 46]
Paragraph[0, 6]
LinkRef[0, 5] textOpen:[0, 0] text:[0, 0] textClose:[0, 0] referenceOpen:[0, 1, "["] reference:[1, 4, "ref"] referenceClose:[4, 5, "]"]
Text[1, 4] chars:[1, 4, "ref"]
Reference[7, 19] refOpen:[7, 8, "["] ref:[8, 11, "ref"] refClose:[11, 13, "]:"] urlOpen:[0, 0] url:[14, 19, "/url1"] urlClose:[0, 0] titleOpen:[0, 0] title:[0, 0] titleClose:[0, 0]
Reference[20, 32] refOpen:[20, 21, "["] ref:[21, 24, "ref"] refClose:[24, 26, "]:"] urlOpen:[0, 0] url:[27, 32, "/url2"] urlClose:[0, 0] titleOpen:[0, 0] title:[0, 0] titleClose:[0, 0]
Reference[33, 45] refOpen:[33, 34, "["] ref:[34, 37, "ref"] refClose:[37, 39, "]:"] urlOpen:[0, 0] url:[40, 45, "/url3"] urlClose:[0, 0] titleOpen:[0, 0] title:[0, 0] titleClose:[0, 0]
````````````````````````````````
The first part is the markdown source, the expected HTML is separated by single .
on the line.
Expected AST is added as a third part to the original spec.txt, also separated by a single .
on the line.
The best way to create a test for an extension is to start with a copy of an existing one and
modify the markdown source for the extension, deleting the expected HTML and AST text but
leaving the .
separator lines. Running the FullSpecTestCase
derived test will create a full
spec file with all section filled in with actual results. These should be validated then copied
to the spec file.