Add renderer documentation (#7)

* Add render documentation
* Remove some JS specific documentation

Author: chrisjsewell

docs/Using_the_api.md

@@ -202,5 +202,116 @@ nested_tokens[0]
 
 ## Renderers
 
+<!-- #region -->
+After the token stream is generated, it's passed to a [renderer](https://github.com/ExecutableBookProject/markdown-it-py/tree/master/markdown_it/renderer.py).
+It then plays all the tokens, passing each to a rule with the same name as token type.
 
-Todo ...
+Renderer rules are located in `md.renderer.rules` and are simple functions
+with the same signature:
+
+```python
+def function(renderer, tokens, idx, options, env):
+  return htmlResult
+```
+<!-- #endregion -->
+
+You can inject render methods into the instantiated render class.
+
+```python
+md = MarkdownIt("commonmark")
+
+def render_em_open(self, tokens, idx, options, env):
+    return '<em class="myclass">'
+
+md.add_render_rule("em_open", render_em_open)
+md.render("*a*")
+```
+
+This is a slight change to the JS version, where the renderer argument is at the end.
+Also `add_render_rule` method is specific to Python, rather than adding directly to the `md.renderer.rules`, this ensures the method is bound to the renderer.
+
+
+You can also subclass a render and add the method there:
+
+```python
+from markdown_it.renderer import RendererHTML
+
+class MyRenderer(RendererHTML):
+    def em_open(self, tokens, idx, options, env):
+        return '<em class="myclass">'
+
+md = MarkdownIt("commonmark", renderer_cls=MyRenderer)
+md.render("*a*")
+```
+
+Plugins can support multiple render types, using the `__ouput__` attribute (this is currently a Python only feature).
+
+```python
+from markdown_it.renderer import RendererHTML
+
+class MyRenderer1(RendererHTML):
+    __output__ = "html1"
+
+class MyRenderer2(RendererHTML):
+    __output__ = "html2"
+
+def plugin(md):
+    def render_em_open1(self, tokens, idx, options, env):
+        return '<em class="myclass1">'
+    def render_em_open2(self, tokens, idx, options, env):
+        return '<em class="myclass2">'
+    md.add_render_rule("em_open", render_em_open1, fmt="html1")
+    md.add_render_rule("em_open", render_em_open2, fmt="html2")
+
+md = MarkdownIt("commonmark", renderer_cls=MyRenderer1).use(plugin)
+print(md.render("*a*"))
+
+md = MarkdownIt("commonmark", renderer_cls=MyRenderer2).use(plugin)
+print(md.render("*a*"))
+```
+
+Here's a more concrete example; let's replace images with vimeo links to player's iframe:
+
+```python
+import re
+from markdown_it import MarkdownIt
+
+vimeoRE = re.compile(r'^https?:\/\/(www\.)?vimeo.com\/(\d+)($|\/)')
+
+def render_vimeo(self, tokens, idx, options, env):
+    token = tokens[idx]
+    aIndex = token.attrIndex('src')
+    if (vimeoRE.match(token.attrs[aIndex][1])):
+
+        ident = vimeoRE.match(token.attrs[aIndex][1])[2]
+
+        return ('<div class="embed-responsive embed-responsive-16by9">\n' +
+               '  <iframe class="embed-responsive-item" src="//player.vimeo.com/video/' +
+                ident + '"></iframe>\n' +
+               '</div>\n')
+    return self.image(tokens, idx, options, env)
+
+md = MarkdownIt("commonmark")
+md.add_render_rule("image", render_vimeo)
+print(md.render("![](https://www.vimeo.com/123)"))
+```
+
+Here is another example, how to add `target="_blank"` to all links:
+
+```python
+from markdown_it import MarkdownIt
+
+def render_blank_link(self, tokens, idx, options, env):
+    aIndex = tokens[idx].attrIndex('target')
+    if (aIndex < 0):
+        tokens[idx].attrPush(['target', '_blank']) # add new attribute
+    else:
+        tokens[idx].attrs[aIndex][1] = '_blank'  # replace value of existing attr
+
+    # pass token to default renderer.
+    return self.renderToken(tokens, idx, options, env)
+
+md = MarkdownIt("commonmark")
+md.add_render_rule("link_open", render_blank_link)
+print(md.render("[a]\n\n[a]: b"))
+```

docs/architecture.md

@@ -50,7 +50,7 @@ The difference is simple:
 - There are special token objects, "inline containers", having nested tokens.
   sequences with inline markup (bold, italic, text, ...).
 
-See [token class](https://github.com/markdown-it/markdown-it/blob/master/lib/token.js)
+See [token class](https://github.com/ExecutableBookProject/markdown-it-py/tree/master/markdown_it/token.py)
 for details about each token content.
 
 In total, a token stream is:
@@ -68,8 +68,8 @@ to an AST.
 
 More details about tokens:
 
-- [Renderer source](https://github.com/markdown-it/markdown-it/blob/master/lib/renderer.js)
-- [Token source](https://github.com/markdown-it/markdown-it/blob/master/lib/token.js)
+- [Renderer source](https://github.com/ExecutableBookProject/markdown-it-py/tree/master/markdown_it/renderer.py)
+- [Token source](https://github.com/ExecutableBookProject/markdown-it-py/tree/master/markdown_it/token.py)
 - [Live demo](https://markdown-it.github.io/) - type your text and click `debug` tab.
 
 
@@ -95,90 +95,67 @@ and tried to do something yourself. We never reject with help to real developers
 
 ## Renderer
 
-After token stream is generated, it's passed to a [renderer](https://github.com/markdown-it/markdown-it/blob/master/lib/renderer.js).
+After the token stream is generated, it's passed to a [renderer](https://github.com/ExecutableBookProject/markdown-it-py/tree/master/markdown_it/renderer.py).
 It then plays all the tokens, passing each to a rule with the same name as token type.
 
 Renderer rules are located in `md.renderer.rules[name]` and are simple functions
 with the same signature:
 
-```js
-function (tokens, idx, options, env, renderer) {
-  //...
-  return htmlResult;
-}
+```python
+def function(renderer, tokens, idx, options, env):
+  return htmlResult
 ```
 
 In many cases that allows easy output change even without parser intrusion.
 For example, let's replace images with vimeo links to player's iframe:
 
-```js
-var md = require('markdown-it')();
+```python
+import re
+md = MarkdownIt("commonmark")
 
-var defaultRender = md.renderer.rules.image,
-    vimeoRE       = /^https?:\/\/(www\.)?vimeo.com\/(\d+)($|\/)/;
+vimeoRE = re.compile(r'^https?:\/\/(www\.)?vimeo.com\/(\d+)($|\/)')
 
-md.renderer.rules.image = function (tokens, idx, options, env, self) {
-  var token = tokens[idx],
-      aIndex = token.attrIndex('src');
+def render_vimeo(self, tokens, idx, options, env):
+    token = tokens[idx]
+    aIndex = token.attrIndex('src')
+    if (vimeoRE.match(token.attrs[aIndex][1])):
 
-  if (vimeoRE.test(token.attrs[aIndex][1])) {
+        ident = vimeoRE.match(token.attrs[aIndex][1])[2]
 
-    var id = token.attrs[aIndex][1].match(vimeoRE)[2];
+        return ('<div class="embed-responsive embed-responsive-16by9">\n' +
+               '  <iframe class="embed-responsive-item" src="//player.vimeo.com/video/' +
+                ident + '"></iframe>\n' +
+               '</div>\n')
+    return self.image(tokens, idx, options, env)
 
-    return '<div class="embed-responsive embed-responsive-16by9">\n' +
-           '  <iframe class="embed-responsive-item" src="//player.vimeo.com/video/' + id + '"></iframe>\n' +
-           '</div>\n';
-  }
-
-  // pass token to default renderer.
-  return defaultRender(tokens, idx, options, env, self);
-};
+md = MarkdownIt("commonmark")
+md.add_render_rule("image", render_vimeo)
+print(md.render("![](https://www.vimeo.com/123)"))
 ```
 
 Here is another example, how to add `target="_blank"` to all links:
 
-```js
-// Remember old renderer, if overridden, or proxy to default renderer
-var defaultRender = md.renderer.rules.link_open || function(tokens, idx, options, env, self) {
-  return self.renderToken(tokens, idx, options);
-};
-
-md.renderer.rules.link_open = function (tokens, idx, options, env, self) {
-  // If you are sure other plugins can't add `target` - drop check below
-  var aIndex = tokens[idx].attrIndex('target');
-
-  if (aIndex < 0) {
-    tokens[idx].attrPush(['target', '_blank']); // add new attribute
-  } else {
-    tokens[idx].attrs[aIndex][1] = '_blank';    // replace value of existing attr
-  }
-
-  // pass token to default renderer.
-  return defaultRender(tokens, idx, options, env, self);
-};
+```python
+from markdown_it import MarkdownIt
+
+def render_blank_link(self, tokens, idx, options, env):
+    aIndex = tokens[idx].attrIndex('target')
+    if (aIndex < 0):
+        tokens[idx].attrPush(['target', '_blank']) # add new attribute
+    else:
+        tokens[idx].attrs[aIndex][1] = '_blank'  # replace value of existing attr
+
+    # pass token to default renderer.
+    return self.renderToken(tokens, idx, options, env)
+
+md = MarkdownIt("commonmark")
+md.add_render_rule("link_open", render_blank_link)
+print(md.render("[a]\n\n[a]: b"))
 ```
 
 Note, if you need to add attributes, you can do things without renderer override.
 For example, you can update tokens in `core` chain. That is slower, than direct
-renderer override, but can be more simple. Let's use
-[markdown-for-inline](https://github.com/markdown-it/markdown-it-for-inline) plugin
-to do the same thing as in previous example:
-
-```js
-var iterator = require('markdown-it-for-inline');
-
-var md = require('markdown-it')()
-            .use(iterator, 'url_new_win', 'link_open', function (tokens, idx) {
-              var aIndex = tokens[idx].attrIndex('target');
-
-              if (aIndex < 0) {
-                tokens[idx].attrPush(['target', '_blank']);
-              } else {
-                tokens[idx].attrs[aIndex][1] = '_blank';
-              }
-            });
-```
-
+renderer override, but can be more simple.
 
 You also can write your own renderer to generate other formats than HTML, such as
 JSON/XML... You can even use it to generate AST.
@@ -194,9 +171,9 @@ This was mentioned in [Data flow](#data-flow), but let's repeat sequence again:
 
 And somewhere between you can apply additional transformations :) . Full content
 of each chain can be seen on the top of
-[parser_core.js](https://github.com/markdown-it/markdown-it/blob/master/lib/parser_core.js),
-[parser_block.js](https://github.com/markdown-it/markdown-it/blob/master/lib/parser_block.js) and
-[parser_inline.js](https://github.com/markdown-it/markdown-it/blob/master/lib/parser_inline.js)
+[parser_core.py](https://github.com/ExecutableBookProject/markdown-it-py/tree/master/markdown_it/parser_core.py),
+[parser_block.py](https://github.com/ExecutableBookProject/markdown-it-py/tree/master/markdown_it/parser_block.py) and
+[parser_inline.py](https://github.com/ExecutableBookProject/markdown-it-py/tree/master/markdown_it/parser_inline.py)
 files.
 
-Also you can change output directly in [renderer](https://github.com/markdown-it/markdown-it/blob/master/lib/renderer.js) for many simple cases.
+Also you can change output directly in [renderer](https://github.com/ExecutableBookProject/markdown-it-py/tree/master/markdown_it/renderer.py) for many simple cases.

docs/development.md

@@ -18,9 +18,6 @@ Before continuing, make sure you've read:
     block and inline rules are usually faster.
   - Sometimes, it's enough to only modify the renderer, for example, to add
     header IDs or `target="_blank"` for the links.
-  - Plugins should not require the `markdown-it` package as dependency in `package.json`.
-    If you need access to internals, those are available via a parser instance,
-    passed on plugin load. See properties of main class and nested objects.
 2. Search existing
    [plugins](https://www.npmjs.org/browse/keyword/markdown-it-plugin)
    or [rules](https://github.com/markdown-it/markdown-it/tree/master/lib),
@@ -34,14 +31,6 @@ Before continuing, make sure you've read:
      Such things should be discussed first on [CommonMark forum](http://talk.commonmark.org/).
 
 
-## Notes for NPM packages
-
-To simplify search:
-
-- add to `package.json` keywords `markdown-it` and `markdown-it-plugin` for plugins.
-- add keyword `markdown-it` for any other related packages.
-
-
 ## FAQ