Open
Description
This is an ambitious idea, and I'm not yet entirely sure of the details.
Checking output data is often tricky: small changes can be meaningful, while large changes can easily be unimportant (e.g. in a PNG plot). It would be nice to have some hooks to communicate with kernels for more intelligent checks (e.g. using plotchecker on matplotlib plots).
How might this work? Could we define a special output mimetype which kernels would produce (e.g. application/x-nbval-output-data), and a way for the test framework to call a check function in the kernel like check_output('saved output data', 'computed output data')?