Support attaching files to open test reports

[mpkorstanje]

🤔 What's the problem you're trying to solve?

The JUnit Team is working on a new reporting format that more accurately represents hierarchical tests (junit-team/junit-framework#4113). Recently support for file attachments was added. However, while Cucumber already supports attaching files to reports, it does yet translate these to the JUnit Platform.

✨ What's your proposed solution?

• Map the embed and log events to JUnits file and report entries respectively.
• Request support for media/mime types in reports from open test reporting
• Request support for log files in reports from open test reporting

⛏ Have you considered any alternatives or workarounds?

No response

📚 Any additional context?

No response

[marcphilipp]

Request support for media/mime types in reports from open test reporting

I wasn't sure whether I should add that to the Open Test Reporting format. Sounds like you'd need it, right? Do you know where/how this is consumed?

[mpkorstanje]

I as framework maintainer have no need for it. But as a consumer of test reports I like them to be richly formatted. The media type allows for a rich representation of the attached file without an externalized file. I imagine that the OTA based reports could also have such a rich representation.

One example be the cucumber-html report report.

But a better, more impressive, example would be Allure. It supports several media types. Including a application/vnd.allure.image.diff which would be hard to replicate by guessing the media type based on a file extension.

Allures API for attaching content to a report looks like this:

...
        // Attach to the test report
        Allure.addAttachment("Screenshot diff",
                "application/vnd.allure.image.diff",
                content);
...

And I thinking out loud, it would be useful if JUnit had a something similar that would attach the bytes to the report, potentially externalizing them into a file.

[mpkorstanje]

And thinking out loud some more, another useful feature for OTA would be the ability to report test steps. For Cucumber this would allow the steps in a scenario to be represented in the OTA report. And these are also present in Allure. https://allurereport.org/docs/attachments/#show-overlay

And with that it comes to mind that to drive adoption of Open Test Reporting format, you may also want to talk to the maintainers of reporting tools to drive adoption there.

[marcphilipp]

And thinking out loud some more, another useful feature for OTA would be the ability to report test steps. For Cucumber this would allow the steps in a scenario to be represented in the OTA report.

This could be done in a (Cucumber-specific) extension schema.

[mpkorstanje]

This could be done in a (Cucumber-specific) extension schema.

Not when used as a JUnit Platform Engine though. Unless I've missed something.

[marcphilipp]

That's right. We'd have to extend TestExecutionListener to make custom events possible and somehow make the listener implementation that writes the XML file extensible. 🤔