Skip to content

Fix test runner failure on PHP tutorials #630

Open
@jcandan

Description

@jcandan

Describe the bug

Getting a Test Runner Failed message on a PHP project.

To Reproduce

Requires PHP 8.2+ and Composer.

Steps to reproduce the behavior:

  1. Open an empty folder in VS Code
    mkdir empty-project1
    code empty-project1
    
  2. Run CodeRoad: Start from the Command Pallete.
  3. Hit Start New Tutorial
  4. Choose URL, and supply https://raw.githubusercontent.com/jcandan/learn-php-basics/refs/heads/main/tutorial.json
  5. Open terminal, and in this empty project folder, run git init.
  6. Hit Start on the tutorial
  7. Supply an empty index.html file, and hit Run

See the Test Runner Failed message.

Expected behavior

To see that the solution should have passed:

  1. Clone the https://github.com/jcandan/learn-php-basics project locally
  2. Checkout the v0.1.0 branch, you'll be on the 1.1S solution commit
  3. Run cd .stack-elevate/ && composer install && ./vendor/bin/phpunit to prove the test runner should have worked

The TAP output shows that the test does pass.

Desktop (please complete the following information):

  • Platfrom: MacBook Pro
  • OS: macOS Sequoia
  • Version 15.3

Node:

  • Version v20.16.0

Git:

  • Version 2.40.0

PHP:

  • Version 8.4.3

Composer:

  • Version 2.8.5

PHPUnit:

  • Version 11.5.6
  • nikeee/phpunit-tap TAP Package required by INIT commit
  • The phpunit.xml configuration ensures the the TAP output
  • The config.testRunner.args.tap required configuration is blank since it is handled by the XML configuration.

Activity

jcandan

jcandan commented on Feb 9, 2025

@jcandan
Author

The nikeee/phpunit-tap outputs TAP compliant reporting. However, it currently does include PHPUnit output as well. I suspect this may be causing the test runner to fail.

According to nikeee/phpunit-tap/issues/31,

having non-TAP-lines between the TAP output is not a problem according to the TAP spec

In that case, the consumer in CodeRoad VS Code extension should be able to parse the combined PHPUnit and TAP compliant output.

However, I have responded to that claim to say that the TAP output stream should be entirely valid TAP. Awaiting his response.

But, in the interrim, we might want to at least confirm via debugging that this is the reason for the test runner failure.

jcandan

jcandan commented on Feb 10, 2025

@jcandan
Author

Okay, nikeee/phpunit-tap#31 (comment) makes a fairly good case that TAP should be able to deal with other output. So, if we do find this to be the cause of the failure, perhaps the TAP handling should be updated to deal with that.

jcandan

jcandan commented on Feb 10, 2025

@jcandan
Author

I've updated the example https://github.com/jcandan/learn-php-basics project with a patch from nikeee/phpunit-tap#32 to ensure the first line of the TAP output is on a new line. So, we definitely should be able to run the test without failure.

Also, I noticed that the Run click returns the Test Runner Failed message fairly quickly. So, I don't think it is even getting to the point of reading the TAP output anyway.

Would really appreciate eyes on this one. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

    Fix test runner failure on PHP tutorials · Issue #630 · coderoad/coderoad-vscode