Skip to content

Defer baseline comparisons until end of test #1451

@DanielRosenwasser

Description

@DanielRosenwasser

Currently if you re-enable all failing fourslash tests (by clearing out failedtests.txt) and run

npm run convertfourslash
hereby baseline-accept

You'll get a baseline file for internal/fourslash/tests/gen/findReferencesAfterEdit_test.go

However, if you run npm run updatefailing, that test is still disabled in failedtests.txt.

I think it is possibly due to a bug in the baselining logic. My guess is that baselining is currently too eager. Any test that requests multiple baselines where an edit has occurred in between probably fails because any of the following might be true:

  1. the baselines were never accepted so the test could never succeed
  2. the second baseline call will always read the results of the first baseline, and is assumed to fail
  3. the second baseline was accepted, so the first call will always fail

I think we look at the original infrastructure for fourslash, every test actually compared against calculated baselines at the end of the test run. (I could be wrong though.)

Originally posted by @DanielRosenwasser in #1450 (comment)

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions