Skip to content

Compiler short circuits when validating schemas, not providing the full list of errors #116

@hysmio

Description

@hysmio

Hey! Noticed something while building while using extensions. It seems that Compile short circuits validation when errors occur in a document. I imagined that it would validate all schemas even if errors occurred in one.

This very well could be a design decision, but wanted to clarify.

4 fairly verbose examples can be found here:
https://goplay.tools/snippet/dhM5tkI82j1

The examples are:

  1. returns meta missing properties

Actual output is the expected output

jsonschema file:///home/user/projects/golang-playground/schema.json compilation failed
[I#] [S#] doesn't validate with file:///home/user/projects/golang-playground/extension_meta.json#
  [I#] [S#/required] missing properties: 'foo', 'foobar'
  1. returns https://json-schema.org/draft/2020-12/schema validation error, but not meta extension validation error

Actual output:

jsonschema file:///home/user/projects/golang-playground/schema2.json compilation failed
[I#] [S#] doesn't validate with https://json-schema.org/draft/2020-12/schema#
  [I#] [S#/allOf/3] allOf failed
    [I#] [S#/allOf/3/$ref] doesn't validate with 'https://json-schema.org/draft/2020-12/meta/validation#'
      [I#/minLength] [S#/properties/minLength/$ref] doesn't validate with '/$defs/nonNegativeIntegerDefault0'
        [I#/minLength] [S#/$defs/nonNegativeIntegerDefault0/$ref] doesn't validate with '/$defs/nonNegativeInteger'
          [I#/minLength] [S#/$defs/nonNegativeInteger/minimum] must be >= 0 but found -1

Expected output:

jsonschema file:///home/user/projects/golang-playground/schema2.json compilation failed
[I#] [S#] doesn't validate with https://json-schema.org/draft/2020-12/schema#
  [I#] [S#/allOf/3] allOf failed
    [I#] [S#/allOf/3/$ref] doesn't validate with 'https://json-schema.org/draft/2020-12/meta/validation#'
      [I#/minLength] [S#/properties/minLength/$ref] doesn't validate with '/$defs/nonNegativeIntegerDefault0'
        [I#/minLength] [S#/$defs/nonNegativeIntegerDefault0/$ref] doesn't validate with '/$defs/nonNegativeInteger'
          [I#/minLength] [S#/$defs/nonNegativeInteger/minimum] must be >= 0 but found -1
[I#] [S#] doesn't validate with file:///home/user/projects/golang-playground/extension_meta.json#
  [I#] [S#/required] missing properties: 'foo', 'foobar'
  1. returns meta2 required properties error, but not meta#/foobar/minLength

Actual output: (this shows two children errors in one message)

jsonschema file:///home/user/projects/golang-playground/schema3.json compilation failed
[I#] [S#] doesn't validate with file:///home/user/projects/golang-playground/extension_meta.json#
  [I#] [S#/required] missing properties: 'foo'
  [I#/foobar] [S#/properties/foobar/minLength] length must be >= 3, but got 1

Expected output:

jsonschema file:///home/user/projects/golang-playground/schema3.json compilation failed
[I#] [S#] doesn't validate with file:///home/user/projects/golang-playground/extension_meta.json#
  [I#] [S#/required] missing properties: 'foo'
  [I#/foobar] [S#/properties/foobar/minLength] length must be >= 3, but got 1
[I#] [S#] doesn't validate with file:///home/user/projects/golang-playground/extension_meta_2.json#
  [I#] [S#/required] missing properties: 'bar'
  1. **Validate returns the same singular meta error when calling .Error() but

DetailedOutput() returned the correct errors**
Actual output:

jsonschema: '' does not validate with file:///home/hayden/projects/golang-playground/schema4.json#/maximum: must be <= 8 but found 9

Expected output: (ValidationError.GoString())

[I#] [S#] doesn't validate with file:///home/hayden/projects/golang-playground/schema4.json#
  [I#] [S#/maximum] must be <= 8 but found 9
  [I#] [S#/foobar] when foo is bar, numbers must be >= 10, but got 9

I've created a PR here for Compile supporting multiple errors.

For ValidationError.Error() I would suggest using GoString or creating a way to show multiple errors.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions