Shrinker is converging on the wrong value #14
Description
First off, my environment is as follows:
- Erlang/OTP 20
- IEx 1.4.5
- quixir 0.9.3
I am testing that a field on my struct is only valid when it is a list where all values are between one and seven:
test "changeset is valid when list values between one and seven" do
ptest [list: list(int(min: 1, max: 8))] do
params = string_params_for(:my_struct, list: list)
changeset = MyStruct.changeset(%MyStruct{}, params)
assert changeset.valid?
end
endI purposely set max to eight to test that it fails and provides the proper failing list. It seems it doesn't however. The first example this test case generates is:
%{list: [1, 1, 6, 6, 8, 3, 6, 2, 5, 2, 3, 7, 6, 2, 3, 5, 7, 1, 8,
5, 2, 6, 3, 8, 5, 6, 7, 3, 5, 6, 6, 2, 2, 8, 4, 2, 7, 1, 5, 3, 3, 3, 4, 1, 2,
8, 1, 5, 6, 1, ...]}This list includes eights so it properly concludes the changeset is not valid. Then it will try to shrink it down to the smallest possible failing test case:
[6, 3, 6, 8, 8, 3, 6, 3, 2, 1, 5]
false
[3, 6, 8, 8, 3, 6, 3, 2, 1, 5]
false
[6, 8, 8, 3, 6, 3, 2, 1, 5]
false
[8, 8, 3, 6, 3, 2, 1, 5]
false
[8, 3, 6, 3, 2, 1, 5]
false
[3, 6, 3, 2, 1, 5]
trueWhat you are looking at is the result of the shrinked list, and the result from changeset.valid? given that list. Up until the final example the list is still invalid since it contains an eight. The final list no longer includes an eight and thus is valid. However, at this point, quixir will stop shrinking and output the following:
Expected truthy, got false
when: list = [3, 6, 3, 2, 1, 5]
This is incorrect, this is the first test case that is passing for a valid reason. Am I overlooking something here? How can I get it to properly report the failing test case? I would also be ok with an option to disable shrinking (since this seems to be the cause) altogether as a quick workaround.