Skip to content

2 errors found: TrueNegatives typos, "get_target" function does not support masking #16

@SimoneSartoni

Description

@SimoneSartoni

Hi Luca, I am actually working on Knowledge Tracing for my thesis. I looked at your code to see if I could find a DKT model as baseline.
While trying it I found 2 errors:

  1. distraction error in metrics.py in TruePositives function, which extends tensorflow TrueNegatives (instead of truePositives) resulting in wrong ACC and AUC.
  2. using "get_target" function before computing loss function and metrics does not support the use of automatic masking.
    I know you changed all -1 s to zeros, making it work, but this way the padded values will be considered in your metrics and loss as correct anwered values, modifying real Accuracy and AUC for example. I suggest you to try incorporating "get_target" function in a final custom layer of the model instead, giving as input the target skill to the model too. This way you can use directly a custom layer (remember to define compute_mask function too) to do the reduce_sum over the skill_true array. I have done it and values of AUC, ACC and loss change a lot in the case you have very different length of the sequences as input (due to large padding).

Despite this errors thanks for the code available, I used to understand better how to implement my own DKT as baseline
Bye ;)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions