remove-lm-mask

[david-thrower]

It appears a phishing_email_detection_gpt2.py line 407 needs the zero mask set to False. This is an AI - introduced error.

embedded = tf.keras.layers.Embedding(
    input_dim=VOCABULARY_SIZE,
    output_dim=EMBEDDING_DIM,
    input_length=max_seq_length,
    mask_zero=True)(tokens)

[david-thrower]

Do not merge

• Counterintuitively, even though the mask is not being passed to the embedding from the tokenizer, setting the parameter zero_mask to False on the Embedding layer appears deleterious to the NLP model's performance, evidence by these trials (https://github.com/david-thrower/cerebros-core-algorithm-alpha/actions/runs/14415992085/job/40432514634):

Trial #     val_binary_accuracy
Trial 1:    0.9499 
Trial 2:    0.9387

Where at least 0.952 was expected, See https://github.com/david-thrower/cerebros-core-algorithm-alpha/actions/runs/14415528912/job/40431381425 (0.97 ) and https://github.com/david-thrower/cerebros-core-algorithm-alpha/releases/tag/v0.10.0-alpha. This change should be tested further, just to see if this face value appearance of performance degradation was spurious, which it may be with a sample size of 2. For now, it appears to be unwise to merged this until there is evidence to discredit this possibility.