How to convert one incoming example into a valid Avalanche-Mini-Dataset

[TheRisenPhoenix]

I am working on a program that compares different CL-Strategies on a specific problem. I already have an infrastructure and can compare multiple approaches.
Now, I'd like to also use Avalanche so I can also evaluate some of the strategies that come with it.
My infrastructure is built, so that the model needs to methods: learn_one and evaluate_many. My data therefore comes in only one datapoint at a time, (n features + task label), and now I'm wondering how to best adopt a wrapper so i can use the Avalanche models and strategies in the same way.
I saw the tutorial about datasets, but since I don't want to load the whole dataset and train/test with it, but only get one specific example at a time from rest of the program, I'm not sure what the best way is to convert this incoming (n features + task label) one example, so I can train an Avalanche model with it.
Any advice would be really appreciated! :)

[EdenBelouadah]

Hello,
I think you can do it with something like this (I don't know if it' the best way though):

scenario = filelist_scenario(
    root="",
    train_file_lists=["/path/to/img1.txt",
                      "/path/to/img2.txt",
                      "/path/to/img3.txt",],

    test_file_lists=["/path/to/batch1.txt",
                     "/path/to/batch2.txt",
                     "/path/to/batch3.txt",],

    task_labels=list(range(0, 3)),
    complete_test_set_only=False,
    train_transform=train_transform,
    eval_transform=test_transform,
)

"/path/to/img1.txt" is a file containing the path of one image.
"/path/to/batch1.txt" is a file containing paths of many images used for evaluations. You can use one image as well

[TheRisenPhoenix]

Hi,
thank you for your answer.
I'm aware of this possibility, however, it doesn't really fit my scenario.

I'm already doing some preprocessing and feature extraction in my program, and eventually come out with a list of features in memory. With an usual keras/pytorch model, I can call a fit method that directly takes a set of values, and I can use it to input one datapoint at a time (from the outside of the program).
My input therefore looks something like this:
input = [0.4, 0.2, 0.1, 0.6]
label = [0]

As avalanche takes (as far as I can see) only scenario-objects (which have a whole bunch of attributes about the total of a dataset), I'm looking for a way to use the underlying model directly with my preprocessed input data for one datapoint, and not to have to provide a whole dataset of any kind.

I hope this helps to clarify my problem, I'm sorry for the confusion

[vlomonaco]

Hi @TheRisenPhoenix thanks for opening this discussion. It is very interesting to us to understand how avalanche is currently used in existing projects. Have you considered using the "tensors_scenario" generator?

pattern_shape = (3, 32, 32)

# Definition of training experiences
# Experience 1
experience_1_x = torch.zeros(100, *pattern_shape)
experience_1_y = torch.zeros(100, dtype=torch.long)

# Experience 2
experience_2_x = torch.zeros(80, *pattern_shape)
experience_2_y = torch.ones(80, dtype=torch.long)

# Test experience
# For this example we define a single test experience,
# but "tensors_scenario" allows you to define even more than one!
test_x = torch.zeros(50, *pattern_shape)
test_y = torch.zeros(50, dtype=torch.long)

generic_scenario = tensors_scenario(
    train_tensors=[(experience_1_x, experience_1_y), (experience_2_x, experience_2_y)],
    test_tensors=[(test_x, test_y)],
    task_labels=[0, 0],  # Task label of each train exp
    complete_test_set_only=True
)

You can read more about this generator here.

[TheRisenPhoenix]

Thank you for your answer, @vlomonaco !
Indeed, it looks like that this is the solution I was looking for - I ignored it when I searched the documentation for the first time, because I thought it would be used in other cases. Thanks for pointing that out!

However, I've run into another problem now. My usecase is about binary classification, therefore my model has one output node that I expect to be 0 or 1. However, when I input my experience_y as to be a tensor of a single one or a single zero, torch gives me an exception, that target 1 is out of bounds (in torch._C._nn.nll_loss), as soon as I hit the positive class example. With the negative class (0) training works fine.
Do I have to use multiclass classification, so expand my output nodes to 2 to get it working, or did I do something wrong?

One experience looks like this:
Negative:
(tensor([[ 2.7774, -1.4017, -1.2465, -0.2646, 0.9403, -0.3448, 2.6328, -0.7428]]), tensor([0]))
Positive:
(tensor([[-0.2695, 0.0816, 0.8156, -0.2646, -0.7135, -0.3448, -0.1993, -0.7428]]), tensor([1]))

[vlomonaco]

I think this really depends on your preferred model/loss implementations. I'd suggest to have two output neurons with cross-entropy loss. To use the basic Pytorch CrossEntropyLoss you need to provide targets as one-hot encodings. Let us know how it goes and if we can mark this question as answered! :)

[TheRisenPhoenix]

Okay, thank you very much for your time and advice! As my original question is answered, I will mark it accordingly.
Thanks again! :)