Multi-Label Dataset in Avalanche

[DavideThird]

Hi everyone.

I would be interested to use Avalanche with a MultiLabel Dataset.
I am interested in the Domain Incremental Scenario where I have always the same classes but from different domains.
However, when I try to pass the multilabel dataset to ni_scenario I receive an error.

avalanche_dataset.py in _count_unique(*sequences)
   2165     for seq in sequences:
   2166         for x in seq:
-> 2167             uniques.add(int(x))
   2168 
   2169     return len(uniques)

ValueError: only one element tensors can be converted to Python scalars

I provide an example of what I would like to have.

import torch
import torchvision
from avalanche.benchmarks.datasets import MNIST
from avalanche.benchmarks.generators import nc_benchmark, ni_benchmark

import matplotlib.pyplot as plt
import numpy as np

from torchvision.transforms import Compose, ToTensor, Normalize, RandomCrop, RandomRotation

def produce_rotated_dataset(rotation_angle):
    rotation = RandomRotation(degrees=(rotation_angle, rotation_angle))

    train_transform = Compose([
        rotation,
        ToTensor(),
        Normalize((0.1307,), (0.3081,))
    ])
    test_transform = Compose([
        ToTensor(),
        Normalize((0.1307,), (0.3081,))
    ])
    mnist_train = MNIST(
        './data/mnist', train=True, download=True, transform=train_transform
    )
    mnist_test = MNIST(
        './data/mnist', train=False, download=True, transform=test_transform
    )
    
    return mnist_train, mnist_test

Now I create 4 MNIST dataset with 4 different rotation angles.

rotation_angles = [0,90,180,270]
list_datasets = [ produce_rotated_dataset(rotation_angle) for rotation_angle in rotation_angles ] 

Now I randomly join two sample to create a new sample with associated two labels that are the original ones of the two samples i.e. I obtain a MultiLabel dataset.

from torch.utils.data import Dataset, DataLoader
class MultiLabelDataset(Dataset):
    """Face Landmarks dataset."""

    def __init__(self, dataset): 
        random_matrix = np.random.randint(low=0, high=len(dataset), size=1000).reshape((500,2))
        
        def produce_img_labels(dataset,random_matrix,i):
            index_0 = random_matrix[i][0]
            index_1 = random_matrix[i][1]
            img_0 = dataset[index_0][0]
            label_0 = dataset[index_0][1]
            img_1 = dataset[index_1][0]
            label_1 = dataset[index_1][1]
            img = torch.hstack([img_0,img_1])
            
            labels = np.asarray([label_0,label_1])  
            labels = torch.from_numpy( labels )      
            
            return img,labels
        
        
        self.lista = [ produce_img_labels(dataset,random_matrix,i) for i in range(random_matrix.shape[0])  ]
        self.data = [ self.lista[i][0]  for i in range(random_matrix.shape[0])   ]
        self.targets = [ self.lista[i][1]  for i in range(random_matrix.shape[0])   ]

    def __len__(self):
        return len(self.lista)

    def __getitem__(self, idx):
        return self.lista[idx]

list_train_datasets = [ MultiLabelDataset(mnist_train) for mnist_train,mnist_test in list_datasets ]
list_test_datasets = [ MultiLabelDataset(mnist_test) for mnist_train,mnist_test in list_datasets ]

scenario = ni_benchmark(
    list_train_datasets, list_test_datasets, n_experiences=4, shuffle=False, seed=1234, task_labels=False
)

However, the scenario creation produce the following error:

avalanche_dataset.py in _count_unique(*sequences)
   2165     for seq in sequences:
   2166         for x in seq:
-> 2167             uniques.add(int(x))
   2168 
   2169     return len(uniques)

ValueError: only one element tensors can be converted to Python scalars

Like I said before, I am interested in the Domain Incremental Scenario.
There is a simple way to solve this problem ?

[AntonioCarta]

targets is supposed is supposed to be a single class, that's why you get an error. Can you try to create your datasets without a target (or using dummy targets)? Most strategies should work as long as you define a proper loss function and metrics since they need to be adapted for your multi-class target.

Unfortunately we still have to update AvalancheDataset to not require targets.

@lrzpellegrini do you have any suggestions?