About tensor_scenario() Method

[highclef]

Hi, I'm quite new to machine learning and I got to know Avalanche library due to my master thesis.
What I am trying to do is to apply diverse continual learning strategies with given custom datasets (.csv).
So I have x_data and y_data, which are defined as tensors and have [67475, 1, 18] and [67475, 1, 6] shapes for each (67475: number of rows in my dataset, 18: number of inputs, 6: number of outputs).
I splitted them using train_test_split() and applied tensor_scenario() as followings:

def zerolistmaker(n):
    listofzeros = [0] * n
    return listofzeros

x_train, x_test, y_train, y_test = train_test_split(x_data, y_data, test_size=0.3, random_state=42)

generic_scenario = tensor_scenario(
    train_data_x=x_train,
    train_data_y=y_train,
    test_data_x=x_test,
    test_data_y=y_test,
    task_labels=zerolistmaker(67475),  # Task label of each train exp
    complete_test_set_only=True
)

# split stream into train and test set
train_stream = generic_scenario.train_stream
test_stream = generic_scenario.test_stream

# Instantiate Continual learning strategy
strategy = Naive(
     model, optimizer, criterion, train_mb_size=32, train_epochs=2,
     eval_mb_size=32)

results = []
 # train and test epochs
for train_task in train_stream:
     # training
     strategy.train(train_task, num_workers=4)
     # evaluate the model and store the results
     results.append(strategy.eval(test_stream))

And here are my questions:

1. Why is the value len(test_stream) equals only 1, while len(train_stream) equals 47232 (70% of the whole dataset)? Where are the rest gone?
2. This way, I'm training my network using 67475 datasets. Am I also generating 67475 experiences and train_task from tensor_scenario()? Am I using the method properly?

By the way, I've read through the Avalanche dataset documentation and got managed to define AvalancheTensordataset element, but I feel like some linkage is missing there between dataset and training. So it would be nice to know how exactly to put AvalancheDataset or AvalancheTensorDataset into the strategy.train() method properly.

Please let me know For any ambiguities. Sorry for basic questions, but I believe this could help others as well 🙏

[highclef]

Update

I resolved this by myself and here is what I did:

• I reshaped my x_data and y_data tensors as [67475, 1, 18] and [67475, 6] for each.
  -> The second index of x_data due to one of the input parameter (sequence length) of my RNN, because this is a LSTM.
  -> I reduced the dimension of my y_data because otherwise I got an error (I am not quite sure why).
• I changed the task_labels parameters according to the number of experiences that I used (see below).
• I changed the method from tensor_scenario() to tensors_scenario()

And here are two different versions of the code:

1. Using only one experience

# create scenario instance
generic_scenario = tensors_scenario(
    train_tensors=[(x_train, y_train)],
    test_tensors=[(x_test, y_test)],
    task_labels=[0],  # Task label of each train exp
    complete_test_set_only=True
)

# split stream into train and test set
train_stream = generic_scenario.train_stream
test_stream = generic_scenario.test_stream

# Instantiate Continual learning strategy
strategy = Naive(
     model, optimizer, criterion, train_mb_size=32, train_epochs=2,
     eval_mb_size=32)

results = []
 # train and test epochs
for train_task in train_stream:
     # training
     strategy.train(train_task, num_workers=4)
     # evaluate the model and store the results
     results.append(strategy.eval(test_stream))

2. Using three experiences

Here I splitted the datasets into three different subsets.

# create scenario instance
generic_scenario = tensors_scenario(
    train_tensors=[(x_train1, y_train1), (x_train2, y_train2), (x_train3, y_train3)],
    test_tensors=[(x_test1, y_test1), (x_test2, y_test2), (x_test3, y_test3)],
    task_labels=[0, 0, 0],  # Task label of each train exp
    complete_test_set_only=False
)

# split stream into train and test set
train_stream = generic_scenario.train_stream
test_stream = generic_scenario.test_stream

# Instantiate Continual learning strategy
strategy = Naive(
     model, optimizer, criterion, train_mb_size=32, train_epochs=2,
     eval_mb_size=32)

results = []
 # train and test epochs
for train_task in train_stream:
     # training
     strategy.train(train_task, num_workers=4)
     # evaluate the model and store the results
     results.append(strategy.eval(test_stream))

I hope it can help someone in the future who's struggling with the same problem :)

[MagicHealer]

@highclef Do you have an example of this method where you use different datasets? Like for first task you use imagenet and other datasets like CUB for second tasks? It would really help me. Thank you.

[highclef]

@MagicHealer Unfortunately I haven't tried it with multiple datasets (experiences) from different data sources. I had a single big dataset and I split it to generate multiple datasets (experiences).

[MagicHealer]

@highclef Thank you for your time in answering my question. I am currently working on a project using 2 separate datasets, but I can't have a successful training. Thanks for your time

[vlomonaco]

Hi @highclef! I was about to answer to this, sorry about the delay :/ and thanks for updating your question with you experience! 💯