Avalanche (PyTorch) reads input images (of my custom dataset) as black images.

[RezaKakooee]

Hi,
I am using Avalanche for a CL problem, but as an initial step I am using JointTrianing to find an upper bound. But, I encountered an strange issue. The final confusion matrix is wrong as all numbers appear only in one of the columns, and the rest columns have just 0 values. I tried to debug the code. Before JointTrianing starts the training process I called one of the images from generic_scenario.test_stream.dataset to visualize it and see what is going wrong. Matplotlib shows me only a completely black image. I decided to test the CL pipeline on another dataset. To do so, instead of calling mnist dataset from Avalanche I downloaded mnist dataset from Kaggle as png to feed it to my CL pipeline without changing anything (indeed only input dataset changed). The confusion matrix is good this time, and Matplotlib shows the right image from mnist!

So, my question is that how Avalanche processes the images in the background which leads to such black images?

I am attaching one of the images from my dataset to this message too.

Many thanks

[vlomonaco]

Hi @RezaKakooee! Sorry for the rough patches, can you share the minimal code to re-produce the error? :)
This seems a data pre-processing problem. Avalanche uses the transformations defined during the dataset creation. How are you creating your own dataset?

[AntonioCarta]

@RezaKakooee we are going to fix this ASAP. In the mean time, you could implement your matrix as a custom PluginMetric. If you look at Avalanche's implementation of the confusion matrix it should be relatively straightforward.

[RezaKakooee]

@AntonioCarta , thanks for your reply. yes I looked at it. but now i have another problem with my training pipeline which is more annoying than CM. The problem is that, JointTraining strategy is able to classify with a reasonable accuracy, when I train it on a small portion of my dataset. However, when I train it on the whole dataset. it does not learn anything!

[AntonioCarta]

Are you sure there's a problem with Avalanche? Are you able to train your model using plain pytorch? JointTraining is the classic offline training so it should be easy.

My guess is that you have to do an hyperparameter search.

[AndreaCossu]

We just merged the PR #714. Please @RezaKakooee let us know if this fixes your bug with the confusion matrix in the Joint Training.

[RezaKakooee]

@AndreaCossu Hi, sorry for the late reply. many thanks.