Should we pass dataset transformations to the pytorch dataset or to the nc_benchmark function ?

[AlbinSou]

Hello,

I started using the library and I am confused about some points. One of them is the fact that the benchmark generator functions accept both train_ and eval_ transform as arguments. I do not understand the purpose of these two arguments given that you can already give these transformations to the pytorch dataset themselves when creating them.

So, how should we use it,

• create the datasets without the transform and pass them only to nc_benchmark ?
• create the datasets with the transforms and pass them to nc_benchmark as well ?
• create the datasets with the transforms and don't pass them to nc_benchmark ?

Of course I could check the behavior myself but I am just wondering what is the purpose of these arguments and what they are used for compared to the classical transformations. Maybe a clarification could be made in the documentation too.

[lrzpellegrini]

Hi Albin,
the recommended way is to pass transformations to nc_benchmark. You can also apply them to datasets and call nc_benchamark without them, but in this way you won't be able to access eval transformations from the training set and vice-versa. For instance, by passing transformations to nc_benchamark you can do something like:

train_dataset_without_augmentation = train_experience.dataset.eval()

test_dataset_with_augmentation = eval_experience.dataset.train()

I know that those features are not properly documented, we are working on aligning the documentation with the actual state of the framework.

One thing you shouldn't do is to apply transformations to the datasets and also pass them to nc_benchamark as this would result in a duplicate stack of transforms.

[AlbinSou]

Ok great, thank you for the precisions !