Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proper conversion of Tensor to list #1524

Merged
merged 2 commits into from
Oct 23, 2023

Conversation

malashinroman
Copy link

list(torch.Tensor) doesn't convert Tensor to list of scalars, but to list of Tensors.

I checked this is True for pytorch versions 1.12 and 2.0

[ins] In [6]: a = torch.Tensor([0,1])
[ins] In [7]: a.tolist()
Out[7]: [0.0, 1.0]
[ins] In [8]: list(a)
Out[8]: [tensor(0.), tensor(1.)]

That lead to the KeyError when I tried to used combined pytorch datsets, like this

    if args.reproduce_bugs:
        # substitute labels 0-9 with 10-19
        fmnist_test.targets += 10
        fmnist_train.targets += 10
        full_test = mnist_test + fmnist_test
        full_train = mnist_train + fmnist_train
        n_classes = 20
    else:
        full_test = mnist_test
        full_train = mnist_train
        n_classes = 10

    benchmark = nc_benchmark(
        full_train,
        full_test,
        n_experiences=n_classes // 2,
        task_labels=list(range(10)),
    )

I probably will open an issue, because the same probably will repeat at least for similar usage in

avalanche/benchmarks/utils/dataset_traversal_utils.py:191

@malashinroman malashinroman mentioned this pull request Oct 19, 2023
@AntonioCarta
Copy link
Collaborator

Thanks! Looks good to me.

can you add a small unit test so that we can avoid a regression?

@lrzpellegrini can you also check this? Do you know if we have other places where we may have the same bug?

@lrzpellegrini
Copy link
Collaborator

Not that I am aware of. Alas, this is a common issue when dealing with elements that may be PyTorch Tensors.

@malashinroman
Copy link
Author

malashinroman commented Oct 20, 2023

@AntonioCarta, I will add unittest tomorrow.

*frankly saying in environment managed as documented I wasn't able to pass couple of fast tests

@AntonioCarta
Copy link
Collaborator

*frankly saying in environment managed as documented I wasn't able to pass couple of fast tests

If you think this is a general issue and not a problem with your environment, feel free to open a separate issue.

For this PR, you can run just your test with python -m unittest tests.your.module.TestClass.test_method

@malashinroman
Copy link
Author

malashinroman commented Oct 21, 2023

@AntonioCarta I've found that scenarios I used before are now deprecated, so I added test to verify type and content of TaskAwareSuperivsedClassificationDataset.targets_val_to_idx

@AntonioCarta AntonioCarta merged commit 3527723 into ContinualAI:master Oct 23, 2023
10 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants