How to use the pretrained SimCLR model to extract image features.

[hosea7456]

❓ Questions and Help

I would like to use the pretrained model of SimCLR to extract image features. I wonder if pl_bolts.models also support pretrained SimCLR model like the CPCV2 as follows?

from pl_bolts.models.self_supervised import CPCV2
# load resnet18 pretrained using CPC on imagenet
model = CPCV2(pretrained='resnet18')
cpc_resnet18 = model.encoder

Thanks a lot!

[github-actions[bot]]

Hi! thanks for your contribution!, great first issue!

[ananyahjha93]

@hosea7456 we are in the process of updating simclr weights for cifar-10 and Imagenet. Also, to use the simclr pretrained weights, you can refer to the docs: https://pytorch-lightning-bolts.readthedocs.io/en/latest/self_supervised_models.html#simclr

The code below just defines the usage pattern. It needs to be updated with the latest trained weights.

from pl_bolts.models.self_supervised import SimCLR

weight_path = 'https://pl-bolts-weights.s3.us-east-2.amazonaws.com/simclr/simclr-cifar10-v1-exp12_87_52/epoch%3D960.ckpt'
simclr = SimCLR.load_from_checkpoint(weight_path, strict=False)

simclr.freeze()

[motoight]

Sorry,when I run the code below, I met an error. Please tell how to fix it, thanks a lot!

from pl_bolts.models.self_supervised import SimCLR
weight_path = 'https://pl-bolts-weights.s3.us-east-2.amazonaws.com/simclr/bolts_simclr_imagenet/simclr_imagenet.ckpt' 
net1 = SimCLR.load_from_checkpoint(weight_path, strict=False)

RuntimeError: Error(s) in loading state_dict for SimCLR:
size mismatch for encoder.conv1.weight: copying a param with shape torch.Size([64, 3, 7, 7]) from checkpoint, the shape in current model is torch.Size([64, 3, 3, 3]).
size mismatch for projection.model.3.weight: copying a param with shape torch.Size([128, 2048]) from checkpoint, the shape in current model is torch.Size([2048]).

[ananyahjha93]

@motoight install bolts from master branch, that should fix it

pip install git+https://github.com/PytorchLightning/pytorch-lightning-bolts.git@master --upgrade

[motoight]

Thanks a lot, this works out for me!

[hosea7456]

@ananyahjha93
Thanks a million.

[MiladInk]

Hi! I think the link for the pretrained SimCLR model here is wrong because clearly it is written in the url that is the imagent net pre-trained model.

Would you mind taking a look?