Not seeing images for labelling in interactive mode

[ElisonSherton]

Hi guys,

I was trying to use finetuner to build a CBIR system and followed this starter but was getting the above error.

Environment Details: python3.8 with torch and finetuner installed.

Would be grateful if someone can clarify this earnestly.

Thanks,
Vinayak.

[bwanglzu]

hi @ElisonSherton i just runned the same code in my mac, I can not reproduce, can you try to install the latest jina using pip install -U jina and run again?

also, can you check you have converted datauri to blob in data_gen like this?

def data_gen():
    for d in from_files('/Users/bo/Downloads/img_align_celeba/*.jpg', size=100, to_dataturi=True):
        d.convert_image_datauri_to_blob(color_axis=0) 
        yield d

and please make sure the data path you send to from_files has been replaced to your local data path

[ElisonSherton]

Hi @bwanglzu

Thanks for the prompt reply.

from jina.types.document.generators import from_files

def data_gen():
    for d in from_files('../data/cleaned_length/womens-length/train/knee/*.jpg', size=100, to_dataturi=True):
        d.convert_image_datauri_to_blob(color_axis=0)  # no need of tf
        yield d

import torchvision
model = torchvision.models.resnet50(pretrained=True)

import finetuner

finetuner.fit(
    model=model,
    interactive=True,
    train_data=data_gen,
    freeze=True,
    to_embedding_model=True,
    input_size=(3, 224, 224),
    output_dim=100
)

Here is my code and I am able to see the UI, can't see any neighbours though as shown in the image below.

The path exists and also the UI is picking query images perfectly from the path, however recommendations for labelling are not shown.

Thanks,
Vinayak.

[bwanglzu]

hi @ElisonSherton

thanks for the reply, I think I know the issue. can you change the file path to absolute path, instead of a relative path? We already noticed this issue in core, and we're fixing it.

Best

Bo

[ElisonSherton]

Hi @bwanglzu

I have changed my path to absolute path, but still I can see the same error and other suggestions are not getting loaded like before :/

Thanks,
Vinayak

[hanxiao]

i think the problem is not about the path, it complains about the data size, so I think your image size is not something that ResNet appreciate. ResNet accepts only input_size=(3, 224, 224), and from your log it says 3,500,500 or 3,900,600

[ElisonSherton]

Hi @hanxiao ,

I have specified input size to be cast to (3, 224, 224) in the finetuner.fit method. Would that not suffice?

Do I need to resize everything beforehand to this size and save the images on disk and then load from there?

Thanks,
Vinayak

[hanxiao]

aha, okay i see what you mean, actually input_size is the shape of the input layer of your network, finetuner requires this if your network is built by pytorch/paddle for doing its magic. But this argument is not about auto reshaping your input into input_size.

Do I need to resize everything beforehand to this size and save the images on disk and then load from there?

No need, let me try something and get back to you

[hanxiao]

hi @ElisonSherton i'm guessing your images are in different sizes right? the first image's size is different than the second, and third, etc.

[ElisonSherton]

Yes, they're not of the same size.

[hanxiao]

that's the problem, i can also reproduce it when the size is not the same. it is a missing feature on our end, you can also normalize them by yourself first, or wait for a day we fix on our end

[hanxiao]

@ElisonSherton solving this issue in upstream jina-ai/serve#3750

[vahuja4]

@hanxiao - has this been resolved now?

[hanxiao]

yes, see next thread.

[hanxiao]

@vahuja4 this is solved, and here is how you can use images of different sizes.

1. I prepared some random jpgs
   
2. upgrade your finetuner, pip install -U finetuner
3. build the constructor with proper resizing:

from jina.types.document.generators import from_files

def data_gen():
    for d in from_files('/Users/hanxiao/Documents/tmpimg/*.jpg'):
        d.convert_image_uri_to_blob()
        d.convert_uri_to_datauri()
        d.resize_image_blob(224, 224)
        d.normalize_image_blob()
        d.set_image_blob_channel_axis(-1, 0)
        yield d

3. Now run the main routine of finetuner.

import torchvision
import finetuner

model = torchvision.models.resnet50(pretrained=True)

finetuner.fit(
    model=model,
    interactive=True,
    train_data=data_gen,
    freeze=True,
    to_embedding_model=True,
    input_size=(3, 224, 224),
    output_dim=100,  # Chop-off the last fc layer and add a trainable linear layer.
)

you will get: