easy-few-shot-learning
easy-few-shot-learning copied to clipboard
sicara
How to get the Prediction for new images?
After training a model based on Prototype networking and evaluating on test data. I want to do prediction on new image set. How can I do that?
Hi! You can create any new dataset for inference with the EasySet interface. You can find an example in the last step of the quickstart tutorial.
The last section of the tutorial explains how to create a dataset for testing. --> 6. Evaluate your model on the test set To use this section, you have to create a json file that contains the corresponding label for each image. In this sense, we are not able to use the model to make an inference. I would also like to know how can we make some predictions on new images.
Hi! Thanks for your feedback, this is a very good point. At this point, there is nothing in EasyFSL to handle partially labeled datasets.
You can't use EasySet or TaskSampler in this case because you would need the labels for query images. What you can use is the methods themselves.
Could you tell me more about your specific use case? Do you have only one support set, or do you need to handle a variety of tasks?
@ebennequin If I give a temporary label to the query dataset, Isn't it predictable? But even if the metrics aren't reliable ? Can I organize dataset for interference only now?
Hi! I'm sorry I don't fully understand your questions. Could you tell me more about your issue? Namely:
- What do you want to do exactly
- What is currently preventing you from doing it
I think what @youngjae-avikus intends to ask is how to run predictions on novel data points.
Given any new unknown image (not a part of train/valid/test split), how to predict it's classes.
I just started tinkering with code in the repo, will add an example in training notebook here https://github.com/INF800/easyfsl_custom_data
I think what @youngjae-avikus intends to ask is how to run predictions on novel data points.
Given any new unknown image (not a part of train/valid/test split), how to predict it's classes.
I think as in #15
I think what @youngjae-avikus intends to ask is how to run predictions on novel data points.
Given any new unknown image (not a part of train/valid/test split), how to predict it's classes.
Thank you this is very clear!
To perform prediction on a single image
Here with artificially generated support and query images:
# 1. Instantiate your classifier
my_classifier = PrototypicalNetworks(my_backbone)
# 2. Process your support set
support_images = torch.rand((6, 3, 32, 32)) # 6 images of size (3, 32, 32)
support_labels = torch.tensor([0, 0, 1, 1, 2, 2]) # 3 classes, 2 shots per class
my_classifier.process_support_set(support_images, support_labels)
# 3. Predict
single_query_image = torch.rand((1, 3, 32, 32)) # 1 image of size (3, 32, 32)
prediction = my_classifier(single_query_image)
Does that answer your question @youngjae-avikus ?
I created an example that might help you.
import torchvision.transforms as tt
import torch
from torchvision.datasets import ImageFolder
from easyfsl.methods import FewShotClassifier
from torch.utils.data import DataLoader
class FewShotPredictor :
"""
This class aims to implement a predictor for a Few-shot classifier.
The few shot classifiers need a support set that will be used for calculating the distance between the support set and the query image.
To load the support we have used an ImageFolder Dataset, which needs to have the following structure:
folder:
|_ class_name_folder_1:
|_ image_1
|_ …
|_ image_n
|_ class_name_folder_2:
|_ image_1
|_ …
|_ image_n
The folder must contain the same number of images per class, being the total images (n_way * n_shot).
There must be n_way folders with n_shot images per folder.
"""
def __init__(self ,
classifier: FewShotClassifier,
device,
path_to_support_images,
n_way,
n_shot,
input_size=224):
"""
:param classifier: created and loaded model
:param device: device to be executed
:param path_to_support_images: path to creating a support set
:param n_way: number of classes
:param n_shot: number of images on each class
:param input_size: size of image
"""
self.classifier = classifier
self.device = device
self.predict_transformation = tt.Compose([
tt.Resize((input_size, input_size)),
tt.ToTensor()
])
self.test_ds = ImageFolder(path_to_support_images, self.predict_transformation)
self.val_loader = DataLoader(
self.test_ds,
batch_size= (n_way*n_shot),
num_workers=1,
pin_memory=True
)
self.support_images, self.support_labels = next(iter(self.val_loader))
def predict (self, tensor_normalized_image):
"""
:param tensor_normalized_image:
Example of normalized image:
pil_img = PIL.Image.open(img_dir)
torch_img = transforms.Compose([
transforms.Resize((224, 224)),
transforms.ToTensor()
])(pil_img)
tensor_normalized_image = tt.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])(torch_img)[None]
:return:
Return
predict = tensor with prediction (mean distance of query image and support set)
torch_max [1] = predicted class index
"""
with torch.no_grad():
self.classifier.eval()
self.classifier.to(self.device)
self.classifier.process_support_set(self.support_images.to(self.device), self.support_labels.to(self.device))
pre_predict = self.classifier(tensor_normalized_image.to(self.device))
predict = pre_predict.detach().data
torch_max = torch.max(predict,1)
class_name = self.test_ds.classes[torch_max[1].item()]
return predict, torch_max[1], class_name
#49
Can somebody tell what model has predicted? it is just showing accuracy however I want to know the prediction result i.e. the class that model predict.
Hi, the first item of the tuple returned by the predict method in @diego91964 's chunk of code contains the predicted scores, the second item contains the class index, and the third contains the class name.
I created an example that might help you.
import torchvision.transforms as tt import torch from torchvision.datasets import ImageFolder from easyfsl.methods import FewShotClassifier from torch.utils.data import DataLoader class FewShotPredictor : """ This class aims to implement a predictor for a Few-shot classifier. The few shot classifiers need a support set that will be used for calculating the distance between the support set and the query image. To load the support we have used an ImageFolder Dataset, which needs to have the following structure: folder: |_ class_name_folder_1: |_ image_1 |_ … |_ image_n |_ class_name_folder_2: |_ image_1 |_ … |_ image_n The folder must contain the same number of images per class, being the total images (n_way * n_shot). There must be n_way folders with n_shot images per folder. """ def __init__(self , classifier: FewShotClassifier, device, path_to_support_images, n_way, n_shot, input_size=224): """ :param classifier: created and loaded model :param device: device to be executed :param path_to_support_images: path to creating a support set :param n_way: number of classes :param n_shot: number of images on each class :param input_size: size of image """ self.classifier = classifier self.device = device self.predict_transformation = tt.Compose([ tt.Resize((input_size, input_size)), tt.ToTensor() ]) self.test_ds = ImageFolder(path_to_support_images, self.predict_transformation) self.val_loader = DataLoader( self.test_ds, batch_size= (n_way*n_shot), num_workers=1, pin_memory=True ) self.support_images, self.support_labels = next(iter(self.val_loader)) def predict (self, tensor_normalized_image): """ :param tensor_normalized_image: Example of normalized image: pil_img = PIL.Image.open(img_dir) torch_img = transforms.Compose([ transforms.Resize((224, 224)), transforms.ToTensor() ])(pil_img) tensor_normalized_image = tt.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])(torch_img)[None] :return: Return predict = tensor with prediction (mean distance of query image and support set) torch_max [1] = predicted class index """ with torch.no_grad(): self.classifier.eval() self.classifier.to(self.device) self.classifier.process_support_set(self.support_images.to(self.device), self.support_labels.to(self.device)) pre_predict = self.classifier(tensor_normalized_image.to(self.device)) predict = pre_predict.detach().data torch_max = torch.max(predict,1) class_name = self.test_ds.classes[torch_max[1].item()] return predict, torch_max[1], class_name#49
I have tried using this for prediction with custom dataset, but I am getting this error as "RuntimeError : Given groups=1, weight of size [64, 3, 7, 7], expected input[1, 2, 3, 4] to have 3 channels, but got 2 channels instead" What could be the issue? could you please help?
Seems like something that would happen if you used this code with grayscale images. You could fix it by converting images to RGB for instance.
Seems like something that would happen if you used this code with grayscale images. You could fix it by converting images to RGB for instance.
Thanks for your response. Should we convert to RGB while training as well? Because I tried converting the test image using below code Instead of this line in the code provided: "pil_img = PIL.Image.open(img_dir)"
I am using cv2 to read and convert to RGB img_array = cv2.imread(img_path) img_rgb = cv2.cvtColor(img_array, cv2.COLOR_BGR2RGB) img_rgb = cv2.resize(img_rgb,(224,224),3)