could you give the code for extract ae and maml featrue, we want to extract feature on own dataset

[tangxiaochu123230]

[xuanbinh-nguyen96]

Hi! I regret not being able to provide feature extraction code because our proposal does not use any feature as input. We simply resized all the images to the suitable size (84x84 for maml and 128x128 for ae) and saved them as one file. If you want to work with your dataset, please resize them and use the pre-trained models we provided to extract the features you want.

[tangxiaochu123230]

do not you subtract mean and divide std. I cannot preduce your result

[xuanbinh-nguyen96]

simply you open the image with the PIL library, resize and convert to the torch tensor, then divide them directly by 255.

[xuanbinh-nguyen96]

from PIL import Image
img_size = [128, 128]
transform = transforms.Compose([lambda x: Image.open(x).convert('L'),
                                         transforms.Resize(img_size),
                                         transforms.ToTensor(),
                                lambda x: x/255.])

[tangxiaochu123230]

thanks you very much [code.]

on rad datset,we get very different feature

[tangxiaochu123230]

if i do not use lambda x: x/255..it may be more closer

[xuanbinh-nguyen96]

I double checked and noticed there is a bit of a mistake here. Very sorry for this. For our proposed dataset - PathVQA:

• for maml (size 84x84), run:
  • Step 1 (resize all image by PIL with LANCZOS), run:

import os

from PIL import Image

path_to_images = 'pathVQA/images'

all_images = []

# Resize images
for r, d, f in os.walk(path_to_images):
    for file in f:
        if '.jpg' in file:
            all_images.append(os.path.join(r, file))

for i, image_file in enumerate(all_images):
    im = Image.open(image_file)
    im = im.resize((84, 84), resample=Image.LANCZOS)
    im.save(image_file)
    if i % 500 == 0:
        print(i)

Step 2, using the bellow "transform" on the images processed in step 1.

img_size = [84, 84]
transform = transforms.Compose([lambda x: Image.open(x).convert('RGB'),
                                         transforms.Resize(img_size),
                                         transforms.ToTensor(),
                                         transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225))
                                         ])

• for ae (size 128x128), run:
  • Step 1 (resize all image by PIL with LANCZOS), run:

import os

from PIL import Image

path_to_images = 'pathVQA/images'

all_images = []

# Resize images
for r, d, f in os.walk(path_to_images):
    for file in f:
        if '.jpg' in file:
            all_images.append(os.path.join(r, file))

for i, image_file in enumerate(all_images):
    im = Image.open(image_file)
    im = im.resize((128, 128), resample=Image.LANCZOS)
    im.save(image_file)
    if i % 500 == 0:
        print(i)

Step 2, using the bellow "transform" on the images processed in step 1.

img_size = [128, 128]
transform = transforms.Compose([lambda x: Image.open(x).convert('L'),
                                         transforms.Resize(img_size),
                                         transforms.ToTensor(),
                                         transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225))
                                         ])

For RAD dataset, we use dataset from https://github.com/aioz-ai/MICCAI19-MedVQA

[tangxiaochu123230]

great,thank you very much for you patience. I want to implemet vqa, if I can skip MMQ Progress and finetune VQA Progress based on model train on RAD dataset