about all_image_batches in load_data.py

[JJ-res101]

trafficstars

Hello, I want to ask why some examples in the same class are different, and some examples of different classes are the same, as shown in block 8-9 of Homework 1 - Step by step.ipynb.

[LecJackS]

Your question has two answers:

why some examples in the same class are different

A class is an alphabet, so this is okay. All different characters on the english alphabet are from one class, all characters on the Japanese alphabet are from another class, etc.

some examples of different classes are the same

This should not happen. I think I made an error while using random.choice from numpy:

numpy.random.choice(a, size=None, replace=True, p=None)

replace should be False , so choosing the same alphabet two times is not allowed.

Need to check that, but I think that's it.

Thank you! :)

[JJ-res101]

Thank you for your reply！

why some examples in the same class are different

You are right. And my question is in the same class how do you make sure that different shots come from the same alphabet.In this sentence:

all_image_batches[b,k,n,:] = image_file_to_array(filename=images_labels[count][1], dim_input=pixel)

As k is the same and n is different, images_labels[count][1] and images_labels[count+1][1] are not necessarily different, because shuffle=true in get_images. Again, when n is the same and k is different, the sample from images_labels doesn't have to be the same class. It really bothers me.

In addition, you can read my modified code if you like. Maybe you can see more clearly what I mean.

        #### YOUR CODE GOES HERE ####
        k_samples = self.num_samples_per_class
        n_classes = self.num_classes
        pixel = self.dim_input       
        all_image_batches = np.ndarray((batch_size, K, N, pixel))
        all_label_batches = np.ndarray((batch_size, K, N, N))
        for b in range(batch_size):
            paths = random.sample(folders, N)
            labels = np.arange(1, N+1)
            images_labels = get_images(paths, labels, K,shuffle=False)#shuffle=False
            for k in range(k_samples):
                for n in range(n_classes):
                    all_image_batches[b, k, n, :] = image_file_to_array(filename=images_labels[k+n*k_samples][1], dim_input=pixel)
                    all_label_batches[b, k, n, :] = np.zeros(n_classes)
                    all_label_batches[b, k, n, n] = 1               
        #############################

[LecJackS]

Hi again @JC-chen97 !

I ended up adding this at the init function:

self.lang_folders = [os.path.join(data_folder, family)
                     for family in os.listdir(data_folder)
                     if os.path.isdir(os.path.join(data_folder, family))]

So now self.lang_folders contains a list of all available classes, as directories, like:

['./omniglot_resized/Burmese_(Myanmar)',
 './omniglot_resized/Gurmukhi',
 './omniglot_resized/Anglo-Saxon_Futhorc',
  .
  .
  .
 './omniglot_resized/Malay_(Jawi_-_Arabic)',
 './omniglot_resized/Kannada']

Doing that, is easier to sample N classes from there, and filter those from the original folders (with all the classes, all characters mixed), which have that each sampled class in their directory.

        for b in range(batch_size):
            # Sample N classes (languages) from all 50 possibles alphabet
            sampled_classes = np.random.choice(self.lang_folders, n_classes, replace=False)
            #print("Sampled classes: ", sampled_classes)
            
            for i, c in enumerate(sampled_classes):
                # 'c' is an string directory to a lang folder (class)
                # Get all character folders within that class c
                all_chars = [f for f in folders if c in f]
                # For each class, sample K examples (characters)
                # All K examples from same class, have same label 'i'
                images = get_images(all_chars, [i]*k_samples, k_samples)
                # For each example, get image as array
                count = 0
                for k in range(k_samples):
                    all_image_batches[b, k, i, :] = image_file_to_array(filename=images[count][1], dim_input=pixels)
                    # Labels as one-hot vectors
                    all_label_batches[b, k, i, :] = np.zeros(n_classes)
                    all_label_batches[b, k, i, i] = 1
                    count += 1
        return all_image_batches, all_label_batches

Let me know what you think, or if something is not clear.

[JJ-res101]

Hi!

The fixed code is cool, which clearly samples different classes(language) . I have no doubt about it now. Thanks!

Besides, I still have 2 questions left.

As you described in issue #1 , you treat a language as a class, so every character in each language has the same label. So in the meta-test phase, the network has seen other characters of the same class, Is that right? For instance, in meta-traing phase, we get images form

'./omniglot_resized/Alphabet_of_the_Magi/character0' ,
 '
 '
 '
 './omniglot_resized/Alphabet_of_the_Magi/character10' 

in meta-test phase, we maybe get images form

'./omniglot_resized/Alphabet_of_the_Magi/character11' ,
 '
 '
 '
 './omniglot_resized/Alphabet_of_the_Magi/character20' 

The character must be different because the variable folders in the code is must different. But as I said earlier, different characters are treated as different examples of the same class. So I feel a little bit that this doesn't follow the spirit of meta-learning. Perhaps one solution is to separate the self.lang_folder into train and val and test as well.

Another question is the same as issue #1, that is, why the ominiglot is treated as 50 classes instead of 1623 classes. I didn't quite understand your answer to that issue, and if CS330 mentions doing that, I'd like you to tell me where it is. In fact, I tend to view ominiglot as 1623 classes.

Thanks a lot!

[LecJackS]

The fixed code is cool, which clearly samples different classes(language) . I have no doubt about it now. Thanks! You're welcome! :)

Besides, I still have 2 questions left.

As you described in issue #1 , you treat a language as a class, so every character in each language has the same label. So in the meta-test phase, the network has seen other characters of the same class, Is that right? For instance, in meta-traing phase, we get images form

'./omniglot_resized/Alphabet_of_the_Magi/character0' ,
 '
 '
 '
 './omniglot_resized/Alphabet_of_the_Magi/character10' 

in meta-test phase, we maybe get images form

'./omniglot_resized/Alphabet_of_the_Magi/character11' ,
 '
 '
 '
 './omniglot_resized/Alphabet_of_the_Magi/character20' 

You're absolutely right.

The character must be different because the variable folders in the code is must different. But as I said earlier, different characters are treated as different examples of the same class. So I feel a little bit that this doesn't follow the spirit of meta-learning. Perhaps one solution is to separate the self.lang_folder into train and val and test as well.

I'm not sure. I think the idea here is to learn a more general class (language) from a lower level classes (characters).

Maybe separating languages in test, train and val is another way to "measure" the performance of the algorithm.

But as the homework was presented, I understand this is not the case, and we only want to classify new unseen characters into languages, based on training done on other characters from the same languages.

Another question is the same as issue #1, that is, why the ominiglot is treated as 50 classes instead of 1623 classes. I didn't quite understand your answer to that issue, and if CS330 mentions doing that, I'd like you to tell me where it is. In fact, I tend to view ominiglot as 1623 classes.

Why it should be 1623 classes? I didn't understand issue #1 question either. If you have 1623 classes, you'll have one example of each. What can you learn from that? besides same digits written in different ways shouldn't be treated as different classes.

In Lecture 3, at 7:57, Chelsea Finn talks about the general formulation, using minilmaginet: https://youtu.be/v7otSgpTc0Q?t=477

There, from the original classes, you select N (like 5), K+1 examples for each one, train on K, test on the left one, to classify it to one of the K trained classes (for that particular meta dataset).

[JJ-res101]

Again, sincerely thanks for your reply！

we only want to classify new unseen characters into languages, based on training done on other characters from the same languages.

I think what you said is that it is reasonable to learn some characters in a language and then recognize other characters in the same language, because different characters in a unified language are actually different, so it reflects the spirit of meta-learning.

Why it should be 1623 classes? I didn't understand issue #1 question either. If you have 1623 classes, you'll have one example of each.

I think If we have 1623 classes, we still have 20 samples for each class, instead of one example of each.

Finally, I think I see the difference between us. You're extracting tasks in one language, and issue #1 and I don't think we should differentiate between languages, which is to throw away the information of the language and treat characters in different languages equally. Which do you think is more appropriate?

[rave78]

Hi,

I think there might be a bug in your code. The tuple images_labels represents (label, image). The label has not been used in line 133. Please note that line 120 returns a shuffled list of (label, image). I supposed the solution is only add all_label_batches[b, k, n, images_labels[count][0]] = 1. Additionally, you can already define np.zeros instead of np.array in line 113

[LecJackS]

Why it should be 1623 classes? I didn't understand issue #1 question either. If you have 1623 classes, you'll have one example of each.

I think If we have 1623 classes, we still have 20 samples for each class, instead of one example of each.

Finally, I think I see the difference between us. You're extracting tasks in one language, and issue #1 and I don't think we should differentiate between languages, which is to throw away the information of the language and treat characters in different languages equally. Which do you think is more appropriate?

I'm commenting on issue #1 and this one to say I were making a mistake considering the classes as languages, and not as one symbol for each class (with 20 examples each).

I found myself on this course again, and reading the SNAIL paper opened my eyes:

The Omniglot and mini-ImageNet datasets for few-shot image classification are the standard benchmarks in supervised meta-learning. Introduced by Lake et al. (2011), Omniglot consists of black and-white images of handwritten characters gathered from 50 languages, for a total of 1632 different classes with 20 instances per class.