bhvRDM for representations in CNNs

[arjung128]

First of all, thank you so much for making this work open source! It is much appreciated! :)

I am interested in calculating RDMs for representations learned by neural networks. For this, I was looking at your bhvRDM function, since the description of the function includes 'This function can also be used to calculate the RDM for computational simulation data'.

My understanding was that for RDMs, correlation distance (i.e. 1 - r where r refers to the Pearson correlation) was used. However, on line 84 (here), I am not sure why you are simply computing the absolute distance. Can you please help me out?

Thanks once again for your phenomenal work!

[ZitongLu1996]

Very nice question! In some cases, the correlation coefficient is used to describe the similarity ignoring the value is positive or negative. So I use a np.abs() function here... In many functions in module rdm_cal, you can find a parameter abs. By this parameter, user can select to calculate the 1-r or 1-abs(r). I' sorry that I didn't add this choice when sub_opt=1 in function bhvRDM. Under this special condition, we cannot calculate the Pearson correlation and we can only calculate the distance. But sorry again, I forget to add a choice for users to select whether computing the absolute distance or not for this condition. I'll update NeuroRA soon! Thanks for your careful reading my codes!

[ZitongLu1996]

Hi, Arjun! I just updated NeuroRA! You can download the new version 1.1.3.4: pip install --upgrade neurora In this version, if you set abs=False, the dissimilarity would be 1-dis not 1-abs(dis).

If you want to put the nodes' values of each layer in CNN, I recommend that you can let the shape of the data_ly_i as [n_conditions, n_nodes, 1] for layer i in a CNN model as the input for bhvRDM function. like: RDM_ly_i = bhvRDM(data, sub_opt=0, abs=False) By this replacement (nodes as the subjects), you can obtain the RDM for each CNN layer with the dissimilarity in a RDM by calculating the 1 - correlation coefficient between vector1=[valueofnode1_condition1, ... , valueofnoden_condition1] and vector2=[valueofnode1_condition2, ... , valueofnoden_condition2]. And so on...

Hope this help! If you have any other questions, you can ask me.

[arjung128]

Hi Zitong!

Thank you very much for your prompt reply. I was very impressed by how quickly you responded to my question and updated the library -- Thanks a ton!

I'm sorry, I do not fully understand: bhvRDM still uses absolute distance and not the Pearson correlation for obtaining the RDM, correct? I am not sure how / whether I can use this function to get RDM values of CNN representations (3D) based on the Pearson correlation using this function... is this possible? Or are you suggesting I can write my own function to do this with the equation in your response?

Thanks once again! :)

[ZitongLu1996]

I think my suggestions above can work. RDM_ly_i = bhvRDM(data, sub_opt=0, abs=False) by this line-code, the calculation is base on Pearson correlation (you can see the codes line 113 - 142, especially line 135 in rdm_cal.py) and the dissimilarities in RDM are calculated by 1-r. I think I have answered your question...

[ZitongLu1996]

Maybe you are puzzled about how to put the data of a CNN into bhvRDM? For example, in a certain CNN, its layer i has 64 feature maps (size of each feature map is 128 by 128). You can spread this 64 by 128 by 128 array into a vector which shape will be [1048576]. So n_nodes=1048576 here, and as the suggestion I told yesterday, the shape of the data as the input of layer i in certain CNN for RDM calculation should be [n_conditions, n_nodes, 1]. Then use RDM_ly_i = bhvRDM(data, sub_opt=0, abs=False), you can get a RDM corresponding to layer i. Hope this more detailed explanation help!

[arjung128]

Oh I see now. Sorry, I had missed line 135. Thanks so much, Zitong!