AIX360
AIX360 copied to clipboard
Trusted-AI
CEM_MAFImageExplainer - broken Example Notebook
First, I want to thank you very much for providing this toolkit! I am eager to use your implementation for my own research!
Unfortunately, as I was working through the example "CEM-MAF-CelebA.ipynb" notebook for contrastive explanations, I was stopped dead while obtaining the pertinent negative explanation. (Code chunk 12)
Error message:
InvalidArgumentError: Conv2DCustomBackpropInputOp only supports NHWC.
[[{{node gradients/G_paper_1_1/cond/ToRGB_lod8/Conv2D_grad/Conv2DBackpropInput}}]]
During handling of the above exception, another exception occurred:
InvalidArgumentError Traceback (most recent call last)
<ipython-input-13-b1a3ab914e94> in <module>
3 arg_max_iterations, arg_initial_const, arg_gamma, None,
4 arg_attr_reg, arg_attr_penalty_reg,
----> 5 arg_latent_square_loss_reg)
6
7 print(info_pn)
c:\workspaces\aix360\aix360\algorithms\contrastive\CEM_MAF.py in explain_instance(self, sess, input_img, input_latent, arg_mode, arg_kappa, arg_binary_search_steps, arg_max_iterations, arg_initial_const, arg_gamma, arg_beta, arg_attr_reg, arg_attr_penalty_reg, arg_latent_square_loss_reg)
95 attr_penalty_reg=arg_attr_penalty_reg, latent_square_loss_reg=arg_latent_square_loss_reg)
96
---> 97 adv_img = attack_pn.attack(input_img, target_label, input_latent)
98 adv_prob, adv_class, adv_prob_str = self._wbmodel.predict_long(adv_img)
99 attr_mod = self.check_attributes_celebA(self._attributes, input_img, adv_img)
c:\workspaces\aix360\aix360\algorithms\contrastive\CEM_MAF_aen_PN.py in attack(self, imgs, labs, latent)
268 # perform the attack
269
--> 270 self.sess.run([self.train])
271 temp_adv_latent = self.sess.run(self.adv_latent)
272 self.sess.run(self.adv_updater, feed_dict={self.assign_adv_latent: temp_adv_latent})
...
InvalidArgumentError: Conv2DCustomBackpropInputOp only supports NHWC.
[[node gradients/G_paper_1_1/cond/ToRGB_lod8/Conv2D_grad/Conv2DBackpropInput (defined at c:\workspaces\aix360\aix360\algorithms\contrastive\CEM_MAF_aen_PN.py:197) ]]
Errors may have originated from an input operation.
Input Source operations connected to node gradients/G_paper_1_1/cond/ToRGB_lod8/Conv2D_grad/Conv2DBackpropInput:
G_paper_1_1/cond/ToRGB_lod8/mul (defined at <string>:27)
My setup:
I tried this example twice. Once on a windows machine (CPU only) and on a linux machine (CPU only). Both systems error out at the same step. The installation of aix360 worked both times according to the setup instructions in the git documentation.
My hypothesis:
I am thinking that the pickled CelebA model (karras2018iclr-celebahq-1024x1024.pkl) is the cause of this error.
Maybe the problem lies with the requirements. AIX360 needs tensorflow=1.14.0 whereas progressive_growing_of_gans requires tensorflow-gpu>=1.6.0.
I would really appreciate it, if you could help me out on this, as I want to know, if it's a model problem, which I can fix with my own models in the future, or if it's something more complicated than that.
Thank you very much in advance!
Hi. Glad to help you figure this out. I believe there was an issue with tensorflow 1.14.0 (I encountered this long ago). I recommend trying tensorflow 1.13.1. And further, I've never run this successfully on CPU - I imagine you will get memory issues. I've only successfully run this notebook using gpu, so if you have access to gpu, please install tensorflow with tensorflow-gpu=1.13.1.
Thank you very much for your fast reply. At the moment, I don't have access to a gpu, but when I do, I will report back.
With tensorflow==1.13.1 (without gpu) the example throws the same error as before.
Yes, please try on GPU when you can. I can confirm that I get the same error when running on CPU (more specifically using a tensorflow-gpu installation but forcing to only use CPU using the command os.environ['CUDA_VISIBLE_DEVICES'] = '-1'). However, when using the same installation and using the GPU, it runs fine. As you suggested, this is likely to do requirements of the GAN model in progressive_growing_of_gans that is used in this example to generate a PN.
Hello, would you mind showing me why the CEM_MAF of the models is empty? I cannot do the step "Download pretrained celebA model" and "Load the downloaded celebA model" (Code chunk 3 and 4 )
Hi. Have you resolved your issue? The models folder is empty because the models are too large to include in the AIX360 repository. Rather the block
Download pretrained celebA model
local_path_models = '../../aix360/models/CEM_MAF' celebA_model_file = dwnld.dwnld_celebA_model(local_path_models)
will download the model from a fixed link and put it in local_path_models. The link for downloading the model as well as all the attribute models that are needed can be found in the file aix360/algorithms/contrastive/dwnld_CEM_MAF_celebA.py. All relevant urls start with http://aix360.mybluemix.net and this might have been down (going through some support/updates) when you tried. Please try again as I have just run it and downloaded all models without issue. If you still have issues, please provide also the relevant errors you are seeing.
Hello, I have downloaded CelebA model, but I don't know the format of the CelebA, I cannot open it and cannot load it. Besides, can I run the code just by cpu?
发件人: rluss @.> 发送时间: 2021年9月14日 3:22 收件人: Trusted-AI/AIX360 @.> 抄送: Zhang, Jianqiao @.>; Comment @.> 主题: Re: [Trusted-AI/AIX360] CEM_MAFImageExplainer - broken Example Notebook (#117)
CAUTION: This email originated from outside of the organisation. Do not click links or open attachments unless you recognise the sender and know the content is safe.
Hi. Have you resolved your issue? The models folder is empty because the models are too large to include in the AIX360 repository. Rather the block
Download pretrained celebA model
local_path_models = '../../aix360/models/CEM_MAF' celebA_model_file = dwnld.dwnld_celebA_model(local_path_models)
will download the model from a fixed link and put it in local_path_models. The link for downloading the model as well as all the attribute models that are needed can be found in the file aix360/algorithms/contrastive/dwnld_CEM_MAF_celebA.py. All relevant urls start with http://aix360.mybluemix.nethttps://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Faix360.mybluemix.net%2F&data=04%7C01%7Cjz482%40exeter.ac.uk%7C5b7aaf5ed8af4c011e1e08d976ebd287%7C912a5d77fb984eeeaf321334d8f04a53%7C0%7C0%7C637671577480734647%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=TkHJsCfrZNvVva7Pm0F1K8dh0rBJvyK%2FtBuEll%2FpCDk%3D&reserved=0 and this might have been down (going through some support/updates) when you tried. Please try again as I have just run it and downloaded all models without issue. If you still have issues, please provide also the relevant errors you are seeing.
― You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FTrusted-AI%2FAIX360%2Fissues%2F117%23issuecomment-918503790&data=04%7C01%7Cjz482%40exeter.ac.uk%7C5b7aaf5ed8af4c011e1e08d976ebd287%7C912a5d77fb984eeeaf321334d8f04a53%7C0%7C0%7C637671577480744639%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=FcH%2BjwjbPgrZaBdwJJsXWii99Y8EURSeB%2B2H62oxOUU%3D&reserved=0, or unsubscribehttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAUDGNYD5IIAN3HJL4HTVBBLUBZFNPANCNFSM4Y6G3TGA&data=04%7C01%7Cjz482%40exeter.ac.uk%7C5b7aaf5ed8af4c011e1e08d976ebd287%7C912a5d77fb984eeeaf321334d8f04a53%7C0%7C0%7C637671577480744639%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=hisnqZLMI31eciHsJT4qZz3fX2fKySW0YiGmvtsZSrU%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Cjz482%40exeter.ac.uk%7C5b7aaf5ed8af4c011e1e08d976ebd287%7C912a5d77fb984eeeaf321334d8f04a53%7C0%7C0%7C637671577480754637%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jqApluwIewBSayFci7uFu4FTBB5KW1e3YMUuGO9L7Bg%3D&reserved=0 or Androidhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Cjz482%40exeter.ac.uk%7C5b7aaf5ed8af4c011e1e08d976ebd287%7C912a5d77fb984eeeaf321334d8f04a53%7C0%7C0%7C637671577480754637%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=pz5XvSiCb0K1LauU13TAyiv8nVEnTM5ElkoORCAZoww%3D&reserved=0.
The errors I am seeing as below
发件人: rluss @.> 发送时间: 2021年9月14日 3:22 收件人: Trusted-AI/AIX360 @.> 抄送: Zhang, Jianqiao @.>; Comment @.> 主题: Re: [Trusted-AI/AIX360] CEM_MAFImageExplainer - broken Example Notebook (#117)
CAUTION: This email originated from outside of the organisation. Do not click links or open attachments unless you recognise the sender and know the content is safe.
Hi. Have you resolved your issue? The models folder is empty because the models are too large to include in the AIX360 repository. Rather the block
Download pretrained celebA model
local_path_models = '../../aix360/models/CEM_MAF' celebA_model_file = dwnld.dwnld_celebA_model(local_path_models)
will download the model from a fixed link and put it in local_path_models. The link for downloading the model as well as all the attribute models that are needed can be found in the file aix360/algorithms/contrastive/dwnld_CEM_MAF_celebA.py. All relevant urls start with http://aix360.mybluemix.nethttps://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Faix360.mybluemix.net%2F&data=04%7C01%7Cjz482%40exeter.ac.uk%7C5b7aaf5ed8af4c011e1e08d976ebd287%7C912a5d77fb984eeeaf321334d8f04a53%7C0%7C0%7C637671577480734647%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=TkHJsCfrZNvVva7Pm0F1K8dh0rBJvyK%2FtBuEll%2FpCDk%3D&reserved=0 and this might have been down (going through some support/updates) when you tried. Please try again as I have just run it and downloaded all models without issue. If you still have issues, please provide also the relevant errors you are seeing.
― You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FTrusted-AI%2FAIX360%2Fissues%2F117%23issuecomment-918503790&data=04%7C01%7Cjz482%40exeter.ac.uk%7C5b7aaf5ed8af4c011e1e08d976ebd287%7C912a5d77fb984eeeaf321334d8f04a53%7C0%7C0%7C637671577480744639%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=FcH%2BjwjbPgrZaBdwJJsXWii99Y8EURSeB%2B2H62oxOUU%3D&reserved=0, or unsubscribehttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAUDGNYD5IIAN3HJL4HTVBBLUBZFNPANCNFSM4Y6G3TGA&data=04%7C01%7Cjz482%40exeter.ac.uk%7C5b7aaf5ed8af4c011e1e08d976ebd287%7C912a5d77fb984eeeaf321334d8f04a53%7C0%7C0%7C637671577480744639%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=hisnqZLMI31eciHsJT4qZz3fX2fKySW0YiGmvtsZSrU%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Cjz482%40exeter.ac.uk%7C5b7aaf5ed8af4c011e1e08d976ebd287%7C912a5d77fb984eeeaf321334d8f04a53%7C0%7C0%7C637671577480754637%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jqApluwIewBSayFci7uFu4FTBB5KW1e3YMUuGO9L7Bg%3D&reserved=0 or Androidhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Cjz482%40exeter.ac.uk%7C5b7aaf5ed8af4c011e1e08d976ebd287%7C912a5d77fb984eeeaf321334d8f04a53%7C0%7C0%7C637671577480754637%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=pz5XvSiCb0K1LauU13TAyiv8nVEnTM5ElkoORCAZoww%3D&reserved=0.
I'm sorry, but I do not see the errors. Could you please try to post them again?
Regarding CelebA, you do not need to know the format. In the notebook, blocks 7 and 8 show how to load and process the data and then to predict given you've loaded the model. R
Regarding CPU, it will not work and GPU will be needed for this tool. Also, make sure you read the lines after block 3 of the notebook beginning with "The following steps must be taken to get open-source code: ". You must download the generator and place it in the proper folder as instructed in the notebook.
Hello, I met a problem about download the sample image when I run the code above in block 6:
img_id = 15
local_path_img = '../../aix360/data/celeba_data'
img_files = dwnld.dwnld_celebA_data(local_path_img, [img_id])
I found the URL in function dwnld_celebA_data is http://aix360.mybluemix.net/static/CEM-MAF/data/
but the URL is can't access due to 404 not found ?
def dwnld_celebA_data(self, local_path, ids):
'''
Download celebA data files
Args:
local path (str): local path to where files are downloaded
ids (int list): list of ids to download data for
Returns:
files (str list): list of files that were downloaded
'''
# This is the link where celebA image data is stored
cdcweb = 'http://aix360.mybluemix.net/static/CEM-MAF/data/'
# Next build list of files to download
cdcfiles = []
for id in ids:
cdcfiles.append(str(id)+'_img.npy')
cdcfiles.append(str(id)+'_latent.npy')
cdcfiles.append(str(id)+'img.png')
files = []
or maybe I can try other images I found myself, but I don't know whether the two .npy file would be useful?
Wish you for reply. Thanks.
I solve the problem by access the URL manually, I can download now by https://aix360.mybluemix.net/static/CEM-MAF/data/15_img.png
