Request preliminary data

[Fhujinwu]

Hello, I sent an email to you to get the preprocessed database, but maybe because you are too busy to check your mailbox or my email is judged as spam, I did not receive your preprocessed database. I hope you can upload the database to github in a form similar to Baidu cloud link or Google cloud link. Thanks！

[Rustastra]

Hi @Beckschen , thanks for this repository! Can't wait to run it.

I think it would be useful to have a sample script for preparing the dataset?

Could you please review it and confirm it looks correct? I haven't run it yet, so your input will be helpful.

Thanks!

Part 1. Download the data from Synapse website using their API, Colab notebook and Google Drive. (Takes some time to run).

from google.colab import drive
drive.mount('/content/drive', force_remount=True)

import synapseclient 
cwd = "/content/drive/My Drive/TransUNet"
syn = synapseclient.Synapse() 
syn.login(
    email="your email here",
    apiKey="your API KEY HERE==")
syn3380218 = syn.get(entity='syn3380218'  )
# https://www.synapse.org/#!Synapse:syn3380218

# filepath where Reg-Training-Testing.zip is saved can be checked via 
# filepath = syn3380218.path
# which in my case was /root/.synapseCache/164/2280164/Reg-Training-Testing.zip

!cp /root/.synapseCache/164/2280164/Reg-Training-Testing.zip '/content/drive/My Drive/TransUNet/Reg-Training-Testing.zip'

!unzip '/content/drive/My Drive/TransUNet/Reg-Training-Testing.zip' -d '/content/drive/My Drive/TransUNet'

2. Following the instructions from ReadMe ** correction based on datasets.py ** ** correction based on the comment from @andife **

import numpy as np
import nibabel as nib
import h5py
import os

cwd = "/content/drive/My Drive/TransUNet/Training-Testing"
data_folder = "/content/drive/My Drive/TransUNet/data"
subfolders = os.listdir(cwd+"/" + "img") 
# subfolders will be like ['0062', '0064', ...]

# I chose subfolder '0066', but maybe you will want to iterate & combine
for subfolder in ['0066']: #subfolders[1:]:
  print(subfolder) 
  tempwd = cwd +"/" + "img" + "/" + subfolder
  files = os.listdir(tempwd) # files will be like ['img0032-0066.nii.gz', 'img0036-0066.nii.gz', ...]

  # iterate over filenames
  for filename in files:
    print(filename)
    righttext = filename[3:] # get the part "xxxx-xxxx.nii.gz"
    subject = righttext[:4]
    img = nib.load(cwd +"/" + "img" + "/" + subfolder + "/" + 'img' + righttext)
    label = nib.load(cwd +"/" + "label" + "/" + subfolder + "/" + 'label' + righttext)

    #Convert them to numpy format, 
    data = img.get_fdata()
    label_data = label.get_fdata()

    #clip the images within [-125, 275], 
    data_clipped = np.clip(data, -125, 275)

    #normalize each 3D image to [0, 1], and 
    data_normalised = (data_clipped - (-125)) / (275 - (-125))

    #extract 2D slices from 3D volume for training cases while 
    # e.g. slice 000
    for i in range(data_normalised.shape[2]):
      formattedi = "{:03d}".format(i)
      slice000 = data_normalised[:,:,i]
      label_slice000 = label_data[:,:,i]
      np.savez(data_folder + "/Synapse/train_npz/case"+subject+"_slice"+formattedi+".npz", 
              image=slice000,
              label=label_slice000)

    #keep the 3D volume in h5 format for testing cases.
    fn = data_folder + '/Synapse/test_vol_h5/case'+subject+'.npy.h5'
    f = h5py.File(fn, 'w')
    dset = f.create_dataset("image", data=data_normalised)
    dset = f.create_dataset("label", data=label_data)
    f.close()

[andife]

Hello, thank you for the code

#keep the 3D volume in h5 format for testing cases. with h5py.File(data_folder + '/Synapse/test_vol_h5/case0062.npy.h5', 'w') as hf: hf.create_dataset("case0062", data=data_normalised)

I think the labels are missing. The file case0001.npy.h5 contains the keys 'image' and 'label'

[Rustastra]

Thanks @andife, I have updated the code as you suggested.

Do you by any chance know which dataset exactly was used? The issue with the one I included as an example is that third dimension (x,y,z) doesn't go beyond 171, but the training sample references e.g. case0040_slice191. Thanks!

[Beckschen]

Hello, I sent an email to you to get the preprocessed database, but maybe because you are too busy to check your mailbox or my email is judged as spam, I did not receive your preprocessed database. I hope you can upload the database to github in a form similar to Baidu cloud link or Google cloud link. Thanks！

@Fhujinwu Hello, sorry for the late reply. Did you receive my email reply? Usually I will response the email ASAP. Good comments! Please drop me a email to get the data. Both Baidu disk and GoogleDrive are fine.

[Beckschen]

@Rustastra Thanks so much for your contribution! I will run it first, and if everything is fine, we will put it on the README for the convenience of those people run code on colab.

[minhoooo1]

Hello, I sent an email to you to get the preprocessed database, but maybe because you are too busy to check your mailbox or my email is judged as spam, I did not receive your preprocessed database. I hope you can upload the database to github in a form similar to Baidu cloud link or Google cloud link. Thanks！

Hello, can I get the data that can run the code from you Thank you!!!

[minhoooo1]

你好，我给你发了一封电子邮件，以获得预处理数据库，但也许是因为你太忙了，检查您的邮箱或我的电子邮件被判断为垃圾邮件，我没有收到您的预处理数据库。我希望你能上传数据库到github的形式类似于百度云链接或谷歌云链接。谢谢！

@Fhujinwu你好，很抱歉这么晚才回复。您是否收到我的电子邮件回复？通常我会尽快回复电子邮件。 好评论！请给我发电子邮件以获取数据。百度磁盘和谷歌驱动器都很好。

Hello, Dr. beckschen! Cound you please send the preprocessed data set that can run the code to my mailbox? Thank you! [email protected]

[minhoooo1]

谢谢@andife，我已经更新了代码，你建议。

您是否知道到底使用了哪个数据集？以我为例，第三维度（x，y，z）的问题并不超过171，但培训样本参考，例如**191 case0040_slice。**谢谢！

Hello! Cound you please send the preprocessed data set that can run the code to my mailbox? Thank you! [email protected]

[liushuang1001]

Hello, I sent an email to you to get the preprocessed database, but maybe because you are too busy to check your mailbox or my email is judged as spam, I did not receive your preprocessed database. I hope you can upload the database to github in a form similar to Baidu cloud link or Google cloud link. Thanks！

@Fhujinwu Hello, sorry for the late reply. Did you receive my email reply? Usually I will response the email ASAP. Good comments! Please drop me a email to get the data. Both Baidu disk and GoogleDrive are fine.

Have you got the data set? Could you please send me a copy? Thank!!! [email protected] @Beckschen@andife

[KouroshGerayeli]

@Beckschen @andife Hello , I tried to get the preprocessed database as well but I did not receive them . I assume you are very busy , but could you please send me the database . My email: [email protected]

[gonyakjelly]

@Beckschen @andife Hello, I'm going to use preprocessed database for research purposes only. Cound you please send the preprocessed data set that can run the code to my mailbox? Thank you so much! my email: [email protected]

[Interesting6]

Has somebody gotten the preliminary data or preprocessed data? If yes, may I have a request for the original/preprocessed data? Thank you so much! my email: [email protected]

[Tensor-king]

Hello,I also need your Processed data ,thank you very much ! my email [email protected]

[jscodecode]

Has somebody gotten the preliminary data or preprocessed data? If yes, may I have a request for the original/preprocessed data? Thank you so much! my email: [email protected]

[minhoooo1]

sorry I didn't get it

------------------ 原始邮件 ------------------ 发件人: @.>; 发送时间: 2022年3月15日(星期二) 晚上10:38 收件人: @.>; 抄送: @.>; @.>; 主题: Re: [Beckschen/TransUNet] Request preliminary data (#24)

Has somebody gotten the preliminary data or preprocessed data? If yes, may I have a request for the original/preprocessed data? Thank you so much! my email: @.***

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you commented.Message ID: @.***>

[jscodecode]

Thank you for your reply. For a while, I won't need it. Thank someone for giving me some references. I've got some raw data, and I've done some processing. But it's different from the data the authors used. You can get back to me if you want.

------------------ 原始邮件 ------------------ 发件人: "Beckschen/TransUNet" @.>; 发送时间: 2022年3月18日(星期五) 下午3:05 @.>; 抄送: "ァDê_/~↘松| @.@.>; 主题: Re: [Beckschen/TransUNet] Request preliminary data (#24)

sorry I didn't get it

------------------ 原始邮件 ------------------ 发件人: @.>;
发送时间: 2022年3月15日(星期二) 晚上10:38 收件人: @.>;
抄送: @.>; @.>;
主题: Re: [Beckschen/TransUNet] Request preliminary data (#24)

Has somebody gotten the preliminary data or preprocessed data? If yes, may I have a request for the original/preprocessed data? Thank you so much! my email: @.***

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you commented.Message ID: @.> — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you commented.Message ID: @.>

[2947582615]

@Beckschen @andife Hello , I also need your Processed data ,thank you very much ! my email [[email protected]]

[liushuang1001]

Hello! TransUnet author replied that my dataset is：The following google drive contains the whole project including preprocessed data.
https://drive.google.com/drive/folders/1ACJEoTp-uqfFJ73qS3eUObQh52nGuzCd?usp=sharing But I can't open it, you can refer to it and try to open it. If it can be opened smoothly, can you compress and package and share with me the preprocessed dataset? It is enough to run through the data of the code. Later, I will modify my own data set according to the modified format for training.

Thanks!

At 2022-03-18 15:32:27, "jscodecode" @.***> wrote:

Thank you for your reply. For a while, I won't need it. Thank someone for giving me some references. I've got some raw data, and I've done some processing. But it's different from the data the authors used. You can get back to me if you want.

------------------ 原始邮件 ------------------ 发件人: "Beckschen/TransUNet" @.>; 发送时间: 2022年3月18日(星期五) 下午3:05 @.>; 抄送: "ァDê_/~↘松| @.@.>; 主题: Re: [Beckschen/TransUNet] Request preliminary data (#24)

sorry I didn't get it

------------------ 原始邮件 ------------------ 发件人: @.>; 发送时间: 2022年3月15日(星期二) 晚上10:38 收件人: @.>; 抄送: @.>; @.>; 主题: Re: [Beckschen/TransUNet] Request preliminary data (#24)

Has somebody gotten the preliminary data or preprocessed data? If yes, may I have a request for the original/preprocessed data? Thank you so much! my email: @.***

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you commented.Message ID: @.> — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you commented.Message ID: @.>

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you commented.Message ID: @.***>

[thestars-maker]

@Beckschen @andife Hello , I also need your Processed data ,thank you very much ! my email [[email protected]]

[GYJ18864525693]

Thanks @andife, I have updated the code as you suggested.谢谢，我已经按照您的建议更新了代码。

Do you by any chance know which dataset exactly was used? The issue with the one I included as an example is that third dimension (x,y,z) doesn't go beyond 171, but the training sample references e.g. case0040_slice191. Thanks!您是否知道到底使用了哪个数据集？我作为示例包含的问题是第三维 （x，y，z） 不会超过 171，但训练样本引用例如 case0040_slice191。谢谢！

Could you give me the code that is used to prepare the dataset?Thank you so much!my email:[email protected]