igneous icon indicating copy to clipboard operation
igneous copied to clipboard

How to start from labeled images (tiff file)

Open BoyuLyu opened this issue 3 years ago • 11 comments

I am having trouble getting started. I am trying to visualize a small labeled volume in neuroglancer. The volume is in tiff file now. How can I transform it into precomputed format?

I guess I need to create the info file first. Can I do that using igneous?

BoyuLyu avatar Mar 01 '22 16:03 BoyuLyu

Hi, you can use CloudVolume.create_new_info to make a new info file. You can then place it in the directory you'd like.

After that, you can follow this guide: https://github.com/seung-lab/cloud-volume/wiki/Example-Single-Machine-Dataset-Upload

I haven't figured out a uniform way to help people create new datasets since everyone organizes their initial files differently. It might be possible to create import/exports from other systems at some point, but raw off-the-microscope is probably always going to be challenging.

william-silversmith avatar Mar 01 '22 18:03 william-silversmith

Thank you so much for the prompt reply! I try using create_new_info and I can obtain the info file. But when I run downsample using igneous, it reported cloudvolume.exceptions.EmptyVolumeException error. I am wondering how should the segmentation files be stored? I understand that for a file with resolution (8nm, 8nm, 40nm) and with chunk size being (100, 100, 40), I should store the first chunk in the folder 8_8_40/0-100_0-100_0-40, but how should I store the chunk? Should I save it as z slices? And how should I name them?

BoyuLyu avatar Mar 01 '22 18:03 BoyuLyu

It's not necessary to store as z slices. Theoretically, CloudVolume should be doing all the work of figuring out the file names for you. Just make sure that you're uploading the right numpy array to the right (chunk aligned) location. You can do non-aligned uploads, but that requires careful planning or single core uploads to avoid conflicts.

Check the location with the EmptyVolumeException and see where the missing area lies. That should give you a clue. You can also try downsampling with fill_missing enabled to get an overview of the volume and see where chunks are missing. Then you can fix them and re-run it.

When fixing, see if you can disable caching on cloud storage when you upload otherwise you may have to wait an hour to see the results.

william-silversmith avatar Mar 01 '22 19:03 william-silversmith

That works! Now I can generate the precomputed files. And I am trying to visualize it using vol.viewer(). But it returns error at HTTP://localhost:1337. vol = CloudVolume('file://local/path/to/segmentation') vol.viewer()

Error response Error code: 404 Message: /: Not Found.

BoyuLyu avatar Mar 01 '22 22:03 BoyuLyu

Don't worry about the error. I should probably add an index.html page with instructions. You can enter the URL into Neuroglancer and it will start working.

Alternatively, just run igneous view /local/path/to/segmentation and it will open your browser to the right location.

william-silversmith avatar Mar 01 '22 22:03 william-silversmith

It works now. But I am wondering if there is a way I can input the path/to/local/segmentation into the Neuroglancer? I tried input URL like precomputed://file://local/path/to/segmentation, but it does work.

BoyuLyu avatar Mar 02 '22 14:03 BoyuLyu

Unfortunately that's not possible due to a security restriction in browsers to prevent them from accessing your files. A server needs to run in order to do that.

On Wed, Mar 2, 2022, 9:01 AM HHHIT @.***> wrote:

It works now. But I am wondering if there is a way I can input the path/to/local/segmentation into the Neuroglancer? I tried input URL like precomputed://file://local/path/to/segmentation, but it does work.

— Reply to this email directly, view it on GitHub https://github.com/seung-lab/igneous/issues/127#issuecomment-1056963331, or unsubscribe https://github.com/notifications/unsubscribe-auth/AATGQSPOROIDG5SUKUKEUVTU55YEPANCNFSM5PUSOFUA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

william-silversmith avatar Mar 02 '22 14:03 william-silversmith

Thanks! That clears my confusion.

BoyuLyu avatar Mar 02 '22 15:03 BoyuLyu

Under this restriction, can I visualize multiple layers? Currently, I can visualize the segmentation layer as well as the mesh. But if I want to visualize the raw data (like segmentation overlaid on the raw EM data), how could I achieve that?

BoyuLyu avatar Mar 02 '22 17:03 BoyuLyu

You can do it by either starting multiple servers with different port numbers or by starting a static file server one level up from multiple layers so they can be specified with file paths

On Wed, Mar 2, 2022, 12:22 PM HHHIT @.***> wrote:

Under this restriction, can I visualize multiple layers? Currently, I can visualize the segmentation layer as well as the mesh. But if I want to visualize the raw data (like segmentation overlaid on the raw data), how could I achieve that?

— Reply to this email directly, view it on GitHub https://github.com/seung-lab/igneous/issues/127#issuecomment-1057183127, or unsubscribe https://github.com/notifications/unsubscribe-auth/AATGQSJRLDHBM7K2D2WLPVDU56PUTANCNFSM5PUSOFUA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

william-silversmith avatar Mar 02 '22 17:03 william-silversmith

Thanks! I tried starting multiple server with different ports and it worked well.

BoyuLyu avatar Mar 02 '22 19:03 BoyuLyu