IDR
idr0026-weigelin-immunotherapy S-BIAD860
Trying to estimate how much space is needed for this conversion.
First image is uint16 (2 bytes), 507 x 507 x 21 x 71 x 4 = approx 3 GB.
Images vary in size for the study, but about 111 .pattern images (see https://github.com/IDR/idr-utils/pull/56) need converting.
Maybe 300 GB or more needed (maybe up to 500 GB)?
Looks like all the pattern files we need to convert are under:
$ ls /uod/idr/metadata/idr0026-weigelin-immunotherapy/patterns/ | wc
111 111 3970
Corresponds to image count from https://github.com/IDR/idr-utils/pull/56
$ screen -S idr0026_bf2raw
$ conda activate bioformats2raw
$ cd /data
$ sudo chown wmoore ./idr0026
$ cd idr0026
$ for i in `ls /uod/idr/metadata/idr0026-weigelin-immunotherapy/patterns/`; do echo $i; /home/wmoore/bioformats2raw-0.6.0-24/bin/bioformats2raw --memo-directory ../memo /uod/idr/metadata/idr0026-weigelin-immunotherapy/patterns/$i ${i%.*}.ome.zarr; done
$ aws --endpoint-url https://uk1s3.embassy.ebi.ac.uk s3 mb s3://idr0026
make_bucket: idr0026
$ aws --endpoint-url https://uk1s3.embassy.ebi.ac.uk s3api put-bucket-policy --bucket idr0026 --policy file://policy.json
$ aws --endpoint-url https://uk1s3.embassy.ebi.ac.uk s3api put-bucket-cors --bucket idr0026 --cors-configuration file://cors.json
$ /home/wmoore/mc cp -r idr0026/ uk1s3/idr0026/zarr
...3.140926_14-52-18.03.ome.zarr/OME/METADATA.ome.xml: 282.79 GiB / 282.79 GiB ━━━━━━━━━━━━━
Checking on s3...
E.g. https://ome.github.io/ome-ngff-validator/?source=https://uk1s3.embassy.ebi.ac.uk/idr0026/zarr/3.50.6-3.140922_11-36-07.00.ome.zarr/0/
This image has only a single omero:channels, so the images appear as single channel in vizarr, even though the zarr array data is 4-channels and they look OK in validator.
cc @sbesson
However, the OME.xml looks OK, e.g. https://uk1s3.embassy.ebi.ac.uk/idr0026/zarr/3.50.6-3.140922_11-36-07.00.ome.zarr/OME/METADATA.ome.xml
This has 4 channels, so maybe the .zattrs omero.channel info is not so critical when we import to OMERO. But it's still wrong!
<Pixels BigEndian="true" DimensionOrder="XYZCT" ID="Pixels:0" Interleaved="false" SignificantBits="16" SizeC="4" SizeT="71" SizeX="507" SizeY="507" SizeZ="21" Type="uint16">
<Channel ID="Channel:0:0" Name="FD6_GREEN" SamplesPerPixel="1">
<LightPath/>
</Channel>
<Channel ID="Channel:0:1" Name="FD5_BLUE" SamplesPerPixel="1">
<LightPath/>
</Channel>
<Channel ID="Channel:0:2" Name="BD8_RED" SamplesPerPixel="1">
<LightPath/>
</Channel>
<Channel ID="Channel:0:3" Name="BD7_RED" SamplesPerPixel="1">
<LightPath/>
</Channel>
On pilot-idr0125...
sudo mkdir /idr0026 && sudo /opt/goofys --endpoint https://uk1s3.embassy.ebi.ac.uk/ -o allow_other idr0026 /idr0026
# copy metadata-only images....
screen -S idr0010_aws_sync
aws s3 sync --no-sign-request --exclude '*' --include "*/.z*" --include "*.xml" --endpoint-url https://uk1s3.embassy.ebi.ac.uk s3://idr0026/zarr .
# import all images into Dataset
for dir in *; do
omero import -d 15352 --transfer=ln_s --depth=100 --name=${dir/.ome.zarr/} --skip=all $dir --file /tmp/$dir.log --errs /tmp/$dir.err;
done
$ python idr-utils/scripts/managed_repo_symlinks.py Dataset:15352 /idr0026/zarr
These look good in OMERO, compared to existing IDR
@will-moore not 100% sure of what went wrong on your conversion but using the converter library shipping with the current IDR version of Bio-Formats, I get
$ /opt/bioformats2raw/bioformats2raw-0.6.0-24/bin/bioformats2raw -p 3.49.6-3.140922_11-33-57.00.pattern 3.49.6-3.140922_11-33-57.00.zarr
OpenJDK 64-Bit Server VM warning: You have loaded library /tmp/opencv_openpnp6732331320596727672/nu/pattern/opencv/linux/x86_64/libopencv_java342.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
..>
[0/0] 99% │███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▉│ 3239/3240 (0:02:38 / 0:00:00)
[0/0] 100% │████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████│ 3240/3240 (0:02:38 / 0:00:00)
[0/1] 100% │████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████│ 3240/3240 (0:00:04 / 0:00:00)
and the omero key contains metadata about all four channels as expected
$ cat 3.49.6-3.140922_11-33-57.00.zarr/0/.zattrs
{
"multiscales" : [ {
"metadata" : {
"method" : "loci.common.image.SimpleImageScaler",
"version" : "Bio-Formats 0.6.10"
},
"axes" : [ {
"name" : "t",
"type" : "time"
}, {
"name" : "c",
"type" : "channel"
}, {
"name" : "z",
"type" : "space"
}, {
"name" : "y",
"type" : "space"
}, {
"name" : "x",
"type" : "space"
} ],
"name" : "11-33-57_PMT - PMT [BD2_GREEN] [00]_Time Time0000.tif",
"datasets" : [ {
"path" : "0",
"coordinateTransformations" : [ {
"scale" : [ 1.0, 1.0, 1.0, 1.0, 1.0 ],
"type" : "scale"
} ]
}, {
"path" : "1",
"coordinateTransformations" : [ {
"scale" : [ 1.0, 1.0, 1.0, 2.0, 2.0 ],
"type" : "scale"
} ]
} ],
"version" : "0.4"
} ],
"omero" : {
"channels" : [ {
"color" : "FF0000",
"coefficient" : 1,
"active" : true,
"label" : "BD2_GREEN",
"window" : {
"min" : 372.0,
"max" : 15788.0,
"start" : 372.0,
"end" : 15788.0
},
"family" : "linear",
"inverted" : false
}, {
"color" : "00FF00",
"coefficient" : 1,
"active" : true,
"label" : "BD8_DEEPR",
"window" : {
"min" : 373.0,
"max" : 16188.0,
"start" : 373.0,
"end" : 16188.0
},
"family" : "linear",
"inverted" : false
}, {
"color" : "0000FF",
"coefficient" : 1,
"active" : true,
"label" : "BD7_RED",
"window" : {
"min" : 759.0,
"max" : 8978.0,
"start" : 759.0,
"end" : 8978.0
},
"family" : "linear",
"inverted" : false
}, {
"color" : "FF0000",
"coefficient" : 1,
"active" : false,
"label" : "FD6_FDRED",
"window" : {
"min" : 237.0,
"max" : 12339.0,
"start" : 237.0,
"end" : 12339.0
},
"family" : "linear",
"inverted" : false
} ],
"rdefs" : {
"defaultT" : 0,
"model" : "color",
"defaultZ" : 13
}
}
}
Sounds like the best way forward would be to redo the whole conversion?
@sbesson Looking at https://github.com/IDR/idr-metadata/issues/648#issuecomment-1598644029, it looks like I used the same version: bioformats2raw-0.6.0-24.
But I'll delete and try again...
Testing... (and failing!)...
(bioformats2raw) [wmoore@pilot-zarr1-dev test]$ pwd
/data/idr0026/test
$ ~/bioformats2raw-0.6.0-24/bin/bioformats2raw /uod/idr/metadata/idr0026-weigelin-immunotherapy/patterns/3.65.9-6.141023_15-45-09.03.pattern 3.65.9-6.141023_15-45-09.03.ome.zarr
cat 3.65.9-6.141023_15-45-09.03.ome.zarr/0/.zattrs
{
"multiscales" : [ {
"metadata" : {
"method" : "loci.common.image.SimpleImageScaler",
"version" : "Bio-Formats 0.6.10"
},
"axes" : [ {
"name" : "t",
"type" : "time"
}, {
"name" : "c",
"type" : "channel"
}, {
"name" : "z",
"type" : "space"
}, {
"name" : "y",
"type" : "space"
}, {
"name" : "x",
"type" : "space"
} ],
"name" : "15-45-09_PMT - PMT [BD7_RED] [03]_Time Time0074.tif",
"datasets" : [ {
"path" : "0",
"coordinateTransformations" : [ {
"scale" : [ 1.0, 1.0, 1.0, 1.0, 1.0 ],
"type" : "scale"
} ]
}, {
"path" : "1",
"coordinateTransformations" : [ {
"scale" : [ 1.0, 1.0, 1.0, 2.0, 2.0 ],
"type" : "scale"
} ]
} ],
"version" : "0.4"
} ],
"omero" : {
"channels" : [ {
"color" : "808080",
"coefficient" : 1,
"active" : true,
"label" : "Channel 0",
"window" : {
"min" : 332.0,
"max" : 15788.0,
"start" : 332.0,
"end" : 15788.0
},
"family" : "linear",
"inverted" : false
} ],
"rdefs" : {
"defaultT" : 0,
"model" : "greyscale",
"defaultZ" : 0
}
}
}
Even using the same lib as @sbesson gives me same result?!
(bioformats2raw) [wmoore@pilot-zarr1-dev test]$ /opt/bioformats2raw/bioformats2raw-0.6.0-24/bin/bioformats2raw /uod/idr/metadata/idr0026-weigelin-immunotherapy/patterns/3.49.6-3.140922_11-33-57.01.pattern 3.49.6-3.140922_11-33-57.01.ome.zarr
$ cat 3.49.6-3.140922_11-33-57.01.ome.zarr/0/.zattrs
{
"multiscales" : [ {
"metadata" : {
"method" : "loci.common.image.SimpleImageScaler",
"version" : "Bio-Formats 0.6.10"
},
"axes" : [ {
"name" : "t",
"type" : "time"
}, {
"name" : "c",
"type" : "channel"
}, {
"name" : "z",
"type" : "space"
}, {
"name" : "y",
"type" : "space"
}, {
"name" : "x",
"type" : "space"
} ],
"name" : "11-33-57_PMT - PMT [FD6_FDRED] [01]_Time Time0028.tif",
"datasets" : [ {
"path" : "0",
"coordinateTransformations" : [ {
"scale" : [ 1.0, 1.0, 1.0, 1.0, 1.0 ],
"type" : "scale"
} ]
}, {
"path" : "1",
"coordinateTransformations" : [ {
"scale" : [ 1.0, 1.0, 1.0, 2.0, 2.0 ],
"type" : "scale"
} ]
} ],
"version" : "0.4"
} ],
"omero" : {
"channels" : [ {
"color" : "808080",
"coefficient" : 1,
"active" : true,
"label" : "Channel 0",
"window" : {
"min" : 376.0,
"max" : 15788.0,
"start" : 376.0,
"end" : 15788.0
},
"family" : "linear",
"inverted" : false
} ],
"rdefs" : {
"defaultT" : 0,
"model" : "greyscale",
"defaultZ" : 0
}
}
I suspect there's something wrong with your environment and particularly the bioformats2raw Conda environment that you're using. Can you try deactivating Conda completely and simply running /opt/bioformats2raw/bioformats2raw-0.6.0-24/bin/bioformats2raw /uod/idr/metadata/idr0026-weigelin-immunotherapy/patterns/3.49.6-3.140922_11-33-57.01.pattern 3.49.6-3.140922_11-33-57.01.ome.zarr ?
That didn't work either!
I wanted to try on a different machine completely...
$ ssh pilot-zarr2-dev
$ cd /data
$ sudo mkdir idr0026
$ sudo chown wmoore idr0026
$ /opt/bioformats2raw/bioformats2raw-0.6.0-24/bin/bioformats2raw -p /uod/idr/metadata/idr0026-weigelin-immunotherapy/patterns/3.49.6-3.140922_11-33-57.01.pattern 3.49.6-3.140922_11-33-57.01.ome.zarr
OpenJDK 64-Bit Server VM warning: You have loaded library /tmp/opencv_openpnp632426155291925201/nu/pattern/opencv/linux/x86_64/libopencv_java342.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
Exception in thread "main" picocli.CommandLine$ExecutionException: Error while calling command (com.glencoesoftware.bioformats2raw.Converter@6ce139a4): java.io.FileNotFoundException: /uod/idr/metadata/idr0026-weigelin-immunotherapy/patterns/3.49.6-3.140922_11-33-57.01.pattern (No such file or directory)
$ cd /uod/idr/metadata/
$ ls
idr0010-doil-dnadamage idr0054-segura-tonsilhyperion
Do I need to clone all of https://github.com/IDR/idr-metadata here?
Just confirming...
(base) [wmoore@pilot-zarr1-dev idr0026]$ mkdir test3
(base) [wmoore@pilot-zarr1-dev idr0026]$ cd test3
(base) [wmoore@pilot-zarr1-dev test3]$ /opt/bioformats2raw/bioformats2raw-0.6.0-24/bin/bioformats2raw /uod/idr/metadata/idr0026-weigelin-immunotherapy/patterns/3.49.6-3.140922_11-33-57.01.pattern 3.49.6-3.140922_11-33-57.01.ome.zarr
OpenJDK 64-Bit Server VM warning: You have loaded library /tmp/opencv_openpnp7313185331649622917/nu/pattern/opencv/linux/x86_64/libopencv_java342.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
2023-06-29 10:14:29,998 [main] WARN loci.formats.in.BaseTiffReader - unknown creation date format: 2014-09-22 11:34:50
$ cat 3.49.6-3.140922_11-33-57.01.ome.zarr/0/.zattrs
{
"multiscales" : [ {
"metadata" : {
"method" : "loci.common.image.SimpleImageScaler",
"version" : "Bio-Formats 0.6.10"
},
"axes" : [ {
"name" : "t",
"type" : "time"
}, {
"name" : "c",
"type" : "channel"
}, {
"name" : "z",
"type" : "space"
}, {
"name" : "y",
"type" : "space"
}, {
"name" : "x",
"type" : "space"
} ],
"name" : "11-33-57_PMT - PMT [FD6_FDRED] [01]_Time Time0028.tif",
"datasets" : [ {
"path" : "0",
"coordinateTransformations" : [ {
"scale" : [ 1.0, 1.0, 1.0, 1.0, 1.0 ],
"type" : "scale"
} ]
}, {
"path" : "1",
"coordinateTransformations" : [ {
"scale" : [ 1.0, 1.0, 1.0, 2.0, 2.0 ],
"type" : "scale"
} ]
} ],
"version" : "0.4"
} ],
"omero" : {
"channels" : [ {
"color" : "808080",
"coefficient" : 1,
"active" : true,
"label" : "Channel 0",
"window" : {
"min" : 376.0,
"max" : 15788.0,
"start" : 376.0,
"end" : 15788.0
},
"family" : "linear",
"inverted" : false
} ],
"rdefs" : {
"defaultT" : 0,
"model" : "greyscale",
"defaultZ" : 0
}
}
}
Do I need to clone all of https://github.com/IDR/idr-metadata here?
For the sake of testing, you might just want to copy the single .pattern file you want to test directly. Otherwise, yes need t clone the whole repository unless @francesw wants to look into extracting idr0026 into a standalone Git repository
Just confirming...
The inconsistency in the output is very concerning. Have you tried after fully deactivating Conda, not just your environment?
Have you tried after fully deactivating Conda, not just your environment?
No. How do you do that?
I already did that. How's that different from deactivating your environment?
I tried on a different machine...
Cloned idr-metadata and moved it to /uod/idr/metadata
cd /uod/idr/metadata/
sudo -Es git clone [email protected]:IDR/idr-metadata.git
cd ../
sudo mv metadata/idr-metadata ./
sudo rm metadata # symlink to /data/idr-metadata
sudo mv idr-metadata metadata
Then tried...
cd /data/idr0026/
wmoore@pilot-zarr2-dev idr0026]$ /opt/bioformats2raw/bioformats2raw-0.6.0-24/bin/bioformats2raw -p /uod/idr/metadata/idr0026-weigelin-immunotherapy/patterns/3.49.6-3.140922_11-33-57.01.pattern 3.49.6-3.140922_11-33-57.01.ome.zarr
$ cat 3.49.6-3.140922_11-33-57.01.ome.zarr/0/.zattrs
{
"multiscales" : [ {
"metadata" : {
"method" : "loci.common.image.SimpleImageScaler",
"version" : "Bio-Formats 0.6.10"
},
"axes" : [ {
"name" : "t",
"type" : "time"
}, {
"name" : "c",
"type" : "channel"
}, {
"name" : "z",
"type" : "space"
}, {
"name" : "y",
"type" : "space"
}, {
"name" : "x",
"type" : "space"
} ],
"name" : "11-33-57_PMT - PMT [FD6_FDRED] [01]_Time Time0028.tif",
"datasets" : [ {
"path" : "0",
"coordinateTransformations" : [ {
"scale" : [ 1.0, 1.0, 1.0, 1.0, 1.0 ],
"type" : "scale"
} ]
}, {
"path" : "1",
"coordinateTransformations" : [ {
"scale" : [ 1.0, 1.0, 1.0, 2.0, 2.0 ],
"type" : "scale"
} ]
} ],
"version" : "0.4"
} ],
"omero" : {
"channels" : [ {
"color" : "808080",
"coefficient" : 1,
"active" : true,
"label" : "Channel 0",
"window" : {
"min" : 376.0,
"max" : 15788.0,
"start" : 376.0,
"end" : 15788.0
},
"family" : "linear",
"inverted" : false
} ],
"rdefs" : {
"defaultT" : 0,
"model" : "greyscale",
"defaultZ" : 0
}
}
}
WAT!?
@will-moore I think I found the source of the issue. Can you try one more test with your last configuration, running sudo /opt/bioformats2raw/bioformats2raw-0.6.0-24/bin/bioformats2raw -p /uod/idr/metadata/idr0026-weigelin-immunotherapy/patterns/3.49.6-3.140922_11-33-57.01.pattern 3.49.6-3.140922_11-33-57.01.ome.zarr (note the sudo at the beginning of the command)?
So the difference is really whether the directory containing the pattern file is writeable or not, meaning that a memo file can be saved or not. I suspect something incorrect is happening in the case where a memo file cannot be written and the wrong reader is used. It might be specific to either FilePatternReader or possibly the IDR customizations.
Either way for the scope of this testing, I think the solution is to re-run the conversion using the same user that owns the patterns directory
OK I've chowned that dir to me...
(base) [wmoore@pilot-zarr1-dev ~]$ cd /uod/idr/metadata/idr0026-weigelin-immunotherapy/
(base) [wmoore@pilot-zarr1-dev idr0026-weigelin-immunotherapy]$ sudo chown wmoore patterns/
I see that in the original conversion I used --memo-directory ../memo which is an alternative solution, right?
Except that probably also specified an existing dir that wasn't mine.
However, the chown above seems to work, so now running full conversion in a screen
cd /data/idr0026/idr0026
for i in `ls /uod/idr/metadata/idr0026-weigelin-immunotherapy/patterns/`; do echo $i; /home/wmoore/bioformats2raw-0.6.0-24/bin/bioformats2raw /uod/idr/metadata/idr0026-weigelin-immunotherapy/patterns/$i ${i%.*}.ome.zarr; done
Meanwhile, deleted invalid data previously uploaded...
$ ./mc rm --force --recursive uk1s3/idr0026/zarr
The above removal of contents of the bucket ran very slowly and has only resulted in the removal of a handful of zarr filesets out of the 111 originally there.
Since we want to delete ALL the filesets uploaded, probably quicker to delete the bucket and recreate..
ran
./mc rb --force uk1s3/idr0026
This seemed to hang/time-out and doesn't seem to have had any affect:
$ ./mc ls uk1s3/idr0026/zarr | wc
95 475 6719
Reverted to running the rm again in a screen
./mc rm --force --recursive uk1s3/idr0026/zarr
@sbesson - Seems that the memo issue is something it would be good to fix (or at least warn) to prevent others suffering the pain above! I can create an issue somewhere, but where?
From my side, the immediate candidates are:
- an issue with https://github.com/IDR/bioformats custom additions to FilePatternReader to support channel names
- an issue with https://github.com/IDR/bioformats being behind the ome/bioformats mainline (i.e. a bug in FilePatternReader that would have been fixed since)
- an issue in https://github.com/ome/bioformats with FilePatternReader
- an issue in https://github.com/glencoesoftware/bioformats2raw
Possibly the outstanding action would be to retest a similar scenario using bioformats2raw 0.7.0, a multi-channel pattern dataset and identify whether it's IDR specific. /cc @melissalinkert
Ah - apologies @sbesson: I just realised you meant that there is probably just 1 issue (not 4) but it needs testing to determine where the issue lies!
Retested with a simpler version of the pattern file with 2 timepoints compatible with upstream Bio-Formats
cat 3.49.6-3.140922_11-33-57.00.pattern
/uod/idr/filesets/idr0026-weigelin-immunotherapy/20170222-symlinks/PNAS_2015/treatment start day 3/mouse 49/day 6-3/time lapse/140922_11-33-57/11-33-57_PMT - PMT [<BD2_GREEN,BD8_DEEPR,BD7_RED,FD6_FDRED>] [00]_Time Time<0000-0001>.tif
Placed a copy of this pattern file under patterns owned by a different user and executed the two following commands:
/opt/bioformats2raw/bioformats2raw-0.6.1/bin/bioformats2raw 3.49.6-3.140922_11-33-57.00.pattern 3.49.6-3.140922_11-33-57.00.zarr
/opt/bioformats2raw/bioformats2raw-0.6.1/bin/bioformats2raw patterns/3.49.6-3.140922_11-33-57.00.pattern 3.49.6-3.140922_11-33-57.00_2.zarr
The .zattrs are identical between both conversion and contain omero metadata for the four channels specified in the pattern file:
(base) [sbesson@pilot-zarr2-dev tmp]$ diff 3.49.6-3.140922_11-33-57.00.zarr/0/.zattrs 3.49.6-3.140922_11-33-57.00_2.zarr/0/.zattrs
(base) [sbesson@pilot-zarr2-dev tmp]$ tail -n 50 3.49.6-3.140922_11-33-57.00.zarr/0/.zattrs
},
"family" : "linear",
"inverted" : false
}, {
"color" : "00FF00",
"coefficient" : 1,
"active" : true,
"label" : "BD8_DEEPR",
"window" : {
"min" : 401.0,
"max" : 16188.0,
"start" : 401.0,
"end" : 16188.0
},
"family" : "linear",
"inverted" : false
}, {
"color" : "0000FF",
"coefficient" : 1,
"active" : true,
"label" : "BD7_RED",
"window" : {
"min" : 801.0,
"max" : 8055.0,
"start" : 801.0,
"end" : 8055.0
},
"family" : "linear",
"inverted" : false
}, {
"color" : "FF0000",
"coefficient" : 1,
"active" : false,
"label" : "FD6_FDRED",
"window" : {
"min" : 250.0,
"max" : 10867.0,
"start" : 250.0,
"end" : 10867.0
},
"family" : "linear",
"inverted" : false
} ],
"rdefs" : {
"defaultT" : 0,
"model" : "color",
"defaultZ" : 13
}
}
}
Based on the above, I am leaning towards options 1 and 2 i.e. it's an IDR/bioformats specific issue which probably will be classified as wontfix as one of the aims of the ongoing conversion work is to get rid of this fork entirely
Started creating zips in a Screen
cd /data/idr0026
for i in */; do zip -r "${i%/}.zip" "$i"; done
With all the previous Filesets deleted from s3, uploaded just a couple of different new ones to test...
(base) [wmoore@pilot-zarr1-dev ~]$ ./mc cp -r /data/idr0026/3.49.6-3.140922_11-33-57.00.ome.zarr uk1s3/idr0026/zarr/3.49.6-3.140922_11-33-57.00.ome.zarr
...zarr/OME/METADATA.ome.xml: 2.78 GiB / 2.78 GiB ━━━━━━━━━━━━━━━ 61.69 MiB/s 46s(base)
(base) [wmoore@pilot-zarr1-dev ~]$ ./mc cp -r /data/idr0026/7.56.10-3.140926_14-52-18.03.ome.zarr uk1s3/idr0026/zarr/7.56.10-3.140926_14-52-18.03.ome.zarr
...zarr/OME/METADATA.ome.xml: 7.69 GiB / 7.69 GiB ━━━━━━━━━━━━━━━ 73.85 MiB/s 1m46s
Ooops - got an extra directory in there, but the images look good:
https://ome.github.io/ome-ngff-validator/?source=https://uk1s3.embassy.ebi.ac.uk/idr0026/zarr/7.56.10-3.140926_14-52-18.03.ome.zarr/7.56.10-3.140926_14-52-18.03.ome.zarr/0/