mountainlab-js icon indicating copy to clipboard operation
mountainlab-js copied to clipboard

Implement mda command in new mountainlab

Open magland opened this issue 6 years ago • 7 comments

should it be called ml-mda?

magland avatar Jul 19 '18 19:07 magland

I'd vote for simply mda.

Edit: unless it's not a full replacement for 'old mda'.

wysota avatar Jul 19 '18 19:07 wysota

It will be a full replacement. So I'll plan on 'mda'.

magland avatar Jul 19 '18 19:07 magland

@wysota. I implemented it and decided to call it mda-info. I will leave this open until you can verify that it is working.

For example, you can test it with this convenient universal kbucket command: mda-info kbucket://b5ecdf1474c5/datasets/synth_datasets/datasets/synth_16ch_30min/firings_true.mda

And also try it on a local .mda file of course.

magland avatar Jul 20 '18 19:07 magland

$ mda-info kbucket://b5ecdf1474c5/datasets/synth_datasets/datasets/synth_16ch_30min/firings_true.mda
Invalid number of dimensions **: 255
$ mda-info `ml-prv-locate dataset/raw.mda.prv`
{
    "num_bytes_per_entry": 4,
    "num_dims": 2,
    "dims": [
        4,
        18000000
    ],
    "dtype": "float32",
    "header_size": 20
}

alexmorley avatar Jul 21 '18 10:07 alexmorley

Should mda-info be a dependency instead? Or moved to system-packages?

alexmorley avatar Jul 21 '18 10:07 alexmorley

It would be nice to have some basic error detection.

$ ./mda-info README.txt Invalid number of dimensions **: 543518313

Something along the lines of "Hey, this doesn't look like an mda file. Use -f to process the file anyway".

wysota avatar Jul 24 '18 06:07 wysota

Also, should mda-info also handle these cases (would not be an "info" command then)?

mda get_chunk file.mda chunk_out.mda --index=0,100 --size=4x50 mda create file.mda --dtype=int16 --size=4x1000 mda set_chunk target_file.mda chunk.mda --index=0,100

wysota avatar Jul 24 '18 06:07 wysota