First-time installation issues: missing `example_contour_and_skeleton_info.mat`

[akihirom2017]

Hello,

I am trying to use this analysis toolbox. I have followed by the instruction as described in README.md. Then I found that the the file example_contour_and_skeleton_info.mat is missing to run this script. I would appreciate if you can provide the matlab file to run the script.

Thank you very much for your help. Sincerely

[MichaelCurrie]

Hello @akihirom2017! Thank you for your interest! May I ask what lab you are from?

We are in the ending stages of a big release to be announced at the C. elegans meeting at UCLA June 25th. Will you be there?

The example file you can use is in this public Google Drive folder: https://drive.google.com/drive/folders/0B7to9gBdZEyGNWtWUElWVzVxc0E?usp=sharing

It is .MAT but it is not a matlab file, just a data file in HDF5 format. In any case, if you put it in the correct folder, it will be processed into features by the OWAT tool.

[MichaelCurrie]

Please reopen this issue if it did not answer your question.

[akihirom2017]

Thank you very much for quick response, Michael.

May I ask what lab you are from? Actually, I am in Bill Schafer lab. When I talked to a couple of people including Ev, Tadas, and Andre, they mentioned and recommended to start using this openWorm program. Since I am in Bill's lab, I thought I could use the original matlab file we have. But it did not work directly, that's why I was asking about the example file.

I downloaded the example file from google drive and will play the open worm script with the file. :)

Thank you very much for your big help.

We are in the ending stages of a big release to be announced at the C. elegans meeting at UCLA June 25th. Will you be there? Unfortunately, I won't attend the worm meeting this time. a couple of lab-mates will attend the conference, though. I have just come to Bill's lab and I have no data to present in the meeting. I plan to come to other worm meeting, so I hope we can meet at some conference in future.

Anyway, Thanks for your help and contributing the open-source project, especially for worm behaviour analysis.

Thanks

Best regards

Aki

On Mon, Jun 5, 2017 at 8:08 AM, Michael Currie [email protected] wrote:

Closed #205 https://github.com/openworm/open-worm-analysis-toolbox/issues/205.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openworm/open-worm-analysis-toolbox/issues/205#event-1109524208, or mute the thread https://github.com/notifications/unsubscribe-auth/AbZ31YGQtigt_IxBhZMvQ7p5UQtj2ZFLks5sA6lYgaJpZM4NvaOO .

[akihirom2017]

Sorry, one more question regarding to run the open-worm-analysis-toolbox.

I have downloaded the .mat example data set and followed the direction in README.md.

I think I successfully conduct to generate bw by "bw = mv.BasicWorm.from_schafer_file_factory("example_contour_and_skeleton_info.mat") "

However, in the next command (nw = mv.NormalizedWorm.from_BasicWorm_factory(bw)

), it gave me error message:

nw = mv.NormalizedWorm.from_BasicWorm_factory(bw) Traceback (most recent call last): File "", line 1, in File "xxx/miniconda/lib/python3.6/site-packages/open_worm_analysis_toolbox/prefeatures/normalized_worm.py", line 97, in from_BasicWorm_factory bw.h_dorsal_contour, File "xxx/miniconda/lib/python3.6/site-packages/open_worm_analysis_toolbox/prefeatures/pre_features.py", line 96, in compute_skeleton_and_widths frames_to_plot=[]) File "xxx/miniconda/lib/python3.6/site-packages/open_worm_analysis_toolbox/prefeatures/skeleton_calculator1.py", line 135, in compute_skeleton_and_widths s1 = _smooth_vec(s1) File "xxx/miniconda/lib/python3.6/site-packages/open_worm_analysis_toolbox/prefeatures/skeleton_calculator1.py", line 128, in _smooth_vec polyorder=SMOOTHING_ORDER) File "xxx/miniconda/lib/python3.6/site-packages/scipy/signal/_savitzky_golay.py", line 339, in savgol_filter _fit_edges_polyfit(x, window_length, polyorder, deriv, delta, axis, y) File "xxx/miniconda/lib/python3.6/site-packages/scipy/signal/_savitzky_golay.py", line 217, in _fit_edges_polyfit polyorder, deriv, delta, y) File "xxx/miniconda/lib/python3.6/site-packages/scipy/signal/_savitzky_golay.py", line 175, in _fit_edge x_edge = axis_slice(x, start=window_start, stop=window_stop, axis=axis) File "xxx/miniconda/lib/python3.6/site-packages/scipy/signal/_arraytools.py", line 45, in axis_slice b = a[a_slice] TypeError: slice indices must be integers or None or have an index method

It is hardly imagine that the this program did not run with example file. Although this error message may be caused by the outside python program (beside open-worm scripts), do you have any idea to solve the issue?

I would appreciate if you gave me a tip.

Thanks

On Mon, Jun 5, 2017 at 9:07 AM, akihiro mori [email protected] wrote:

Thank you very much for quick response, Michael.

May I ask what lab you are from? Actually, I am in Bill Schafer lab. When I talked to a couple of people including Ev, Tadas, and Andre, they mentioned and recommended to start using this openWorm program. Since I am in Bill's lab, I thought I could use the original matlab file we have. But it did not work directly, that's why I was asking about the example file.

I downloaded the example file from google drive and will play the open worm script with the file. :)

Thank you very much for your big help.

We are in the ending stages of a big release to be announced at the C. elegans meeting at UCLA June 25th. Will you be there? Unfortunately, I won't attend the worm meeting this time. a couple of lab-mates will attend the conference, though. I have just come to Bill's lab and I have no data to present in the meeting. I plan to come to other worm meeting, so I hope we can meet at some conference in future.

Anyway, Thanks for your help and contributing the open-source project, especially for worm behaviour analysis.

Thanks

Best regards

Aki

On Mon, Jun 5, 2017 at 8:08 AM, Michael Currie [email protected] wrote:

Closed #205 https://github.com/openworm/open-worm-analysis-toolbox/issues/205.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openworm/open-worm-analysis-toolbox/issues/205#event-1109524208, or mute the thread https://github.com/notifications/unsubscribe-auth/AbZ31YGQtigt_IxBhZMvQ7p5UQtj2ZFLks5sA6lYgaJpZM4NvaOO .

[ver228]

Hello Aki,

Maybe you can try the code below. It uses a file generated by our lab that you can get here. However you can use any set of skeletons/contours as long as they are list of (2, number_of_points) numpy arrays. I used something similar for simulated data before.

The code only uses the OPTION 2 to calculate features from a given set of skeletons (and ignore features related to the contour, like the area or width). If you want to use the contour data uncomment the code lines in OPTION 1. This is a fairly slow step because it would calculate the skeletons from the contours (it is computational heavy and has not been optimised). I have a faster version of the skeletonization code in the Tierpsy Tracker, but it requires cython (C compiler) and it is a bit noisier.

In the Tierpsy Tracker I use the OPTION 3 but it requires the contour width too, and i did not store it in the example file. I have additionally other functions to calculate the features means here, but they are part of the tierpsy-tracker and need documentation.

import open_worm_analysis_toolbox as mv
import tables

if __name__ == '__main__':
    fname = 'N2 2 on food L_2011_06_20__10_56_54___2___1_features.hdf5'
    
    #read sample data
    with tables.File(fname, 'r') as fid:
        #the data is already in micrometers no need to do a conversion here
        cnt_dorsal = fid.get_node('/coordinates/ventral_contours')[:]
        cnt_ventral = fid.get_node('/coordinates/dorsal_contours')[:]
        skeletons = fid.get_node('/coordinates/skeletons')[:]
    
    #OPTIONS FOR INPUTS
    
    #OPTION 1: 
    #this function can accept a list of contours of different size 
    #and will calculate the skeletons.
    
    #this function requires as input a list of contours (2, contour_points)
    cnt_dorsal = [x.T for x in cnt_dorsal]
    cnt_ventral = [x.T for x in cnt_ventral]
    
    #unfortunately is severly limited since it does require that the contour is subdivided before in dorsal and ventral
    #bw1 = mv.BasicWorm.from_contour_factory(cnt_dorsal, cnt_ventral)
    #this function normalize the skeletons into an array of 49,2,n_points
    #it might take a while because if no skeleton was given it is calculating it now.
    #nw1 = mv.NormalizedWorm.from_BasicWorm_factory(bw1)
    
    #OPTION 2:
    #use this option if you only have the skeletons.
    
    #this function requires as input a list of contours (2, skeleton_points)
    skeletons = [x.T for x in skeletons]
    
    #Features that depends on the contour. like width or area will not be calculated.
    bw2 = mv.BasicWorm.from_skeleton_factory(skeletons)
    #this function normalize the skeletons into an array of 49,2,n_points
    nw2 = mv.NormalizedWorm.from_BasicWorm_factory(bw2)
    
    #OPTION 3:
    #this assumes you have a normalize skeleton, ventral, dorsal and contour width.
    #By default the _features.hdf5 file does not save the widths
    #nw3 = mv.NormalizedWorm.from_normalized_array_factory(skeleton, widths, ventral_contour, dorsal_contour)
    
    
    #ALL THIS IS THE SAME ONCE YOU HAVE THE NormalizedWorm OBJECT
    
    nw = nw2
    
    #this data is at 30 frames per seconds
    nw.video_info.fps = 30 
    ventral_side = "clockwise" #Valid options are "clockwise", "anticlockwise" and "unknown".
    nw.video_info.set_ventral_mode(ventral_side)
    
    wf = mv.WormFeatures(nw)

[akihirom2017]

Thanks for your help.

I tested two things, but I still had error messages. I would appreciate if you can help me.

1.

I just copied and paste the script you have sent in previous e-mail. I named the file "test.py" and it located in the prefeature directory under open_worm_analysis_toolbox (e.g. open-worm-analysis-toolbox/ open_worm_analysis_toolbox/prefeatures).

when I run the script, here is what I got: amori@hex:~/programs/python/open-worm-analysis-toolbox/ open_worm_analysis_toolbox/prefeatures$ python test.py xxx/miniconda/lib/python3.6/site-packages/open_worm_ analysis_toolbox/features/worm_features.py:815: UserWarning: morphology.width.head was NOT calculated. 'NormalizedWorm' object has no attribute 'widths' warnings.warn(msg_warn) xxx/miniconda/lib/python3.6/site-packages/open_worm_ analysis_toolbox/features/worm_features.py:815: UserWarning: morphology.width.midbody was NOT calculated. 'NormalizedWorm' object has no attribute 'widths' warnings.warn(msg_warn) xxx/miniconda/lib/python3.6/site-packages/open_worm_ analysis_toolbox/features/worm_features.py:815: UserWarning: morphology.width.tail was NOT calculated. 'NormalizedWorm' object has no attribute 'widths' warnings.warn(msg_warn) xxx/miniconda/lib/python3.6/site-packages/open_worm_ analysis_toolbox/features/worm_features.py:815: UserWarning: morphology.area was NOT calculated. 'NormalizedWorm' object has no attribute 'signed_area' warnings.warn(msg_warn) xxx/miniconda/lib/python3.6/site-packages/open_worm analysis_toolbox/features/worm_features.py:815: UserWarning: morphology.area_per_length was NOT calculated. 'NormalizedWorm' object has no attribute 'signed_area' warnings.warn(msg_warn) xxx/miniconda/lib/python3.6/site-packages/open_worm analysis_toolbox/features/worm_features.py:815: UserWarning: morphology.width_per_length was NOT calculated. 'NormalizedWorm' object has no attribute 'widths' warnings.warn(msg_warn) xxx/miniconda/lib/python3.6/site-packages/open_worm_ analysis_toolbox/features/worm_features.py:815: UserWarning: locomotion.crawling_bends.head was NOT calculated. slice indices must be integers or None or have an index method warnings.warn(msg_warn) xxx/miniconda/lib/python3.6/site-packages/open_worm_ analysis_toolbox/features/worm_features.py:815: UserWarning: locomotion.crawling_bends.head.amplitude was NOT calculated. slice indices must be integers or None or have an index method warnings.warn(msg_warn) xxx/miniconda/lib/python3.6/site-packages/open_worm_ analysis_toolbox/features/worm_features.py:815: UserWarning: locomotion.crawling_bends.head.frequency was NOT calculated. slice indices must be integers or None or have an index method warnings.warn(msg_warn) xxx/miniconda/lib/python3.6/site-packages/open_worm_ analysis_toolbox/features/worm_features.py:815: UserWarning: locomotion.crawling_bends.midbody was NOT calculated. slice indices must be integers or None or have an index method warnings.warn(msg_warn) xxx/miniconda/lib/python3.6/site-packages/open_worm_ analysis_toolbox/features/worm_features.py:815: UserWarning: locomotion.crawling_bends.midbody.amplitude was NOT calculated. slice indices must be integers or None or have an index method warnings.warn(msg_warn) xxx/miniconda/lib/python3.6/site-packages/open_worm_ analysis_toolbox/features/worm_features.py:815: UserWarning: locomotion.crawling_bends.midbody.frequency was NOT calculated. slice indices must be integers or None or have an index method warnings.warn(msg_warn) xxx/miniconda/lib/python3.6/site-packages/open_worm_ analysis_toolbox/features/worm_features.py:815: UserWarning: locomotion.crawling_bends.tail was NOT calculated. slice indices must be integers or None or have an index method warnings.warn(msg_warn) xxx/miniconda/lib/python3.6/site-packages/open_worm_ analysis_toolbox/features/worm_features.py:815: UserWarning: locomotion.crawling_bends.tail.amplitude was NOT calculated. slice indices must be integers or None or have an index method warnings.warn(msg_warn) xxx/miniconda/lib/python3.6/site-packages/open_worm_ analysis_toolbox/features/worm_features.py:815: UserWarning: locomotion.crawling_bends.tail.frequency was NOT calculated. slice indices must be integers or None or have an index method warnings.warn(msg_warn) xxx/miniconda/lib/python3.6/site-packages/open_worm_ analysis_toolbox/features/worm_features.py:815: UserWarning: path.range was NOT calculated. 'NoneType' object is not subscriptable warnings.warn(msg_warn) xxx/miniconda/lib/python3.6/site-packages/open_worm_ analysis_toolbox/features/worm_features.py:815: UserWarning: path.duration was NOT calculated. 'NormalizedWorm' object has no attribute 'widths' warnings.warn(msg_warn) xxx/miniconda/lib/python3.6/site-packages/open_worm_ analysis_toolbox/features/worm_features.py:815: UserWarning: path.duration.worm was NOT calculated. 'NormalizedWorm' object has no attribute 'widths' warnings.warn(msg_warn) xxx/miniconda/lib/python3.6/site-packages/open_worm_ analysis_toolbox/features/worm_features.py:815: UserWarning: path.duration.head was NOT calculated. 'NormalizedWorm' object has no attribute 'widths' warnings.warn(msg_warn) xxx/miniconda/lib/python3.6/site-packages/open_worm_ analysis_toolbox/features/worm_features.py:815: UserWarning: path.duration.midbody was NOT calculated. 'NormalizedWorm' object has no attribute 'widths' warnings.warn(msg_warn) xxx/miniconda/lib/python3.6/site-packages/open_worm_ analysis_toolbox/features/worm_features.py:815: UserWarning: path.duration.tail was NOT calculated. 'NormalizedWorm' object has no attribute 'widths' warnings.warn(msg_warn) xxx/miniconda/lib/python3.6/site-packages/open_worm_ analysis_toolbox/features/posture_features.py:662: RuntimeWarning: divide by zero encountered in true_divide all_wavelengths = 1 / frequency_values

I am not sure this is successully done.

additionally, Since the script directly give me 'wf' which is the 'Obtain features' in original pipeline. So, I tried to create a new class which defined in init.py by adding an extra line "from .prefeatures.test import NewBasicWorm" and modified test.py as required and (I think) I followed the same way, but the NewBasicWorm.test() was not successfully provide the wf.

#2. I thought these error message mentioned above might be caused by a particular file (e.g. N2 2 on food L_2011_06_20__10_56_54___2___1 _features.hdf5). So, I downloaded another file from google drive (e.g. unc-63 (ok1075) on food L_2010_04_09__11_44_51___4___5_features.hdf5). Unfortunately, this file does not contain the "/coordinates/ventral_contours" which gave an error message and stop the process.

Traceback (most recent call last): File "test.py", line 12, in cnt_dorsal = fid.get_node('/coordinates/ventral_contours')[:] File "/lmb/home/amori/miniconda/lib/python3.6/site-packages/tables/file.py", line 1593, in get_node node = self._get_node(nodepath) File "/lmb/home/amori/miniconda/lib/python3.6/site-packages/tables/file.py", line 1541, in _get_node node = self._node_manager.get_node(nodepath) File "/lmb/home/amori/miniconda/lib/python3.6/site-packages/tables/file.py", line 437, in get_node node = self.node_factory(key) File "/lmb/home/amori/miniconda/lib/python3.6/site-packages/tables/group.py", line 1160, in _g_load_child node_type = self._g_check_has_child(childname) File "/lmb/home/amori/miniconda/lib/python3.6/site-packages/tables/group.py", line 395, in _g_check_has_child % (self._v_pathname, name)) tables.exceptions.NoSuchNodeError: group / does not have a child named /coordinates/ventral_contours

I would appreciate if you can give any advice to avoid these problems.

Thank you very much for your big help.

Best regards

Aki

On Mon, Jun 5, 2017 at 1:53 PM, ver228 [email protected] wrote:

Hello Aki,

Maybe you can try the code below. It uses a file generated by our lab that you can get here https://imperiallondon-my.sharepoint.com/personal/ajaver_ic_ac_uk/_layouts/15/guestaccess.aspx?docid=0fcd1cd4566c4484b92eb2b1849056b45&authkey=AYfYwtDFR2BzJx-hQ-xIRTw. However you can use any set of skeletons/contours as long as they are list of (2, number_of_points) numpy arrays. I used something similar for simulated data before. I have other functions to calculate the feature means but I only have them in the Tierpsy Tracker https://github.com/ver228/tierpsy-tracker. Let me know how it goes.

import open_worm_analysis_toolbox as mvimport tables if name == 'main': fname = 'N2 2 on food L_2011_06_20__10_56_54___2___1_features.hdf5'

#read sample data
with tables.File(fname, 'r') as fid:
    #the data is already in micrometers no need to do a conversion here
    cnt_dorsal = fid.get_node('/coordinates/ventral_contours')[:]
    cnt_ventral = fid.get_node('/coordinates/dorsal_contours')[:]
    skeletons = fid.get_node('/coordinates/skeletons')[:]

#OPTIONS FOR INPUTS

#OPTION 1:
#this function can accept a list of contours of different size
#and will calculate the skeletons.

#this function requires as input a list of contours (2, contour_points)
cnt_dorsal = [x.T for x in cnt_dorsal]
cnt_ventral = [x.T for x in cnt_ventral]

#unfortunately is severly limited since it does require that the contour is subdivided before in dorsal and ventral
#bw1 = mv.BasicWorm.from_contour_factory(cnt_dorsal, cnt_ventral)
#this function normalize the skeletons into an array of 49,2,n_points
#it might take a while because if no skeleton was given it is calculating it now.
#nw1 = mv.NormalizedWorm.from_BasicWorm_factory(bw1)

#OPTION 2:
#use this option if you only have the skeletons.

#this function requires as input a list of contours (2, skeleton_points)
skeletons = [x.T for x in skeletons]

#Features that depends on the contour. like width or area will not be calculated.
bw2 = mv.BasicWorm.from_skeleton_factory(skeletons)
#this function normalize the skeletons into an array of 49,2,n_points
nw2 = mv.NormalizedWorm.from_BasicWorm_factory(bw2)

#OPTION 3:
#this assumes you have a normalize skeleton, ventral, dorsal and contour width.
#By default the _features.hdf5 file does not save the widths
#nw3 = mv.NormalizedWorm.from_normalized_array_factory(skeleton, widths, ventral_contour, dorsal_contour)


#ALL THIS IS THE SAME ONCE YOU HAVE THE NormalizedWorm OBJECT

nw = nw2

#this data is at 30 frames per seconds
nw.video_info.fps = 30
ventral_side = "clockwise" #Valid options are "clockwise", "anticlockwise" and "unknown".
nw.video_info.set_ventral_mode(ventral_side)

wf = mv.WormFeatures(nw)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openworm/open-worm-analysis-toolbox/issues/205#issuecomment-306180068, or mute the thread https://github.com/notifications/unsubscribe-auth/AbZ31S8iJr9qIQjLdfPZkuImuH2SU8AMks5sA_ozgaJpZM4NvaOO .

[ver228]

Hello Aki,

Those are not errors. They are warnings informing you that those features are not calculated. This is because in OPTION 1 it is only using the skeletons, so it does not have information about the area and width, and several features depend on it. If you use the OPTION 2 this warnings show not appear. It might be good to suppress those warnings in the future, and either force the user to explicitly select what features they want to calculate or skip some features if there is only the skeleton present, but for the moment this works.

You can access to the features from wf._features. It is a dictionary, and the fields should be the once in the column in this table.

I am not sure what you want to do with __init__.py, but I suggest that to work with this package you first install the module using python setup.py develop in the main folder, and then create new classes deviated from the original ones.

import open_worm_analysis_toolbox as mv
class NewWorm(mv.NormalizedWorm):
   ...

From where did you downloaded the N2 2 on food L_2011_06_20__10_56_54___2___1 _features.hdf5? It seems that it was not produced by my software or it was an older version. I am sure it has some skeletons/contours you can play with. Use pytables or h5py to explore it. If you want more examples I can send you some. Unfortunately we haven't upload the files in the single worm database to Zenodo as we plan, but we will do it at some point.

Finally, a little comment about how to "comment" in github. It would be nice if you could use the character ` to separate code from raw text. It is easier to read. Take a look here.

[akihirom2017]

Thank you for your help.

I will follow your direction. Thanks

Finally, a little comment about how to "comment" in github. Sorry, I was using a regular e-mail instead of github message board. But I will follow your suggestion regarding to comment and code. Sorry about that. I am still new to use github....

Best regards

Aki

On Tue, Jun 6, 2017 at 9:27 AM, ver228 [email protected] wrote:

Hello Aki,

Those are not errors. They are warnings informing you that those features are not calculated. This is because in OPTION 1 it is only using the skeletons, so it does not have information about the area and width, and several features depend on it. If you use the OPTION 2 this warnings show not appear. It might be good to suppress those warnings in the future, and either force the user to explicitly select what features they want to calculate or skip some features if there is only the skeleton present, but for the moment this works.

You can access to the features from wf._features. It is a dictionary, and the fields should be the once in the column in this table https://github.com/ver228/tierpsy-tracker/blob/master/tierpsy/extras/features_names.csv .

I am not sure what you want to do with init.py, but I suggest that to work with this package you first install the module using python setup.py develop in the main folder, and then create new classes deviated from the original ones.

import open_worm_analysis_toolbox as mv class NewWorm(mv.NormalizedWorm): ...

From where did you downloaded the N2 2 on food L_2011_06_20__10_56_54___2___1 _features.hdf5? It seems that it was not produced by my software or it was an older version. I am sure it has some skeletons/contours you can play with. Use pytables or h5py to explore it. If you want more examples I can send you some. Unfortunately we haven't upload the files in the single worm database to Zenodo as we plan, but we will do it at some point.

Finally, a little comment about how to "comment" in github. It would be nice if you could use the character ` to separate code from raw text. It is easier to read. Take a look here https://guides.github.com/features/mastering-markdown/.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openworm/open-worm-analysis-toolbox/issues/205#issuecomment-306417711, or mute the thread https://github.com/notifications/unsubscribe-auth/AbZ31Xhn5LAUL-v9XhtY0beRgwvRXkwSks5sBQ19gaJpZM4NvaOO .

[akihirom2017]

Hello,

It is progressing, but I still have problem. and I overlooked the question in your previous e-mail, so here is the answer.

From where did you downloaded the N2 2 on food L_2011_06_20__10_56_54___2___1 _features.hdf5? I have just downloaded the file from the "here" in "It uses a file generated by our lab that you can get here https://imperiallondon-my.sharepoint.com/personal/ajaver_ic_ac_uk/_layouts/15/guestaccess.aspx?docid=0fcd1cd4566c4484b92eb2b1849056b45&authkey=AYfYwtDFR2BzJx-hQ-xIRTw . " (from e-mail on 5th Jun).

Now, the script seems to work to obtain "wf" features(?). However, it cannot process further. I mean, I would like to make figures based on wf features. particularly, I would like to run the following command lines (from README.md):

experiment_histograms = mv.HistogramManager([wf, wf])

control_histograms = mv.HistogramManager(control_worms)

stat = mv.StatisticsManager(experiment_histograms, control_histograms)

stat[0].plot(ax=None, use_alternate_plot=True)

`print("Nonparametric p and q values are %.2f and %.2f, respectively." %

  (stat.min_p_wilcoxon, stat.min_q_wilcoxon))`

However, I have got error message with the first line.

`Traceback (most recent call last):

File "test.py", line 217, in

experiment_histograms = mv.HistogramManager(wf)

File "/lmb/home/amori/miniconda/lib/python3.6/site-packages/open_worm_analysis_toolbox/statistics/histogram_manager.py", line 104, in init

new_histogram_set = self.init_histograms(worm_features)

File "/lmb/home/amori/miniconda/lib/python3.6/site-packages/open_worm_analysis_toolbox/statistics/histogram_manager.py", line 203, in init_histograms

return np.array([Histogram.create_histogram(f) for f in worm_features])

TypeError: 'Length' object is not iterable`

Note that "test.py" contained the script you gave me.

As far as I understood after google search, I think I need to add the iterable (values?) to Length. but I have no idea how it can be done.

Do you have any idea?

I would appreciate if you suggestion some idea.

Thank you very much

Best

On Tue, Jun 6, 2017 at 9:39 AM, akihiro mori [email protected] wrote:

Thank you for your help.

I will follow your direction. Thanks

Finally, a little comment about how to "comment" in github. Sorry, I was using a regular e-mail instead of github message board. But I will follow your suggestion regarding to comment and code. Sorry about that. I am still new to use github....

Best regards

Aki

On Tue, Jun 6, 2017 at 9:27 AM, ver228 [email protected] wrote:

Hello Aki,

Those are not errors. They are warnings informing you that those features are not calculated. This is because in OPTION 1 it is only using the skeletons, so it does not have information about the area and width, and several features depend on it. If you use the OPTION 2 this warnings show not appear. It might be good to suppress those warnings in the future, and either force the user to explicitly select what features they want to calculate or skip some features if there is only the skeleton present, but for the moment this works.

You can access to the features from wf._features. It is a dictionary, and the fields should be the once in the column in this table https://github.com/ver228/tierpsy-tracker/blob/master/tierpsy/extras/features_names.csv .

I am not sure what you want to do with init.py, but I suggest that to work with this package you first install the module using python setup.py develop in the main folder, and then create new classes deviated from the original ones.

import open_worm_analysis_toolbox as mv class NewWorm(mv.NormalizedWorm): ...

From where did you downloaded the N2 2 on food L_2011_06_20__10_56_54___2___1 _features.hdf5? It seems that it was not produced by my software or it was an older version. I am sure it has some skeletons/contours you can play with. Use pytables or h5py to explore it. If you want more examples I can send you some. Unfortunately we haven't upload the files in the single worm database to Zenodo as we plan, but we will do it at some point.

Finally, a little comment about how to "comment" in github. It would be nice if you could use the character ` to separate code from raw text. It is easier to read. Take a look here https://guides.github.com/features/mastering-markdown/.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openworm/open-worm-analysis-toolbox/issues/205#issuecomment-306417711, or mute the thread https://github.com/notifications/unsubscribe-auth/AbZ31Xhn5LAUL-v9XhtY0beRgwvRXkwSks5sBQ19gaJpZM4NvaOO .

[akihirom2017]

Sorry, my explanation was short.

the result in previous e-mail was run by experiment_histograms = mv.HistogramManager(wf) (I assumed this is the control_experimental case)

If I run "experiment_histograms = mv.HistogramManager([wf, wf])" (I assume this is the experimental/mutant case), then the error message was Traceback (most recent call last): File "test.py", line 217, in <module> experiment_histograms = mv.HistogramManager([wf, wf]) File "/lmb/home/amori/miniconda/lib/python3.6/site-packages/open_worm_analysis_toolbox/statistics/histogram_manager.py", line 104, in __init__ new_histogram_set = self.init_histograms(worm_features) File "/lmb/home/amori/miniconda/lib/python3.6/site-packages/open_worm_analysis_toolbox/statistics/histogram_manager.py", line 203, in init_histograms return np.array([Histogram.create_histogram(f) for f in worm_features]) File "/lmb/home/amori/miniconda/lib/python3.6/site-packages/open_worm_analysis_toolbox/statistics/histogram_manager.py", line 203, in <listcomp> return np.array([Histogram.create_histogram(f) for f in worm_features]) File "/lmb/home/amori/miniconda/lib/python3.6/site-packages/open_worm_analysis_toolbox/statistics/histogram.py", line 149, in create_histogram return cls(feature) File "/lmb/home/amori/miniconda/lib/python3.6/site-packages/open_worm_analysis_toolbox/statistics/histogram.py", line 113, in __init__ self.compute_covering_bins() File "/lmb/home/amori/miniconda/lib/python3.6/site-packages/open_worm_analysis_toolbox/statistics/histogram.py", line 283, in compute_covering_bins assert(min_data >= self.bin_boundaries[0] - bin_width) IndexError: index 0 is out of bounds for axis 0 with size 0

Any helps would be appreciated.

Thank you.

Best

On Thu, Jun 8, 2017 at 3:37 PM, akihiro mori [email protected] wrote:

Hello,

It is progressing, but I still have problem. and I overlooked the question in your previous e-mail, so here is the answer.

From where did you downloaded the N2 2 on food L_2011_06_20__10_56_54___2___1 _features.hdf5? I have just downloaded the file from the "here" in "It uses a file generated by our lab that you can get here https://imperiallondon-my.sharepoint.com/personal/ajaver_ic_ac_uk/_layouts/15/guestaccess.aspx?docid=0fcd1cd4566c4484b92eb2b1849056b45&authkey=AYfYwtDFR2BzJx-hQ-xIRTw . " (from e-mail on 5th Jun).

Now, the script seems to work to obtain "wf" features(?). However, it cannot process further. I mean, I would like to make figures based on wf features. particularly, I would like to run the following command lines (from README.md):

experiment_histograms = mv.HistogramManager([wf, wf])

control_histograms = mv.HistogramManager(control_worms)

stat = mv.StatisticsManager(experiment_histograms, control_histograms)

stat[0].plot(ax=None, use_alternate_plot=True)

`print("Nonparametric p and q values are %.2f and %.2f, respectively." %

  (stat.min_p_wilcoxon, stat.min_q_wilcoxon))`

However, I have got error message with the first line.

`Traceback (most recent call last):

File "test.py", line 217, in

experiment_histograms = mv.HistogramManager(wf)

File "/lmb/home/amori/miniconda/lib/python3.6/site-packages/ open_worm_analysis_toolbox/statistics/histogram_manager.py", line 104, in init

new_histogram_set = self.init_histograms(worm_features)

File "/lmb/home/amori/miniconda/lib/python3.6/site-packages/ open_worm_analysis_toolbox/statistics/histogram_manager.py", line 203, in init_histograms

return np.array([Histogram.create_histogram(f) for f in

worm_features])

TypeError: 'Length' object is not iterable`

Note that "test.py" contained the script you gave me.

As far as I understood after google search, I think I need to add the iterable (values?) to Length. but I have no idea how it can be done.

Do you have any idea?

I would appreciate if you suggestion some idea.

Thank you very much

Best

On Tue, Jun 6, 2017 at 9:39 AM, akihiro mori [email protected] wrote:

Thank you for your help.

I will follow your direction. Thanks

Finally, a little comment about how to "comment" in github. Sorry, I was using a regular e-mail instead of github message board. But I will follow your suggestion regarding to comment and code. Sorry about that. I am still new to use github....

Best regards

Aki

On Tue, Jun 6, 2017 at 9:27 AM, ver228 [email protected] wrote:

Hello Aki,

Those are not errors. They are warnings informing you that those features are not calculated. This is because in OPTION 1 it is only using the skeletons, so it does not have information about the area and width, and several features depend on it. If you use the OPTION 2 this warnings show not appear. It might be good to suppress those warnings in the future, and either force the user to explicitly select what features they want to calculate or skip some features if there is only the skeleton present, but for the moment this works.

You can access to the features from wf._features. It is a dictionary, and the fields should be the once in the column in this table https://github.com/ver228/tierpsy-tracker/blob/master/tierpsy/extras/features_names.csv .

I am not sure what you want to do with init.py, but I suggest that to work with this package you first install the module using python setup.py develop in the main folder, and then create new classes deviated from the original ones.

import open_worm_analysis_toolbox as mv class NewWorm(mv.NormalizedWorm): ...

From where did you downloaded the N2 2 on food L_2011_06_20__10_56_54___2___1 _features.hdf5? It seems that it was not produced by my software or it was an older version. I am sure it has some skeletons/contours you can play with. Use pytables or h5py to explore it. If you want more examples I can send you some. Unfortunately we haven't upload the files in the single worm database to Zenodo as we plan, but we will do it at some point.

Finally, a little comment about how to "comment" in github. It would be nice if you could use the character ` to separate code from raw text. It is easier to read. Take a look here https://guides.github.com/features/mastering-markdown/.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openworm/open-worm-analysis-toolbox/issues/205#issuecomment-306417711, or mute the thread https://github.com/notifications/unsubscribe-auth/AbZ31Xhn5LAUL-v9XhtY0beRgwvRXkwSks5sBQ19gaJpZM4NvaOO .

[ver228]

I have never use that part I think @MichaelCurrie or @JimHokanson might have some feedback there. It is likely it got broken at some point during the refactoring.

[akihirom2017]

Hello,

I think the Linux had a problem to install the open_worm_analysis_toolbox, so I started from scratch. I installed Ubuntu 16.04 on Windows through VMWare.

I mostly follow the INSTALL.md lines, but found some correction which I need to put some comment to avoid some issues (these issues may happen only on my environmental condition) Please find them in "I followed this line." file I think I have successfully installed the open_worm_analysis_toolbox (I just assumed this because I have not get any error during installation process.)

Then, I tried to use test_setup.py in /tools. However, in this process, I got an error message. akihirom@ubuntu:~/Documents/Programs/open-worm-analysis-toolbox-master/tools$ python test_setup.py File "test_setup.py", line 120 from open - worm - analysis - toolbox import user_config ^ SyntaxError: invalid syntax

I would appreciate if you can give me suggestion to avoid this error.

Thank you very much. Sincerely

On Fri, Jun 9, 2017 at 10:07 AM, ver228 [email protected] wrote:

I have never use that part I think @MichaelCurrie https://github.com/michaelcurrie or @JimHokanson https://github.com/jimhokanson might have some feedback there. It is likely it got broken at some point during the refactoring.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openworm/open-worm-analysis-toolbox/issues/205#issuecomment-307337301, or mute the thread https://github.com/notifications/unsubscribe-auth/AbZ31bUVzrvM5zMAk1HMB3xxz6Foslonks5sCQtCgaJpZM4NvaOO .

[JimHokanson]

It appears as though 1) we named our package with hyphens instead of underscores :/ and 2) auto code cleanup translated the package name into separate names with subtraction signs

The correct code should presumably be: from open-worm-analysis-toolbox import user_config

[akihirom2017]

Thanks, Jim

The correct code should presumably be: from open-worm-analysis-toolbox import user_config

When I run test_setup.py script with the line, I have still got error message. aki@ubuntu:~/github/open-worm-analysis-toolbox/tools$ python test_setup.py File "test_setup.py", line 121 from open-worm-analysis-toolbox import user_config ^ SyntaxError: invalid syntax

Am I missing some key step to avoid this issue? Just in case, I am currently use python 3 (3.5.3) on ubuntu 16.04.

Please let me know. Thanks

On Mon, Jun 12, 2017 at 4:27 AM, Jim Hokanson [email protected] wrote:

It appears as though 1) we named our package with hyphens instead of underscores :/ and 2) auto code cleanup translated the package name into separate names with subtraction signs

The correct code should presumably be: from open-worm-analysis-toolbox import user_config

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openworm/open-worm-analysis-toolbox/issues/205#issuecomment-307682277, or mute the thread https://github.com/notifications/unsubscribe-auth/AbZ31VIk7uXu7T9grVsq29YpfG-pDKn1ks5sDLAWgaJpZM4NvaOO .

[akihirom2017]

I have tested few things additionally, hopefully this help to figure out the issue.

#1 After installation of open-worm-analysis-toolbox,I run following command lines: python3 setup.py build python3 setup.py install --user python3 setup.py develop

All results seem to be fine (I did not get any error message).

Then I tried python3 setup.py test Then, it gave me: ['build', 'setup.py', '.git', 'INSTALL.md', 'OpenWorm Analysis Toolbox logo.ai', 'jim_matlab_testing', 'dist', 'tests', 'README.md', '.openworm.yml', 'mkdocs.yml', 'examples', 'open_worm_analysis_toolbox', '.travis.yml', 'open_worm_analysis_toolbox.egg-info', 'LICENSE.md', 'OpenWorm Analysis Toolbox logo.png', 'THANKS.md', 'INSTALL-OSX.md', 'documentation', 'tools', '.gitignore'] running test running egg_info writing top-level names to open_worm_analysis_toolbox.egg-info/top_level.txt writing dependency_links to open_worm_analysis_toolbox.egg-info/dependency_links.txt writing open_worm_analysis_toolbox.egg-info/PKG-INFO writing requirements to open_worm_analysis_toolbox.egg-info/requires.txt reading manifest file 'open_worm_analysis_toolbox.egg-info/SOURCES.txt' writing manifest file 'open_worm_analysis_toolbox.egg-info/SOURCES.txt' running build_ext /home/aki/miniconda2/lib/python3.5/site-packages/matplotlib/font_manager.py:280: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment. 'Matplotlib is building the font cache using fc-list. ' examples (unittest.loader._FailedTest) ... ERROR open_worm_analysis_toolbox (unittest.loader._FailedTest) ... ERROR

====================================================================== ERROR: examples (unittest.loader._FailedTest)

ImportError: Failed to import test module: examples Traceback (most recent call last): File "/home/aki/miniconda2/lib/python3.5/unittest/loader.py", line 462, in _find_test_path package = self._get_module_from_name(name) File "/home/aki/miniconda2/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name import(name) File "/home/aki/github/open-worm-analysis-toolbox/examples/init.py", line 1, in from .validate_features import main as validate_features ImportError: No module named 'examples.validate_features'

====================================================================== ERROR: open_worm_analysis_toolbox (unittest.loader._FailedTest)

ImportError: Failed to import test module: open_worm_analysis_toolbox Traceback (most recent call last): File "/home/aki/miniconda2/lib/python3.5/unittest/loader.py", line 462, in _find_test_path package = self._get_module_from_name(name) File "/home/aki/miniconda2/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name import(name) File "/home/aki/github/open-worm-analysis-toolbox/open_worm_analysis_toolbox/init.py", line 26, in from .features.worm_features import WormFeatures File "/home/aki/github/open-worm-analysis-toolbox/open_worm_analysis_toolbox/features/worm_features.py", line 42, in from . import posture_features File "/home/aki/github/open-worm-analysis-toolbox/open_worm_analysis_toolbox/features/posture_features.py", line 16, in import cv2 ImportError: No module named 'cv2'

---

Ran 2 tests in 0.000s

FAILED (errors=2)

somehow, examples.validate_features and cv2 were not recognised.

#2 I tried to run some example scripts in /examples python feature_processing_options_testing.py this gave me: File "feature_processing_options_testing.py", line 16 import open-worm-analysis-toolbox as mv ^ SyntaxError: invalid syntax This problem is similar to the one in previous e-mail.

python generate_stats_plots.py this gave me: Traceback (most recent call last): File "generate_stats_plots.py", line 15, in import open_worm_analysis_toolbox as mv File "/home/aki/.local/lib/python3.5/site-packages/open_worm_analysis_toolbox-3.0.0-py3.5.egg/open_worm_analysis_toolbox/init.py", line 19, in ImportError: No module named 'open_worm_analysis_toolbox.prefeatures'

Here is some information about PATH, PYTHONPATH. aki@ubuntu:~/github/open-worm-analysis-toolbox/examples$ echo $PATH /home/aki/miniconda2/bin:/home/aki/bin:/home/aki/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin aki@ubuntu:~/github/open-worm-analysis-toolbox/examples$ echo $PYTHONPATH /home/ubuntu/opencv/build/lib/python3.3/site-packages:

Thank you for your help.

Best

On Mon, Jun 12, 2017 at 10:45 AM, akihiro mori [email protected] wrote:

Thanks, Jim

The correct code should presumably be: from open-worm-analysis-toolbox import user_config

When I run test_setup.py script with the line, I have still got error message. aki@ubuntu:~/github/open-worm-analysis-toolbox/tools$ python test_setup.py File "test_setup.py", line 121 from open-worm-analysis-toolbox import user_config ^ SyntaxError: invalid syntax

Am I missing some key step to avoid this issue? Just in case, I am currently use python 3 (3.5.3) on ubuntu 16.04.

Please let me know. Thanks

On Mon, Jun 12, 2017 at 4:27 AM, Jim Hokanson [email protected] wrote:

It appears as though 1) we named our package with hyphens instead of underscores :/ and 2) auto code cleanup translated the package name into separate names with subtraction signs

The correct code should presumably be: from open-worm-analysis-toolbox import user_config

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openworm/open-worm-analysis-toolbox/issues/205#issuecomment-307682277, or mute the thread https://github.com/notifications/unsubscribe-auth/AbZ31VIk7uXu7T9grVsq29YpfG-pDKn1ks5sDLAWgaJpZM4NvaOO .

[JimHokanson]

I just looked at it a bit more closely. The package is actually named with underscores: from open_worm_analysis_toolbox import user_config

The other errors I see are:

1. The examples directory is not on your path. Apparently this was made a package as well.
2. OpenCV is not properly installed, thus the cv2 error.

[akihirom2017]

Thanks for your help, Jim

from open_worm_analysis_toolbox import user_config Thanks. I fixed it. and there is not more error on the section.

OpenCV is not properly installed, thus the cv2 error. Yes, I have installed cv2 (3.2.0) and work. the section do not give me more error message

The examples directory is not on your path. Apparently this was made a package as well. This is bit confusing me. the location of example_data is aki@ubuntu:~/example_data$ pwd /home/aki/example_data So, I think the EXAMPLE_DATA_PATH in open_worm_analysis_toolbox/user_config.py should look like: EXAMPLE_DATA_PATH = r'/home/aki/example_data' (I just followed the example although I do not know the meaning of 'r' in front of path)

The other parameters in the same file remain. (e.g. PERFORM_SLOW_ECCENTRICITY_CALC = True)

Then I run python3 test_setup.py; it gave me:

aki@ubuntu:~/github/open-worm-analysis-toolbox/tools$ python test_setup.py
Python version: 3.6.1 |Continuum Analytics, Inc.| (default, May 11 2017,
13:09:58)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
The version used is acceptable, since it is >=2.7
'Unable to import shapely
Unable to import open-worm-analysis-toolbox/user_config.py module
Traceback (most recent call last):
  File "test_setup.py", line 125, in <module>
    if not hasattr(user_config, 'EXAMPLE_DATA_PATH'):
NameError: name 'user_config' is not defined

of course, I did not move any file from the original folder.

why this line cause error? Please let me know.

Thanks

On Mon, Jun 12, 2017 at 2:07 PM, Jim Hokanson [email protected] wrote:

I just looked at it a bit more closely. The package is actually named with underscores: from open_worm_analysis_toolbox import user_config

The other errors I see are:

1. The examples directory is not on your path. Apparently this was made a package as well.
2. OpenCV is not properly installed, thus the cv2 error.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openworm/open-worm-analysis-toolbox/issues/205#issuecomment-307783817, or mute the thread https://github.com/notifications/unsubscribe-auth/AbZ31UuaPPTPGFRw7o4SBHSaa3neFN5Qks5sDTgugaJpZM4NvaOO .

[MichaelCurrie]

Hi @akihirom2017 are you sure you have created a file called user_config.py? That error seems to say that you do not have such a file in your open-worm-analysis-toolbox/open_worm_analysis_toolbox folder.

From the installation instructions:

In the open-worm-analysis-toolbox/open-worm-analysis-toolbox folder there should be a file user_config_example.txt. Rename this file as user_config.py. It will be ignored by GitHub since it is in the .gitignore file. So in user_config.py, specify your computer's specific Google Drive root directory and other settings.

If you can confirm this and that you still have the issue I will try to replicate the error on my end to see if a bug was introduced inadvertently recently.

Thanks again for all your work on getting this to work. Having others go through the installation will help us to simplify the process for the future. :)

[akihirom2017]

Thank you.

are you sure you have created a file called user_config.py? Yes, I think so.

The file location is: aki@ubuntu:~/Documents/Programs/ open-worm-analysis-toolbox/open_worm_analysis_toolbox$ ls config.py prefeatures travis_config.txt utils.py features __pycache__ user_config.py version.py __init__.py statistics user_config.py~

and the inside of user_config.py is: `aki@ubuntu:~/Documents/Programs/open-worm-analysis-toolbox/open_worm_analysis_toolbox$ cat user_config.py #!/usr/bin/env python

-- coding: utf-8 --

""" user_config.py: a module to specify installation-specific settings for the open-worm-analysis-toolbox repository

This file is in the .gitignore file for the open-worm-analysis-toolbox repo, so you can put configuration settings here that apply to just your computer.

"""

#EXAMPLE_DATA_PATH = r'/home/ubuntu/example_data' EXAMPLE_DATA_PATH=r'/home/aki/example_data'

There is a step that is very slow that can be disabled

if you are just debugging other parts of the code.

For production, this should be True.

PERFORM_SLOW_ECCENTRICITY_CALC = True`

[MichaelCurrie]

@akihirom2017 it sure looks like you have it just the way it is supposed to! It must be something else. Tomorrow I will try to replicate your issue. Thanks again

[akihirom2017]

I think I found what the cause is. This may not be the solution (& I am sure you can find a better way.)

In my way, I made two things changed.

(1) location of user_config.py. the "user_config.py" originally created under /open-worm-analysis-toolbox/open-worm-analysis-toolbox. But I moved the user_config.py to /open-worm-analysis-toolbox (one directory above)

(2) modifying a line in test_setup.py in tools

#from open_worm_analysis_toolbox import user_config import user_config

Once I have changed these two things, I successfully ran the python tools/test_setup.py.

aki@ubuntu:~/github/open-worm-analysis-toolbox$ python tools/test_setup.py Python version: 3.6.1 |Continuum Analytics, Inc.| (default, May 11 2017, 13:09:58) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] The version used is acceptable since it is >=2.7 Finished running test_setup.py

[MichaelCurrie]

Hi @akihirom2017, I've run tools/test_setup.py and I found a couple of things were out of date, like testing for shapely which we no longer use. If you try running it now it should work.

One thing to confirm is you should have your user_config.py NOT in the directory above but in /open-worm-analysis-toolbox/open-worm-analysis-toolbox/user_config.py Here's my setup:

[ver228]

Sometimes pip breaks conda...

[akihirom2017]

Thanks for the tips, Michel and ver228

I think I could installed open-worm-analysis toolbox on ubuntu. however, the same problem reimans to run the program.

I mean, I following the usage example describe in https://github.com/ openworm/open-worm-analysis-toolbox.

aki@ubuntu:~/Documents/Programs/open-worm-analysis- toolbox/open_worm_analysis_toolbox/prefeatures$ python Python 3.6.1 |Continuum Analytics, Inc.| (default, May 11 2017, 13:09:58) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux Type "help", "copyright", "credits" or "license" for more information.

import open_worm_analysis_toolbox as mv bw = mv.BasicWorm.from_schafer_file_factory("example_contour_ and_skeleton_info.mat") Traceback (most recent call last): File "", line 1, in File "/home/aki/Documents/Programs/open-worm-analysis-toolbox/ open_worm_analysis_toolbox/prefeatures/basic_worm.py", line 212, in from_schafer_file_factory with h5py.File(data_file_path, 'r') as h: File "/home/aki/miniconda/lib/python3.6/site-packages/h5py/_hl/files.py", line 271, in init fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr) File "/home/aki/miniconda/lib/python3.6/site-packages/h5py/_hl/files.py", line 101, in make_fid fid = h5f.open(name, flags, fapl=fapl) File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/home/ilan/minonda/conda-bld/h5py_1490028290543/work/h5py/_objects.c:2846) File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/home/ilan/minonda/conda-bld/h5py_1490028290543/work/h5py/_objects.c:2804) File "h5py/h5f.pyx", line 78, in h5py.h5f.open (/home/ilan/minonda/conda-bld/h5py_1490028290543/work/h5py/h5f.c:2123) OSError: Unable to open file (File signature not found)

it seems that a file (example_contour_and_skeleton_info.mat) cannot be recognised even though it is located in example_data under my home directory (I also changed the corresponding line in user_config.py (e.g. EXAMPLE_DATA_PATH = r'/home/aki/example_data')).

aki@ubuntu:~/example_data$ ls /home/aki/example_data/* /home/aki/example_data/example_contour_and_skeleton_info.mat /home/aki/example_data/example_video_feature_file.mat /home/aki/example_data/example_video_norm_worm.mat

So, I back to the script which provided by ver228 previously. I named the script test.py.

aki@ubuntu:~/Documents/Programs/open-worm-analysis-toolbox/open_worm_analysis_toolbox/prefeatures$ python test.py

Although I have got many warnings in the result, it seems to be fine to finish the calculation to get "wf" at the end of the script.

Now, I would like to use the wf data to create a figures.

I have done two things: (1) I assumed I have got wf in usage example. I added a line in the test.py script as follow.

` ###This is the last line of ver228's script wf = mv.WormFeatures(nw)

add this line followed by usage example

experiment_histograms = mv.HistogramManager([wf,wf]) ` here is the result: Traceback (most recent call last): File "test.py", line 145, in experiment_histograms = mv.HistogramManager([wf,wf]) File "/home/aki/Documents/Programs/open-worm-analysis-toolbox/open_worm_analysis_toolbox/statistics/histogram_manager.py", line 104, in init new_histogram_set = self.init_histograms(worm_features) File "/home/aki/Documents/Programs/open-worm-analysis-toolbox/open_worm_analysis_toolbox/statistics/histogram_manager.py", line 203, in init_histograms return np.array([Histogram.create_histogram(f) for f in worm_features]) File "/home/aki/Documents/Programs/open-worm-analysis-toolbox/open_worm_analysis_toolbox/statistics/histogram_manager.py", line 203, in return np.array([Histogram.create_histogram(f) for f in worm_features]) File "/home/aki/Documents/Programs/open-worm-analysis-toolbox/open_worm_analysis_toolbox/statistics/histogram.py", line 149, in create_histogram return cls(feature) File "/home/aki/Documents/Programs/open-worm-analysis-toolbox/open_worm_analysis_toolbox/statistics/histogram.py", line 113, in init self.compute_covering_bins() File "/home/aki/Documents/Programs/open-worm-analysis-toolbox/open_worm_analysis_toolbox/statistics/histogram.py", line 283, in compute_covering_bins assert(min_data >= self.bin_boundaries[0] - bin_width) IndexError: index 0 is out of bounds for axis 0 with size 0

(2) I assumed I have got control worm (I just pretend control_worm = wf) so the script has modified as follow:

` wf = mv.WormFeatures(nw)

I am assuming control_worms = wf

control_histograms = mv.HistogramManager(wf) ` Then, what I got was: Traceback (most recent call last): File "test.py", line 146, in control_histograms = mv.HistogramManager(wf) File "/home/aki/Documents/Programs/open-worm-analysis-toolbox/open_worm_analysis_toolbox/statistics/histogram_manager.py", line 104, in init new_histogram_set = self.init_histograms(worm_features) File "/home/aki/Documents/Programs/open-worm-analysis-toolbox/open_worm_analysis_toolbox/statistics/histogram_manager.py", line 203, in init_histograms return np.array([Histogram.create_histogram(f) for f in worm_features]) TypeError: 'Length' object is not iterable

Do you have any idea to avoid this issue?

Thank you very much for your help.

Best

[MichaelCurrie]

Hi @akihirom2017 I have been at the worm meeting this past week and I am traveling for the next two days but I will respond when I get back. Thanks!

[akihirom2017]

Thanks, Michael.

I really appreciate about this.

Best

[akihirom2017]

Is there any update? or can I try different datasets rather than example_contour_and_skeleton_info.mat or the files shown in example section?

Thank you very much.

Sincerely

[MichaelCurrie]

Sorry for the delay, I will hope to look at this on the weekend in 3 or 4 days. Otherwise perhaps @JimHokanson can help.

[akihirom2017]

Thank you.

Best regards

On Wed, Jul 5, 2017 at 3:50 PM, Michael Currie [email protected] wrote:

Sorry for the delay, I will hope to look at this on the weekend in 3 or 4 days. Otherwise perhaps @JimHokanson https://github.com/jimhokanson can help.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openworm/open-worm-analysis-toolbox/issues/205#issuecomment-313126464, or mute the thread https://github.com/notifications/unsubscribe-auth/AbZ31f7US7iW2DRUOACwV0SstWf5abVmks5sK6KmgaJpZM4NvaOO .

[JimHokanson]

@akihirom2017

Sorry for the delay, things have been way too hectic on my end.

Regarding the file, our code doesn't automatically append the example path to your inputs. The example path is just used to facilitate running of the examples, which internally look for the example path and then append various file names, assuming those to be present in that folder.

For example in "examples/get_pandas_features.py"

# Set up the necessary file paths for file loading
    base_path = os.path.abspath(mv.user_config.EXAMPLE_DATA_PATH)
    matlab_generated_file_path = os.path.join(
        base_path, 'example_video_feature_file.mat')
    data_file_path = os.path.join(base_path, "example_video_norm_worm.mat")

    # Load the normalized worm from file
    nw = mv.NormalizedWorm.from_schafer_file_factory(data_file_path)

To run your code, specify the full path to the file:

import open_worm_analysis_toolbox as mv
bw = mv.BasicWorm.from_schafer_file_factory("/home/aki/example_data/example_contour_and_skeleton_info.mat")

I'm not sure what is in the contents of test.py. Is this in our repo? My guess is most likely something is not correct further upstream and it is manifesting itself as an error downstream, for example if there are no valid worm features for some reason.