Make the data more accessible to other users

[astraw]

This would be beneficial for all flydra/humdra users and seems like it would help get Steve up and running with the various data formats the flydra currently produces. Furthermore, it would support the goal of allowing us (the biologists) to focus on gathering the data and doing the experiments, and quite possibly encourage computer vision and machine learning people, for example, to help us analyze our data. Currently flydra emits realtime output in .h5 files with both the 2D data and rough 3D data and .ufmf movie files, which are then analyzed and cleaned up through a series of further scripts to produce more final data including things like Kalman smoothed 3D position and orientation trajectories. Documenting the intermediate and final file formats (and potentially what the various processing steps do) would be a really useful goal.

[AndreaCensi]

Here is my proposal for how to better organize the DB structure:

https://github.com/strawlab/flydra_db/wiki

(I'll send you a DB to play with)

Note that this does not get into the actual data format used, just how to organize the collection of data in tables, groups, etc.

[astraw]

A canonical final flydra output format (FFOF) has now been invented! The output of "flydra_analysis_data2smoothed --hdf5 --time-data $INPUT.h5 $INPUT.h5" is actually documented. This is a very simple HDF5 schema, and it is documentated at http://strawlab.org/schemas/flydra/1.1/ This can be expanded to include other output data (e.g. orientations), but the plan is that what is there stays as-is.