AllenSDK icon indicating copy to clipboard operation
AllenSDK copied to clipboard

Release updated VBO dataset

Open danielsf opened this issue 2 years ago • 2 comments

Tasks

  • [ ] Generate new metadata and NWB files for VBO released-sessions using something like VisualBehaviorOphysProjectCache.from_lims(data_release_date=['2021-03-25', '2021-08-12'])
  • [ ] Perform whatever validation Marina and company wanted done on a new release.
  • [ ] Upload to S3 using informatics_data_release_tools
  • [ ] Update readme for data release (see below) making special note of sessions that were updated in the course of #2395
  • [ ] Try to reconstruct all of the changes to the data that have been made since summer 2021
  • [ ] Respond to issue #2366 that we have released new data with a fix
  • [ ] Talk to Adam about the PDF link in the documentation that needed to be updated; see if we think it works.

VBO release readme is here (need to find the source code for this) https://allensdk.readthedocs.io/en/latest/visual_behavior_optical_physiology.html

danielsf avatar May 10 '22 21:05 danielsf

Here is a list of issues that are candidates to resolve in an updated VBO release (roughly in priority order):

Affects cache tables only:

VB cells table has more experiments than experiments table #2242

add imaging_plane_group and imaging_plane_group_count to cache tables #2219

Affects NWB files:

reads session type from pkl file instead of from mtrain #2488
(This issue tracks several instances of errors regarding the session_type - we should make sure that the fix is applied to the cache tables AND the metadata attribute of SDK BehaviorOphysExperiment and BehaviorSession objects (which comes from the NWB file I assume))

These three are also related to session_type / image_set issues, but are on me to get data updated in lims so that the re-released NWB files are accurate: Remove incorrect session from Visual Behavior data release #2215 mismatch in image set listed in SDK dataset and pkl file for one VisualBehaviorMultiscope session #2216 Visuasl Behavior sessions with incorrect image set #2202

Filter pupil_width and other eye tracking columns similarly to pupil_area #2046

Omission times are logged incorrectly by 1 frame #1539

mismatch in data types for mouse_id column for from_s3_cache vs from_lims methods #2093

ncorrect naming of key in metatdata. 'experiment_container_id' should be 'ophys_container_id' #2197

remove unecessary columns in stimulus_presentations attribute of BehaviorOphysSession and BehaviorSession #1946

consistent naming of change column across tables #2180

standardize column name for auto reward across AllenSDK tables #1949

consistency in time columns of stimulus_presentations table #2195

Would be very nice to have:

[dd licks, rewards, run speed and pupil to stimulus presentations #2226

(these 4 are related) functions to work with additional Visual Behavior stimuli #1191

Add fingerprint stimulus to stim table #1190

Add receptive field movies to Visual Behavior stim table #1802

Receptive field mapping sessions throw KeyError when retrieving stimulus presentations #1759

matchings avatar Jul 06 '22 20:07 matchings

Also potentially relevant: https://github.com/AllenInstitute/AllenSDK/issues/2213

matchings avatar Jul 06 '22 20:07 matchings