MIN1PIPE (reads "minipipe")

A MINiscope 1-photon-based Calcium Imaging Signal Extraction PIPEline.

MIN1PIPE is a fully automatic, Matlab-based toolbox, solving the full range problems in 1-photon calcium imaging in one package: data enhancement → movement morrection → signal extraction. It requires minimal parameter-tuning and integrates the semi-auto options. Each inidividual module can also be easily adapted for the 2-photon imaging setting.

Contents

1. Updates
2. Introduction and Features
3. Dependencies
4. Usage
5. Dataset
6. Custom Data
7. Practical Suggestions
8. References
9. Questions

Updates

3/25/2022 New version released (v3.1 & v4.0.1 (beta)): v4.0.1: MIN1PIPE now automatically detects se and spatialr. Please refer to the release notes for syntax. v3.1: latest stable version, with relatively liberal standards of selecting ROIs. Feedbacks regarding the bugs and/or suggestions are welcome.

11/19/2019 New version released (v2-alpha): new neural enhancing module with noise suppression: reduce the effect of sharp background structures. Add new output variable dff for dF/F. Feedbacks regarding the bugs and/or suggestions are welcome.

11/01/2018 New version released: updated movement correction module - balanced the running time for extremely large or shading videos; updated neural enhancing module - introduced dirt-cleaning function for dirty videos (potentially with dirts on the imaging sensor); updated seeds cleansing module - better seeds cleansing filters for seeds selection. Feedbacks regarding the bugs and/or suggestions are welcome.

09/26/2018 Updated data loading interface, for Doric scope videos with bit depth of 16.

07/16/2018 Patch version released. The program auto-detects available memory and processes data in chunk. Integrate fast read&write and memory mapping at key steps. The toolbox is undergoing some beta tests, so please expect frequent updates recently.

08/02/2018 Added guide for manual_seeds_select and real_neuron_select under the secion Usage.

08/16/2018 Added reading interface for .mat file format.

08/23/2018 Fixed bug of manual_seeds_select.

Introduction and Features

MIN1PIPE contains the following three modules:

• Neural Enhancing: remove spatial noise and then adaptively remove non-neural background in the field of view in a framewise manner.
• Movement Correction: remove field of view movement with a specially designed hierarchical nonrigid movement correction module (integrating KLT Tracker and LogDemons deformation registration method), which is free of assumption about movement type and amplitude.
• Neural Signal Extraction: identify neuronal ROIs and corresponding calcium traces with minimal false positive rates (incorporating GMM, LSTM as true neuron selector and modified CNMF as spatiotemporal calcium signal identifier)

Additional Features

• Semi-auto options: we also provide semi-auto options, including
  • Automated manual seeds selection module: for users who want to manually select seeds of neuron ROIs that will result in ZERO false positives
  • Post-process exclusion of "bad" neural components
• RNN-LSTM classifier training module: we provide a module of Recursive Neural Network (RNN) with Long-short Term Memory (LSTM) structure fully implemented in Matlab with which the users can train their specific calcium-dynamical classifier.

Dependencies

This Matlab implementation has the following dependencies (included under utilities):

• Modified CNMF
  • CVX: ~~for OS other than Windows, users should download correpsonding CVX toolbox and replace the folder CVX in the codes.~~ the package automatically determines the platform, then downloads and installs the appropriate version.

Additional Matlab toolboxes:

• Computer Vision System Toolbox
• Curve Fitting Toolbox
• Fuzzy Logic Toolbox
• Image Processing Toolbox
• Optimization Toolbox
• Parallel Computing Toolbox
• Signal Processing Toolbox
• Statistics and Machine Learning Toolbox
• Symbolic Math Toolbox
• Wavelet Toolbox

Other modified functions adapted from others are credited the original sources in code documentations.

Usage

• Do not add the package to the Matlab path before running, the package will add everything automatically.
• Download/clone the git repository of the codes
• Open Matlab and set the MIN1PIPE folder as the current folder in Matlab
• Run min1pipe.m.
  • The code automatically sets the package to the path, and processes the data the user specifies.
  • For argument options, please see min1pipe.m
• The only manual intervention is to select the data for processing.
  • A modal dialogue is popped up right after the execution of the code, and the users can select the folder containing the data.
  • For multi-video datasets automatically divided by the data acquisition softwares, only the first (or a random video of the session) needs choosing.
  • The algorithm will automatically judge the format of the datasets.
  • Currently support: .avi, .tif and .tiff.

Manual Options

• manual_seeds_select

  • A figure will pop up, including: max progection of the video after the neural enhancing and movement correction.
  • The user needs to click on the center of potentail neurons once
  • A red dot appears at the same position indicating that the seed has been successfully selected.
  • Press 'Enter' if all seeds are selected.

• real_neuron_select

  • The goal of this function is to collect the set of false-positive ROIs that the user selects.
  • A figure will pop up, including: 2 X max progection of the video after the neural enhancing and movement correction + 10 calcium traces/session.
  • The user can freely click on the calcium traces, or the max projection map (bottom max map) to indicate the false positive that should be removed.
  • To do this, move the mouse cursor at the corresponding position, until a red contour of that ROI is shown on the map (bottom max map).
  • Click to select that ROI, either on the calcium trace area or the max map area.
  • A red dot of the center of the selected ROI is shown on the top max map.
  • To terminate the current session, press 'Enter'.
  • The function will automatically open the next session, until all sessions are looped.
  • A .mat file with 'filename_refined.mat' will be created in the same folder with refined key output variables.

Key Parameters:

• Fsi: frame rate of original video
• Fsi_new: frame rate of temporally downsampled video
• spatialr: spatial downsampling scalar, e.g. 0.5 for a spatial downsampling rate of 2 (now optional)
• se: structure element size, estimated from typical half-neuron size after spatial downsampling (now optional)

Procedure Parameters:

• ismc: whether use movement correction module
• flag: whether use automatic or semi_automatic seeds selection
• ~~isvis: whether visualize after processing, including results of each step~~
• ifpost: whether use post-process

Key Outputs:

• roifn: processed vectorized ROI footprints; contains single cell in each column (vectorized spatial map)
• sigfn: processed calcium traces of corresponding ROIs; contains single cell in each row (calcium trace)
• spkfn: spike train inferred from refine_sig.m
• ~~roifnr: processed vectorized ROI footprints without calcium deconvolution~~
• ~~sigfnr: processed calcium traces without calcium deconvolution, meaning "no artificial cleaning"~~
• seedsfn: ROI centers in pixel coordinates; indices of all ROIs and be converted to (h, w) position using ind2sub
• Params: used parameters
• reg: data after neural enhancing (and movement correction), saved for reprocessing

Other fixed preset parameters can be found in min1pipe.m, and the table in the paper.

Dataset

As a demo, we demonstrate the use of 1-photon calcium imaging video recorded with UCLA miniscope:

demo_min1pipe.m

The same code can also be adapted to custom scripts for the processing.

Here are the visualization of the demo dataset processing:

Custom Data

To use the code on a custom dataset, no specific requirements are needed. The processed data and the data after movement correction are saved in the same folder of the raw data in ".mat" format, with "_data_processed" and "_reg" as endings separately.

If post-process is selected, there will be an additional ".mat" file created with "_data_processed_refined".

Practical Suggestions

• Data tips
  • Data should be arranged in sessions.
  • Each session contains multiple videos automatically divided by the recording softwares.
  • For Inscopix data, data are divided and renamed with a pattern of adding "-" + "indices". We suggest sticking to this format for .tif and .tiff data.
  • For UCLA miniscope, data are named with "msCam"/letter(s) + "indices", and we suggest sticking to this format for .avi data.
  • For best practice, remove apparent artifects such as bright edges of the grin lenses, even though the algorithm can handle these conditions.
  • For sessions with only a few neurons and possibly huge artifects/contaminations, semi-auto seeds selection can be considered at first hand;
  • Usually it poses more difficulties (more time spent/less accuracy) on the movement correction module under such (few neurons/key information) circumstances.
  • For sessions with small translational movement, other traditional rigid correction methods can also be used. The option of rigid correction would also be integrated in future releases.
• Software
  • Matlab R2017 and later.
• Hardware
  • Better hardwares are always preferred, for professional data analysis such as in the regular lab environment, even though the algorithms can be adapted to personal computers.
  • Typically, ~4 times of the size of a single session dataset (after downsampling) of memory is recommanded for processing.
  • ~~Memory mapping will be integrated in future~~ Future arrived.
  • Matlab parallel computing only supports NVIDIA graphic cards.
  • Patch version requires sufficient space in local hard disk (especially the disk containing the dataset currently running).

References

Updates

Please cite the MIN1PIPE journal paper if it helps your research.

@article{lu2018MIN1PIPE,
  title={MIN1PIPE: A Miniscope 1-photon-based Calcium Imaging Signal Extraction Pipeline.},
  author={Lu, J., Li, C., Singh-Alvarado, J., Zhou, Z., Fröhlich, F., Mooney, R., & Wang, F.},
  journal={Cell reports},
  volume={23},
  number={12},
  pages={3673--3684},
  year={2018},
  publisher={Elsevier}
}

Archives

The paper is now accepted by Cell Reports.

Please cite the MIN1PIPE paper if it helps your research.

@article{lu2018MIN1PIPE,
  title={MIN1PIPE: A Miniscope 1-photon-based Calcium Imaging Signal Extraction Pipeline.},
  author={Lu, J., Li, C., Singh-Alvarado, J., Zhou, Z., Fröhlich, F., Mooney, R., & Wang, F.},
  journal={bioRxiv, 311548},
  year={2018}
}

or the related arXiv version:

@article{lu2017seeds,
  title={Seeds Cleansing CNMF for Spatiotemporal Neural Signals Extraction of Miniscope Imaging Data},
  author={Lu, Jinghao and Li, Chunyuan and Wang, Fan},
  journal={arXiv preprint arXiv:1704.00793},
  year={2017}
}

Questions?

Please email to [email protected] for additional questions.