ciatah icon indicating copy to clipboard operation
ciatah copied to clipboard

Error when trying to view miniscope avi

Open hsw28 opened this issue 3 years ago • 60 comments

I am trying to view a .avi file created using miniscope software. I am on a mac and have tried in matlab 2016b and 2018b. I keep getting the error:

Error using VideoReader/init (line 619)
Could not read file due to an unexpected error. Reason: Cannot Decode

I am able to open the video in VLC though not in quicktime. I thought perhaps it was an issue with matlab 2018b based on this thread but going down to 2016b also did not work. Any advice? Thanks!

hsw28 avatar Dec 01 '20 01:12 hsw28

Couple question to help narrow down the issue:

  • Which miniscope software were the AVIs created in?
  • If you press Ctrl+J in VLC to bring up the codec information, could you copy and paste that information or put a screenshot?
    • Or alternatively, paste output from aviinfo('FILENAME'), https://www.mathworks.com/help/matlab/ref/aviinfo.html.
  • Are you able to open it in ImageJ (e.g. as a virtual stack)?

It is possible that we can use ffmpeg or another video processing tool to convert your file to compatible codec for mac (assuming loading works fine on Windows/Linux).

bahanonu avatar Dec 01 '20 11:12 bahanonu

Ok, I figured that initial problem out which is that I couldn't read that codec on my mac with matlab. So I took the video and converted it into a tiff stack using imageJ, and now I can read the file in matlab.

I am now trying to run preprocessing to remove horizontal lines. I have entered these parameters image

>  MAIN______________: {'====================='}
>                       checkConcurrentAnalysis: 0
>                             resetParallelPool: 0
>                               analyzeFromDisk: 0
>                    calcDroppedFramesFromMovie: []
>                    REGISTRATION______________: {'====================='}
>                                      parallel: 1
>                               registrationFxn: 'transfturboreg'
>                              turboregRotation: 0
>                                     RegisType: 1
>                         numTurboregIterations: 1
>                       turboregNumFramesSubset: 5000
>                                      pxToCrop: 14
>                      motionCorrectionRefFrame: 100
>      REGISTRATION_NORMALIZATION______________: {'====================='}
>                         normalizeMeanSubtract: 1
>                normalizeMeanSubtractNormalize: 1
>                     normalizeComplementMatrix: 1
>                                 normalizeType: 'bandpass'
>                              normalizeFreqLow: 70
>                             normalizeFreqHigh: 100
>                         normalizeBandpassType: 'bandpass'
>                         normalizeBandpassMask: 'gaussian'
>                                       SmoothX: 10
>                                       SmoothY: 10
>                                       zapMean: 0
>             MOVIE_NORMALIZATION______________: {'====================='}
>                          filterBeforeRegister: []
>                      saveBeforeFilterRegister: 0
>                      saveFilterBeforeRegister: []
>                        filterBeforeRegFreqLow: 0
>                       filterBeforeRegFreqHigh: 10
>                 filterBeforeRegImagejFFTLarge: 10000
>                 filterBeforeRegImagejFFTSmall: 80
>                              medianFilterSize: 3
>              MOVIE_DOWNSAMPLING______________: {'====================='}
>                          downsampleFactorTime: 4
>                         downsampleFactorSpace: 2
>     IO_and_MOVIE_IDENTIFICATION______________: {'====================='}
>                              inputDatasetName: [1×0 char]
>                             outputDatasetName: [1×0 char]
>                              fileFilterRegexp: 'testvideo'
>                       processMoviesSeparately: 0
>                        loadMoviesFrameByFrame: 0
>                 treatMoviesAsContinuousSwitch: 1
>                         loadMovieInEqualParts: 0
>                                   useParallel: 1
>                              nParallelWorkers: 7
>                  STRIPE_REMOVAL______________: {'====================='}
>                        stripOrientationRemove: 'horizontal'
>                                     stripSize: 7
>                           stripfreqLowExclude: 20
>                          stripfreqHighExclude: 20
>                         stripfreqBandpassType: 'highpass'
>                                  refCropFrame: 100

When it finished running I got this output:

> 
> Elapsed time is 15.546693 seconds.
> Changing calciumImagingAnalysis input HDF5 dataset name ""->""

Yet even thought I requested a new video be saved as HDF5 I do not have that video anywhere, and when the preview opens to see the video there is no video.

However, if in the parameters I choose "save movie to NWB format" with all other parameters the same, an NWB movie does save in the directory and is available to view in imageJ, although the movie is entirely black.

I am unsure if this is something to do with my parameters or how I am using the software (both the HDF5 problem as well as the video being black with NWB). I can upload the tiff if that would make it easier.

Many thanks

hsw28 avatar Dec 01 '20 22:12 hsw28

Good to hear you solved the codec issue.

Does the raw video appear as expected if you run playMovie('PATH_TO_TIFF');?

If you could send a link to the tiff, I could take a quick look and get back with what the issue might be or any settings that need to be changed.

bahanonu avatar Dec 01 '20 22:12 bahanonu

great, thank you!! here is a (temporary) link to the tiff file https://t.co/FvJ25lmDBO?amp=1

hsw28 avatar Dec 01 '20 23:12 hsw28

and yes the raw movie plays

hsw28 avatar Dec 01 '20 23:12 hsw28

Great! Quick question, are the dark stripes in the original video before conversion as well? I've seen this before with data from UCLA miniscopes, can remove to a certain degree using removeStripsFromMovie, see https://bahanonu.github.io/calciumImagingAnalysis/help_stripe_removal/. It's also an option on the 2nd page of modelPreprocessMovie.

image image

bahanonu avatar Dec 01 '20 23:12 bahanonu

the stripes are an artifact from one of the focusing lenses on the miniscope. that was what i was attempting to remove but I wasnt successful (see above). am I using the software incorrectly? thanks again!

hsw28 avatar Dec 01 '20 23:12 hsw28

👍 Good to know. Below is some info on running the movie, I've also downsampled 3x in space as that normally speeds up processing without sacrificing downstream analysis when looking at cells, you can skip that step if needed. Can then run cell extraction (would need to do some optimization for this movie, due to stripes) as below. If anything's confusing, let me know and I can look to update the GUI accordingly.

Processed movie

result-1

Settings

image

Cell extraction

image

modelPreprocessMovie settings

options.turboreg:
                           MAIN______________: {'====================='}
                      checkConcurrentAnalysis: 0
                            resetParallelPool: 0
                              analyzeFromDisk: 0
                   calcDroppedFramesFromMovie: []
                   REGISTRATION______________: {'====================='}
                                     parallel: 1
                              registrationFxn: 'transfturboreg'
                             turboregRotation: 0
                                    RegisType: 1
                        numTurboregIterations: 1
                      turboregNumFramesSubset: 5000
                                     pxToCrop: 14
                     motionCorrectionRefFrame: 100
     REGISTRATION_NORMALIZATION______________: {'====================='}
                        normalizeMeanSubtract: 1
               normalizeMeanSubtractNormalize: 1
                    normalizeComplementMatrix: 1
                                normalizeType: 'bandpass'
                             normalizeFreqLow: 70
                            normalizeFreqHigh: 100
                        normalizeBandpassType: 'bandpass'
                        normalizeBandpassMask: 'gaussian'
                                      SmoothX: 10
                                      SmoothY: 10
                                      zapMean: 0
            MOVIE_NORMALIZATION______________: {'====================='}
                         filterBeforeRegister: 'divideByLowpass'
                     saveBeforeFilterRegister: 0
                     saveFilterBeforeRegister: []
                       filterBeforeRegFreqLow: 0
                      filterBeforeRegFreqHigh: 7
                filterBeforeRegImagejFFTLarge: 10000
                filterBeforeRegImagejFFTSmall: 80
                             medianFilterSize: 3
             MOVIE_DOWNSAMPLING______________: {'====================='}
                         downsampleFactorTime: 8
                        downsampleFactorSpace: 3
    IO_and_MOVIE_IDENTIFICATION______________: {'====================='}
                             inputDatasetName: '/1'
                            outputDatasetName: '/1'
                             fileFilterRegexp: 'testmovie'
                      processMoviesSeparately: 0
                       loadMoviesFrameByFrame: 0
                treatMoviesAsContinuousSwitch: 1
                        loadMovieInEqualParts: 0
                                  useParallel: 1
                             nParallelWorkers: 7
                 STRIPE_REMOVAL______________: {'====================='}
                       stripOrientationRemove: 'horizontal'
                                    stripSize: 7
                          stripfreqLowExclude: 20
                         stripfreqHighExclude: 20
                        stripfreqBandpassType: 'highpass'
                                 refCropFrame: 100

bahanonu avatar Dec 02 '20 00:12 bahanonu

Ahh that looks awesome!! thank you so much. I am still having some issues that are probably just due to my inaptitude at the program. I attempted to use the same parameters as you did above:

First, I can't seem to get out of this window even with double clicking image

If I remove any of the movement adjustment processing steps it seems to work ok!

I dont have these toolboxes installed (secondary toolboxes), could it be something with them? As I said before probably me just not using the software correctly.

2nd tier toolbox check (not required for main pre-processing pipeline). Warning: Toolbox license available but toolbox is NOT installed. Computer Vision System Toolbox (Video_and_Image_Blockset). Warning: Toolbox license available but toolbox is NOT installed. Financial Toolbox (Financial_Toolbox).

hsw28 avatar Dec 02 '20 01:12 hsw28

and thanks again for all your help!! this has been a good day

hsw28 avatar Dec 02 '20 01:12 hsw28

Sure thing! Regarding that window, you are supposed to double click to move on (just a function of how imrect works in Matlab, may change that to be a button going forward).

Is the image you showed representative of where you've been selecting for cropping to determine motion correction coordinates? What is likely happening (if you send me the _preprocess.log file in the processing_info sub-folder I can confirm) is that coordinates outside the movie are being passed to downstream functions, leading to the motion correction functions to crash. Preferably you'd have a rectangle like below (I'd added a constraint to prevent this from happening in the GUI).

Those toolboxes aren't needed for the pre-processing (but looks like you already have the license, so you could go ahead and install them using the Add-On explorer).

image

bahanonu avatar Dec 02 '20 02:12 bahanonu

@hsw28 Got email notification of your message but don't see it here. Could you send me the .log file in the processing_info folder? Will help me debug. Thanks!

bahanonu avatar Dec 02 '20 21:12 bahanonu

Sorry, I realized my error right after I posted the message! working through it now :)

hsw28 avatar Dec 02 '20 22:12 hsw28

OK, so everything seems to work if I do a single video, but if I try to do a folder with 18 videos I run into this issue:

> 00
> -------
> pre-allocating single ones matrix...
> @@@@@@@
> Error using ones
> Requested 608x608x17871 (24.6GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long time and cause MATLAB to become unresponsive. See array size limit or preference panel for more information.
> 
> Error in loadMovieList (line 390)
> 				outputMovie = ones([xDimMax yDimMax zDimLength],imgClass);
> 
> Error in calciumImagingAnalysis/modelPreprocessMovieFunction (line 628)
> 					thisMovie = loadMovieList(movieList,'convertToDouble',0,'frameList',thisFrameList,'inputDatasetName',options.datasetName,'treatMoviesAsContinuous',options.turboreg.treatMoviesAsContinuousSwitch,'loadSpecificImgClass','single');
> 
> Error in calciumImagingAnalysis/modelPreprocessMovie (line 37)
> 	obj.modelPreprocessMovieFunction('folderListPath',folderListInfo,'fileFilterRegexp',options.fileFilterRegexp,'datasetName',options.datasetName,'frameList',[]);
> 
> Error in calciumImagingAnalysis/runPipeline (line 202)
> 				obj.(thisFxn{1});
> 
> Error in calciumImagingAnalysis/display (line 817)
> 			obj.runPipeline;
> @@@@@@@
> pre-allocating movies to display...
> +++++++
> no movie!
> --------------------------------------------------------------
> MIJ 1.3.6-fiji: Matlab to ImageJ Interface
> --------------------------------------------------------------
> More Info: http://bigwww.epfl.ch/sage/soft/mij/
> Help: MIJ.help
> JVM> 1.3.6-fiji
> JVM> Version: 1.8.0_152
> JVM> Total amount of memory: 246784 Kb
> JVM> Amount of free memory: 108079 Kb
> ImageJ> Version:1.50e
> ImageJ> Memory:144MB of 2325MB (6%)
> ImageJ> Directory plugins: /Users/Hannah/Programming/calciumImagingAnalysis-master/_external_programs/Fiji.app/plugins/
> ImageJ> Directory macros: /Users/Hannah/Programming/calciumImagingAnalysis-master/_external_programs/Fiji.app/macros/
> ImageJ> Directory luts: /Users/Hannah/Programming/calciumImagingAnalysis-master/_external_programs/Fiji.app/luts/
> ImageJ> Directory image: Not specified
> ImageJ> Directory imagej: /Users/Hannah/Programming/calciumImagingAnalysis-master/_external_programs/Fiji.app/
> ImageJ> Directory startup: /Users/Hannah/Programming/calciumImagingAnalysis-master/_external_programs/Fiji.app/
> ImageJ> Directory home: /Users/Hannah/
> --------------------------------------------------------------
> Status> ImageJ is running.
> --------------------------------------------------------------
> Warning: The DrawMode property will be removed in a future release. Use the SortMethod property
> instead. 
> > In msgbox (line 284)
>   In calciumImagingAnalysis/modelPreprocessMovieFunction>playOutputMovies (line 2122)
>   In calciumImagingAnalysis/modelPreprocessMovieFunction (line 984)
>   In calciumImagingAnalysis/modelPreprocessMovie (line 37)
>   In calciumImagingAnalysis/runPipeline (line 202)
>   In calciumImagingAnalysis/display (line 817) 
> +++++++
> no movie!
> ImageJ instance ended cleanly
> 
> ostruct = 
> 
>   struct with fields:
> 
>         folderList: {'/Volumes/VERBATIM/Miniscope/tiff'}
>     savedFilePaths: {[]}
>        fileNumList: {}
> 
> 
> options = 
> 
>   struct with fields:
> 
>                    showFigures: 1
>                 folderListPath: {'/Volumes/VERBATIM/Miniscope/tiff'}
>        checkConcurrentAnalysis: 0
>     concurrentAnalysisFilename: '_currentlyAnalyzingFolderCheck.mat'
>        processMoviesSeparately: 0
>                  processMovies: 1
>            loadOptionsFromFile: 0
>        turboregNumFramesSubset: 5000
>                   turboregType: 'preselect'
>                       turboreg: [1×1 struct]
>                      dfofMovie: 1
>                       dfofType: 'dfof'
>               downsampleFactor: 4
>                       pxToCrop: 14
>               fileFilterRegexp: 'mov'
>                    inputPCAICA: 0
>                    askForPCICs: 0
>                      frameList: []
>                    datasetName: '/1'
>              outputDatasetName: '/1'
>                   refCropFrame: 100
>                   deflateLevel: 1
>                 supportedTypes: {'.h5'  '.hdf5'  '.tif'  '.tiff'  '.avi'  '.isxd'}
>     calcDroppedFramesFromMovie: []
>                     saveMovies: 0
>                  saveDfofMovie: 0
>                  turboregMovie: 1
>                 normalizeMovie: 0
>                downsampleMovie: 1
>                  logFileRegexp: '(recording.*.(txt|xml)|.*_metadata.mat)'
>         nwbSettingsDatasetname: '/general/optophysiology/imaging_plane/description'
> 
> Elapsed time is 5.998427 seconds.
> Changing calciumImagingAnalysis input HDF5 dataset name "/1"->"/1"
> 
> >> 

hsw28 avatar Dec 02 '20 22:12 hsw28

I realize this might just be a pure memory issue. Is there a way to avoid this, such as by down sampling (time and space) extensively before doing any other processing?

hsw28 avatar Dec 02 '20 22:12 hsw28

also what cell extraction method/parameters did you use above? thanks again

hsw28 avatar Dec 02 '20 22:12 hsw28

Also, I am using the same parameters as you are above, on the same video, and my post processing video looks completely different. Here is a gif from it and you can see there is like no contrast or anything remotely like your gif :)

test

hsw28 avatar Dec 02 '20 22:12 hsw28

Yep, you can downsample in advance using several ways below. In the near future this will be seamless (modelPreprocessMovie will read from disk like other parts of ciatah pipeline to get around memory issues).

option 1

Use modelPreprocessMovie, run first using the below parameters then run again on the downsampled movies.

image

Change the settings so it will process each movie separately as below:

image

option 2

Use modelDownsampleRawMovies module before modelPreprocessMovie, see:

  • https://bahanonu.github.io/calciumImagingAnalysis/pipeline_detailed/#spatially-downsample-raw-movies-or-convert-to-hdf5-with-modeldownsamplerawmovies

option 3

Via the command-line using the below code or see:

  • https://github.com/bahanonu/calciumImagingAnalysis/blob/master/%2Bciapkg/%2Bdemo/cmdLinePipeline.m#L36-L37
thisFileList = getFileList('PATH_TO_FOLDER');
nFiles = length(thisFileList);
for fileNo = 1:nFiles
	thisFilePath = thisFileList{fileNo};
	
	% Downsample in space
	inputMovieD = downsampleMovie(thisFilePath,'downsampleDimension','space','downsampleFactor',4);
	
	% Downsample in time
	inputMovieD = downsampleMovie(inputMovieD,'downsampleDimension','time','downsampleFactor',4);
	
	% Save out as HDF5 file
	[path,file,ext] = fileparts(thisFilePath);
	saveMatrixToFile(inputMovie,[path filesep file '_dsSpaceTime.h5']);
end

bahanonu avatar Dec 02 '20 22:12 bahanonu

thank you thank you!! Sorry for all the questions, just want to make sure I thoroughly understand everything if I will be using this for my analysis pipeline. Sorry also that I am cross-talk asking new questions as you answer

hsw28 avatar Dec 02 '20 22:12 hsw28

Also, I am using the same parameters as you are above, on the same video, and my post processing video looks completely different. Here is a gif from it and you can see there is like no contrast or anything remotely like your gif :)

test

Great! If you adjust the contrast in ImageJ or in Matlab (press J in playMovie interface), should look similar to the movie I'd attached :D

bahanonu avatar Dec 02 '20 22:12 bahanonu

thank you thank you!! Sorry for all the questions, just want to make sure I thoroughly understand everything if I will be using this for my analysis pipeline. Sorry also that I am cross-talk asking new questions as you answer

No problem! I'll also update the guide to make this more straight forward, as others have emailed/asked me about similar questions.

bahanonu avatar Dec 02 '20 22:12 bahanonu

Also, I am using the same parameters as you are above, on the same video, and my post processing video looks completely different. Here is a gif from it and you can see there is like no contrast or anything remotely like your gif :) test

Great! If you adjust the contrast in ImageJ or in Matlab (press J in playMovie interface), should look similar to the movie I'd attached :D

Ah ok!! I thought your gif was unedited after post processing. Got it, thank you!!

hsw28 avatar Dec 02 '20 22:12 hsw28

also what cell extraction method/parameters did you use above? thanks again

The results I showed were from a method (CELLMax) we haven't released yet; I'd have to look in more detail to find parameters that work with PCA-ICA or CNMF(-E) given the stripes caused issues with those methods on first pass. You could try to run PCA-ICA with 100-200 ICs, but will likely get a lot of non-cells/artifacts that you have to manually sort.

bahanonu avatar Dec 02 '20 22:12 bahanonu

also what cell extraction method/parameters did you use above? thanks again

The results I showed were from a method (CELLMax) we haven't released yet; I'd have to look in more detail to find parameters that work with PCA-ICA or CNMF(-E) given the stripes caused issues with those methods on first pass. You could try to run PCA-ICA with 100-200 ICs, but will likely get a lot of non-cells/artifacts that you have to manually sort.

great thanks! your results look great but the methods I tried all looked questionable. I await the cellmax method release!

hsw28 avatar Dec 02 '20 23:12 hsw28

Hi, it's me again!! I am trying to do modelExtractSignalsFromMovie and i get this error/output: Is this just because of the type of extraction I am trying to do? thanks for any tips, yet again!

> 
> !!!!!!!!!!!!!!!!!!!!!
> Running: obj.modelExtractSignalsFromMovie
> @@@@@@@
> Error using dir
> Characters adjacent to a ** wildcard must be file separators.
> 
> Error in calciumImagingAnalysis/modelExtractSignalsFromMovie/getAlgorithmRootPath (line 466)
> 			foundFiles = dir(fullfile([obj.defaultObjDir filesep obj.externalProgramsDir], ['**\' algorithmFile '']));
> 
> Error in calciumImagingAnalysis/modelExtractSignalsFromMovie (line 85)
> 				getAlgorithmRootPath('runCELLMax.m','CELLMax',obj,1);
> 
> Error in calciumImagingAnalysis/runPipeline (line 202)
> 				obj.(thisFxn{1});
> 
> Error in calciumImagingAnalysis/display (line 817)
> 			obj.runPipeline;
> @@@@@@@
> /Volumes/VERBATIM/Rat1/2020_12_07/19_00_35/Miniscope/processing_info
> Adding folders: /Volumes/VERBATIM/Rat1/2020_12_07/19_00_35/Miniscope/processing_info:
> =====================
> Subject: m000
> ###
> Getting movie info for 1\1: /Volumes/VERBATIM/Rat1/2020_12_07/19_00_35/Miniscope/2020_12_07_p000_m000_NULL000_strpRm_dsSpace_spFltBfReg_treg_crop_fxFrms_dfof_dsTime_1.h5
> loading chunk | offset: 0  0  1 | block: 304  304    1
> movie size: 0.35254Mb | 304  304 | single
> checking movies: 100|
> 
> ans =
> 
>   1×1 cell array
> 
>     {[250]}
> 
> 
> ans =
> 
>   1×1 cell array
> 
>     {[1]}
> 
> -------
> pre-allocating single NaN matrix...
> -------
> loading 1/1: /Volumes/VERBATIM/Rat1/2020_12_07/19_00_35/Miniscope/2020_12_07_p000_m000_NULL000_strpRm_dsSpace_spFltBfReg_treg_crop_fxFrms_dfof_dsTime_1.h5
> loading movie as contiguous chunk: 0  0  0 | 304  304  250
> loading chunk | offset: 0  0  0 | block: 304  304  250
> movie size: 88.1347Mb | 304  304  250 | single
> movie class: single
> movie size: 304  304  500
> x-dims: 304
> y-dims: 304
> z-dims: 250
> movie size: 176.2695Mb | 304  304  500 | single
> Elapsed time is 0.802823 seconds.
> ###
> =====================
> Subject: m000
> Warning: Converting NaN inputs to 0. 
> > In imhist (line 88)
>   In getHistogramData>computeHistogramData (line 126)
>   In getHistogramData (line 21)
>   In imcontrast>isDisplayRangeOutsideDataRange (line 1576)
>   In imcontrast (line 116)
>   In calciumImagingAnalysis/modelExtractSignalsFromMovie/subfxnSignalSizeSpacing (line 1554)
>   In calciumImagingAnalysis/modelExtractSignalsFromMovie (line 163)
>   In calciumImagingAnalysis/runPipeline (line 202)
>   In calciumImagingAnalysis/display (line 817) 
> Warning: Cannot set Position while WindowStyle is 'docked' 
> > In imcontrast>createHistogramPalette (line 196)
>   In imcontrast (line 135)
>   In calciumImagingAnalysis/modelExtractSignalsFromMovie/subfxnSignalSizeSpacing (line 1554)
>   In calciumImagingAnalysis/modelExtractSignalsFromMovie (line 163)
>   In calciumImagingAnalysis/runPipeline (line 202)
>   In calciumImagingAnalysis/display (line 817) 
> Warning: Cannot set Position while WindowStyle is 'docked' 
> > In imcontrast>createHistogramPalette (line 239)
>   In imcontrast (line 135)
>   In calciumImagingAnalysis/modelExtractSignalsFromMovie/subfxnSignalSizeSpacing (line 1554)
>   In calciumImagingAnalysis/modelExtractSignalsFromMovie (line 163)
>   In calciumImagingAnalysis/runPipeline (line 202)
>   In calciumImagingAnalysis/display (line 817) 
> gridWidth:
>     m000: 20
> 
> gridSpacing:
>     m000: 27
> 
> =====================
> 1/1 (1/1): m000 NULL000
> -------
> CELLMax
> @@@@@@@
> Reference to non-existent field 'CELLMax'.
> 
> Error in calciumImagingAnalysis/modelExtractSignalsFromMovie (line 289)
> 						saveID = {obj.extractionMethodStructSaveStr.(obj.signalExtractionMethod)};
> 
> Error in calciumImagingAnalysis/runPipeline (line 202)
> 				obj.(thisFxn{1});
> 
> Error in calciumImagingAnalysis/display (line 817)
> 			obj.runPipeline;
> @@@@@@@
> 
> errorList =
> 
>      1
> 
> =====================
> Error in the following folders: 1
> Re-run after checking below warning tips.
> =====================
> Warning:
> =====================
> Error during cell extraction with "modelExtractSignalsFromMovie". Some tips:
> - Make sure REGEXP for finding movie is correct.
> - Make sure folder has files in it and is accessible to MATLAB.
> - Try and avoid running cell extraction of extremely small movies (e.g. 100x100 movie with 25
> frames).
> - Make sure folder added with "modelAddNewFolders" points to an actual folder and not a file.
> - CNMF and CNMF-E: check that they are installed under "_external_programs" folder.
> - PCA-ICA: make sure there are MORE frames than PCs and ICs requested, else PCA-ICA will not run.
> - ROI: make sure you have run a previous cell-extraction method.
> =====================
>  
> > In calciumImagingAnalysis/modelExtractSignalsFromMovie/subfxnRunWarning (line 432)
>   In calciumImagingAnalysis/modelExtractSignalsFromMovie (line 427)
>   In calciumImagingAnalysis/runPipeline (line 202)
>   In calciumImagingAnalysis/display (line 817) 
> 
> 
> Run processing pipeline by typing below (or clicking link) into command window (no semi-colon!):
> obj
> >> 

hsw28 avatar Dec 08 '20 04:12 hsw28

@hsw28 hey! Yep, that's likely because you chose CELLMax instead of CNMF or PCA-ICA. e.g. select PCA-ICA as below, let me know if any of the subsequent steps are confusing or see https://bahanonu.github.io/calciumImagingAnalysis/pipeline_detailed/#extracting-cells-with-modelextractsignalsfrommovie.

image

bahanonu avatar Dec 10 '20 05:12 bahanonu

Hi, sorry to bother you again! Two more questions:

  1. When will cellmax be available?
  2. I have a video where the cells are 'fainter' / less in focus. When I process it the program adds a bunch of speckled noise. Here is a pic from the video before processing and after processing: image image

Here are the parameters I am using. Thanks for any help! image image image

hsw28 avatar Jan 13 '21 23:01 hsw28

By process of elimination I found it introduces the noise at df/f0

hsw28 avatar Jan 14 '21 00:01 hsw28

hey! To your first question: hopefully soon, I can likely add you to the private repo if you'd like to try it out before then.

re: noise, there should generally not be noise added. See below for an explanation, let me know if have any follow-up questions.

  • It is already there in the movie, just normally quantitatively it is low relative to the baseline with 1P miniscope movies, so you'll see it more easily after dF/F0 or spatial filtering.
    • In many cases the contrast is automatically adjusted in GUIs to boost likelihood users will see cells, etc. (even though underlying matrix values are not changed), which can sometimes lead to perception noise is being added depending on viewing raw vs. dF/F0.
  • Any pixels/speckles that are consistently white/same value across frames are likely dead pixels on the camera sensor. These can be fixed multiple ways.

Raw movie

For example, take one of the example movies in the CIAtah repository: 2014_04_01_p203_m19_check01.

A frame from the raw movie looks like: image

Applying a simple bandpass or highpass filter (to remove the low frequency background) leads to the below (keeping contrast/brightness the same as the raw movie image): image

However, if we adjust the contrast, we can now see some of the noise present in the higher frequency components of the raw movie: image

dF/F0

Now compare to dF/F0 of that same raw movie without any motion correction, etc. we see the below: image

However, if you adjust the contrast: image

bahanonu avatar Jan 14 '21 05:01 bahanonu

@hsw28 Wanted to check that the above explanation made sense.

bahanonu avatar Jan 24 '21 20:01 bahanonu