Temporal components with negative values (in estimates.C)

[NataliaFavila]

For better support, please use the template below to submit your issue. When your issue gets resolved please remember to close it.

Sometimes errors while running CNMF occur during parallel processing which prevents the log to provide a meaningful error message. Please reproduce your error with setting dview=None.

If you need to upgrade CaImAn follow the instructions given in the documentation.

• Tell us a bit about your setup:

1. Operating system (Linux/macOS/Windows): Windows

2. Python version (3.x): Python 3.8

3. Working environment (Python IDE/Jupyter Notebook/other): Jupyter Notebook

4. Which of the demo scripts you're using for your analysis (if applicable):

5. CaImAn version*: 1.9.7

6. CaImAn installation process (pip install ./pip install -e ./conda): conda

*You can get the CaImAn version by creating a params object and then typing params.data['caiman_version']. If the field doesn't exist, type N/A and consider upgrading)

• Describe the issue that you are experiencing

We are running CNMF to extract the calcium traces from 1p videos. We noticed that we get negative values in the temporal traces (in CNMF.estimates.C). Is this normal?

Is the baseline not suppose to be non negative?

Thank you very much for your help, Natalia

• Copy error log below

• If you're not reporting an error, type your message below

[j-friedrich]

Yes, that is rather normal if you do not set the temporal parameter bas_nonneg to True. estimates.C-estimates.bl[:,None] will always be non negative, but if bas_nonneg=False then estimates.bl can be negative, as can be estimates.C.

See #783, #738

[NataliaFavila]

Thank you for the clarification!

Follow up question, we are struggling to understand if the traces we are getting in C from the CNMF.fit are the raw fluorescence traces (which we expected to be non negative) or if there is already a step of deconvolution integrated?

We have been looking at the parameters and are confused on how we will affect the deconvolution step when modifying the paramteres p = 2, bas_nonneg = True and deconv_flag = True.

Would simply changing deconv_flag to False or p = 0 give us the raw fluorescence traces?

Or is it that fluorescence traces are in estimates.C and deconvolved traces in estimates.S?

[j-friedrich]

esimates.S is the deconvolved trace, estimates.C the denoised trace. To get the noisy raw trace you can add the residual, estimates.C+estimates.R. (Or solve the least squares problem min_{C,f} |[A,b]*[C;f] - Y|^2 via np.linalg.lstsq(np.hstack([cnm2.estimates.A.toarray(), cnm2.estimates.b]), Yr)[0])

If you set p=0 the trace is not devonvolved/denoised, estimates.C is only constrained to be non-negative and estimates.S is a copy of estimates.C.

I just realized, the deconv_flag doesn't have any effect. Indeed, there are some arguments that could be passed when creating the CNMF object, that are irrelevant or relevant to OnACID. However, the strongly recommeded method is to pass a CNMFParams object.

@pgunn I wonder whether we should force the user to provide a CNMFParams object, as we do for the OnACID class? Currently cnm1 = cnmf.CNMF(n_processes) and cnm2 = cnmf.CNMF(n_processes, params=params.CNMFParams()) yield different default parameters.

[pgunn]

I think it would make sense to require that of the user, although we probably should do a warning for several versions before making it a requirement so people can correct any existing code they might have.

[EricThomson]

I had previously left this open because the discussion was useful but I'll just open a separate issue for this thought so it doesn't get buried here.

I wonder whether we should force the user to provide a CNMFParams object, as we do for the OnACID class? Currently cnm1 = cnmf.CNMF(n_processes) and cnm2 = cnmf.CNMF(n_processes, params=params.CNMFParams()) yield different default parameters.

[zhouyi0812]

I am a bit confused, I run two version with and without bas_nonneg=True, and all the parameters are the same. And the accepted number is different, with bas_nonneg=True having 3 more components than without one. It is normal? Thank you!

My para are: p = 1 # order of the autoregressive system, it depends on what ca2+ indicator K = None # upper bound on number of components per patch, in general None gSig = (6, 6) # gaussian width of a 2D gaussian kernel, which approximates a neuron gSiz = (25, 25) # average diameter of a neuron, in general 4gSig+1 Ain = None # possibility to seed with predetermined binary masks merge_thr = .7 # merging threshold, max correlation allowed rf = 40 # half-size of the patches in pixels. e.g., if rf=40, patches are 80x80 stride_cnmf = 20 # amount of overlap between the patches in pixels tsub = 1 # downsampling factor in time for initialization. Increase if you have memory problems ssub = 2 # downsampling factor in space for initialization, low_rank_background = None # None leaves background of each patch intact, gnb = 0 # number of background components (rank) if positive, nb_patch = 0 # number of background components (rank) per patch if gnb>0, min_corr = .8 # min peak value from correlation image min_pnr = 10 # min peak to noise ration from PNR image ssub_B = 1 # additional downsampling factor in space for background ring_size_factor = 1.4 # radius of ring is gSizring_size_factor bord_px = 0 bas_nonneg=True

[EricThomson]

@zhouyi0812 when you set nonnegative baseline, you can expect to see different results (including number of components), as the optimization problem being solved is different with an additional constraint added. See #783