phy
phy copied to clipboard
Assertion Error when opening spike sorted file
Hi,
I recently ran a recording through Kilosort2.5, where it reported to find spikes and good units (if few), but when I try to open it up in phy I get this error:
[33m16:07:42.861 [W] model:545 Unreferenced clusters found in templates (generally not a problem)[0m
[33m16:07:42.938 [W] model:567 Unreferenced clusters found in spike_clusters (generally not a problem)[0m
[33m16:07:42.965 [W] model:54 4279/40000 values are nan in C:\Users\idavalik\Data\Neuropixels\Socal_track_day5_g0\Socal_track_day5_g0_imec0\similar_templates.npy, replacing by zero.[0m
[31m16:07:43.826 [E] __init__:62 An error has occurred (AssertionError):
Traceback (most recent call last):
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\idavalik\AppData\Local\Continuum\anaconda3\envs\phy2\Scripts\phy.exe\__main__.py", line 7, in <module>
sys.exit(phycli())
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 782, in main
rv = self.invoke(ctx)
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 610, in invoke
return callback(*args, **kwargs)
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\click\decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\apps\__init__.py", line 159, in cli_template_gui
template_gui(params_path, **kwargs)
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\apps\template\gui.py", line 199, in template_gui
gui = controller.create_gui()
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1638, in create_gui
self.supervisor.attach(gui)
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 942, in attach
gui=gui, sort=gui.state.get('ClusterView', {}).get('current_sort', None))
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 760, in _create_views
gui, data=self.cluster_info, columns=self.columns, sort=sort)
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 916, in cluster_info
return [self.get_cluster_info(cluster_id) for cluster_id in self.clustering.cluster_ids]
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 916, in <listcomp>
return [self.get_cluster_info(cluster_id) for cluster_id in self.clustering.cluster_ids]
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 745, in get_cluster_info
out[key] = func(cluster_id)
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1150, in get_best_channel_label
return self._get_channel_labels([self.get_best_channel(cluster_id)])[0]
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\utils\context.py", line 154, in memcached
out = f(*args, **kwargs)
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1144, in get_best_channel
channel_ids = self.get_best_channels(cluster_id)
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\utils\context.py", line 154, in memcached
out = f(*args, **kwargs)
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\apps\template\gui.py", line 149, in get_best_channels
template = self.model.get_template(template_id)
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 855, in get_template
return self._get_template_dense(template_id, channel_ids=channel_ids)
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 798, in _get_template_dense
channel_ids_, amplitude, best_channel = self._find_best_channels(template)
File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 787, in _find_best_channels
assert best_channel in channel_ids
AssertionError
[0m
The recording was very noisy and I fear/suspect some probe damage, but I have had no problems opening noisy files before.
I also get this error recently after running kilosort on a full session with a few noisy channels disabled. If I run it on a 10-second segment of the data I don't get this error in phy.
So no workaround to look at the entire session? Anyone knows the reason for this?
I'm not sure if Kilosort2 rewrites params.py after sorting for a second or third time. Did you try to delete all files generated by Kilosort2.5 and start over again? It's working in my case.
I tried re-running after deleting any of the files generated by KS2.5 but I still get this error. I don't know the exact cause of this error but I suspect it has something to do with the fact that in my dataset (and maybe @Idavr as well?), a few channels are disabled. Then assert best_channel in channel_ids
fails because the best channel for one cluster is mis-assigned to one of the disabled channels. @marius10p do you think this is possible?
I've also tried re-running it multiple times, getting the same error. Since this happened after something funky happened with the channels during the actual recording (the above mentioned noise) I am unable to open any sessions done after this weirdness, even if the raw stream of the recording does not have the same kind of noise. There are still very - very - few good units being detected by KS2.5, so I can not say whether that is a confounding factor but when there has been no spikes or units previously I haven't been able to run KS at all. A sudden channel mislabeling (for whatever reason) would make sense, but I am very interested to hear what Marius has to say on the subject.
I think the key point is when using probes with blanked channels. Now when I use old probes with blanked channels, Phy show the channels numbered from zero to (n - number of subtracted channels). For example, if I subtract 2 and 3 from a 10-line probe (form 0-9), it will show probes between 0 and 7 (previously 2 and 3 were simply not shown). I'm not sure if this is related to this issue in Kilosort2.5
If someone was able to send me a full dataset (even tiny) that either does not open, or that has mislabeled channels (with indications of where the actual labels should be), I could have a look
@rossant Appreciate it very much! I could send you one of mine to take a look at? How would you want it sent?
Any file-sharing like Dropbox, at cyrille.rossant at gmail.com
Le sam. 14 nov. 2020 à 16:28, Idavr [email protected] a écrit :
@rossant https://github.com/rossant Appreciate it very much! I could send you one of mine to take a look at? How would you want it sent?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cortex-lab/phy/issues/1051#issuecomment-727223292, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAO2GVZAV2SB3SGSA7YSR7LSP2O3FANCNFSM4TQZDDGA .
Sent.
I'm now also getting a similar error trying to run phy on a KS 2.5 sorted session where a reference channel on an NPx probe was disabled. Any insights since the last posting?
Exactly, it happens when some channels are disconnected.
Update: It does seem like there is some kind of issue with the KS2.5 output, as even if I specify the correct channels the params.py file gets saved with 383 channels (instead of 385) and the dat-path is set to be the temp_wh.dat file instead of the raw .bin file. Running the same files through KS2 I had no trouble accessing them with phy and params.py was saved correctly. @Hankslab-ABG have you checked the params.py file for inconsistencies compared to when you could run phy on your data?
@ldavr I think that works as intended. The dat path should be set to the drift corrected file, otherwise you'll see the raw spikes moving with drift. The 383 channels are probably the connected ones on your probe, although it should normally be 383.
So to replicate this problem, I should just set connected(n) = 0 for some channel n on NP 1.0 ?
@marius10p Ah, I see. But why does KS2 find 385 channels on the probe when running the analysis then? Also, I was trying to find the connected variable in the rez.mat, but it was not there?
I am also getting the exact same error and it appears to occur regardless if connected(:) = 1 or connected(anyN)=0.
Steps to reproduce
- Installed software: Windows 10, Open Ephys 0.5.2, Kilosort 2.5, phy2.0b1
- Open Ephys: record a session of FPGA data stored with recording note in binary format with a small data set (210 sec) of 32 channels + 8ADC (40 channels).
- Build channel map with
connected(:)=1;
ORconnected(1:8)=0; % the other 32 channels being =1
, save into .mat file - Open KS2.5 GUI; select continuous.dat binary file recorded in Step 2 and channel map created in Step 3.
- Run All.
- In the output directory:
phy template-gui params.py
End of my stack appears the same as the OPs:
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 787, in _find_best_channels
assert best_channel in channel_ids
AssertionError
[0m
Wonder if @rossant was able to examine the data set of @ladvr and offer feedback?
Thanks!
@Idavr I'm getting 383 channels in params.py, as I should (since I have channel 192 disconnected). What's strange is that that's the same setup that's used in neuropixPhase3B1_kilosortChanMap.mat and neuropixPhase3B2_kilosortChanMap.mat, so in theory if @marius10p has used those channel maps with KS2.5 he should be getting the same errors with Phy that we are, correct?
Oddly, though, I'm not able to reproduce this issue with Phy for every recording. I just ran Phy successfully on a recording I did on Friday that has channel 192 disconnected, so I'm wondering if its actually related to the channel connections.
Also, for what it's worth, that channel does contribute to sorted clusters, so I'm not sure why that would be happening.
@Hankslab-ABG, does it happen with some datasets 100% of the time and other datasets not at all? Is there a difference between the size of those datasets? I am wondering if the datasets are too small (in terms of time - as my test data sets are) that the error will occur more often.
Oddly, though, I'm not able to reproduce this issue with Phy for every recording. I just ran Phy successfully on a recording I did on Friday that has channel 192 disconnected, so I'm wondering if its actually related to the channel connections.
Also, for what it's worth, that channel does contribute to sorted clusters, so I'm not sure why that would be happening.
Oddly enough I wasn't able to reproduce the issue on a dataset I've been sent. Could someone else send me a full dataset (raw data, .npy files, no need for the .phy subfolder) via dropbox or similar at my email address?
@linussun , I'm still trying to suss out a pattern, but it seems like it happens more often for one of my rats than the other. They have a slightly different channel mapping as one rat has the NPx probe implanted slightly deeper than the other. Both run sessions that go from about 45 to 60 minutes, so they're not particularly short.
For what it's worth, here is the exact errors that are being thrown when I try to open Phy:
AssertionError [0m [90m12:36:58.872 [D] context:100 Initialize joblib cache dir at
D:\Recordings\Rat78_2020-11-23_11-40-47\experiment1\recording1\continuous\Neuropix-PXI-100.0.phy`.[0m
[90m12:36:58.872 [D] context:100 Initialize joblib cache dir at D:\Recordings\Rat78_2020-11-23_11-40-47\experiment1\recording1\continuous\Neuropix-PXI-100.0\.phy
.[0m
[90m12:36:58.872 [D] context:101 Reducing the size of the cache if needed.[0m
[90m12:36:58.872 [D] context:101 Reducing the size of the cache if needed.[0m
[90m12:36:58.874 [D] base:102 Add filter high_pass
.[0m
[90m12:36:58.874 [D] base:102 Add filter high_pass
.[0m
[90m12:36:58.874 [D] config:31 Load config file C:\Users\cns-hankslab\.phy\phy_config.py
.[0m
[90m12:36:58.874 [D] config:31 Load config file C:\Users\cns-hankslab\.phy\phy_config.py
.[0m
[90m12:36:58.875 [D] plugin:145 Loading 0 plugins.[0m
[90m12:36:58.875 [D] plugin:145 Loading 0 plugins.[0m
[90m12:36:58.877 [D] context:209 The file D:\Recordings\Rat78_2020-11-23_11-40-47\experiment1\recording1\continuous\Neuropix-PXI-100.0\.phy\new_cluster_id.pkl
doesn't exist.[0m
[90m12:36:58.877 [D] context:209 The file D:\Recordings\Rat78_2020-11-23_11-40-47\experiment1\recording1\continuous\Neuropix-PXI-100.0\.phy\new_cluster_id.pkl
doesn't exist.[0m
[90m12:36:58.985 [D] context:185 Save data to D:\Recordings\Rat78_2020-11-23_11-40-47\experiment1\recording1\continuous\Neuropix-PXI-100.0\.phy\spikes_per_cluster.pkl
.[0m
[90m12:36:58.985 [D] context:185 Save data to D:\Recordings\Rat78_2020-11-23_11-40-47\experiment1\recording1\continuous\Neuropix-PXI-100.0\.phy\spikes_per_cluster.pkl
.[0m
[90m12:36:59.043 [D] gui:463 Creating GUI.[0m
[90m12:36:59.043 [D] gui:463 Creating GUI.[0m
[90m12:36:59.047 [D] state:46 Load C:\Users\cns-hankslab.phy\TemplateGUI\state.json for GUIState.[0m
[90m12:36:59.047 [D] state:46 Load C:\Users\cns-hankslab.phy\TemplateGUI\state.json for GUIState.[0m
[31m12:37:04.782 [E] init:62 An error has occurred (AssertionError): Traceback (most recent call last): File "c:\programdata\anaconda3\envs\phy2\lib\runpy.py", line 193, in _run_module_as_main "main", mod_spec)
File "c:\programdata\anaconda3\envs\phy2\lib\runpy.py", line 85, in _run_code exec(code, run_globals)
File "C:\ProgramData\Anaconda3\envs\phy2\Scripts\phy.exe_main_.py", line 7, in
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 829, in call return self.main(*args, **kwargs)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 782, in main rv = self.invoke(ctx)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx))
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 610, in invoke return callback(*args, **kwargs)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\decorators.py", line 21, in new_func return f(get_current_context(), *args, **kwargs)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps_init_.py", line 159, in cli_template_gui template_gui(params_path, **kwargs)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps\template\gui.py", line 199, in template_gui gui = controller.create_gui()
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1638, in create_gui self.supervisor.attach(gui)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 942, in attach gui=gui, sort=gui.state.get('ClusterView', {}).get('current_sort', None))
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 760, in _create_views gui, data=self.cluster_info, columns=self.columns, sort=sort)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 916, in cluster_info return [self.get_cluster_info(cluster_id) for cluster_id in self.clustering.cluster_ids]
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 916, in
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 745, in get_cluster_info out[key] = func(cluster_id)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1150, in get_best_channel_label return self._get_channel_labels([self.get_best_channel(cluster_id)])[0]
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\utils\context.py", line 154, in memcached out = f(*args, **kwargs)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1144, in get_best_channel channel_ids = self.get_best_channels(cluster_id)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\utils\context.py", line 154, in memcached out = f(*args, **kwargs)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps\template\gui.py", line 149, in get_best_channels template = self.model.get_template(template_id)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 855, in get_template return self._get_template_dense(template_id, channel_ids=channel_ids)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 798, in get_template_dense channel_ids, amplitude, best_channel = self._find_best_channels(template)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 787, in _find_best_channels assert best_channel in channel_ids
AssertionError [0m [31m12:37:04.782 [E] init:62 An error has occurred (AssertionError): Traceback (most recent call last): File "c:\programdata\anaconda3\envs\phy2\lib\runpy.py", line 193, in _run_module_as_main "main", mod_spec)
File "c:\programdata\anaconda3\envs\phy2\lib\runpy.py", line 85, in _run_code exec(code, run_globals)
File "C:\ProgramData\Anaconda3\envs\phy2\Scripts\phy.exe_main_.py", line 7, in
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 829, in call return self.main(*args, **kwargs)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 782, in main rv = self.invoke(ctx)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx))
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 610, in invoke return callback(*args, **kwargs)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\decorators.py", line 21, in new_func return f(get_current_context(), *args, **kwargs)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps_init_.py", line 159, in cli_template_gui template_gui(params_path, **kwargs)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps\template\gui.py", line 199, in template_gui gui = controller.create_gui()
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1638, in create_gui self.supervisor.attach(gui)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 942, in attach gui=gui, sort=gui.state.get('ClusterView', {}).get('current_sort', None))
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 760, in _create_views gui, data=self.cluster_info, columns=self.columns, sort=sort)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 916, in cluster_info return [self.get_cluster_info(cluster_id) for cluster_id in self.clustering.cluster_ids]
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 916, in
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 745, in get_cluster_info out[key] = func(cluster_id)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1150, in get_best_channel_label return self._get_channel_labels([self.get_best_channel(cluster_id)])[0]
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\utils\context.py", line 154, in memcached out = f(*args, **kwargs)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1144, in get_best_channel channel_ids = self.get_best_channels(cluster_id)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\utils\context.py", line 154, in memcached out = f(*args, **kwargs)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps\template\gui.py", line 149, in get_best_channels template = self.model.get_template(template_id)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 855, in get_template return self._get_template_dense(template_id, channel_ids=channel_ids)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 798, in get_template_dense channel_ids, amplitude, best_channel = self._find_best_channels(template)
File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 787, in _find_best_channels assert best_channel in channel_ids AssertionError [0m`
Also, two lines that appears in the cmd window but not in the Phy log is
`model:54 2928/135424 values are nan in D:\Recordings\Rat78_2020_11-23_11-40-47\experiment1\recording1\continuous\Neuropix-PXI-100.0\similar_templates.npy, replacing by zero.
c:\programdata\anaconda3\envs\Phy2\lib\site-packages\phylib\io\model.py:771: RunetimeWarning: invalid value encountered in greater_equal peak_channels = np.nonzero(amplitude >= self.amplitude_threshold * max_amp)`
@rossant would a Google Drive link work for you?
sure!
@rossant Sent!
Hi, Had the same problem in the past days, I just realised that a dataset will raise Assertion error when kilosort runs with minFR = 0 but not if minFR = 1/100. It seems like minFR has to be different than 0. @rossant I send you the datasets.
Hi, I am having the same problem in multiple files. The files are about 30-40min long and recorded with NPX 1.0. I have tried some of the suggestions included in the issue without success:
- setting minFR to 1/100
- setting all channels to connected
- using the default channelmap included in the Kilosort 2.5 repo wherein 192 is disconnected.
Let me know if sending the datasets might be useful. Thanks!
Any updates on this?
For what it's worth, I re-ran a couple of the datasets that wouldn't open in Phy when using KS2.5. I used KS2 instead of 2.5 and was able to open them in Phy. However, what I found was that all of the clusters were based around very ill-defined waveforms that did not look spike-like at all. Has anyone else tried this? I know that some had mentioned noisy datasets might be behind this issue, so I'd be interested to see if anyone was able to get sensible-looking waveforms when re-running in KS2.
What works for me now in Kilosort2.5:
- setting
ops.minFR = 1/600;
- setting
ops.nblocks = 0
; - no removing any channel in the probe (all connected)
I am experiencing this issue as well. I tried the workaround suggested by @JoseGuzman without success. @rossant, is there any update? I am also willing to share the recording I am using via dropbox if that would be helpful.