Automatic EEG localization and labelling using Revopoint 3D

[chinmaychinara91]

Tutorial: https://neuroimage.usc.edu/brainstorm/Tutorials/TutDigitize3dScanner#Digitize_using_Revopoint_Digitize_GUI_in_Brainstorm

Test Data: Can be downloaded from here.

Goal is to allow automatic localization and labelling of EEG caps using 3D scanned mesh of a person wearing the cap (captured using Revopoint 3D). This represents a new low-cost, fast, easy-to-use approach for digitizing heads and electrodes as compared to the traditional digitizer.

NOTE: Currently tested well with two configurations of EEG caps: ~~https://github.com/brainstorm-tools/brainstorm3/pull/716/commits/d40c719ede6ee167c290c7b526836d57811761ca~~ https://github.com/brainstorm-tools/brainstorm3/pull/716/commits/c49ebc6cc474979bfd634731f5be67dcca33e6bc

Proposed GUI within Brainstorm LEGACY MODE

2024 MODE

• [x] Load OBJ file into brainstorm (mesh with vertices, faces and color)
• [x] Add color field to brainstorm surface structure
• [x] Color should be updated when mesh downsampled in BST as well
• [x] Refactor the Digitize menu to add Revopoint
• [x] Refactor the Digitizer GUI for Revopoint tasks
• [x] Menu to load default available caps in BST as Montages in Revopoint GUI
• [x] Handle surface color update for Views menu option on 3D figure (Clone figure, etc.)
• Test nothing breaks for the already available Digitizer functionalities:
  • [x] Simulation mode
  • [x] With Polhemus connected
• [x] 19th Aug 2024: Make it work on both the "legacy" and 2024 Digitize panel (https://github.com/brainstorm-tools/brainstorm3/pull/682)

@tmedani @ajoshiusc @yashvakilna

[chinmaychinara91]

tutorials in GUI and tutorial scripts

Can you point me to this ?

[rcassani]

Can you point me to this ?

@chinmaychinara91

1. Running tutorials on GUI

There are many tutorials in this page, that you can follow to verify the changes are backwards compatible https://neuroimage.usc.edu/brainstorm/Tutorials/

2. Running tutorials with scripts

Most tutorials can be run with scripts, located in main Brainstorm repository. Look for the tutorial_*.m scripts: https://github.com/brainstorm-tools/brainstorm3/tree/master/toolbox/script

There is also this private repo: https://github.com/brainstorm-tools/bst-tests

[rcassani]

Hey guys, unfortunately, we do not have a revopoint yet. Can you provide the following to tests the code in the PR?

• ~~Some raw files to test the import of the new surface format~~ Already in the PR description
• A protocol where the sensor localization has been already performed

In addition, have you had the chance to run the digitization with Polhemus, to see that everything is fine with that?

[chinmaychinara91]

Hey guys, unfortunately, we do not have a revopoint yet. Can you provide the following to tests the code in the PR?

• Some raw files to test the import of the new surface format
• A protocol where the sensor localization has been already performed

In addition, have you had the chance to run the digitization with Polhemus, to see that everything is fine with that?

@rcassani:

1. Based on the tutorial in the description above, raw test data is attached here as sub1_Waveguard_ANT_65.zip
2. The test protocol can also be found in the link above as TutorialDigitizer.zip

I ran it in the simulated mode without the Polhemus connected. Functionalities looked good and nothing broke. Polhemus here at USC has gone for some repair. Once it is back I will check it.

Also ran the exhaustive testing with various tutorials as you instructed above. All looked good.

[tmedani]

In addition, have you had the chance to run the digitization with Polhemus, to see that everything is fine with that?

we can test that at USC, we have the digitizer around.

but if I understand it, this will not be an issue since the Digitizer simulation was working fine.

[rcassani]

@chinmaychinara91, thank you for the data

but if I understand it, this will not be an issue since the Digitizer simulation was working fine.

Correct, it should not be an issue. Though we must test it before merging the PR

[chinmaychinara91]

@chinmaychinara91, thank you for the data

but if I understand it, this will not be an issue since the Digitizer simulation was working fine.

Correct, it should not be an issue. Though we must test it before merging the PR

Yes definitely. We will test it here and let you know.

[tmedani]

@chinmaychinara91, thank you for the data

but if I understand it, this will not be an issue since the Digitizer simulation was working fine.

Correct, it should not be an issue. Though we must test it before merging the PR

Yes definitely. We will test it here and let you know.

It may require some time to do that on the digitizer itself. we need to set it up and run it.

[rcassani]

@Moo-Marc, can you give us a hand with the test with the Polhemus?

[chinmaychinara91]

It may require some time to do that on the digitizer itself. we need to set it up and run it.

Will be testing the PR with Polhemus today. Will keep posted.

[chinmaychinara91]

The Polhemus here at USC is being bit of a task as it just got back from repairs. @Moo-Marc could you help us in testing this PR out at your end ? Just want to make sure already available functionalities do not break.

[Moo-Marc]

Hi, not sure why but I had not received notifications for your messages. Sure I'll help test. Hopefully we don't get conflicts with my other digitizer PR.

[chinmaychinara91]

@Moo-Marc thank you so much. Do let me know once you test it out.

[chinmaychinara91]

d753abd commit takes into account that the New Digitize Panel changes (https://github.com/brainstorm-tools/brainstorm3/pull/682) are merged to this PR.

[chinmaychinara91]

@rcassani this is ready for review. Both the legacy and new panel support Revopoint now. Do test it out on your end based on the tutorial in description and let me know if you face any issues.

[chinmaychinara91]

@rcassani thank you for your comments. I have replied to them individually. Do go through them. But to summarize yes it needs another round of cleanup from my end which I will work on this week. Definitely it is not generalized well and there are some hard coded values but we are yet to receive more samples with variety of caps to get a good sense of generalizing them.

[rcassani]

@chinmaychinara91, related to 53f43dd.

Do not patch code just to make it work. You have to understand why there was a problem, and how it can be solved from origin.

Moreover, remember to keep previous behaviours otherwise:

• All the calls must be updated. E.g. the proposed changes brakes the calls brainstorm digitize https://github.com/brainstorm-tools/brainstorm3/blob/23b805bea0cecaa09ef41de416fd95b8e65176b1/brainstorm.m#L141

• If user's code rely on this they will need to change their codes.

In addition, it would need to update the usage strings for panel_digitize and panel_digitize_2024

What about solving the bug from root like below? This solution is compatible with former ways using 'Start' with panel_digitize and panel_digitize_2024

--- a/toolbox/sensors/panel_digitize.m
+++ b/toolbox/sensors/panel_digitize.m
@@ -40,7 +40,7 @@ end
 %#function icinterface
 
 %% ===== START =====
-function Start(DigitizerType) %#ok<DEFNU>
+function Start(varargin) %#ok<DEFNU>
     global Digitize    
     % ===== INITIALIZE CONNECTION =====
     % Intialize global variable
@@ -68,10 +68,11 @@ function Start(DigitizerType) %#ok<DEFNU>
             'trans',    []));
     
     % ===== PARSE INPUT =====
-    Digitize.Type = 'Digitize';
-    if nargin > 0 && ~isempty(DigitizerType)
-        Digitize.Type = DigitizerType;
+    DigitizerType = 'Digitize';
+    if nargin > 0 && ~isempty(varargin{1})
+        DigitizerType = varargin{1};
     end
+    Digitize.Type = DigitizerType;
     switch DigitizerType
         case 'Digitize'
             % Do nothing

Same thing must be done for panel_digitize_2024

[chinmaychinara91]

@rcassani

I have made all the changes based on the comments and discussions above and on Slack. Let me know if you find any errors on your side.

To summarize:

1. User can now just right click on the imported texture surface > Digitize (3D Scanner) to start the digitization.
2. Regarding There is not a clear workflow for the case where the subject has its own anatomy, the user can right click on the subject > Import surfaces and import the textured surface. Then right click on the imported textured surface > Digitize (3D Scanner) to start the digitization. follow step 1 above.
3. User now gets to choose what to do when starting 3D Scanner digitization (Use existing surface or Add a new surface).
4. Better way of sampling without having to use reducepath.
5. auto_3dscanner now handles all the automatic detection and labelling of 3D Scanner acquired mesh. For new caps, we just need to change the getEegCapLandmarkLabels function and that should be it.
6. Did some rigorous testing and was able to come up with a single range for imfindcircles with the caps we have available. Check https://github.com/brainstorm-tools/brainstorm3/pull/716/commits/3403c9ef4359220ca9b189425b14ea901ef3f079.
7. Updated the cap templates for Colin27 and ICBM152. Check https://github.com/brainstorm-tools/brainstorm3/pull/716/commits/c49ebc6cc474979bfd634731f5be67dcca33e6bc.
8. The textured surface in database is updated with Digitize.Transf transformation once the fiducials are digitized.
9. Had forgot about this https://forum.revopoint3d.com/t/revopoint-generated-mesh-coordinate-system/25662. The Revopoint vertices are in 'mm'. ~~Did some digging here https://www.fieldtriptoolbox.org/tutorial/electrode/ and I think we need this approach https://github.com/brainstorm-tools/brainstorm3/pull/716/commits/b1124bf6e0fc558fd81ffb303ff36d0c4210859d if we want to support any generic head surface acquired through other devices.~~ Check https://github.com/brainstorm-tools/brainstorm3/pull/716/commits/06df2526657c09def91b5418ff32d3b4c9e9d958 https://github.com/brainstorm-tools/brainstorm3/pull/716/commits/cedc8affd4852a161db8b3a486b3524d18043024 (fieldtrip dependency removed).
10. Needed to do some ordering changes in the way the initialization points were captured for scalability. Updated the tutorial with the instructions.
11. For https://github.com/brainstorm-tools/brainstorm3/pull/716/commits/86e8d830c69c514b82b2569632854d9797543757, the data to test with can be found in the Test data link in description.

• ft_obj: test data provided by fieldtrip here
• 3d_software_obj: random freely available head scans online

Tutorial updated partially. Should be good for your testing. Will do a final update of tutorial after it is ready to merge.

[chinmaychinara91]

For importing an OBJ into brainstorm:

1. The in_tess_wftobj function in actually a slightly modified submodule of https://www.fieldtriptoolbox.org/reference/fileio/ft_read_headshape. Should we use the fieldtrip function directly or since the license allows us, just continue to do it as it is right now (and just put a reference to the fieldtrip function).
2. We also use the https://www.fieldtriptoolbox.org/reference/forward/ft_convert_units function to determine the units of the vertices based on the shape of the head. This is a good function to have.

Both these functions are available as part of spm12 plugin (/external/fieldtrip) and also the fieldtrip plugin. How do we go about this ?

@tmedani

[rcassani]

In short, it does not makes too much sense to request the user to install FieldTrip as plugin and Load it to perform trivial operations and things that already in the Brainstorm source code. Moreover, not all functions in FieldTrip are available in the compiled version of Brainstorm.

---

1. The in_tess_wftobj function in actually a slightly modified submodule of https://www.fieldtriptoolbox.org/reference/fileio/ft_read_headshape. Should we use the fieldtrip function directly or since the license allows us, just continue to do it as it is right now (and just put a reference to the fieldtrip function).

• The waveform .obj files are simple text files that can be parsed quickly, take a look to a Brainstorm function such as in_tess_off.m. Do you have any trouble reading this format of file? Do we want to install full FieldTrip for this?

2. We also use the https://www.fieldtriptoolbox.org/reference/forward/ft_convert_units function to determine the units of the vertices based on the shape of the head. This is a good function to have.

• Same thing as the point above. Do we want to install full FieldTrip for this?

• A conversion of surface from Brainstorm > FieldTrip > Brainstorm would be needed.

• Choosing the right scaling for distance units is shown in the Brainstorm tutorials. This is done with a function that is already in Brainstorm, channel_fixunits

[chinmaychinara91]

@rcassani thanks for pointing out the resources. That makes things much simpler. I don't think we will need FT now.

[chinmaychinara91]

Code looks good, almost ready to merge.

• Most of the code duplication mentioned in the previous review has been addressed.
• @chinmaychinara91 I have addressed several of the comments in this review. Please take a look to those changes so you can improve future PRs

@rcassani thank you for the review. I will work on the comments you gave and get back.

[rcassani]

@chinmaychinara91, @tmedani It seems this PR is ready.

Please add a warning in the tutorials and in the GUI (e.g. when clicking on the Auto button) that the Auto feature is experimental, and results must be carefully verified.

[chinmaychinara91]

Please add a warning in the tutorials and in the GUI (e.g. when clicking on the Auto button) that the Auto feature is experimental, and results must be carefully verified.

Agreed will do

[chinmaychinara91]

Did the tests. All look good now. It can be merged.

[chinmaychinara91]

Tutorials updated. Found some more bugs while doing some tests and fixed them in https://github.com/brainstorm-tools/brainstorm3/pull/716/commits/53f04ca38eab3370a2b707eef5485ed7af60cd44 and https://github.com/brainstorm-tools/brainstorm3/pull/716/commits/bcacbcaea9a71017d82ecb3f112bb3f89c1a59b4. It is good to go now.