Facial Landmark Annotation

[tobiasbaur]

A nice addition would be to visualize and edit facial landmarks in a video. How exactly is up for discussion

[ThomasSmith3]

Think this will be covered by the points scheme I am working on,

[tobiasbaur]

yes, thats why i assigned it to you :)

[ThomasSmith3]

I have just pushed my current version which should all this to work. It works by adding a new teir and selecting the Point teir, from here you can select how many points you want. Then you have to select the frame and point you want to use. With the selected point you can click on the video and it will update the x and y coords of this point. You can also name each point if you want to. There is also functionality to copy forward a selected frame. There is currently a bug where clicking the time line does not select that frame. I will be looking into this as it may just be that I haven't merged with the current master and this has the fix already. The save format is 'frame name';('point name':'xcoord':'ycoord':'point confidence');(...);(...);confidence (...) = is just another point, there is on per point.

Let me know if there is anything else that is needed for this. A future addition could be preset points, but I am going to work on some of the other geometric annotations.

https://github.com/hcmlab/nova/commit/d204e34e8135aa2df28a7c2b26145572c3009058

[tobiasbaur]

hey Thomas, i had a quick look at it, and it's already looking very promsing. Do you think you can handle merging it with the latest changes in the master branch?

[ThomasSmith3]

Think I should be able too. Also remembered that Michel wanted a feature where you can move all points at once. Think the best way will be using the right click button and mouse drag. I will let you know how I get on with all of this, might be next week as I have a meeting to prep for this week.

[ThomasSmith3]

Hi guys,

I have added additional functionality to the points, you can now delete points (set them back to -1,-1), move point/s using right click and mouse drag (uses the points selected in the list).

I am having trouble merging the current master into my version as so much has changed and keep getting close but then finding I have removed something I shouldn't have. I have been using my local hotfixes branch to do this, by making it up today with master then merging the geometric branch in. Any help would be greatly appreciated.

Cheers, Tom

[tobiasbaur]

Hey Tom, currently merging (on a new branch) when i'm done fixing some minor issues, please have a look at it if everything's included (and working). I think after some testing we're fine to put this in the master branch as well!

[tobiasbaur]

Ok, fixed some minor issues (the annolist always changed and the video got small for any other annotation than points, the interface was not cleared properly). Please test the merge branch if everything is working as you would expect.

I think this works pretty well already.

Some issues we still should adress for points:

• I think when a point is relabelled, all points on that positiuon in the annolist should have that name (forward copy to the whole list). Because, this is basically a marker to know which point it is, right?

• Database storage, we can leave this to future work for now, but when working on the mongodb, we should create a scheme in the db, or at lease for now forbid to save it to the db.

[ThomasSmith3]

Thanks Tobi :D

There are a few thing Michel wants as well,

• change colour of points on fly
• have multiple teirs on one video

The problem I found was when click play it doesn't display the current frame's points only the last selected so going to fix that.

Will also keep this up to date when you push new update :)

[ThomasSmith3]

Updated with changes mentioned above. Now just needs testing for edge cases that I haven't checked yet.

[tobiasbaur]

Johannes big merge is added to master, please branch from master for future development. He changed some parts of your code as well, so please check it out

[ThomasSmith3]

Has the recent point bug been fixed? where it had too many frames. https://github.com/hcmlab/nova/commit/64e85cd9165bd89c1b41cb13977b87149796ed21

and the others since 24th? https://github.com/hcmlab/nova/commits/geometric

[tobiasbaur]

yes should be, used the latest version from geometric

[tobiasbaur]

can you check if everything is here? The code is in the MainHandlerGeometric now, and the painting is done a bit easier so zooming also works

[ThomasSmith3]

There are a few broken features, but I will look into this over the next few days.

Liking the new layout though

[tobiasbaur]

Hey Thomas, whats the intention that when you click the signal track it always jumps to the next annotation? On discrete and regular tiers, this is not the desired behaviour. Any reason this is done? (For now it requires the shift button to jump to the next label, otherwise it's back to the default behaviour). Maybe there is some reason i dont understand

[ThomasSmith3]

Hi,

I mean to add an if statement that only allows this for geometric or continuous. It allows you to read the actual values from the tables. Will add it in now ☺ Tom

[ThomasSmith3]

Updated it, should stop interfering now :) Did you get my email about point sync issue?

[tobiasbaur]

having a look into it..

On other news it's now possible to store and load point annotations to the database.

The bad thing is that they get pretty huge and mongodb has a limit of 16 mb. Will need to think of a solution for this.

I made minor changes to the file format as (only the headers) so please update existing files, tjhats how they should look like (before the minscore was abused to hold the num_points ;) )

[ThomasSmith3]

Liking the new file format for the points.

When I first started doing the geometric stuff I did think I should change it but wasn’t too familiar with the framework yet.

Quick question, I am trying to initialise the segmentation values in AnnoTier.cs and I need the width and height of the video but struggling to find a way of getting it.

Thanks for looking into the sync issue.