snorkel
snorkel copied to clipboard
snorkel-team
Overlap and conflict rate between all labeling functions (LFs)
Problem I Want To solve
Hello, I would like to thank you for your contribution and this project. First, I would like to check if my point of view is right. As far as I understand, if for example you have 5 labeling functions (LFs) each one of them will have 1 overlap and 1 conflict rate. I suppose that besides the fact that maybe all the LFs overlap between each other the only rate that is illustrated is the higher. The same for the conflict rate. it will be insightful and helpful to provide a matrix with 4 such rates for each function (if there are 5 LFs) and to know the pairwise overlap and conflict rates. For example, if there are 5 LFs, the conflict and overlap rates will be two matrices 5 x 5.
Describe the solution you'd like
The solution is the illustration of the aforementioned matrices. I suppose it will provide a better understanding of the interactions between the LFs.
Describe alternatives you've considered
Depends on the problem, sometimes only the higher rates of conflict and overlap, if the higher one is the only illustrated is enough.
Additional context
As far as now I have not seen any code of the Snorkel project, thus, I do not have an outline of an implementation. I just illustrated an idea for further discussion.
Thank you in advance, Manos.
This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 7 days.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Hello @gionanide , I think the functionality that you are looking for is in this method, let us know if that solves your problem. https://github.com/snorkel-ai/strap/blob/303dc8b36327f1fac8afe4d237751044af2cc046/src/python/lf_analysis/metrics.py#L243
Hello @vkrishnamurthy11 thank you for your answer. I think that I do not have access to the page/ link you sent me.
@gionanide My apologies. Can you please see if this link works: https://github.com/snorkel-ai/strap/blob/main/src/python/lf_analysis/metrics.py#L306
@vkrishnamurthy11 I do not think that I have access in those links. Or maybe I am doing something wrong
This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 7 days.