deeprank2 icon indicating copy to clipboard operation
deeprank2 copied to clipboard

Add performances comparisons with previous packages

Open gcroci2 opened this issue 2 years ago • 4 comments

Table implemented in PR #493 shows the timings obtained for generating graphs/graphs+grids, atomic resolution, with all features except for the ones in the conservation module, because we don't have the pssm files for the data in the tutorials (for computing the performances of deeprank2, I used the raw data available at this address).

We need some discussion here:

  1. Is this a satisfying way of showing performances? Do we need to generate all the features possible (by adding conservation module features), and to add performances for residue resolution as well?
  2. How do we do a fair comparison with the previously developed packages? Features are different in number and in how they are calculated, so if we use all features in all packages we can't know if the comparison is fair. Maybe we could just pick a couple of them which are the same in all packages (e.g., distance, residue type)?

When we'll have clearer ideas/plans about 1. and 2., compare deeprank2 with:

  • deeprank
    • [ ] PPIs, grid
  • deeprank-gnn
    • [ ] PPIs, graph
  • deeprank-mut
    • [ ] variants, grid

gcroci2 avatar Sep 21 '23 09:09 gcroci2

How would you advice to proceed here, especially for question 2.? @sonjageorgievska, @DaniBodor

gcroci2 avatar Sep 21 '23 14:09 gcroci2

I don't have a great idea about this, apart from just doing a "not fair" comparison and being open about it and explaining the differences.

DaniBodor avatar Sep 27 '23 13:09 DaniBodor

I don't have a great idea about this, apart from just doing a "not fair" comparison and being open about it and explaining the differences.

Actually I agree, I also think this is the only realistic option we have.

gcroci2 avatar Sep 27 '23 16:09 gcroci2

This issue is stale because it has been open for 30 days with no activity.

github-actions[bot] avatar Oct 30 '23 03:10 github-actions[bot]