embeddings-benchmark
Paper Writing: Methods - The Open-source Effort [in need of review]
@imenelydiaker already did good work here. I added a few additional things.
@imenelydiaker do you mind taking a look at this again?
I can help with review and also added a few changes, but here are some general comments.
- Some sentences read difficult (I recomend using Hemingway app to check for clarify), see my rewrite of the first sentence.
- "Datasets Quality Assurance" should that be really part of this section? Apart from "asking contributors to fill in metadata fields" fragment there's little relevance to the healine above.
- I know that point-based system is detailed in
sec:contributionsappendix, but this sounds like important enough to move it, or at least briefly outline in the main section of the paper. - “near-perfect scores” what does it mean? Should we specify threshold?
- I'm afraid that description of collecting metadata fields for dataset may not be clear for someone who has never done it, maybe we should consider providing a brief example of the metadata field as listing or figure?
And the overall goal of this section is a bit unclear to me. A lot of things are mentioned superficially, or referred to only in appendix. How detailed we want it to be? Another question is whether entire "Methods" should really be before "Related Work" as we draw some inspiration from earlier community driven initiatives that are discussed there, so it could read more easily building upon that.
I can help with review and also added a few changes, but here are some general comments.
Would be lovely!
Some sentences read difficult (I recomend using Hemingway app to check for clarify), see my rewrite of the first sentence.
I agree. We should probably do this at the end though. Focus on the main narrative points to start with
"Datasets Quality Assurance" should that be really part of this section? Apart from "asking contributors to fill in metadata fields" fragment there's little relevance to the healine above.
I believe it is related to the open-source effort. Do you have another suggestion on where to put it
I know that point-based system is detailed in sec:contributions appendix, but this sounds like important enough to move it, or at least briefly outline in the main section of the paper.
I would welcome a rewrite if you have a good one - One thing I am afraid of is reviewers getting too caught up on the specifics of a non-perfect point system (we know it is - it can't be perfect).
“near-perfect scores” what does it mean? Should we specify threshold?
We can probably be more clear I agree. However near perfect scores vary from metric to metric though I think most people would understand it as a score close to the maximum of what would be possible for the task (often close to 1). Again a reformulation is very welcome
I'm afraid that description of collecting metadata fields for dataset may not be clear for someone who has never done it, maybe we should consider providing a brief example of the metadata field as listing or figure?
Again def. welcome to suggest a rewrite here
Okay, I gave it another pass, you may have a look. I decided to split Open source efforts and Quality Assurance. Still tempted by moving "Related Work" before "Methods" but as this is a substantial change, decided to leave it right now for later discussion.
I started reviewing the method section, did the first half of it, plan to do the 2nd part this week
Thanks for these suggestion @mariyahendriksen - I have read through the suggestions and applied them.
It should be noted that the benchmark section is stilll a work in progress. I have just started working on that today (though we are still waiting on a few datasets). I can ping you here once the section is in a reasonable state.
@KennethEnevoldsen sounds great, please do ping me! Also, feel free to let me know if you there is any priority I should be aware of. In the meantime, I will continue on the methods section.
@mariyahendriksen I believe this section (on the open-source effort) is done. Would love a pair of eyes on the second part of the method section
@KennethEnevoldsen I restructured the 2nd part of the methods section! fyi I also went over the abstract, the introduction and the appendices a couple of times. let me know if there are any urgent tasks that require attention!