MedLLMsPracticalGuide icon indicating copy to clipboard operation
MedLLMsPracticalGuide copied to clipboard

Great Resource

Open MattStammers opened this issue 1 year ago • 2 comments

Firstly well done. I have read through nearly all the links in your readme and they mostly work. I have also read your paper and the ones you have acknowledged at the bottom. Your paper is probably the most clearly structured.

However, there are a few issues. Firstly several of the links appear under multiple headings but some of the headings don't really make sense. For example the Korean Journal of Medical Education 2023 article appears both in medical education (where it fits) and medical translation where it fits less well. I understand why have done this but it makes the guide less robust. It would be better to have each article tagged only once under a heading and for the sometimes arbitrary duplication to be reduced.

Secondly some of the links already fail such as healthcare magic where even the derivative of this no longer seems to exist.

Thirdly, there are some articles which are very heavily vision based such as the VisionFM paper. I really enjoyed this paper but it is very different in emphasis compared to say the RadFM paper which is very much focused on text. It's up to you whether you continue to include it but this paper did feel slightly out of place and probably would have been excluded if this was a systematic review.

Finally, after reading all of it I was left with the distinct impression that the 'what's missing' aspect of this is far bigger than is currently realised. However, you have done a great job to start to repair that deficit and for that I am grateful. Thus, if you only choose to address point 2 I would fully understand your decision because none of the rest of it is actually broken and most of these are stylistic things.

MattStammers avatar Aug 09 '24 14:08 MattStammers

I am also happy to do a pull request to fix a few of these things if you want but this is your guide and I don't want to act presumptively.

MattStammers avatar Aug 09 '24 14:08 MattStammers

Many thanks for your interest in our work and for providing your comment.

  1. Thank you. We have removed the Korean Journal of Medical Education 2023 article from medical translation.
  2. Thank you. We have updated the links.
  3. Thank you. We have moved VisionFM to the Multi-modal LLM section.
  4. Thank you. We hope the "BOX: Keypoints" of our latest version of the paper (https://arxiv.org/pdf/2311.05112) can answer your question about the 'what's missing' aspect of LLMs in medicine.

Thank you again for your comments. We greatly welcome and appreciate any requests you may have. Please feel free to make a pull request.

AI-in-Health avatar Aug 12 '24 09:08 AI-in-Health

The other thing I am noticing is that some models which were 'open source' are then suddenly dissapearing making it very hard to keep the guide up to date!

It seems that the leading models on the Huggingface leaderboard mostly fall into this category. You guys are doing a great job though despite this

MattStammers avatar Aug 20 '24 13:08 MattStammers

I'm going to close this issue for now because you are doing your best and this is going to be an iterative project as it is :)

MattStammers avatar Aug 23 '24 16:08 MattStammers

Thank you. We look forward to your further comments to help us improve our repository.

AI-in-Health avatar Aug 24 '24 01:08 AI-in-Health