DatasetHelpers [Feature request] Support for florence models

This is my go to tool for lora training the thing missing is support for natural language models totrain flux models

Jan 07 '25 15:01 FozeAhmad

I have plans to add Gemini thought the free API (with each user generating their own API Key) and I've found a project that implements Florence2 which I could learn from.

Jan 07 '25 17:01 Particle1904

Awesome

Jan 07 '25 18:01 FozeAhmad

Added Gemini captioning for the next release (v2.9.3), Florence2 is next; no ETA for Florence2 yet.

Jan 21 '25 10:01 Particle1904

https://huggingface.co/spaces/fancyfeast/joy-caption-alpha-two https://moondream.ai/playground

They are good. Support NSFW & SFW. Gemini and all other online APIs don't support NSFW

Mar 08 '25 18:03 snowbedding

Already planned for Florence, just haven't had the time.

Mar 08 '25 21:03 Particle1904

https://huggingface.co/mnemic/paligemma-longprompt-v1-safetensors https://huggingface.co/gokaygokay/paligemma-rich-captions

This seems like a combination of tags + long descriptive caption.

Mar 09 '25 09:03 snowbedding

I've added support for Florence2 based on this repo: https://github.com/Particle1904/DatasetHelpers/commit/db21a7069922e06abe4d7d29d6d067f499b191fe

Both captioning and automatic text/watermark removal should be available in the next release (2.9.5). I finally found a reason to sit down and implement it (Florence2 has support for OCR with Regions, and I need to clean up watermarks from a large dataset). These new features are mostly feature complete but they aren't available in the user interface yet until 2.9.5.

May 25 '25 11:05 Particle1904

2.9.5 is now available, which includes Florence2 captioning. I still think that Gemini is miles ahead of Florence. I have plans to add LlamaSharp in the future to make any .GGUF model available. I'll close this thread now. Please open a new one for any issues related to 2.9.5 or other releases.

May 27 '25 02:05 Particle1904