huggingface
Add other controlnet conditioning models
Some other conditioning methods are not yet included, such as Semantic Segmentation. These are not in the Annotators but it could be worth it adding here. For example, the Semantic Segmentation is easy to add through Hugging Face.
I'm interested to get parity with the annotators in the A1111 extension. After we get through the current refactoring PRs, I have depth_leres on a branch ready to go. I believe there are 3 segmentation annotators in A1111 to consider.
How do they implement the annotators in A1111? Do they just have a bunch of files that implements them like we did here? Could be interesting if they want to switch their processors to this repo. Are there any other img2img processors that you know of (such as the depth_leres, segmentations)? We could put it in a backlog
A1111 Implementation is here: https://github.com/Mikubill/sd-webui-controlnet/tree/main/annotator. It is derived from the ControlNet sources similar to this package.
There are design differences which I outlined in #43, but two major points would be harder to resolve:
- Resizing and format conversions are handled elsewhere. The canny annotator is simply a call to cv2.Canny.
- They have processors as another layer on annotators which do additional transformations. E.g. the safe/scribble parameters to HED are handled at the processor layer and are not in the annotator.
Unless there is a major compatibility break for controlnet_aux, code sharing would probably need to be another package that both depend on.