ControlNet
ControlNet copied to clipboard
lllyasviel
ADE20K categories and color codes?
Hello, I am wondering which categories of ADE20K are supported and if there is a translation table with color codes? Thank you
https://docs.google.com/spreadsheets/d/1se8YEtb2detS7OuPE86fXGyD269pMycAWe2mtKUj2W8/edit#gid=0
Thank you. I knew that table already. So you can confirm that ControlNet is trained on only a very small subset of ADE20K?
Just in case it might help you or anyone having the same question, I prepared a c4d scene with 150 materials and 150 text objects all named and colored according to the ADE20 Color150 reference.
You can find the post with the Cinema4d scene, a prerendered color chart showing what's in the c4d scene, and everything else over here:
https://github.com/Mikubill/sd-webui-controlnet/discussions/445
I would not call it ADE20 color reference because the official ADE20K project is using very different colors (and 3200 categories btw.). I have checked it and this is confusing people. Better name it something like ControlNet segmentation colors. On this page they wrote about ADE20K protocol they are using and not ADE20K colors/categories.
The documentation where I took the data to make this called the model Color150, based on ADE20k most frequently encountered objects. The other categories, and the sub categories for object parts, are not color-coded like this. Their data can be interpreted as colors, but there is no official rules or reference for those colors. You must chose the right palette for the circumstances. You'll never get 3200 different objects on a single image, so it's better to have adaptive solutions. But I had no idea what this whole semantic segmentation thing was just a few weeks ago, so if that's not correct, please teach me, I love to learn and I know I know next to nothing about all this.
I got my information from here: ADE20k where matlib CSS4 colors are used. there is a objects.txt in official dataset which consists of 1599 categories (half the size of the 3200 mentioned ones).
Thanks for linking to this information about ADE20k.
Where is the official color mapping reference that will tell me what color corresponds to each of those 1599 categories? Isn't Css4 just a color managing library for matplotlib on python? How do you go from that to get a spreadsheet with 1599 categories and 1599 individual colors ?
Here is information about the origin of the color150 model I used and how it related to ADE20k :
3.1 Scene parsing benchmark We select the top 150 categories ranked by their total pixel ratios2 in the ADE20K dataset and build a scene parsing benchmark of ADE20K, termed as SceneParse150. Among the 150 categories, there are 35 stuff classes (i.e., wall, sky, road) and 115 discrete object classes (i.e., car, person, ta- ble). The annotated pixels of the 150 classes occupy 92.75% of all the pixels of the dataset, where the stuff classes occupy 60.92%, and discrete object classes occupy 31.83%.
page 7 and following from : https://arxiv.org/pdf/1608.05442.pdf