"ObjectNet" - another dataset

[dobkeratops]

http://news.mit.edu/2019/object-recognition-dataset-stumped-worlds-best-computer-vision-models-1210?fbclid=IwAR3SnpB8P5ecs4Z8KrOVw1i-WedYYPqOtV5DuT4rszkyBUcvgsP0WLqmGdQ

Only just discovered this (the article is from december 19)

• 50,000 images

• Supposedly it's full of pathalogical cases, with the best current NN's only scoring 55%

• focussed on domestic objects

• 313 object classes, perhaps we can ensure we've got the same labels covered in the graph. (still looking for the class list), "113 overlap with the imagenet challenge" (those would be the most important ones to replicate?)

• There's something deliberate where they dont want people to train on it ("a distinctive solid 1 pixel border and the license demands that if you publish the images, you must include that, so people know they're testing fairly")

[dobkeratops]

Here's their label list, from the paper:

Air freshener, Alarm clock, Backpack, Baking sheet, Banana, Bandaid, Baseball bat, Baseball glove, Basket,
Bathrobe, Bath towel, Battery, Bed sheet, Beer bottle, Beer can, Belt, Bench, Bicycle, Bike pump, Bills
(money), Binder (closed), Biscuits, Blanket, Blender, Blouse, Board game, Book (closed), Bookend, Boots,
Bottle cap, Bottle opener, Bottle stopper, Box, Bracelet, Bread knife, Bread loaf, Briefcase, Brooch, Broom,
Bucket, Butcher’s knife, Butter, Button, CD/DVD case, Calendar, Can opener, Candle, Canned food, Cellphone,
Cellphone case, Cellphone charger, Cereal, Chair, Cheese, Chess piece, Chocolate, Chopstick, Clothes hamper,
Clothes hanger, Coaster, Coffee beans, Coffee grinder, Coffee machine, Coffee table, Coin (money), Comb,
Combination lock, Computer mouse, Contact lens case, Cooking oil bottle, Cork, Cutting board, DVD player,
Deodorant, Desk lamp, Detergent, Dishrag or hand towel, Dish soap, Document folder (closed), Dog bed,
Doormat, Drawer (open), Dress, Dress pants, Dress shirt, Dress shoe (men), Dress shoe (women), Drill, Drinking
Cup, Drinking straw, Drying rack for clothes, Drying rack for plates, Dust pan, Earbuds, Earring, Egg, Egg carton,
Envelope, Eraser (white board), Extension cable, Eyeglasses, Fan, Figurine or statue, First aid kit, Flashlight,
Floss container, Flour container, Fork, French press, Frying pan, Glue container, Hair brush, Hair clip, Hair
dryer, Hair tie, Hammer, Hand mirror, Handbag, Hat, Headphones (over ear), Helmet, Honey container, Ice, Ice
cube tray, Iron, Ironing board, Jam, Jar, Jeans, Kettle, Keyboard, Key chain, Ladle, Lampshade, Laptop (open),
Laptop charger, Leaf, Leggings, Lemon, Letter opener, Lettuce, Light bulb, Lighter, Lipstick, Loofah, Magazine,
Makeup, Makeup brush, Marker, Match, Measuring cup, Microwave, Milk, Mixing/Salad Bowl, Monitor, Mouse
pad, Mouthwash, Mug, Multitool, Nail, Nail clippers, Nail file, Nail polish, Napkin, Necklace, Newspaper, Night
light, Nightstand, Notebook, Notepad, Nut for a screw, Orange, Oven mitts, Padlock, Paintbrush, Paint can,
Paper, Paper bag, Paper plates, Paper towel, Paperclip, Peeler, Pen, Pencil, Pepper shaker, Pet food container,
Landline phone, Photograph, Pill bottle, Pill organizer, Pillow, Pitcher, Placemat, Plastic bag, Plastic cup, Plastic
wrap, Plate, Playing cards, Pliers, Plunger, Pop can, Portable heater, Poster, Power bar, Power cable, Printer,
Raincoat, Rake, Razor, Receipt, Remote control, Removable blade, Ribbon, Ring, Rock, Rolling pin, Ruler,
Running shoe, Safety pin, Salt shaker, Sandal, Scarf, Scissors, Screw, Scrub brush, Shampoo bottle, Shoelace,
Shorts, Shovel, Skateboard, Skirt, Sleeping bag, Slipper, Soap bar, Soap dispenser, Sock, Soup Bowl, Sewing kit,
Spatula, Speaker, Sponge, Spoon, Spray bottle, Squeegee, Squeeze bottle, Standing lamp, Stapler, Step stool,
Still Camera, Sink Stopper, Strainer, Stuffed animal, Sugar container, Suit jacket, Suitcase, Sunglasses, Sweater,
Swimming trunks, T-shirt, TV, Table knife, Tablecloth, Tablet, Tanktop, Tape, Tape measure, Tarp, Teabag,
Teapot, Tennis racket, Thermometer, Thermos, Throw pillow, Tie, Tissue, Toaster, Toilet paper roll, Tomato,
Tongs, Toothbrush, Toothpaste, Tote bag, Toy, Trash bag, Trash bin, Travel case, Tray, Trophy, Tweezers,
Umbrella, USB cable, USB flash drive, Vacuum cleaner, Vase, Video camera, Walker, Walking cane, Wallet,
Watch, Water bottle, Water filter, Webcam, Weight (exercise), Weight scale, Wheel, Whisk, Whistle, Wine bottle,
Wine glass, Winter glove, Wok, Wrench, Ziploc bag

[dobkeratops]

just flicking through..

• variations on bottle ("spray bottle, squeeze bottle,water bottle,pill bottle,beer bottle") some of which we already have
• variations on brush ("toothbrush","makeup brush","scrub brush")
• sporting items ("tennis racket)
• household appliances ("vacuum cleaner"..)
• variations on bag ("ziploc bag","trash bag")
• lots of specific clothing types
• cooking apppliances
• several specific <something> container (i've used container-> as a graph node already, ->bin , ->bottle etc)
• variations on camera ("video camera","webcam","still camera")
• variations on knife ("table knife","butchers knife")
• tool->multitool ,wrench,..(i've used tool-> as a graph node)
• a few different names to cases i've used "Trash bag"="bin bag", plus "backpack" instead of "rucksack" (alias?)
• I notice "figurine", i've used this as an label in some of yours. they merge "figurine or statue" as one. I tried a graph node "humanoid shape". I hope this can be done with an alias e.g. their "figurine or statue" = "statue/figurine" in our terms
• Irritatingly a few are slightly ambiguous -
• magazine ..I presume that means printed magazine but we've got firearms (box magazine);
• tablet .. by now that probably means ipad/MS surface to most people but it could mean medicine, or stone tablet in archaeology perhaps;
• keyboard (probably means computer keyboard but there's also piano keyboard, synthesizer keyboard
• they list "ring", i bet they mean ring (jewewllery), but there's other general ring shaped components IMO (also inflatable rubber ring). maybe "ring_shape"->ring (jewellery),etc would avoid ambiguity

[bbernhard]

very cool, many thanks for sharing! :)

I like the fact that they try to "break" neural nets. :D

I am wondering if one could gamify collecting those type of images. e.g: by combining random labels and asking the user to snap a photo that shows that. e.g: snap a photo that shows an "apple in a plant pot", "spoon on the television", etc.