Cannot add images with single color lettering on top of a solid background to Augmented Images database

[haroldmthyng]

When building the Augmented Images database, whether programmatically or using the arcoreimg tool, most reference images that are comprised of single color text on top of a solid background color fail to be added to the images database.

Tested this using a simple b&w "stop sign" image and a company logo. ARCore fails for both with the error Failed to get enough keypoints from target image. when using the arcoreimg tool, and of course throws a ImageInsufficientQualityException in Java.

Here is a link to an example of the images ARCore cannot parse and add to the images database. https://www.dropbox.com/sh/rileeu0bgunb916/AABhqc21cWK8Nk2HbU9zobdwa?dl=0

I understand it's best to use images with high contrast, and try to avoid sparse features but most logos and signage are comprised of single color text over a solid background, which is a definitive use for Augmented Images. Both of these text images should be easily parsable for inclusion as references into an Augmented Images target database. Also, keep in mind these same images are easily recognizable within ARKit version 1.5 and newer on iOS.

[inio]

Thanks for pointing this out. I've recorded this as a feature/enhancement request as it doesn't really seem like a defect.

[haroldmthyng]

Thanks @inio . I'm curious what amount of letters are required before a text-based image is considered to have a sufficient amount of key points. Otherwise the recognition is pretty solid.

[raymond-ffs]

I've found that adjusting the white space around the logo drastically affects the score. Example: I doubled the canvas size of your stop sign, padding it with white space. I ended up getting a score of 85. Perhaps the algorithm is just being a little too finicky?

[haroldmthyng]

Prior to posting this issue I tried adding white space, black space. Adding multi-color gradient to the background and around the edges. Adding noise. Adding pixelation. It flat out just doesn't like images with large color blocks.

[raymond-ffs]

I should clarify that I increased the size of the canvas, and left the logo the same size (so the logo itself is a sitting in a large white space. Perhaps this may have altered the contrast ratio of the image itself enough to allow arcoreimg to get a score. It is not an ideal solution, but just thought I'd mention it in case someone was waiting around for this feature request.

[haroldmthyng]

Very interesting. Not an ideal solution at all, but definitely a work-around. Played around with power of 2 sizes along with using multiples of common power of 2 sizes. Managed to get a score of 100 increasing canvas size to 1536 or (6 X 256).

But that's just playing around with sizes and seeing what scores come back. It's interesting that increasing to 1664 yielded a score of 75, 1792 a score of 90, and 2048 a score of 0.

All of this was against the Uniqlo logo.

[sgnm]

Hi, I faced the same problem too. As for me, scaling the size of the canvas while leaving the logo the same size didn't work. Resizing the logo to 95% worked well.

In my experience, square is better than rectangle.

[sgnm]

Uhh sorry... I tried another image but my way didn't work... I guess each image depends on white space, resolution and scale. So, It's better to resize image over and over...

[michaelvogt]

Yes, my experience, too. So far, every marker needed quite a bit of experimentation to get good quality scores from arcoreimg tool.

Glad that I'm not the only one,. Gives me hope that this will be fixed.

[haroldmthyng]

@inio per all the comments above, hoping you reconsider treating this as a defect vs a feature request. The level of image manipulation required to receive a usable score for these simpler images seems excessive and a common issue other developers are running into.

[AlbertSnow]

Thanks for pointing this out. I've recorded this as a feature/enhancement request as it doesn't really seem like a defect.

@inio Hello inio, I meet the same problem. Our team purpose is track sentence, and draw translated word on that. But the arcore not work on single sentence. I mean maybe google can release some branch for text track. Tranks about that.

[schematical]

Having the same issue. Any word on the status of Text/Basic Logo/Symbol tracking?

[ajwilkinson]

I have also just run into this. Here, we have a very "curvy" white on orange company logo, and we have some big display boards with this on so the idea was to use this as an anchor. However, as per everyone's experience above, this doesn't work at the moment. Perhaps this is because freak feature extraction needs a lot more corners and edges that a standard two colour logo would contain?

[McBrideMusings]

Chiming in as well, I'm having the same issue at my job. We're attempting to track warning labels/stickers on motor vehicles, some of which are just a symbol not unlike a logo. Many of the stickers we're attempting to add do not have enough feature points, according to the arcoreimg tool.

[fredsa]

As noted in this issue. Augmented Images currently work best when using images that contain lots of visual features. That is, images like the samples included in the SDK will likely work better than simple pictograms.

For additional approaches that may work for your project, see https://firebase.google.com/products/ml-kit

[machinesarenotpeople]

Any updates on this? The problem with the current requirements is that anchor "photos" are not as common as symbols such as logos, severly limiting where ARCore can be used.

[kushallokesh]

Any updates on this? The problem with the current requirements is that anchor "photos" are not as common as symbols such as logos, severly limiting where ARCore can be used.

Having the same issue, let me know if you come across any useful workarounds

[jankais3r]

+1 for having this issue resolved after 3 years... Images that work without any issue in ARKit are outright being refused by ARCore… What is the workaround when real world does not allow any tweaking to the target image?

[fredsa]

Augmented Images is implemented in such a way that might lead to somewhat counter-intuitive or unexpected constraints/behavior.

Augmented Images does not perform straight-up image-detection as one might guess, but instead works as follows:

Creating an entry in the Augment Image database:

1. ARCore reads in the image
2. ARCore extract the grayscale information (discarding RGB data)
3. ARCore identifies key "features" or feature points in the image; these are the same features that are expected to be picked up by ARCore's tracking algorithm if the image were place in front of the device's camera at runtime

Locating an Augmented Image:

1. ARCore identifies visual "features" in the full camera image. You can think of these "features" as the points in the point cloud.
2. ARCore finds features that appear to lie on a plane (=flat surface)
3. ARCore compares these features to the features in the Augmented Image database that is currently active
4. If there's a sufficient match of features, ARCore performs the math calculation to determine the orientation of the image relative to the device (and thus the world)

It's because of the above that images with very few features, such as some signs or logos, cannot be added to the database, or are given a low quality score.

There are two potential workarounds:

1. If the area surrounding the target of interest does have sufficient detail, then you can add a larger image to the database as a target, and do the math in your app to determine the smaller rectangle where the desired image is.
2. Do not use Augmented Images and instead use the CPU image to run your own image detection ML model

[shacharoz]

if standard, old fashioned, black n white AR markers do not work, it is quite disturbing. i saw that if i take a photo of the marker with my phone, it technically works, although bad tracking. please try to figure out how to use black n white markers.

[GinoValerio]

I've found that adjusting the white space around the logo drastically affects the score. Example: I doubled the canvas size of your stop sign, padding it with white space. I ended up getting a score of 85. Perhaps the algorithm is just being a little too finicky?

Solved my problem thanks!!

[sinianyutian]

这是来自QQ邮箱的假期自动回复邮件。   您好，谢谢你的邮件！

[fancychendong]

I've found that adjusting the white space around the logo drastically affects the score. Example: I doubled the canvas size of your stop sign, padding it with white space. I ended up getting a score of 85. Perhaps the algorithm is just being a little too finicky?

Solved my problem thanks!!

I tried it, still doesn't work. I just want to recognize the letter logo icon on the shoe box. I have no clue now, can anyone help me

[fredsa]

I'd say that recognizing the letters in a logo on a shoe box is not a great use case for Augmented Images.

I'd recommend using a ML model that is trained for this purpose. You can combine it with ARCore, see https://developers.google.com/ar/develop/machine-learning