uadetector
uadetector copied to clipboard
arouel
http://user-agent-string.info/ has closed down
They issued an xml file dated 11/30 which does not contain any user agent strings. So if you use any of the updating parsers, it cant detect any user agents. As a work around, I used the non updating parser.
That is very inconvenient... we have servers running in production with the updating parser :-(
Any plans or ideas? (other than temporarily switch to the non-updating parser)
We are also switching to the non-updating parser. I am just wondering about the impact of this in the medium term. I assume that the vast majority of detection is pattern based, so even without updates we will continue to correctly detect the vast majority of UAs for some time to come. Only if some completely new browser type comes along will it fail to classify it. Am I correct in this thinking?
We are trying to figure out if we can block the updating of the database on network level, so that it doesn't get the useless empty database. Uadetector should then fall back to the included database.
Update: that works and either clean the tmp folder with the empty uas.xml or replace it with the October version from https://github.com/before/uadetector/blob/master/modules/uadetector-resources/src/main/resources/net/sf/uadetector/resources/uas.xml.
@before Maybe put a note on uadetector.sourceforge.net, the github project page and on Twitter.
@HaraldWalker: Good idea, will do it.
@all: How do we want to deal with the status quo?
- The new service uses a different URL which could be controlled by a System Property.
- Users that are not willing to logon to udger.com should work with the existing UAS data that is shipped with uadetector-resources. We should recommend to use
UADetectorServiceFactory.getResourceModuleParser()then.
What do you think?
I'm still not clear how it will work with udger.com. Is the url controlled with username/password? Presumably then you would have to code the authentication mechanism into UADetector. That would work for me and I'm not unwilling to pay for the data at the prices they have published.
But am sure lots of people will not want to purchase a licence -- there is always the option to use the current last known good resources, but how quickly does it age and become less useful I wonder?
40€/month is more than I pay Amazon for my beefy web server. The pricing kind of excludes anyone who has been using the user-agent-string data for non-commercial purposes. :(
@before
Short term: Consider modifying the code so you don't fetch the empty database any longer. For some projects it will be a quick fix to update the jar dependency. Easier than adjusting the code and going through a complete release and deployment cycle. By returning the empty database they overwrote the last working version people had in the local cache. That was certainly not a nice move.
Long term: Do we and can we use the commercial database of udger.com or is there an alternative or can we create an open source alternative? Not sure how difficult it would be to maintain the UAS database. We can probably manage some small fixes and additions.
All the other libraries that used the user-agent-string.info database have the same problem. A lot of open source projects and implementations are being affected by this situation.
@jugglingcats depends on how detailed you need the detection to be. In our product we care more about the higher level categorizations (the UA family, OS family etc.) for reporting purposes. Those should be fine for quiet a while with the last available database. Enough time to find a solution.
I agree with @HaraldWalker on all points. Our goal is to have high level classification: to identify mainstream bots and between common mobile vs. desktop browsers. We don't need it to be 100% accurate.
Before I used uadetector, I've used http://browscap.org/. I switched for performance reasons and the lack of a Java library. But this could be a usable data source as a replacement for user-agent-string. Just my two cents.
I checked some time ago the browscap database and it contains a lot of unverified patterns. Perhaps it changed these days but it would require a new library. At the moment the domain model of UADetector is bound tightly to the database of http://user-agent-string.info.
I experimented some months ago with a different approach of reading databases and become much more flexible by using Scala and the awesome library scalaz-stream which runs under Java as well. It comes with the overhead depending on the Scala Libraries but the runtime performance of this implementation was surprisingly quick.
If I would write something new to work on a different database as well than only by implementing it in Scala.
As far as I can tell @HaraldWalker is right, short term. Turn off automatic updates now and hope you have a good version in the on disk cache (or backups). The new owners at udger.com at least have a compatible data format download available after (free) registration so you won't be totally stuck if you have to go that route.
uadetector medium term needs to make clear we don't curate the data, and that you can get some (all the major desktop and mobile clients ?) for free from udger.com who will also be happy to sell you data with bots etc. in, as well as fresher updates.
any updates on this? i'd like to second @Funzinator's suggestion of using the data provided by http://browscap.org/
As it turns out, the old style database from udger.com is not 100% compatible. First of all the DTD changed, and second, I notice already one change in the XML. E.g. browser (family) with id 394 is now 'Safari mobile' and previously it was 'Mobile Safari'. That means the mapping between the database and uadetector enums is broken. If we fix this, then we are no longer compatible with the old format.
So sad to see this library succumb to what seems like closed-sourcing of the UA database. I've used UAdetector on several projects with great success. Is there any hope for the future? Is there any free alternative?
