2700+ User-Agents that have missing brand/model + new browsers and bots

[liviuconcioiu]

missing.txt Browsers:

DuckDuckGo Privacy Browser - DDG-Android-3.1.1
Faux Browser - Faux/20151018 CFNetwork/808.1.4 Darwin/16.1.0
Meizu Browser - Mozilla/5.0 (Linux; U; Android 6.0; zh-CN; MZ-M5c Build/MRA58K) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/40.0.2214.89 MZBrowser/6.9.410-2017122215 UWS/2.11.0.33 Mobile Safari/537.36
Crosswalk Browser - Mozilla/5.0 (Linux; Android 7.1.1; Z983 Build/NMF26F) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.94 Mobile Crosswalk/11.45.2454.20170222 Mobile Safari/537.36
EUI Browser - Mozilla/5.0 (Linux; U; Android 6.0.1;en_gb; C103 Build/ZIXOSOP5801803011S) AppleWebKit/537.36 (KHTML, like Gecko)Version/4.0 Chrome/49.0.0.0 Mobile Safari/537.36 EUI Browser/5.8.018S
Turbo Browser - Mozilla/5.0 (Linux; Android 6.0; es-us; 5044O Build/MRA58K) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/55.0.2883.91 Mobile Safari/537.36 Hawk/TurboBrowser/2.0.51
Baidu Mobile Browser - Mozilla/5.0 (Linux; Android 6.0.1; 6055P Build/MMB29M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/63.0.3239.111 Mobile Safari/537.36 bdbrowser/1.9.0.2

Bots:

Mozilla/5.0 (compatible; evc-batch/2.0)
Mozilla/5.0 (compatible; Cloudflare-AMP/1.0; +https://amp.cloudflare.com/doc/fetcher.html) AppleWebKit/534.34
Twitter/7.14 CFNetwork/893.14.2 Darwin/17.3.0
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Sparkler/0.2.0-SNAPSHOT
Tarmot Gezgin/1.0 (compatible; TarmotGezgin/1.1; +http://www.tarmot.com/gezgin)
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.34 (KHTML, like Gecko) Qt/4.8.2
node-urllib/2.25.3 Node.js/6.12.2 (Linux 4.4; x64)
SiteSucker for macOS/2.10.5
Validator.nu/LV http://validator.w3.org/services
Octopus 1.0.2
Mozilla/5.0 (compatible; ips-agent)

Devices/Models:

See attached file.

[sgiehl]

Thanks for your data. I'll try to add some more detections with it...

[sgiehl]

Twitter/7.14 CFNetwork/893.14.2 Darwin/17.3.0

This is more likely a Twitter app on an ios device. Guess it shouldn't be detected as bot. Or does it fetch data without any user action?

[sgiehl]

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Sparkler/0.2.0-SNAPSHOT

Can't find enough information to prove it is a bot

[sgiehl]

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.34 (KHTML, like Gecko) Qt/4.8.2

Why should that be a bot? The Qt might mean QuickTime

[Findus23]

Why should that be a bot? The Qt might mean QuickTime

I think this is rather a program based on Qt that displays a website in a webview.

[liviuconcioiu]

@Findus23 is right. That one is based on Qt. Regarding Sparkler, here is a link https://github.com/USCDataScience/sparkler

[liviuconcioiu]

Browsers:

Phantom Browser - Mozilla/5.0 (LG-C205 AppleWebkit/531 Browser/Phantom/V2.0 Widget/LGMW/3.0 MMS/LG-MMS-V1.0/1.2 Java/ASVM/1.1 Profile/MIDP-2.1 Configuration/CLDC-1.1) - Details here: https://trademarks.justia.com/850/69/lg-phantom-browser-85069626.html
Orca Browser - Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009030821 Firefox/3.0.7 Orca/1.1 build 2

Bots:

NetTrack Anonymous Web Statistics https://nettrack.info/support.php
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5 Build/MRA58N) AppleWebKit/537.36(KHTML, like Gecko) Chrome/61.0.3116.0 Mobile Safari/537.36 Chrome-Lighthouse
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.134 Safari/537.36 http://notifyninja.com/monitoring
Mozilla/5.0 (compatible; alexa site audit/1.0; +http://www.alexa.com/help/webmasters; )
Quora Link Preview/1.0 (http://www.quora.com)

[e-nouri]

@sgiehl This is not a twitter app, it is an apple bot, that checks for the apple-app-site-association file to link it to the installed app.

[AnilGupte]

Let me add one more not recognized: Mozilla/5.0 (CrKey armv7l 1.5.16041) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.0 Safari/537.36

This is a ChromeCast device

[sgiehl]

result of the useragent is:

user_agent: Mozilla/5.0 (CrKey armv7l 1.5.16041) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.0 Safari/537.36
os: [ ]
client:
  type: browser
  name: Chrome
  short_name: CH
  version: 31.0.1650.0
  engine: Blink
  engine_version: ""
device:
  type: tv
  brand: GO
  model: Chromecast
os_family: Unknown
browser_family: Chrome

[AnilGupte]

I should have clarified. The OS is not recognized. Also, how do you get type: tv - I only get integers?

[GregOriol]

@AnilGupte I might be wrong, but there don't seem to be any OS information in that user-agent.

To get a device type name ("tv") instead an integer (5), check the DeviceParserAbstract class, which has the values and the associated names. You may want to use DeviceParserAbstract::getDeviceName($deviceType); to get the name in an easy way (where the $deviceType parameter is the number you got).

[sgiehl]

you can also directly use ->getDeviceName() instead of ->getDevice() to get the name instead of an integer. And yes, the useragent does not include any OS information. But afaik the OS in chromecast is always the same, so we could set it as soon as chromecast is detected. But actually I'm not sure what the ChromeCast has as OS. Some are saying it's more an Android, others it's Chrome OS.

[AnilGupte]

Thanx @sgiehl - yes that works! @GregOriol - I have no idea which is why I posted it here so that people who do can include this in the issues to be addressed. Thanx!