Parser-PHP icon indicating copy to clipboard operation
Parser-PHP copied to clipboard

Undetected user agents

Open JuMiLo opened this issue 6 years ago • 10 comments

Hi,

the following user agents doesn't get detected.

Agents with "browser->using" object: Mozilla/5.0 (Linux; Android 8.0.0; SM-G950F Build/R16NW; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/69.0.3497.100 Mobile Safari/537.36

Mozilla/5.0 (Linux; Android 6.0.1; SM-G800F Build/MMB29K; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/69.0.3497.100 Mobile Safari/537.36

Agents with empty "browser" object: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/601.7.8 (KHTML, like Gecko)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14) AppleWebKit/605.1.15 (KHTML, like Gecko)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/534.57.7 (KHTML, like Gecko)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.59.10 (KHTML, like Gecko)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/605.1.15 (KHTML, like Gecko)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/603.3.8 (KHTML, like Gecko)

Mozilla/5.0 (iPhone; CPU iPhone OS 12_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/16A366

Mozilla/5.0 (iPad; CPU OS 12_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/16A366

Mozilla/5.0 (Linux; NetCast; U) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/34.0.1847.137 Safari/537.31 SmartTV/6.0

Mozilla/5.0 (iPad; CPU OS 9_3_5 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Mobile/13G36

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/605.1.15 (KHTML, like Gecko)

Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; tb-gmx/2.6.12; (gmx/1.5.0.0); rv:11.0) like Gecko

Mozilla/5.0 (iPhone; CPU iPhone OS 12_0_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Snapchat/10.43.0.25 (iPhone7,2; iOS 12.0.1; gzip)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/605.1.15 (KHTML, like Gecko)

Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.9.0.1) Gecko/2008070208

Mozilla/5.0 (Windows; U; Windows NT 6.1; rv:2.2) Gecko/20110201

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko)

Mozilla/5.0 (iPad; CPU OS 10_3_3 like Mac OS X) AppleWebKit/603.3.8 (KHTML, like Gecko) Mobile/14G60

Mozilla/5.0 (iPad; CPU OS 11_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E218

Mozilla/5.0 (iPhone; CPU iPhone OS 12_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) (Ecosia [email protected])

Mozilla/5.0 (iPad; CPU OS 11_2_6 like Mac OS X) AppleWebKit/604.5.6 (KHTML, like Gecko) Mobile/15D100

Mozilla/5.0 (iPhone; CPU iPhone OS 10_3_3 like Mac OS X) AppleWebKit/603.3.8 (KHTML, like Gecko) Mobile/14G60

Mozilla/5.0 (iPhone; CPU iPhone OS 12_0_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/16A405

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/600.8.9 (KHTML, like Gecko)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/603.1.30 (KHTML, like Gecko)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.78.2 (KHTML, like Gecko)

Mozilla/5.0 (Macintosh; Intel Mac OS X) AppleWebKit (KHTML, like Gecko)

Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; tb-webde/2.7.4; (webde/1.4.0.0); rv:11.0) like Gecko

swcd (unknown version) CFNetwork/974.2.1 Darwin/18.0.0 swcd (unknown version) CFNetwork/758.5.3 Darwin/15.6.0 swcd (unknown version) CFNetwork/889.9 Darwin/17.2.0 swcd (unknown version) CFNetwork/894 Darwin/17.4.0 swcd (unknown version) CFNetwork/897.15 Darwin/17.5.0 swcd (unknown version) CFNetwork/902.2 Darwin/17.7.0 swcd (unknown version) CFNetwork/901.1 Darwin/17.6.0 swcd (unknown version) CFNetwork/893.14.2 Darwin/17.3.0 swcd (unknown version) CFNetwork/975.0.3 Darwin/18.2.0 SafeBrowser/1.11.0.38 CFNetwork/974.2.1 Darwin/18.0.0

JuMiLo avatar Oct 18 '18 12:10 JuMiLo

  • Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_3) AppleWebKit/601.4.4 (KHTML, like Gecko)

  • Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/601.7.7 (KHTML, like Gecko)

  • Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/604.3.5 (KHTML, like Gecko)

  • Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/605.1.15 (KHTML, like Gecko)

  • Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; Touch; (webde/1.5.0.0); rv:11.0) like Gecko

  • Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; Touch; rv:11.0; BR-Safe-Browsing) like Gecko

  • Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; tb-gmx/2.6.9; (gmx/1.4.0.0); rv:11.0) like Gecko

  • Mozilla/5.0 (Windows NT 6.1; Trident/7.0; tb-gmx/2.7.5; (gmx/1.4.0.0); rv:11.0) like Gecko

  • Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; tb-webde/2.7.4; (web.de/1.3.0.0); rv:11.0) like Gecko

  • Mozilla/5.0 (Windows NT 6.1; Win64; x64; Trident/7.0; (webde/1.4.0.0); rv:11.0) like Gecko

  • Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.1) Gecko/20100122 firefox/3.6.1

  • Mozilla/5.0 (iPad; CPU OS 10_3_2 like Mac OS X) AppleWebKit/603.2.4 (KHTML, like Gecko) Mobile/14F89

  • Mozilla/5.0 (iPad; CPU OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko)

  • Mozilla/5.0 (iPhone; CPU iPhone OS 10_2 like Mac OS X) AppleWebKit/602.3.12 (KHTML, like Gecko) Mobile/14C92

  • Mozilla/5.0 (iPhone; CPU iPhone OS 11_2_2 like Mac OS X) AppleWebKit/604.4.7 (KHTML, like Gecko) Mobile/15C202

  • Mozilla/5.0 (iPhone; CPU iPhone OS 11_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15F79

  • Mozilla/5.0 (iPhone; CPU iPhone OS 9_3_5 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Mobile/13G36

  • swcd (unknown version) CFNetwork/808.2.16 Darwin/16.3.0

  • swcd (unknown version) CFNetwork/811.4.18 Darwin/16.5.0

JuMiLo avatar Nov 02 '18 18:11 JuMiLo

I made a test for Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.59.10 (KHTML, like Gecko). I get:

WhichBrowser\Parser Object
(
    [browser] => WhichBrowser\Model\Browser Object
        (
            [using] => 
            [family] => 
            [channel] => 
            [stock] => 1
            [hidden] => 
            [mode] => 
            [type] => 
            [name] => 
            [alias] => 
            [version] => 
        )

    [engine] => WhichBrowser\Model\Engine Object
        (
            [name] => Webkit
            [alias] => 
            [version] => WhichBrowser\Model\Version Object
                (
                    [value] => 534.59.10
                    [hidden] => 
                    [nickname] => 
                    [alias] => 
                    [details] => 
                    [builds] => 
                )

        )

    [os] => WhichBrowser\Model\Os Object
        (
            [family] => 
            [edition] => 
            [hidden] => 
            [name] => OS X
            [alias] => Mac OS X
            [version] => WhichBrowser\Model\Version Object
                (
                    [value] => 10.6.8
                    [hidden] => 
                    [nickname] => 
                    [alias] => 
                    [details] => 2
                    [builds] => 
                )

        )

    [device] => WhichBrowser\Model\Device Object
        (
            [manufacturer] => Apple
            [model] => Macintosh
            [series] => 
            [carrier] => 
            [identifier] => 
            [flag] => 
            [type] => desktop
            [subtype] => 
            [identified] => 1
            [generic] => 1
            [hidden] => 1
        )

    [camouflage] => 
    [features] => Array
        (
        )

    [cache:WhichBrowser\Parser:private] => 
    [expires:WhichBrowser\Parser:private] => 
    [cached] => 
)

It says stock, so it's not empty. What result do you expect?

rvalitov avatar Nov 11 '18 12:11 rvalitov

With empty I mean that no "browser->name" or "browser->version" is detected.

JuMiLo avatar Nov 12 '18 15:11 JuMiLo

Is there a way to add user agents manually?

JuMiLo avatar Nov 12 '18 15:11 JuMiLo

I can see myself quite plainly that the reason you're not getting any browser information back is that there's no browser information in most of the above user agent strings. You can't have what's not there. There was one that I thought would be caught, e.g. Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.1) Gecko/20100122 firefox/3.6.1 but wasn't, but all of the others that I tried that have any browser information came up fine.

etetlow avatar Jan 28 '19 11:01 etetlow

Hi,

the following user agents doesn't get detected.

atc/1.0 watchOS/5.1.3 model/Watch3,4 hwp/t8004 build/16S5535a (6; dt:156)

jfarcas avatar Jan 29 '19 17:01 jfarcas

Outlook on iOS does not get detected. The value in parenthesis is the iOS outlook app version I believe.

Outlook-iOS/687.32416.prod.iphone (2.81.0)

ericmillsio avatar Feb 04 '19 17:02 ericmillsio

Some bots that are not added in the library that I had to add them manually

Would be nice to add them in the repo for future versions

    [ 'name' => 'Cron Job',                     'id'    => 'cron-job',      'regexp' => '/cron-job.org/u' ],
    [ 'name' => 'Google Stackdriver Monitoring - Uptime Checks', 'id'    => 'googlestackdrivermonitoring',      'regexp' => '/GoogleStackdriverMonitoring-UptimeChecks/u' ],
    [ 'name' => 'Byte Spider',                  'id'    => 'bytespider',      'regexp' => '/Bytespider/u' ],
    [ 'name' => 'Dot Bot',                      'id'    => 'dotbot',      'regexp' => '/DotBot\/([0-9.]*)/u' ],
    [ 'name' => 'Nimbostratus Bot',             'id'    => 'nimbostratus-bot',      'regexp' => '/Nimbostratus-Bot\/([0-9.]*)/u' ],
    [ 'name' => 'Uptime Bot',                   'id'    => 'uptimebot',      'regexp' => '/Uptimebot\/([0-9.]*)/u' ],
    [ 'name' => 'Netcraft Survey Agent',        'id'    => 'netcraftsurveyagent',      'regexp' => '/NetcraftSurveyAgent\/([0-9.]*)/u' ],
    [ 'name' => 'Domain Crawler',               'id'    => 'domaincrawler',      'regexp' => '/domaincrawler.com/u' ],
    [ 'name' => 'Zoominfo Bot',                 'id'    => 'zoominfobot',      'regexp' => '/zoominfobot/u' ],
    [ 'name' => 'FullStory Bot',                'id'    => 'fullstorybot',      'regexp' => '/FullStoryBot\/([0-9.]*)/u' ],
    [ 'name' => 'Semrush Bot',                  'id'    => 'semrushbot',      'regexp' => '/SemrushBot\/([0-9.]*)/u' ],
    [ 'name' => 'MJ12 Bot',                     'id'    => 'mj12bot',      'regexp' => '/MJ12bot/u' ],
    [ 'name' => 'Survey Bot',                   'id'    => 'surveybot',      'regexp' => '/SurveyBot\/([0-9.]*)/u' ],
    [ 'name' => 'BLEX Bot',                     'id'    => 'blexbot',      'regexp' => '/BLEXBot\/([0-9.]*)/u' ],
    [ 'name' => 'Mail.RU Bot',                  'id'    => 'mail\.ru_bot',      'regexp' => '/Mail.RU_Bot\/([0-9.]*)/u' ],
    [ 'name' => 'Researchscan',                 'id'    => 'researchscan',      'regexp' => '/Researchscan/u' ],
    [ 'name' => 'OrgProbe',                     'id'    => 'orgprobe',      'regexp' => '/OrgProbe\/([0-9.]*)/u' ],
    [ 'name' => 'Grapeshot Crawler',            'id'    => 'grapeshotcrawler',      'regexp' => '/GrapeshotCrawler\/([0-9.]*)/u' ],
    [ 'name' => 'SP Bot',                       'id'    => 'spbot',      'regexp' => '/spbot\/([0-9.]*)/u' ],
    [ 'name' => 'KomodiaBot',                   'id'    => 'komodiabot',      'regexp' => '/KomodiaBot\/([0-9.]*)/u' ],
    [ 'name' => 'WhatsApp',                     'id'    => 'whatsapp',      'regexp' => '/WhatsApp\/([0-9.]*)/u' ],
    [ 'name' => 'Bugsnag',                      'id'    => 'bugsnag',      'regexp' => '/Bugsnag/u' ],
    [ 'name' => 'SP Bot',                       'id'    => 'spbot',      'regexp' => '/spbot\/([0-9.]*)/u' ],
    [ 'name' => 'MASSCAN',                      'id'    => 'masscan',      'regexp' => '/masscan\/([0-9.]*)/u' ],
    [ 'name' => 'Cloudflare-AMP',               'id'    => 'cloudflare-amp',      'regexp' => '/Cloudflare-AMP\/([0-9.]*)/u' ],
    [ 'name' => 'aiHit Bot',                    'id'    => 'aihitbot',      'regexp' => '/aiHitBot\/([0-9.]*)/u' ],
    [ 'name' => 'Coccocbot Image',              'id'    => 'coccocbot-image',      'regexp' => '/coccocbot-image\/([0-9.]*)/u' ],
  

also there is a spelling mistake in CloudFlare, it is written CloudFare

[ 'name' => 'CloudFlare Always Online', 'id' => 'cloudflare', 'regexp' => '/CloudFlare-AlwaysOnline\/([0-9.]*)/u' ],

maraouf avatar May 05 '20 21:05 maraouf

Here's another one that is not detected: "check_http/v2.3.1 (nagios-plugins 2.3.1)"

Not a big deal, since the only usual use case for this one is checking that a site is up, but FYI. That is the complete user-agent string. Browscap seems to be the only library that picks it up. http://thadafinser.github.io/UserAgentParserComparison/v4/user-agent-detail/dc/d5/dcd5e41b-7bc9-4a3b-b233-135826d90385.html

Xaraxia avatar May 12 '20 04:05 Xaraxia

This is not my repo, but the last few comments are hijacking this issue.

You should open a separate issue and create a pr for any missing bots, crawlers, devices or browsers.

You can see some pr's I've opened as an example.

summercms avatar Jun 11 '20 18:06 summercms