a suggestion for increased performance.

[visualperception]

It occurs to me that when regex is being used to check for search engines that by using the full domain name beginning with ^ will increase performance since if the first character is not matched then regex can exit and save checking the whole string. However, since the full domain may well begin www. I would also suggest stripping www. out of the regex check once it has been read into memory levaing example ^google.com$ instead of ^www.google.com$ and then stripping www. from the beginning of each refer domain before sending to regex for testing. This should, I think, improve performance (possibly only marginally but a tad I think). And put a note in docs that wherever possible in search_engines.pm, to use full domain beginning with ^ when entering a new search engine into list.

I may try and amend search_engines.pm when/if I have time but can only do it for saerch engines I have log records for and know what full refer domain is. And of course there are a lot of generic ones in the list already which don't have full ^domain$ format. Having said that, since I added the google domains which cover the vast majority of searches and do have have full ^domain$ format, they should have some performance benefit even with www. included. That could well be the reason why performance has not been significantly degraded by adding so many extra search engine checks. Not 100% sure but I think so.

[edit] Have thought some more about this and maybe it won't make much difference since so many searches are from google that the real identifier for google searches is the country TLD. Now if the regex could start at end of domain and search back to beginning of domain it would be faster but I don't think that's possible with regex without doing a string reverse on the domain and reversing all the regex. Maybe not such a good idea after all.