Add tests

[cxw42]

Per discussion with @gregsadetsky. It would be nice to have some!

• [ ] Consolidate the list of sites people have reported here and on the Web Store
  • Report these sites via the wiki page
  • Edit To clarify, this thread is not the place to report specific sites :) . This thread is for figuring out how we are going to implement the tests.
• [ ] Add tests to detect when the extension stops working on those sites

It's tricky, though, since we can't access (or even navigate to!) the pertinent settings page (link here - go there and try clicking! see what happens).

We can, however, loop over a list of testcases (e.g., document.querySelector('.wiki-content table') in the wiki page) and, for each:

• open that site in a new window
• onload (i.e., after our content script has run), check to see if we touched the page
• if not, report an error

I have used Jasmine, which I believe would not have a problem with this type of test. It has asynchronous support, so we could bounce messages through the background between a Jasmine popup and the windows it creates.

Better ideas welcome!

[snowbound]

I briefly had this installed but noticed that it has issues with bleepingcomputer.com where if you search for anything on the site then bleepingcomputer.com is added to your search engines.

[cxw42]

@snowbound Thanks for reporting! The branch from #6, or the current Chrome Web Store version (0.0.1)? The #6 version works for me on bleepingcomputer.

[snowbound]

I downloaded it via https://github.com/gregsadetsky/chrome-dont-add-custom-search-engines and loaded it via UnPacked extension in Chrome via Developer Mode. I went to Bleeping Computer, entered garbage in their search box. Went into Chrome's Search Engine list and bleepingconputer.com was in the list.

[cxw42]

@snowbound OK - that's the old version, which is known to fail. Thanks for confirming! The pull-6 version (zip) should work if you unload the version you have, and then load the pull-6 version unpacked.

For the record, thanks to @gregsadetsky for finding this article showing that Selenium can access chrome://* pages. He says "we'd have to see if the automated testing sites out there (travis, circle ci, etc.) support this."

[cxw42]

@gregsadetsky Any objection to a gh-pages branch to hold tests? I ran a combinatorial test suite here and would like to move it out of my personal site and put it in better condition to automate. Source.

[gregsadetsky]

@cxw42 great idea, go for it! thank you!

[cxw42]

@gregsadetsky I added the gh-pages branch. Would you please turn it on in the settings? Thanks!

[gregsadetsky]

sorry, what do you mean by turn it on? I see that "Your GitHub Pages site is currently being built from the gh-pages branch" but I'm not sure which URL this repo would appear on

it's a bit of a mess, as my gregsadetsky.github.io points to another repo right now

thoughts?

[cxw42]

OK - how about this different approach? A test extension so that pressing a button will perform searches on all the known problematic sites. After you press that button, you can visit chrome://settings/searchEngines to see if any were added.

Implementation outline:

• In the main source tree, two extensions: the usual one, and a test extension.
• The test extension pulls a test plan via XHR from the main repo (not the wiki --- we need stronger control to drive automated actions)
• The test extension, when activated, opens a tab for each site listed in the test plan.
• The test extension's content script, when loaded on a site, performs a search by issuing clicks or other events listed in the test plan for that site.

[cxw42]

I am updating the wiki page to list full Query URL rather than just site. That way we will have a database on which to run automated tests.