Added the Stackoverflow fetcher: bring stackoverflow to the console where you already are

[augnustin]

Added the Stackoverflow fetcher: if a query did not return any result, it fetches google with the query and displays the first Stackoverflow result.

Backward compatibility should be ok, but many changes were involved. Among others:

• added a config.rb file for personal API credentials
• added a processor.rb file that takes care of formatting displayed texts in the console
• this involves changes in main: say and created method handle_responses

Readme.md was updated too, with installation details (to get API Key). The config.rb.example file has a working API KEY to get an example working straightaway.

Hope you enjoy it, I already love it!

[lpanebr]

this seems interesting but implemented as a fallback like now it is conflicting with the new behavior of of Betty giving help and sample messages.

Can you make it like a normal Module that is driven by queries like:

$ betty search stack for how to install cmus
$ betty stackoverflow more votes for how to install cmus
$ betty stackoverflow best answer for how to install cmus
$ betty stackover best answer for how to install cmus

Also make it in a new branch from current dev.

[augnustin]

Yes, I tested it around a little bit and I agree that I encountered weird behaviors sometimes. I'll make an update :smile:

[lpanebr]

Great to hear that!

[djbutler]

I was really excited to see this StackOverflow feature, but the API key is a bit of a pain. I have a similar tool for pulling SO snippets (https://github.com/djbutler/oflow) which uses one of Google's undocumented JSON endpoints and does not require an API key, it might be nice to use the same setup here.

[augnustin]

I updated my PR with a regular module for the queries

betty so how to post json data with curl
betty stackoverflow get local time in command line
betty stack open the browser from the console in linux

Among the possible improvements:

• remove the API endpoint as suggested by DJbutler --> I need to check how that is possible, but if it is, it really makes sense
• being able to fetch results: instead of asking "Do you want to execute this command?", I would ask "What do you want to do?" and eg. q=next question, Q=previous question, a=next answer, A=previous answer and finally o would open it in the browser --> But it needs another indepth analysis of how the code work not to mess the existing structure ...

[augnustin]

@djbutler I checked on how your code work. The only sad thing about this is that your code uses an API that is depreciated:

Note: The Google Web Search API has been officially deprecated as of November 1, 2010. It will continue to work as per our deprecation policy, but the number of requests you may make per day will be limited. Therefore, we encourage you to move to Custom Search, which provides an alternative solution.

That could be a decent workaround to use it as long as it is still working, but it won't last forever ... Another possibility would be to get and install the api key during the install.rb script, though it must not be super trivial...

It is super interesting to see how close your oflow script is from this one! It confirms that there's a need for this!

[pickhardt]

This is cool, but whose api key is this?

• GOOGLE_SEARCH_API_KEY = "AIzaSyCxPfXTDuuTjBCTmDlQB5HJsbKfANFlwso"

[djbutler]

@aug-riedinger Yeah it's not ideal. There's yet another incarnation of this idea here: https://github.com/gleitz/howdoi They spoof the user agent field to query www.google.com and then parse the HTML. It is also possible to query SO's search API directly, although it's not as good as Google.

[augnustin]

@pickhardt Everyone with a google account can create as many projects as possible and each project can activate its own API keys.

The one in the example file is given away to test it around, but it is not the one I use on my computer.

@djbutler I thought crawling the web was the worst practice ever (though in this case, it is only about automating the query we would make in the browser anyway ...) And yes, SO's search is nothing comparable to google search.

I agree those solutions are not ideal. Here are the possible solutions I see:

• remain like this
• find a decent API limitless search engine
• use the depreciated API
• crawl the web
• use the SO search engine
• automate the project creation + API key fetching

Any insight which one you'd rather have? I like the last one if hopefully it is achievable ...

Stupid API limitations ...

[pickhardt]

When you say given away, who is giving it away? Wouldn't everyone need to just fetch their own API key?

In that case, I agree with your last solution: "automate the project creation + API key fetching"

Assuming that is straight-forward to do. Mineswell have everyone get their own API key.

[augnustin]

My account, but on an unpaid one so it'll stop making calls when the limit is reached.

I'll have a look at the last one then.