[WIP] V3.0.0-alpha.1

[brettallred]

This is the first cut getting this gem working again.

Changes

• Code extracted into modules Quotes, Historical, Splits, Symbols
• Test/Unit moved over to Minitest
• Tests extracted out my module
• Add Byebug for debugging
• Add Yard for documentation
• Change Quotes module to use the V7 Api
• Fix Historical per https://github.com/herval/yahoo-finance/pull/53
• Switch http client to https://github.com/nahi/httpclient

TODO / Questions

• Dividends - Not sure how this was ever working. I don't see a period that related to :dividends_only
• Splits is not working . Need to figure out a new endpoint for that.
• N/A as nil still needs some work
• Documentation - This isn't a CSV endpoint so some of the concepts no longer exist such as column selection.

[herval]

That's amazing! Thanks for taking the time on this

Would you have a second to check what's up with the build? https://travis-ci.org/herval/yahoo-finance/builds/305367376?utm_source=github_status&utm_medium=notification

[bcoles]

Nice work.

I took a super quick look at the build error.

All 5 errors are due to Errno::ENAMETOOLONG: File name too long @ rb_sysopen

This is due to using CSV.foreach. The first argument needs to be a file path. However, the CSV.foreach call in each of the following is parsing the HTTP response as the first argument:

• YahooFinance::FinanceUtilsTest#test_finance_utils_companies
• YahooFinance::FinanceUtilsTest#test_finance_utils_industries
• YahooFinance::FinanceUtilsTest#test_finance_utils_stock_by_market
• YahooFinance::FinanceUtilsTest#test_finance_utils_companies_by_market
• YahooFinance::FinanceUtilsTest#test_finance_utils_sectors

As such, the foreach method is trying to open a huge chunk of text as a file name, which is doomed to failure.

[bcoles]

Without looking at the code, you can probably fix the tests by doing something like CSV.parse(http_response, headers: true) and iterating over that.

Also, the HTTP response should be stored in a variable and checked for validity, rather than cramming the whole HTTP call into the first argument. So, something like the following untested spaghetti pseudo-code:

Before

        companies = []
        next unless MARKETS[country][market]

        CSV.foreach(http_client.get(MARKETS[country][market].url).body) do |row|
          next if row.first == "Symbol"
          companies << map_company(row, market)
        end

After

        companies = []
        next unless MARKETS[country][market]

        http_response = http_client.get(MARKETS[country][market].url).body
        return 'something went horribly wrong fetching the data' if http_response.nil?

        begin
          csv = CSV.parse(http_response, headers: true)
        rescue
          return 'something went horribly wrong parsing the CSV'
        end

        CSV.each(csv) do |row|
          # next if row.first == "Symbol" # don't need this line any more, because we're using `header: true`
          companies << map_company(row, market)
        end

Keep in mind the above is pseudo-code which should resolve the CSV parsing, but whether it achieves what you want I have no idea.

[brettallred]

Had some time today to keep working on this. I've been using this branch in a prod environment for some time now. Travis is having issues today so we will see if the build passes. Everything looks good locally.