Data download missing data

[robertmartin8]

Based on some feedback received and subsequent experiments, it seems that the data download is missing out a lot of tickers (and if it's missing out the SPY, there will be an error in parsing_keystats.py).

This project downloads price data for free from Yahoo Finance, via pandas-datareader (and fix-yahoo-finance). However, I've noticed lately that the data is becoming a lot more inconsistent, and sometimes just fails completely. This is because Yahoo seems to be dropping their support for this API.

The data on yahoo is still there, it's just a problem of accessing it. In the past I wrote a blog post about downloading data from the linked source, but 'deprecated it' once I realised that pandas-datareader with fix-yahoo-finance did the same thing but much better. My method still works, but it won't be trivial to integrate it with the project (and anyway it's a very clunky solution). I suppose that the easiest solution is to find another data source, so suggestions would be welcome.

As a temporary fix, I have added the csv files (containing all the data) to this repo.

[ictinc]

Hi Robert, Maybe instead of using Yahoo finance use IEX trading instead? https://iextrading.com/developer/docs/

http://pandas-datareader.readthedocs.io/en/latest/readers/iex.html

Cheers, Ronald.

[robertmartin8]

@ictinc

Hi Ronald,

Thank you for the suggestion, it looks like a very clean API and it's great that there's a pandas-datareader interface – it also has adjusted closing prices which is one of my veto factors for considering a service. I'll definitely look into it, it should just be a matter of changing the pandas-datareader call.

Thanks again,

Robert

[ictinc]

Hey Robert, No worries, glad to help in any way I can. Unfortunately I'm not a developer or I would have done it myself.

Cheers, ronald

On Tue, Jun 5, 2018, 05:22 Robert Martin [email protected] wrote:

@ictinc https://github.com/ictinc

Hi Ronald,

Thank you for the suggestion, it looks like a very clean API and it's great that there's a pandas-datareader interface – it also has adjusted closing prices which is one of my veto factors for considering a service. I'll definitely look into it, it should just be a matter of changing the pandas-datareader call.

Thanks again,

Robert

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/robertmartin8/MachineLearningStocks/issues/14#issuecomment-394568931, or mute the thread https://github.com/notifications/unsubscribe-auth/ADoJnJBX0rX6Thc7I300mbfK9xw2eaWpks5t5fl1gaJpZM4ULZ7U .

[jackmoody11]

@robertmartin8 There is a package that already exists to help out with this. It also outputs the data in a pandas format. It's named iexfinance.

[alvarobartt]

Hi @robertmartin8, congratulations on your great work!

I was just checking your GitHub profile when I got to this issue, so I will suggest you to use investpy since it is a Python package I developed for retrieving financial data from Investing.com which is a consistent reliable source and I think investpy can provide you the data you need.

Any further query or doubt feel free to ask me! Good luck with your studies @ Cambridge!

[robertmartin8]

Hi @alvarobartt, thanks for the kind words.

Investpy looks very cool – I am currently not actively maintaining MachineLearningStocks anymore, but I think it's perfect for some of the personal projects I've been doing!

[alvarobartt]

Oh it's ok @robertmartin8! If you need help just let me know!

[robertmartin8]

I've just been recommended another data source called SimFin, which has a really nice python API. Worth taking a look!