YFinance changed their api's (or their security)

[TobiasFP]

Hi. There seams to be an issue with the finance-go, since yFinance changed its encryption keys. Apparantly they change their encryption keys every day or even sometimes more than once a day. This has lead to finance-go not working for a while. yFinance has the same issue: https://github.com/ranaroussi/yfinance/issues/1407

I just wanted to bring it to your attention. I have been very happy with this library, so thank you very much for your work.

[ktpx]

v6 of the API seems to work, not 7.

[md-dev-lab]

@ktpx You saved me! :)

[ktpx]

I think they pulled the plug on v6 now. :(

[joce]

All projects I see using the Yahoo API are facing this. There is this nugget I got from the ticker issue report on the subject:

https://stackoverflow.com/questions/76065035/yahoo-finance-v7-api-now-requiring-cookies-python

Not sure how/if that helps...

[TiagoNH]

Saw somewhere that for instance, an url like this: https://query2.finance.yahoo.com/v6/finance/quoteSummary/AAPL?modules=financialData is working, but I'm not sure if the data structures are the same. Is someone working on a patch?

[joce]

https://github.com/achannarasappa/ticker/issues/250#issuecomment-1563423747 Looks like we're out of luck. 😢

[ktpx]

Saw somewhere that for instance, an url like this: https://query2.finance.yahoo.com/v6/finance/quoteSummary/AAPL?modules=financialData is working, but I'm not sure if the data structures are the same. Is someone working on a patch?

Totally different structure than the quotes. This show financial data. But atleast one can grab the currentprices.

[LandRover]

Crumb is a param that is easy to gain, not sure how it will behave in long term.

For now, it is possible to generate crumb via https://query2.finance.yahoo.com/v1/test/getcrumb and then pass crumb value as param https://query2.finance.yahoo.com/v7/finance/quote?symbols=AAPL&crumb=abcdef.g. (Works with v7 only)

I have had the same crumb for a few days now, so it's possible to cache it locally on disk to avoid this call.

[ktpx]

If you inspect the cookie header, it says one year. "Expires=Wed, 5 Jun 2024 22:54:09 GMT"

[rosimildo]

Hi there, is there any workaround I could use to test ? This is broken for a while.

[joce]

So I have a fix for this and I have opened a PR for it. Not sure who should be reviewing it. I also have "modernized" the finance-mock project used to test finance-go. PR can be found here: https://github.com/piquette/finance-mock/pull/1

[TobiasFP]

I would love to review it, but i don't think we can get it in as i think the project is dead here. Maybe fork?

[joce]

Is it, though? There were a couple of PR that were merged a month ago by @ackleymi .

[TobiasFP]

My bad. Just add me to the pr if you want

[joce]

Doesn't look like I can add you, but feel free to head over the the PR and comment on it: https://github.com/piquette/finance-go/pull/28

[TobiasFP]

Doesn't look like I can add you, but feel free to head over the the PR and comment on it: https://github.com/piquette/finance-go/pull/28

Done. Good job!

[ktpx]

Did they add some kind of rate limit? I don't get a crumb anymore, just "Too many requests".

[joce]

I don't have access to a machine with the library on it atm. I'll check about this later today.

On Thu, Jul 20, 2023 at 3:32 PM ktpx @.***> wrote:

Did they add some kind of rate limit? I don't get a crumb anymore, just "Too many requests".

— Reply to this email directly, view it on GitHub https://github.com/piquette/finance-go/issues/25#issuecomment-1644483451, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAATT2NOHUSYZVJRSGVCHALXRGBTHANCNFSM6AAAAAAVGDRPD4 . You are receiving this because you commented.Message ID: @.***>

[ktpx]

I don't have access to a machine with the library on it atm. I'll check about this later today. … On Thu, Jul 20, 2023 at 3:32 PM ktpx @.> wrote: Did they add some kind of rate limit? I don't get a crumb anymore, just "Too many requests". — Reply to this email directly, view it on GitHub <#25 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAATT2NOHUSYZVJRSGVCHALXRGBTHANCNFSM6AAAAAAVGDRPD4 . You are receiving this because you commented.Message ID: @.>

Not a software issue really, API.

curl  https://query2.finance.yahoo.com/v1/test/getcrumb
Too Many Requests

Thats atleast the URL i used in my progs

[joce]

You need to have a cookie set when you call getcrumb. Just calling to it from curl won't work.

On Thu, Jul 20, 2023 at 4:19 PM ktpx @.***> wrote:

I don't have access to a machine with the library on it atm. I'll check about this later today. … <#m_9094901114993369745_> On Thu, Jul 20, 2023 at 3:32 PM ktpx @.> wrote: Did they add some kind of rate limit? I don't get a crumb anymore, just "Too many requests". — Reply to this email directly, view it on GitHub <#25 (comment) https://github.com/piquette/finance-go/issues/25#issuecomment-1644483451>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAATT2NOHUSYZVJRSGVCHALXRGBTHANCNFSM6AAAAAAVGDRPD4 https://github.com/notifications/unsubscribe-auth/AAATT2NOHUSYZVJRSGVCHALXRGBTHANCNFSM6AAAAAAVGDRPD4 . You are receiving this because you commented.Message ID: @.>

Not a software issue really, API.

curl https://query2.finance.yahoo.com/v1/test/getcrumb Too Many Requests

— Reply to this email directly, view it on GitHub https://github.com/piquette/finance-go/issues/25#issuecomment-1644548867, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAATT2M2VWMKEK3GUPQ4VOTXRGHG7ANCNFSM6AAAAAAVGDRPD4 . You are receiving this because you commented.Message ID: @.***>

[ktpx]

I do set cookie in the code, curl was bad example. But I still get same error. It worked, and literary didnt work half hour later, no code changed, so gotta be something yahoo changed.

[joce]

Hum. I don't know what to say.

I just ran the "examples" program (examples/main.go) from my branch, and everything works nicely.

On Fri, Jul 21, 2023 at 10:54 AM ktpx @.***> wrote:

I do set cookie in the code, curl was bad example. But I still get same error.

— Reply to this email directly, view it on GitHub https://github.com/piquette/finance-go/issues/25#issuecomment-1645718376, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAATT2KW72BSDMITUYRSZTLXRKJZNANCNFSM6AAAAAAVGDRPD4 . You are receiving this because you commented.Message ID: @.***>

[ktpx]

Which site do you query to get the cookie? Seems might be the problem. Ive used fc.yahoo.com, as its the only one that worked for me. For regular api calls, query2.

[joce]

 cookieURL = "https://login.yahoo.com"
userAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0)

Gecko/20100101 Firefox/113.0"

func fetchCookies() (string, time.Time, error) { client := http.Client{} request, err := http.NewRequest("GET", cookieURL, nil) if err != nil { return "", time.Time{}, err }

request.Header = http.Header{
    "Accept":                   {"*/*"},
    "Accept-Encoding":          {"gzip, deflate, br"},
    "Accept-Language":          {"en-US,en;q=0.5"},
    "Connection":               {"keep-alive"},
    "Host":                     {"login.yahoo.com"},
    "Sec-Fetch-Dest":           {"document"},
    "Sec-Fetch-Mode":           {"navigate"},
    "Sec-Fetch-Site":           {"none"},
    "Sec-Fetch-User":           {"?1"},
    "TE":                       {"trailers"},
    "Update-Insecure-Requests": {"1"},
    "User-Agent":               {userAgent},
}

func fetchCrumb(cookies string) (string, error) { client := http.Client{} request, err := http.NewRequest("GET", crumbURL, nil) if err != nil { return "", err }

request.Header = http.Header{
    "Accept":          {"*/*"},
    "Accept-Encoding": {"gzip, deflate, br"},
    "Accept-Language": {"en-US,en;q=0.5"},
    "Connection":      {"keep-alive"},
    "Content-Type":    {"text/plain"},
    "Cookie":          {cookies},
    "Host":            {"query1.finance.yahoo.com"},
    "Sec-Fetch-Dest":  {"empty"},
    "Sec-Fetch-Mode":  {"cors"},
    "Sec-Fetch-Site":  {"same-site"},
    "TE":              {"trailers"},
    "User-Agent":      {userAgent},
}

On Fri, Jul 21, 2023 at 10:58 AM ktpx @.***> wrote:

Which site do you query to get the cookie? Seems might be the problem. Ive used fc.yahoo.com, as its the only one that worked for me. For regular api calls, query2.

— Reply to this email directly, view it on GitHub https://github.com/piquette/finance-go/issues/25#issuecomment-1645724393, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAATT2ON5ODH36WILOHXD6DXRKKKFANCNFSM6AAAAAAVGDRPD4 . You are receiving this because you commented.Message ID: @.***>

[ktpx]

Ok using login subdomain again got me the cookie, but still no crum. Mind showing your crumb URL.....? I didnt find it in the source

[joce]

I will refer you to my pull request:

https://github.com/piquette/finance-go/pull/28

You should be able to get everything you need there.

[ktpx]

Thanks, its same url. Im guessing they are doing some IP based limiting, or blocking, im just getting the too many requests from here (EU).

[joce]

Have you sync'd my branch and run the examples program?

[ktpx]

Ok, confirmed is a location issue. So ill assume people using this tool wil also have problem if aren't in US.

[TiagoNH]

So, that means this solution is only working if the request is made in the US?