yahoo-finance-api

yahoo-finance-api copied to clipboard

Hallo Christian,

I've just started playing around with your package and like it. Thank you! I've managed to get basic quote information such as trailing PE, however I fail to access the information that is found on the statistics page such as payout ratio. Looking at past issues, I have found #35 which is marked as completed, however I fail to fi /nd any of it in the source code.

Also the query link shared, fails with

{
    "finance": {
        "result": null,
        "error": {
            "code": "Unauthorized",
            "description": "Invalid Crumb"
        }
    }
}

Update: I actually figured it out, adding &crumb=mdeVssfeRhi to the request url works. I got the crumb value from a browser session on yahoo finance.

To get the crumb, you need to provide two things: cookies and user agent:

curl 'https://query2.finance.yahoo.com/v1/test/getcrumb' \
  -H 'cookie: GUC=AQEBCAFlULBleUIhXwTg&s=AQAAAC76j7jx&g=ZU9e-w; A1=d=AQABBK0Lr2ICEI-bBud4SuIDLaB4bqaMNbAFEgEBCAGwUGV5Zc3ibmUB_eMBAAcIrQuvYqaMNbA&S=AQAAAggpViKy189d2OkdWsFnK_Y; A3=d=AQABBK0Lr2ICEI-bBud4SuIDLaB4bqaMNbAFEgEBCAGwUGV5Zc3ibmUB_eMBAAcIrQuvYqaMNbA&S=AQAAAggpViKy189d2OkdWsFnK_Y; A1S=d=AQABBK0Lr2ICEI-bBud4SuIDLaB4bqaMNbAFEgEBCAGwUGV5Zc3ibmUB_eMBAAcIrQuvYqaMNbA&S=AQAAAggpViKy189d2OkdWsFnK_Y; gpp=DBAA; gpp_sid=-1; gam_id=y-rpk1JRJE2uL6EDgVwC0_in.GgdLv339c~A; axids=gam=y-rpk1JRJE2uL6EDgVwC0_in.GgdLv339c~A&dv360=eS00Mm1IMHpoRTJ1RXhhNVlTUGtSVWRPdkZCWExrbjFkNH5B; tbla_id=ba643052-93dc-4ae8-a277-ee1cb7f7dab5-tuctc48e47a; cmp=t=1699788664&j=0&u=1---; PRF=t%3DZURN.SW%252BSREN.SW%26newChartbetateaser%3D0%252C1700910092680' \
  -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0'

From the cookies, only A1 is necessary. So this is the smallest request I could find:

curl 'https://query2.finance.yahoo.com/v1/test/getcrumb' \
  -H 'cookie: A1=d=AQABBK0Lr2ICEI-bBud4SuIDLaB4bqaMNbAFEgEBCAGwUGV5Zc3ibmUB_eMBAAcIrQuvYqaMNbA&S=AQAAAggpViKy189d2OkdWsFnK_Y' \
  -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0'

And to get A1 cookie:

curl -v -s 'https://finance.yahoo.com/quote/ZURN.SW' \
  -H 'accept-language: en-US,en;q=0.9,fr;q=0.8' \
  -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0' \
   1> /dev/null

Here's the response to the last request:

< HTTP/1.1 200 OK
< referrer-policy: no-referrer-when-downgrade
< strict-transport-security: max-age=31536000
< x-frame-options: SAMEORIGIN
< content-security-policy: frame-ancestors 'self' https://*.yahoo.com https://*.engadget.com https://*.pnr.ouryahoo.com https://pnr.ouryahoo.com https://*.search.aol.com https://*.onesearch.com https://*.publishing.oath.com https://*.aol.com; sandbox allow-downloads allow-forms allow-modals allow-same-origin allow-scripts allow-popups allow-popups-to-escape-sandbox allow-top-navigation-by-user-activation allow-presentation;
< cache-control: private, no-store, no-cache, max-age=0
< content-type: text/html; charset=utf-8
< vary: Accept-Encoding
< date: Mon, 13 Nov 2023 05:04:59 GMT
< x-envoy-upstream-service-time: 479
< server: ATS
< x-envoy-decorator-operation: finance-nodejs--mtls-production-ir2.finance-k8s.svc.yahoo.local:4080/*
< Age: 0
< Transfer-Encoding: chunked
< Connection: keep-alive
< Expect-CT: max-age=31536000, report-uri="http://csp.yahoo.com/beacon/csp?src=yahoocom-expect-ct-report-only"
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< Set-Cookie: A1=d=AQABBHuuUWUCECcV2H5ND6rsZYO4zo0HI2gFEgEBAQH_UmVbZc1o0CMA_eMAAA&S=AQAAAumF6MTURk-DLOB53xHM7Bk; Expires=Tue, 12 Nov 2024 11:04:59 GMT; Max-Age=31557600; Domain=.yahoo.com; Path=/; SameSite=Lax; Secure; HttpOnly
< Set-Cookie: A3=d=AQABBHuuUWUCECcV2H5ND6rsZYO4zo0HI2gFEgEBAQH_UmVbZc1o0CMA_eMAAA&S=AQAAAumF6MTURk-DLOB53xHM7Bk; Expires=Tue, 12 Nov 2024 11:04:59 GMT; Max-Age=31557600; Domain=.yahoo.com; Path=/; SameSite=None; Secure; HttpOnly
< Set-Cookie: A1S=d=AQABBHuuUWUCECcV2H5ND6rsZYO4zo0HI2gFEgEBAQH_UmVbZc1o0CMA_eMAAA&S=AQAAAumF6MTURk-DLOB53xHM7Bk; Domain=.yahoo.com; Path=/; SameSite=Lax; Secure

The feature was discussed in #35, but it never got implemented. If you'd like to see that feature, I'm open to PRs.

For getting a crumb value, there should already be a function in the library for that purpose.

I tried to code the request that gets the A1-Cookie in in c#. My response was correct but it didn't contain a A1-Cookie

@RobertSchiefele Just tested my code today to see if anything changed and I found out that it is still working. So there is some issue on your side.

@8ctopus I've tried the curl command for the cookie and it's not present. Here is what I get. Can you try again on your side to see if you still have it or can see a problem on my side? Thank you

curl -v -s 'https://finance.yahoo.com/quote/ZURN.SW' \                                                                                                   ✔  3s 
  -H 'accept-language: en-US,en;q=0.9,fr;q=0.8' \
  -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0' \
   1> /dev/null
* Connected to finance.yahoo.com (2a00:1288:7c:800::4001) port 443 (#0)
* ALPN: offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
} [322 bytes data]
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* (304) (IN), TLS handshake, Unknown (8):
{ [19 bytes data]
* (304) (IN), TLS handshake, Certificate (11):
{ [4371 bytes data]
* (304) (IN), TLS handshake, CERT verify (15):
{ [79 bytes data]
* (304) (IN), TLS handshake, Finished (20):
{ [36 bytes data]
* (304) (OUT), TLS handshake, Finished (20):
} [36 bytes data]
* SSL connection using TLSv1.3 / AEAD-AES128-GCM-SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: C=US; ST=California; L=Sunnyvale; O=Oath Holdings Inc.; CN=*.fantasysports.yahoo.com
*  start date: Dec 12 00:00:00 2023 GMT
*  expire date: Jan 31 23:59:59 2024 GMT
*  subjectAltName: host "finance.yahoo.com" matched cert's "*.yahoo.com"
*  issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert SHA2 High Assurance Server CA
*  SSL certificate verify ok.
* using HTTP/2
* h2 [:method: GET]
* h2 [:scheme: https]
* h2 [:authority: finance.yahoo.com]
* h2 [:path: /quote/ZURN.SW]
* h2 [accept: */*]
* h2 [accept-language: en-US,en;q=0.9,fr;q=0.8]
* h2 [user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0]
* Using Stream ID: 1 (easy handle 0x147809800)
> GET /quote/ZURN.SW HTTP/2
> Host: finance.yahoo.com
> Accept: */*
> accept-language: en-US,en;q=0.9,fr;q=0.8
> user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0
>
< HTTP/2 200
< referrer-policy: no-referrer-when-downgrade
< strict-transport-security: max-age=31536000
< x-frame-options: SAMEORIGIN
< content-security-policy: frame-ancestors 'self' https://*.yahoo.com https://*.engadget.com https://*.pnr.ouryahoo.com https://pnr.ouryahoo.com https://*.search.aol.com https://*.onesearch.com https://*.publishing.oath.com https://*.aol.com; sandbox allow-downloads allow-forms allow-modals allow-same-origin allow-scripts allow-popups allow-popups-to-escape-sandbox allow-top-navigation-by-user-activation allow-presentation;
< cache-control: private, no-store, no-cache, max-age=0
< content-type: text/html; charset=utf-8
< vary: Accept-Encoding
< date: Thu, 21 Dec 2023 10:50:02 GMT
< x-envoy-upstream-service-time: 662
< server: ATS
< x-envoy-decorator-operation: finance-nodejs--mtls-production-ir2.finance-k8s.svc.yahoo.local:4080/*
< age: 5
< expect-ct: max-age=31536000, report-uri="http://csp.yahoo.com/beacon/csp?src=yahoocom-expect-ct-report-only"
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
<
{ [8372 bytes data]
* Connection #0 to host finance.yahoo.com left intact

@Vladutu I tried the YahooManager.cs class from @8ctopus and it worked fine on my side. Thanks to the author.

@RobertSchiefele what YahooManager.cs are you refering to? Maybe I'm not looking where I should but the yahoo-finance-api is PHP project, not c#. I saw that he was doing a call to https://fc.yahoo.com to get the cookie (which I was also doing) but that doesn't work anymore, the website is not responding (I got 502 connection timeout)

@Vladutu sorry I got this Classs from another site, cannot remember where. But anyway here is the code

using Microsoft.Extensions.Caching.Memory; using System.Net;

namespace My.YahooFinanceAPI { public class YahooManager { private static readonly object yahooCredentialsLocker = new object();

    private const string STOCK_MARKET_URL_SUMMARY = "https://query1.finance.yahoo.com/v10/finance/quoteSummary/{0}?formatted=true&lang=en-US&region=US&modules=assetProfile%2CbalanceSheetHistory%2CbalanceSheetHistoryQuarterly%2CcalendarEvents%2CcashflowStatementHistory%2CcashflowStatementHistoryQuarterly%2CdefaultKeyStatistics%2Cearnings%2CearningsHistory%2CearningsTrend%2CesgScores%2CfinancialData%2CfundOwnership%2CincomeStatementHistory%2CincomeStatementHistoryQuarterly%2CindexTrend%2CindustryTrend%2CinsiderHolders%2CinsiderTransactions%2CinstitutionOwnership%2CmajorDirectHolders%2CmajorHoldersBreakdown%2CnetSharePurchaseActivity%2Cprice%2CrecommendationTrend%2CsecFilings%2CsectorTrend%2CsummaryDetail%2CsummaryProfile%2CupgradeDowngradeHistory%2Cpageviews%2Cquotetype&ssl=true";
    
    private const string YahooFcUrl = "https://fc.yahoo.com";
    private const string YahooGetCrumbUrl = "https://query2.finance.yahoo.com/v1/test/getcrumb";
    private const string CacheKeyCookieContainer = "YahooCookieContainer";
    private const string CacheKeyCrumb = "YahooCrumb";

    private readonly IHttpClientFactory _httpClientFactory;
    private readonly IMemoryCache _memCache;

    public YahooManager(IHttpClientFactory httpClientFactory, IMemoryCache memCache)
    {
        _httpClientFactory = httpClientFactory;
        _memCache = memCache;
    }


    public async Task<string> GetYahooSymbolSummary(string symbol)
    {
        return await WebRequestGet(string.Format(STOCK_MARKET_URL_SUMMARY, symbol));
    }

    public async Task<string> WebRequestGet(string url)
    {
        (CookieContainer cookie, string crumb) credentials = new(null, null);
        lock (yahooCredentialsLocker)
        {
            int tryCounter = 0;
            while ((credentials.crumb == null || credentials.cookie == null) && tryCounter < 10)
            {
                Task.Delay(1000 * tryCounter).Wait();
                credentials = GetYahooCookie().ConfigureAwait(false).GetAwaiter().GetResult();
                tryCounter++;
            }//while
        }//lock

        string sAgent = $"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36";
        var httpRequestMessage = new HttpRequestMessage(HttpMethod.Get, url + "&crumb=" + credentials.crumb)
        {
            Headers =
                    {
                        { "Cookie",  credentials.cookie.GetCookieHeader(new Uri(YahooFcUrl))},
                        { "User-Agent",sAgent }
                    }
        };
        var httpClient = _httpClientFactory.CreateClient();
        var webResponse = await httpClient.SendAsync(httpRequestMessage);
        //webResponse.EnsureSuccessStatusCode();
        string responseContent = await webResponse.Content.ReadAsStringAsync();
        return responseContent;
    }//WebRequestGet

    private async Task<(CookieContainer cookie, string crumb)> GetYahooCookie()
    {
        CookieContainer m_cookieContainer = null;
        string m_crumb = null;
        _memCache.TryGetValue(CacheKeyCookieContainer, out m_cookieContainer);
        _memCache.TryGetValue(CacheKeyCrumb, out m_crumb);

        if (m_cookieContainer == null || m_crumb == null)
        {
            string sAgent = $"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36";

            m_cookieContainer = new CookieContainer();
            var uri = new Uri(YahooFcUrl);
            var httpRequestMessage = new HttpRequestMessage(HttpMethod.Get, YahooFcUrl)
            {
                Headers =
                    {
                        { "User-Agent", sAgent }
                    }
            };
            var httpClient = _httpClientFactory.CreateClient();
            var httpResponseMessage = await httpClient.SendAsync(httpRequestMessage);
            IEnumerable<string> cookies;
            httpResponseMessage.Headers.TryGetValues("Set-Cookie", out cookies);
            if (cookies == null)
                return (null, null);

            foreach (var cookieValue in cookies)
                m_cookieContainer.SetCookies(uri, cookieValue);
            httpResponseMessage.Dispose();

            httpRequestMessage = new HttpRequestMessage(HttpMethod.Get, YahooGetCrumbUrl)
            {
                Headers =
                    {
                        { "Cookie",  m_cookieContainer.GetCookieHeader(uri) },
                        { "User-Agent",sAgent}
                    }
            };
            httpClient = _httpClientFactory.CreateClient();
            var crumbResponse = await httpClient.SendAsync(httpRequestMessage);
            crumbResponse.EnsureSuccessStatusCode();
            string responseContent = await crumbResponse.Content.ReadAsStringAsync();
            m_crumb = responseContent.Trim('"');

            var cacheEntryOptions = new MemoryCacheEntryOptions()
                .SetSlidingExpiration(TimeSpan.FromSeconds(60*5))
                .SetAbsoluteExpiration(TimeSpan.FromSeconds(60*10))
                .SetPriority(CacheItemPriority.Normal);
            _memCache.Set(CacheKeyCookieContainer, m_cookieContainer, cacheEntryOptions);
            _memCache.Set(CacheKeyCrumb, m_crumb, cacheEntryOptions);
        }//endif
        return (m_cookieContainer, m_crumb);
    }//GetYahooCookie


}

}

@RobertSchiefele thank you. I will have a look.

@RobertSchiefele Seems it's doing the same using "https://fc.yahoo.com" which for some reason doesn't work anymore for me. I even asked a friend to try it out (thought maybe my IP was blocked) but it was the same thing