ranaroussi
Downloading Subheading Values (raw, fmt, longFmt) and Pfolio list
Hello, Thanks for the great utility. Need some advice (or possible fixes, if missing) on getting it working for specific items:
While downloading mutual funds data, the following fields are not getting retrieved:
-
The common fields, like "AnnualExpenseRatio and TotalAssets are available (under view-source:https://finance.yahoo.com/quote/DCMIX/holdings). But since they have multiple components under different sub-headings (raw, fmt, longFmt), they are not getting downloaded:
"annualReportExpenseRatio":{"raw":0.0075,"fmt":"0.75%"},"totalAssets":{"raw":832159936,"fmt":"832.16M","longFmt":"832,159,936"},What's the way to download these "raw", "fmt" values? -
Holdings List (like in the view-source of view-source:https://finance.yahoo.com/quote/SPY/holdings, the following is available and I need to download this list of top 10 holdings:
"holdings":[{"symbol":"AAPL","holdingName":"Apple Inc","holdingPercent":{"raw":0.059,"fmt":"5.90%"}},{"symbol":"MSFT","holdingName":"Microsoft Corp","holdingPercent":{"raw":0.055999998,"fmt":"5.60%"}},
This is the code I am trying (which is NOT working for above download):
for str in t:
try:
stock = yf.Ticker(str)
a.append(stock.info.get('symbol'))
b.append(stock.info.get('annualReportExpenseRatio'))
c.append(stock.info.get('annualReportExpenseRatio(fmt)'))
d.append(stock.info.get('holdingName'))
etc....
If the above code can be tweaked, please let me know. If not, can the yfinance utility be fixed to enable download of the above fields listed in 1 and 2 above, please?
Thanks
May want to break this into two separate issues. TLDR - question 1 - I may have a satisfactory answer for your. TLDR - question 2 - Does not seem to be available. Maybe a bug.
Question 1: "raw", "fmt" values?
- It is just another way to display the raw information. fmt and longfmt is typically for display. So if it's a number it may have a comma or be abbreviated.
- Warning: fmt and longfmt may not always be there. So you may be better off referring ONLY to raw.
- I've had some issues NOT being able to get the fmt/longfmt value from yFinance in Python Versions > 3.5.4 . However when it works in 3.5.4 it works like this.
import yfinance as yfTicker Symbol = yfTicker.Ticker("TSLA") print(Symbol.info["grossProfits"]['raw']) print( Symbol.info["grossProfits"]['raw']['fmt']) print( Symbol.info["grossProfits"]['raw']['longFmt'])
Again it above will not work in Python 3.7. - https://github.com/ranaroussi/yfinance/issues/804
I haven't heard from the community, but I think fmt, longFmt is being ignored in later versions of yFinance/Python. It kind of makes sense to just use "raw" version which is always there. Then use python functions/casting/styling to format the way you need.
Question 2: Retrieving Holdings
Bug? I tried the following, it does not return the holder, but previous high/low and other stats for SPY. import yfinance as yfTicker
if name == 'main': Symbol = yfTicker.Ticker('SPY') print(Symbol.get_institutional_holders()) print(Symbol.get_mutualfund_holders()) print(Symbol.get_major_holders())
Hope the above helps.
Many Thanks @clhubert for your inputs. I tried both in 3.7.6 and a downgraded version. Unfortunately, the above solution you provided is not working even in lower versions of Python.
Which version did you downgrade to?
Error Msg?
FYI. The version that is working for me is python version 3.5.3.
On Sun, Aug 15, 2021 at 12:46 AM user631 @.***> wrote:
Many Thanks @clhubert https://github.com/clhubert for your inputs. I tried both in 3.7.6 and a downgraded version. Unfortunately, the above solution you provided is not working even in lower versions of Python.
— You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/ranaroussi/yfinance/issues/811#issuecomment-898999436, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAS52LLYLSRYAIH6RLAZCWTT45IFDANCNFSM5B54BDHA .
Hello @clhubert It was returning a NULL output till yesterday, and then I upgraded as I wanted to do some other work.
Somehow, today it is not allowing me to even run "pip" on the Python 3.5.3 shell. Without that, today I can't even reinstall yfinance.
Below is the error message log. Please let me know if you can provide any help.
Python 3.5.3 (v3.5.3:1880cb95a742, Jan 16 2017, 16:02:32) [MSC v.1900 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>>
======================== RESTART: D:\get-pip.py ========================
[33mDEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.[0m
Collecting pip<21.0
Using cached pip-20.3.4-py2.py3-none-any.whl (1.5 MB)
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 20.3.4
Uninstalling pip-20.3.4:
Successfully uninstalled pip-20.3.4
Successfully installed pip-20.3.4
>>> py -m ensurepip --default-pip
SyntaxError: invalid syntax
>>>
>>> py -m pip install --upgrade pip setuptools wheel
SyntaxError: invalid syntax
>>>
Many Thanks
Unfortunately Python 3.5.3 is EOL. I am only using it because of a framework I am using hasn't upgraded to a later version of Python yet.
You should use the latest stable version of python if your project allows it.
Although fmt, longfmt is being returned by the underlying yahoo api (sometimes). yFinance appears to ignore it and just use raw. You should probably do the same.
Solution: Leverage python to display the format the way you need. By default, yFinance is returning the raw format. Ignore the fact that Fmt, LongFmt is sometimes returned but not accessible through the yFinance API.
Per what I mentioned in the beginning of this thread.. There is no guarantee that Yahoo will provide a fmt or LongFmt.
It appears to be a good decision for the API to not depend on a field that may or may not be there. It makes the api less fragile.
Thanks again @clhubert The problem is that even the "raw" value is not coming up in the higher versions. Here is the output of your code when I run it in Jupyter with version:
Your code (I am using a stock Tesla, a ETF SPY and a mutual fund DCMIX):
import yfinance as yfTicker
Symbol = yfTicker.Ticker("TSLA","SPY","DCMIX")
print(Symbol.info["annualReportExpenseRatio"]['raw'])
print( Symbol.info["annualReportExpenseRatio"]['raw']['fmt'])
print( Symbol.info["annualReportExpenseRatio"]['raw']['longFmt'])
Output Error:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-3-89206bef8646> in <module>
1 import yfinance as yfTicker
----> 2 Symbol = yfTicker.Ticker("TSLA","SPY","DCMIX")
3 print(Symbol.info["annualReportExpenseRatio"]['raw'])
4 print( Symbol.info["annualReportExpenseRatio"]['raw']['fmt'])
5 print( Symbol.info["annualReportExpenseRatio"]['raw']['longFmt'])
TypeError: __init__() takes from 2 to 3 positional arguments but 4 were given
And when I try only with a list of 2 mutual funds and only printing "raw" values:
import yfinance as yfTicker
Symbol = yfTicker.Ticker("DCMIX","FRUSX")
print(Symbol.info["annualReportExpenseRatio"]['raw'])
The error output:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-9-d390f482ec3a> in <module>
1 import yfinance as yfTicker
2 Symbol = yfTicker.Ticker("DCMIX","FRUSX")
----> 3 print(Symbol.info["annualReportExpenseRatio"]['raw'])
C:\ProgramData\Anaconda3\lib\site-packages\yfinance\ticker.py in info(self)
141 @property
142 def info(self):
--> 143 return self.get_info()
144
145 @property
C:\ProgramData\Anaconda3\lib\site-packages\yfinance\base.py in get_info(self, proxy, as_dict, *args, **kwargs)
497
498 def get_info(self, proxy=None, as_dict=False, *args, **kwargs):
--> 499 self._get_fundamentals(proxy=proxy)
500 data = self._info
501 if as_dict:
C:\ProgramData\Anaconda3\lib\site-packages\yfinance\base.py in _get_fundamentals(self, kind, proxy)
303
304 # get info and sustainability
--> 305 data = utils.get_json(ticker_url, proxy, self.session)
306
307 # holders
C:\ProgramData\Anaconda3\lib\site-packages\yfinance\utils.py in get_json(url, proxy, session)
54
55 session = session or _requests
---> 56 html = session.get(url=url, proxies=proxy, headers=user_agent_headers).text
57
58 if "QuoteSummaryStore" not in html:
AttributeError: 'str' object has no attribute 'get'
If this can be fixed in yfinance itself, a great deal of data content will become available.
Below are the 3 issues with the code, plus a modified working version of it at the bottom. Hopefully this clears things up for you.
Issue 1: Syntax Issue - See Documentation
- Yfinance.Ticker accepts only one ticker (see documentation/library source code)
- Correct Syntax is Symbol = yfTicker.Ticker("DCMIX")
- Check the documentation for the other functions that accept multiple symbols..
Issue 2: Original Issue in Thread:
- See earlier in this thread.... raw, fmt, or LongFMT will not work on later version of Python
- Correct Syntax is Symbol.info['annualReportExpenseRatio']
- This will automatically retrieve the "Raw" value in the background.
Issue 3: TypeError/AttributeError
- Use python's type() function to figure out which object you are working with.
- This way you can look up the documentation (or library source code) and know exactly what attributes and functions are available.
- The correct syntax in Issue 2: will be clear when you realize what type of object you are using
import yfinance as yfTicker if name == 'main': Symbol = yfTicker.Ticker("DCMIX") # Confirms you have a dict object. print(type(Symbol.info)) #See everything available in info dict object. print(Symbol.info) # Use the dict object to retrieve the one value you want.. print(Symbol.info['annualReportExpenseRatio']) # Note: the annualReportExpenseRation for TSLA and SPY is "NONE"
On Thu, Aug 19, 2021 at 2:53 AM user631 @.***> wrote:
Thanks again @clhubert https://github.com/clhubert The problem is that even the "raw" value is not coming up in the higher versions. Here is the output of your code when I run it in Jupyter with version:
Your code (I am using a stock Tesla, a ETF SPY and a mutual fund DCMIX):
import yfinance as yfTicker Symbol = yfTicker.Ticker("TSLA","SPY","DCMIX") print(Symbol.info["annualReportExpenseRatio"]['raw']) print( Symbol.info["annualReportExpenseRatio"]['raw']['fmt']) print( Symbol.info["annualReportExpenseRatio"]['raw']['longFmt'])
Output Error:
TypeError Traceback (most recent call last)
in 1 import yfinance as yfTicker ----> 2 Symbol = yfTicker.Ticker("TSLA","SPY","DCMIX") 3 print(Symbol.info["annualReportExpenseRatio"]['raw']) 4 print( Symbol.info["annualReportExpenseRatio"]['raw']['fmt']) 5 print( Symbol.info["annualReportExpenseRatio"]['raw']['longFmt']) TypeError: init() takes from 2 to 3 positional arguments but 4 were given
And when I try only with a list of 2 mutual funds and only printing "raw" values:
import yfinance as yfTicker Symbol = yfTicker.Ticker("DCMIX","FRUSX") print(Symbol.info["annualReportExpenseRatio"]['raw'])
The error output:
AttributeError Traceback (most recent call last)
in 1 import yfinance as yfTicker 2 Symbol = yfTicker.Ticker("DCMIX","FRUSX") ----> 3 print(Symbol.info["annualReportExpenseRatio"]['raw']) C:\ProgramData\Anaconda3\lib\site-packages\yfinance\ticker.py in info(self) 141 @property 142 def info(self): --> 143 return self.get_info() 144 145 @property
C:\ProgramData\Anaconda3\lib\site-packages\yfinance\base.py in get_info(self, proxy, as_dict, *args, **kwargs) 497 498 def get_info(self, proxy=None, as_dict=False, *args, **kwargs): --> 499 self._get_fundamentals(proxy=proxy) 500 data = self._info 501 if as_dict:
C:\ProgramData\Anaconda3\lib\site-packages\yfinance\base.py in _get_fundamentals(self, kind, proxy) 303 304 # get info and sustainability --> 305 data = utils.get_json(ticker_url, proxy, self.session) 306 307 # holders
C:\ProgramData\Anaconda3\lib\site-packages\yfinance\utils.py in get_json(url, proxy, session) 54 55 session = session or _requests ---> 56 html = session.get(url=url, proxies=proxy, headers=user_agent_headers).text 57 58 if "QuoteSummaryStore" not in html:
AttributeError: 'str' object has no attribute 'get'
If this can be fixed in yfinance itself, a great deal of data content will become available.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ranaroussi/yfinance/issues/811#issuecomment-901691979, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAS52LOYP5NXLSTQFXSEGFDT5SZ7HANCNFSM5B54BDHA .
Many Thanks @clhubert - it fills my need partially. I'll need to customize my code further to make it perfect. But again, really great insights and help you offered. Thanks again.
There's a lot here which I won't attempt to understand now ... but if someone wants to submit a PR I can merge in.
