Add ETF support for portfolio allocation command

[montezdesousa]

Description

This PR adds ETF support to the alloc command in the portfolio menu. This means that users that do not provide any country, sector or region for their ETF trades will still be able to see their allocations.

The method is similar on what was used to fetch company data for stocks. In this case the allocation_model module will automatically search for the ETF sector/country/region weights and assign the proportional value to the portfolio allocations.

Example:

Portfolio X holdings, %: ETF ABC: 1000$, 50% STOCK XYZ: 1000$, 50%

Suppose ETF ABC has 50% stocks in US and 50% in Brazil, so it will contribute 500$ to the US allocation and 500$ to Brazil. Suppose STOCK XYZ is from US.

Final country allocs, %: US: 1500$, 75% Brazil: 500$, 25%

• [x] Summary of the change / bug fix.
• [ ] Link # issue, if applicable.
• [ ] Screenshot of the feature or the bug before/after fix, if applicable.
• [x] Relevant motivation and context.
• [ ] List any dependencies that are required for this change.

How has this been tested?

• Please describe the tests that you ran to verify your changes.
• Provide instructions so we can reproduce.
• Please also list any relevant details for your test configuration.

Others

• [x] I have performed a self-review of my own code.
• [x] I have commented my code, particularly in hard-to-understand areas.
• [x] My code passes all the checks pylint, flake8, black, ... To speed up development you should run pre-commit install.
• [x] New and existing unit tests pass locally with my changes. You can test this locally using pytest tests/....

[JerBouma]

I can not test this because it requires me to set 2FA (which I haven't enabled). Furthermore, the process of getting this code is far too difficult for the user (see: https://github.com/Chavithra/degiro-connector#36-how-to-find-your--totp_secret_key-)

The following needs to be done (can you look into this @Chavithra):

• Set username and password via keys for DEGIRO
• Set username, password and 2FA via keys for DEGIRO without needing to do anything with a QR code and similar. It shouldn't be complicated.
• I shouldn't need to rely on and .env file or config_terminal.py file to make this work. That's not user friendly.

[Chavithra]

Hi @JerBouma,

are you suggesting letting the user manually adding 2FA password each time they use the Terminal ?

Thanks

[JerBouma]

Hi @Chavithra,

That sounds good. You gotta try and make it accessible for people that just know how to navigate the terminal. We can not ask them to edit some file or extract some code from an URL. Therefore the ideal solution here would be:

1. Let them set-up their DEGIRO account directly from keys (without the option to provide the 2FA here)
2. When they wish to log-in into the DEGIRO menu we have inside the portfolio menu, if they have 2FA enabled they input the 2FA key in here. Otherwise, simply running login should suffice.
3. They can now use the DEGIRO functionalities.

[montezdesousa]

Hi @Chavithra,

That sounds good. You gotta try and make it accessible for people that just know how to navigate the terminal. We can not ask them to edit some file or extract some code from an URL. Therefore the ideal solution here would be:

1. Let them set-up their DEGIRO account directly from keys (without the option to provide the 2FA here)
2. When they wish to log-in into the DEGIRO menu we have inside the portfolio menu, if they have 2FA enabled they input the 2FA key in here. Otherwise, simply running login should suffice.
3. They can now use the DEGIRO functionalities.

ok done with help from @Chavithra

[JerBouma]

The current issue with the implementation is that it doesn't recognize that I bought ETFs on a different exchange. For example the ETF VUSA is this product on Yahoo Finance: https://finance.yahoo.com/quote/VUSA.AS/ and the U.S. equivalent is VOO. To make this work, the tickers need to be converted into ISIN properly.

2022 Aug 03, 05:52 (🦋) /portfolio/ $ load 20220803_105213_paexport_degiro.csv
 Preprocessing orderbook: ..........
    Loading company data: ......
      Loading price data: 
4 Failed downloads:
- IWDA: No data found, symbol may be delisted
- VUSA: No data found for this date range, symbol may be delisted
- VWRL: No data found for this date range, symbol may be delisted
- ESP0: No data found, symbol may be delisted
.Error: Index contains duplicate entries, cannot reshape


     Calculating returns: Error: 'Quantity'

Error: 'End Value'


Portfolio: 20220803_105213_paexport_degiro.csv
Risk Free Rate: 0

[montezdesousa]

The current issue with the implementation is that it doesn't recognize that I bought ETFs on a different exchange. For example the ETF VUSA is this product on Yahoo Finance: https://finance.yahoo.com/quote/VUSA.AS/ and the U.S. equivalent is VOO. To make this work, the tickers need to be converted into ISIN properly.

2022 Aug 03, 05:52 (🦋) /portfolio/ $ load 20220803_105213_paexport_degiro.csv
 Preprocessing orderbook: ..........
    Loading company data: ......
      Loading price data: 
4 Failed downloads:
- IWDA: No data found, symbol may be delisted
- VUSA: No data found for this date range, symbol may be delisted
- VWRL: No data found for this date range, symbol may be delisted
- ESP0: No data found, symbol may be delisted
.Error: Index contains duplicate entries, cannot reshape


     Calculating returns: Error: 'Quantity'

Error: 'End Value'


Portfolio: 20220803_105213_paexport_degiro.csv
Risk Free Rate: 0

Yes, thanks for testing this, I'll try to fix that

[DidierRLopes]

@montezdesousa @JerBouma let's try to merge these PRs. As we refactor code, it will be harder to come back to these files to get them merged.

[montezdesousa]

@montezdesousa @JerBouma let's try to merge these PRs. As we refactor code, it will be harder to come back to these files to get them merged.

I'll try this sprint

[JerBouma]

How far is this one coming along? @montezdesousa

[montezdesousa]

How far is this one coming along? @montezdesousa

Been afk these last 7 days. So sprint starting tomorrow for me, will try to deliver anyways @JerBouma

[JerBouma]

There are still some things off. As you can see the return calculation we do is not the same as on DEGIRO. It is fine that it differs a little but this is too much. Furthermore, it is impossible that the benchmark has such poor return given that my current portfolio consists primarily of S&P500 and World (thus the benchmark is closely related). Could you look into this? @montezdesousa