mplfinance icon indicating copy to clipboard operation
mplfinance copied to clipboard

Feature Request: Candlestick Footprint Chart

Open quantfreedom opened this issue 2 years ago • 39 comments

i want to create a footprint chart like this one where i am able to make an open high low close but also show how much trading volume was at each y value of the candle

here is an example of what i am talking about

image

its very easy to get the data ... getting and understanding how to organize the data on our end is very very easy ... you can see what a csv file would look like here ... https://public.bybit.com/trading/BTCUSD/

lets say i wanted to create a 5 min candle ... for the info i would just add up all the sell orders at each price point of the candle and do the same for the buy orders ... then i have a dataframe within the candle that tells me the sell column and buy column ... and you can get the open high low close even from that data ... the open is the first trade at the start of the candle then the high is the highest point ... low is the lowest point and the close is the last candle ...

this info helps us visualize at what price point in the candle has the most trade volume ... footprint charts are used in high level trading ... I will be making a ton of videos on this subject if we can get one of these open source plotting platforms to help create something like this

all we would need is send dataframes inside a master dataframe ... so you have the overall dataframe that houses all of the candles and inside each candle is a dataframe of all of the sell and buy data at each price of the candle

ultimately would be nice for it to look like this ... but for now the top one would be ultra sexy image

quantfreedom avatar Nov 25 '22 20:11 quantfreedom

@The1iAmNeo Very interesting! I will take a closer look in a week or so (busy with another project for the next week or so). At that time I will look into how best to implement this, and where it should fall in priorities. How are your python coding skills? Perhaps you can help with some of the coding? Thanks. All the best. --Daniel

DanielGoldfarb avatar Nov 26 '22 23:11 DanielGoldfarb

@DanielGoldfarb hey thanks for taking an interest ... it would really help out the algo trading community so much.

my coding skills aren't the greatest but would love to chat and see what i could help with ... only been coding for like 2 months so not sure what i could do lol ... but i am willing to try because having this would be a life saver

quantfreedom avatar Nov 27 '22 03:11 quantfreedom

@DanielGoldfarb hey ... wanted to check back in and see if you were still interested in doing this?

quantfreedom avatar Dec 09 '22 13:12 quantfreedom

Would be interested to see this up and running, please let know how I can help , not a good coder in python though

kzaheer09 avatar Feb 02 '23 09:02 kzaheer09

@QuantFreedom1022 I'd like to help to code this Candlestick Footprint Chart method, but I need some clarification. Are we treating the bid-traded and ask-traded volumes as those indicated by the Buy and Sell labels in the 'side' column of the provided csv files? Also, is there a specific number of market orders that must be plotted for each footprint or a rule associated with such?

andrewrgarcia avatar Mar 07 '23 05:03 andrewrgarcia

@DanielGoldfarb ... wanted to check back in with you on this ... would like to start working on this soon

@andrewrgarcia yeah i am down to work together on this ... i already have the code and the csv files needed ... just need the plotting to do its job

quantfreedom avatar Apr 01 '23 13:04 quantfreedom

I am very involved right now working on a project for Jupytercon. I am happy for you two to begin moving forward with this, and I can put some time into it. I would like to see some details regarding design and what you are planning, so as to fit nicely into the big picture and long term plans for mplfinance. You can post here, or if you prefer, email me at [email protected]

The first thing I'd like know is what is the minimum expectation for the data (what columns are a must; which are optional). If the data is going to look like this, https://public.bybit.com/trading/BTCUSD/, then it seems to me that it doesn't necessarily fit into the standard mpf.plot(df,**kwargs) function, which expects simply OHLC(V) data. Perhaps we write a wrapper around mpf.plot()? Or, if you have in mind just adding a new type= to mpf.plot(), maybe that can work as well, but I'd like to see what that might look like inside.

Looking forward to hearing from you. All the best. --Daniel

DanielGoldfarb avatar Apr 02 '23 01:04 DanielGoldfarb

@DanielGoldfarb ... wanted to check back in with you on this ... would like to start working on this soon

@andrewrgarcia yeah i am down to work together on this ... i already have the code and the csv files needed ... just need the plotting to do its job

@QuantFreedom1022 Nice. You can start by attaching the files and the code I'll see what I can do. If you're familiar with Git, I can open a new branch and you can add your material on the fork, your choice.

andrewrgarcia avatar Apr 03 '23 17:04 andrewrgarcia

@andrewrgarcia and @DanielGoldfarb ... here is the link to my github repo for the footprints ... and also here is a quick loom video of me explaining a little of how it works https://github.com/QuantFreedom1022/Footprints https://www.loom.com/share/76ccbf0a46374256b798f2a237f41f78

quantfreedom avatar Apr 12 '23 15:04 quantfreedom

@andrewrgarcia and @DanielGoldfarb ... here is the link to my github repo for the footprints ... and also here is a quick loom video of me explaining a little of how it works https://github.com/QuantFreedom1022/Footprints https://www.loom.com/share/76ccbf0a46374256b798f2a237f41f78

@QuantFreedom1022 I forked your repository here to shorten and revise your footprint code. My revised version does not use numba yet it runs faster than your former code. It also removes the sell / buy count issue you mention in your video and thus, appropriately reduces the trade count (see csv output screen captures for the revised code and your code). Still, please check the code and provide me some feedback if needed before I submit a PR to merge it to your repository

andrewrgarcia avatar Apr 13 '23 20:04 andrewrgarcia

@andrewrgarcia WOW ... ummm so this code is way way way above my skill level lol ... so i feel it almost impossible to give any feedback lol ...

but i did take your code and add it to my repo so that way we have your code and i also have my code ... but yeah lets move forward with yours seeing as how it is faster and better in all aspects.

also i would love to jump on a quick 10 or so screen share call to talk about what we are both thinking as far as moving the project forward now that we have a function that can organize the data to start to build the footprint charts

quantfreedom avatar Apr 13 '23 21:04 quantfreedom

@andrewrgarcia WOW ... ummm so this code is way way way above my skill level lol ... so i feel it almost impossible to give any feedback lol ...

but i did take your code and add it to my repo so that way we have your code and i also have my code ... but yeah lets move forward with yours seeing as how it is faster and better in all aspects.

also i would love to jump on a quick 10 or so screen share call to talk about what we are both thinking as far as moving the project forward now that we have a function that can organize the data to start to build the footprint charts

@QuantFreedom1022 Hey, it's possible you're new to open source. Can you please remove the andrew_footprint.py file you created by copying my code and instead approve my Pull Request (PR) ? I just sent you one. (My PR keeps your .ipynb code intact)

Best.

andrewrgarcia avatar Apr 13 '23 21:04 andrewrgarcia

@andrewrgarcia ok i just deleted your file and approved your pull request ... so everything should be up to date ... again for me i would prefer a quick 10 min screen share call for next steps but its up to you ... if you would like to setup a time to talk you can let me know your email and we can set that up or we can just do it here.

if not then what are your thoughts on what to do next

quantfreedom avatar Apr 13 '23 22:04 quantfreedom

@andrewrgarcia WOW ... ummm so this code is way way way above my skill level lol ... so i feel it almost impossible to give any feedback lol ...

but i did take your code and add it to my repo so that way we have your code and i also have my code ... but yeah lets move forward with yours seeing as how it is faster and better in all aspects.

also i would love to jump on a quick 10 or so screen share call to talk about what we are both thinking as far as moving the project forward now that we have a function that can organize the data to start to build the footprint charts

@QuantFreedom1022 I think the biggest challenge now would be on how to lump the timestamps into groups with the volume data so that footprints are made. My first thought was to use an argmax and/or argmin function, but if you have any other ideas and/or can sketch it, that would help.

The other thing would be on the side of the new footprints.py code to make it more lean. Your initial algorithm used several global variables, and because of the lengthiness of it, the new code inherits all these variables as constructors to the Printer class i.e. the variables below the init() function; some of these variables do not need to be constructors as some of them may be local to the specific code section they are used on. It would be great if you could identify the ones which do not need to be so "global" as this was originally your algorithm.

andrewrgarcia avatar Apr 13 '23 22:04 andrewrgarcia

@andrewrgarcia okay well I'm going to have to take some time to read through your code and figure out exactly what it's doing because I've never done object-oriented programming before so I have to educate myself a little bit and then once I feel like I have a grasp of that I'll make a loom video explaining anything that I see and also maybe some suggestions as well on how to put the footprints together

But this would be good because I'm at a point with my other project that I have to learn an object oriented programming so this is a good simple code to practice figuring out what's going on

quantfreedom avatar Apr 13 '23 22:04 quantfreedom

@andrewrgarcia okay well I'm going to have to take some time to read through your code and figure out exactly what it's doing because I've never done object-oriented programming before so I have to educate myself a little bit and then once I feel like I have a grasp of that I'll make a loom video explaining anything that I see and also maybe some suggestions as well on how to put the footprints together

But this would be good because I'm at a point with my other project that I have to learn an object oriented programming so this is a good simple code to practice figuring out what's going on

@QuantFreedom1022 That's great. The second paragraph is more of a request. I would prioritize the stuff I mentioned on the first paragraph (ideas/algos/drawn sketches for the lumping of timestamps with volumes). This can be done without learning the new code and more by looking at the csv that you generate with your former code and how to put the info together for the footprints

andrewrgarcia avatar Apr 13 '23 22:04 andrewrgarcia

@andrewrgarcia ok sounds good ... ill have a look at it as soon as I can and let u know any ideas I have

quantfreedom avatar Apr 13 '23 22:04 quantfreedom

i want to create a footprint chart like this one where i am able to make an open high low close but also show how much trading volume was at each y value of the candle

here is an example of what i am talking about

image

its very easy to get the data ... getting and understanding how to organize the data on our end is very very easy ... you can see what a csv file would look like here ... https://public.bybit.com/trading/BTCUSD/

lets say i wanted to create a 5 min candle ... for the info i would just add up all the sell orders at each price point of the candle and do the same for the buy orders ... then i have a dataframe within the candle that tells me the sell column and buy column ... and you can get the open high low close even from that data ... the open is the first trade at the start of the candle then the high is the highest point ... low is the lowest point and the close is the last candle ...

this info helps us visualize at what price point in the candle has the most trade volume ... footprint charts are used in high level trading ... I will be making a ton of videos on this subject if we can get one of these open source plotting platforms to help create something like this

all we would need is send dataframes inside a master dataframe ... so you have the overall dataframe that houses all of the candles and inside each candle is a dataframe of all of the sell and buy data at each price of the candle

ultimately would be nice for it to look like this ... but for now the top one would be ultra sexy

image

I did this but in a more elegant way (my last PR in candle builder.py). Instead of a dataframe within a master dataframe, the code creates a "3-d" dataframe so to speak, where each open and close time is now unique and each entry for every other column is now a list of ask bid and volumes thereof. The next steps would be making the actual graphs and answering the questions I posted in this new PR.

andrewrgarcia avatar Apr 14 '23 03:04 andrewrgarcia

Hi all, my name us philip and loojs like im comming in late. I was planning on coding some footprint logic and came accross this project. Do u guys still need help? Im an intermediate developer at best but very resourceful and have a tendency to get things done. Look forward to you reply/ies.

Much thanks

Philip030601 avatar Nov 14 '23 01:11 Philip030601

Hi all, my name us philip and loojs like im comming in late. I was planning on coding some footprint logic and came accross this project. Do u guys still need help? Im an intermediate developer at best but very resourceful and have a tendency to get things done. Look forward to you reply/ies.

Much thanks

@Philip030601 yeah man that would be great ... i need to get back into this and working with someone else on it would help me get back into this project ... could you send me your discord or telegram or email or something and then we can setup a time to chat about what is needed to be done and share some ideas on what we want it to look like

quantfreedom avatar Nov 15 '23 02:11 quantfreedom

Hi quantfreedom.

Hope your doing well. Im in pst time and available to talk after 6pm almost every night. Let me know of your availability and will plan on it.

Thanks

On Tue, Nov 14, 2023, 6:48 PM Quant Freedom 1022 @.***> wrote:

Hi all, my name us philip and loojs like im comming in late. I was planning on coding some footprint logic and came accross this project. Do u guys still need help? Im an intermediate developer at best but very resourceful and have a tendency to get things done. Look forward to you reply/ies.

Much thanks

@Philip030601 https://github.com/Philip030601 yeah man that would be great ... i need to get back into this and working with someone else on it would help me get back into this project ... could you send me your discord or telegram or email or something and then we can setup a time to chat about what is needed to be done and share some ideas on what we want it to look like

— Reply to this email directly, view it on GitHub https://github.com/matplotlib/mplfinance/issues/572#issuecomment-1811725537, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYZZYII7KWKIOV7FZKURBNLYEQUP5AVCNFSM6AAAAAASLTJOBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJRG4ZDKNJTG4 . You are receiving this because you were mentioned.Message ID: @.***>

Philip030601 avatar Nov 15 '23 18:11 Philip030601

@Philip030601 ok ... let me know a contact for you that would be good ... either telegram, email or discord

quantfreedom avatar Nov 15 '23 18:11 quantfreedom

@Philip030601 ok ... let me know a contact for you that would be good ... either telegram, email or discord

Hi quant. My email is [email protected].

Philip030601 avatar Nov 16 '23 01:11 Philip030601

@andrewrgarcia not sure if you are still interested in this but i am picking the project back up ... and i now understand about classes and all the stuff i didn't know about 6 months ago ... so i can actually be useful this time ... let me know if you are still interested in trying to finish this

quantfreedom avatar Nov 17 '23 16:11 quantfreedom

I found a person who has something close to what i want ... you can check out the project here https://github.com/murtazayusuf/OrderflowChart

quantfreedom avatar Nov 17 '23 17:11 quantfreedom

I am happy to contribute with you guys, but contingent on 1) that we all follow good open-source / version control practices so that we all get credit for our contributions and 2) we start from scratch with an object-oriented structure with concise code with little to no additional dependencies (packages). Mplfinance is a lean package and does not use numba, which in principle can make it quite heavy. In addition, I didn't find the implementation of numba helpful as I made it just as fast without it. Code cannot be clunky, it needs to be sleek

andrewrgarcia avatar Nov 17 '23 17:11 andrewrgarcia

@andrewrgarcia yeah i am happy to do so

1: this will be my first time trying to work with someone over the internet with only typing and also using github for version controls and practices and all that stuff ... so if you are willing to work with someone that is 100% new to this then i am fine but i have absolutely no idea about what good version control practices are ... i don't even know rule number one

2: yes i can do this but the concise code with little to no dependencies not sure ... all i know is i want to use numpy and as little pandas as possible because pandas is so slow ... but you are smarter than me so i am ok with using what ever dependencies that you want

We already have the project open at https://github.com/QuantFreedom1022/Footprints .. but i have no idea where to start ... or where should we start talking about this idea?

if you could lead this that would be great because i had no idea how to lead a project with only typing over the internet ... i am more of a lets hope on zoom ... talk for 15 mins and then get to work

quantfreedom avatar Nov 17 '23 17:11 quantfreedom

Hi there, I will need some help with this, where can I get this type of data? I was trying Tradermade.com, but there are Bid/Ask/Mid columns

KMey-create avatar Nov 24 '23 21:11 KMey-create

@KMey-create i get mine from binance https://www.binance.com/en/landing/data

quantfreedom avatar Nov 24 '23 22:11 quantfreedom

Quant,

Happy belated thanksgiving. I thought the data had to be time and sales type in tick format.

Regards,

On Fri, Nov 24, 2023, 2:23 PM Quant Freedom 1022 @.***> wrote:

@KMey-create https://github.com/KMey-create i get mine from binance https://www.binance.com/en/landing/data

— Reply to this email directly, view it on GitHub https://github.com/matplotlib/mplfinance/issues/572#issuecomment-1826114089, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYZZYIMW337VL2V47EVVNWDYGEM4NAVCNFSM6AAAAAASLTJOBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRWGEYTIMBYHE . You are receiving this because you were mentioned.Message ID: @.***>

Philip030601 avatar Nov 26 '23 23:11 Philip030601