FinRL
FinRL copied to clipboard
AI4Finance-Foundation
Keras models with Attention and bidirectional lstm
Is your feature request related to a problem? Please describe. I wrote this up https://gist.github.com/thistleknot/0edafe7c477290e300e5cac08e7a4984 Can be used with multivariate data to forecast a sequence out.
Describe the solution you'd like Would like to see a library that can use such a multivariate set to do RL
Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.
Additional context Add any other context or screenshots about the feature request here.
Is your feature request related to a problem? Please describe. I wrote this up https://gist.github.com/thistleknot/0edafe7c477290e300e5cac08e7a4984 Can be used with multivariate data to forecast a sequence out.
Describe the solution you'd like Would like to see a library that can use such a multivariate set to do RL
Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.
Additional context Add any other context or screenshots about the feature request here.
Hi, would you like to connect your codes with FinRL and test its performance?
I'm not sure if the code will work as it's setup for supervised? I know this method of rl uses Ann's, so maybe the model itself can be used but the way I'm using the model in that example is with a supervised problem
On Tue, Aug 23, 2022, 12:41 AM Joshua Laferriere @.***> wrote:
The code currently works with tabular data. I'd have to fit it for RL and that's where I'm at a loss. I've written an RL algorithm once using q learning in ai gym (basic taxi tutorial). I was hoping to get some pointers or if someone else could just use this code to augment RL.
On Tue, Aug 23, 2022, 12:18 AM Xiao-Yang Liu @.***> wrote:
Is your feature request related to a problem? Please describe. I wrote this up https://gist.github.com/thistleknot/0edafe7c477290e300e5cac08e7a4984 Can be used with multivariate data to forecast a sequence out.
Describe the solution you'd like Would like to see a library that can use such a multivariate set to do RL
Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.
Additional context Add any other context or screenshots about the feature request here.
Hi, would you like to connect your codes with FinRL and test its performance?
— Reply to this email directly, view it on GitHub https://github.com/AI4Finance-Foundation/FinRL/issues/701#issuecomment-1223654269, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHKKOXL56E4464JAL7YWO3V2R3N3ANCNFSM57JZ44JA . You are receiving this because you authored the thread.Message ID: @.***>
I'm not sure if the code will work as it's setup for supervised? I know this method of rl uses Ann's, so maybe the model itself can be used but the way I'm using the model in that example is with a supervised problem On Tue, Aug 23, 2022, 12:41 AM Joshua Laferriere @.> wrote: … The code currently works with tabular data. I'd have to fit it for RL and that's where I'm at a loss. I've written an RL algorithm once using q learning in ai gym (basic taxi tutorial). I was hoping to get some pointers or if someone else could just use this code to augment RL. On Tue, Aug 23, 2022, 12:18 AM Xiao-Yang Liu @.> wrote: > Is your feature request related to a problem? Please describe. I wrote > this up > https://gist.github.com/thistleknot/0edafe7c477290e300e5cac08e7a4984 Can > be used with multivariate data to forecast a sequence out. > > Describe the solution you'd like Would like to see a library that can > use such a multivariate set to do RL > > Describe alternatives you've considered A clear and concise > description of any alternative solutions or features you've considered. > > Additional context Add any other context or screenshots about the > feature request here. > > Hi, would you like to connect your codes with FinRL and test its > performance? > > — > Reply to this email directly, view it on GitHub > <#701 (comment)>, > or unsubscribe > https://github.com/notifications/unsubscribe-auth/ABHKKOXL56E4464JAL7YWO3V2R3N3ANCNFSM57JZ44JA > . > You are receiving this because you authored the thread.Message ID: > @.***> >
Thanks for sharing. It is possible, e.g., I tested some cases when first using supervised/unsupervised learning to train a policy network, and then using RL to further train it. Some code adaptions are needed, while in principle, it is possible.
Well tell me what you need from me. If you think you can implement it with what I've provided then by all means. Else I presume i need to read up on q learning implementation using an Ann (something i havent done yet)
On Tue, Aug 23, 2022, 2:28 PM Xiao-Yang Liu @.***> wrote:
I'm not sure if the code will work as it's setup for supervised? I know this method of rl uses Ann's, so maybe the model itself can be used but the way I'm using the model in that example is with a supervised problem On Tue, Aug 23, 2022, 12:41 AM Joshua Laferriere @.
> wrote: … <#m_1030488903780535123_> The code currently works with tabular data. I'd have to fit it for RL and that's where I'm at a loss. I've written an RL algorithm once using q learning in ai gym (basic taxi tutorial). I was hoping to get some pointers or if someone else could just use this code to augment RL. On Tue, Aug 23, 2022, 12:18 AM Xiao-Yang Liu @.> wrote: > Is your feature request related to a problem? Please describe. I wrote > this up > https://gist.github.com/thistleknot/0edafe7c477290e300e5cac08e7a4984 Can
be used with multivariate data to forecast a sequence out. > > Describe the solution you'd like Would like to see a library that can > use such a multivariate set to do RL > > Describe alternatives you've considered A clear and concise > description of any alternative solutions or features you've considered. > > Additional context Add any other context or screenshots about the > feature request here. > > Hi, would you like to connect your codes with FinRL and test its > performance? > > — > Reply to this email directly, view it on GitHub > <#701 (comment) https://github.com/AI4Finance-Foundation/FinRL/issues/701#issuecomment-1223654269>, or unsubscribe > https://github.com/notifications/unsubscribe-auth/ABHKKOXL56E4464JAL7YWO3V2R3N3ANCNFSM57JZ44JA . > You are receiving this because you authored the thread.Message ID: > @.***> >
Thanks for sharing. It is possible, e.g., I tested some cases when first using supervised/unsupervised learning to train a policy network, and then using RL to further train it. Some code adaptions are needed, while in principle, it is possible.
— Reply to this email directly, view it on GitHub https://github.com/AI4Finance-Foundation/FinRL/issues/701#issuecomment-1224904971, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHKKOQOCJ5L2PKX3WHTLJLV2U66NANCNFSM57JZ44JA . You are receiving this because you authored the thread.Message ID: @.***>
I just revisited q-learning. I think what I need to do is use a q-table (with bidirectional attention?) as a multi-variate input to a multi-variate/step output
"Multiple Parallel Input and Multi-Step Output" https://machinelearningmastery.com/how-to-develop-lstm-models-for-time-series-forecasting/
I touched up on reinforcement learning and found a few nice resources.
I can likely augment this with attention https://www.geeksforgeeks.org/implementing-deep-q-learning-using-tensorflow/
Custom environments https://blog.paperspace.com/creating-custom-environments-openai-gym/
To figure out how to load stock data as my state space. Action space = buy/sell Rewards will be >0 return = True, vice versa.
The code currently works with tabular data. I'd have to fit it for RL and that's where I'm at a loss. I've written an RL algorithm once using q learning in ai gym (basic taxi tutorial). I was hoping to get some pointers or if someone else could just use this code to augment RL.
On Tue, Aug 23, 2022, 12:18 AM Xiao-Yang Liu @.***> wrote:
Is your feature request related to a problem? Please describe. I wrote this up https://gist.github.com/thistleknot/0edafe7c477290e300e5cac08e7a4984 Can be used with multivariate data to forecast a sequence out.
Describe the solution you'd like Would like to see a library that can use such a multivariate set to do RL
Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.
Additional context Add any other context or screenshots about the feature request here.
Hi, would you like to connect your codes with FinRL and test its performance?
— Reply to this email directly, view it on GitHub https://github.com/AI4Finance-Foundation/FinRL/issues/701#issuecomment-1223654269, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHKKOXL56E4464JAL7YWO3V2R3N3ANCNFSM57JZ44JA . You are receiving this because you authored the thread.Message ID: @.***>
Instead of a sequence. I think it needs to be a multivariate to multivariate single step output?
Idk but this is what confuses me. I've never done an Ann for rl especially for a multivariate sequence series
On Tue, Aug 23, 2022, 3:44 PM Joshua Laferriere @.***> wrote:
Well tell me what you need from me. If you think you can implement it with what I've provided then by all means. Else I presume i need to read up on q learning implementation using an Ann (something i havent done yet)
On Tue, Aug 23, 2022, 2:28 PM Xiao-Yang Liu @.***> wrote:
I'm not sure if the code will work as it's setup for supervised? I know this method of rl uses Ann's, so maybe the model itself can be used but the way I'm using the model in that example is with a supervised problem On Tue, Aug 23, 2022, 12:41 AM Joshua Laferriere @.
> wrote: … <#m_-102830034417399649_m_1030488903780535123_> The code currently works with tabular data. I'd have to fit it for RL and that's where I'm at a loss. I've written an RL algorithm once using q learning in ai gym (basic taxi tutorial). I was hoping to get some pointers or if someone else could just use this code to augment RL. On Tue, Aug 23, 2022, 12:18 AM Xiao-Yang Liu @.> wrote: > Is your feature request related to a problem? Please describe. I wrote > this up > https://gist.github.com/thistleknot/0edafe7c477290e300e5cac08e7a4984 Can
be used with multivariate data to forecast a sequence out. > > Describe the solution you'd like Would like to see a library that can > use such a multivariate set to do RL > > Describe alternatives you've considered A clear and concise > description of any alternative solutions or features you've considered. > > Additional context Add any other context or screenshots about the > feature request here. > > Hi, would you like to connect your codes with FinRL and test its > performance? > > — > Reply to this email directly, view it on GitHub > <#701 (comment) https://github.com/AI4Finance-Foundation/FinRL/issues/701#issuecomment-1223654269>, or unsubscribe > https://github.com/notifications/unsubscribe-auth/ABHKKOXL56E4464JAL7YWO3V2R3N3ANCNFSM57JZ44JA . > You are receiving this because you authored the thread.Message ID: > @.***> >
Thanks for sharing. It is possible, e.g., I tested some cases when first using supervised/unsupervised learning to train a policy network, and then using RL to further train it. Some code adaptions are needed, while in principle, it is possible.
— Reply to this email directly, view it on GitHub https://github.com/AI4Finance-Foundation/FinRL/issues/701#issuecomment-1224904971, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHKKOQOCJ5L2PKX3WHTLJLV2U66NANCNFSM57JZ44JA . You are receiving this because you authored the thread.Message ID: @.***>