I think there is a look-ahead bias

[mg64ve]

Hi there, nice work. However I think there is a look-ahead bias. Every timestep, you get state and this state includes the current closeprice. Then with step method you calculate profit as:

self.exit_price = self.closingPrice
self.reward += ((self.entry_price - self.exit_price)/self.exit_price + 1)*(1-self.fee)**2 - 1 # calculate reward

In this case you are using the same information that you already used to predict the next action. What do you think about it?

[miroblog]

state_n <- updateState() action_n <- network(state_n) reward_n <- compute_reward(action_n, state_n)

---

state_n_plus_1 <- updateState() action_n_plus_1 <- network(state_n_plus_1) reward_n_plus_1 <- compute_reward(action_n_plus_1, state_n_plus_1)

reward is computed from the current state, thus there is no lookahead bias. Put it simply, (e.g.) one decides to sell the stock based on past&current price and if one does sell, then one would calculate the earning based on current price.