Balance, order volume is not explained in the docs. Reward gaming behavior observed in some models.

[astrologos]

Hi @AminHP @bionicles @super-pirata , could you please update the docs to include an explanation of the agent's asset pool and how the volume of an order is determined?
An example on how to change these would also be helpful, as the README is opaque in this regard.

I have observed what I suspect is reward-gaming behavior in my trained agents, and I'm wondering if it is due to an under-specified environment. My agents (tested on DQN, PPO and A2C) fall into a stable valley where their optimal behavior is to first sell, then buy and continue buying. This results in profit slightly shy of 1.

Clearly this should not be valid behavior in any trading environment.

Thanks!
Jack

[astrologos]

Training loss associated with this policy is 0.0008.

[astrologos]

[astrologos]

Hi, I am writing to report that I've created an alternative trading environment that is more advanced, highly customizable, and simpler to use, render and evaluate. It dodges the issues above.

You can find the repo here:
https://www.github.com/astrologos/tradinggym