Check out the ray docker image, it has the whole ML / RL stack pre built and configured. Maybe easier than building your own. All the best