[Question] Why does network initialization use jax's default network initialization?

[liruiluo]

Question

Why does network initialization use jax's default network initialization?
Would it be more reasonable to use He initialization or orthogonal initialization?

Checklist

• [√ ] I have read the documentation (required)
• [√ ] I have checked that there is no similar issue in the repo (required)

[araffin]

Hello,

Why does network initialization use jax's default network initialization?

For simplicity and because it worked so far.

Would it be more reasonable to use He initialization or orthogonal initialization?

I would be open to have a pull request that changes it if you show empirical results (benchmark on multiple envs using at least 5-10 random seeds) that it is actually better defaults.