The current inverseDynamicsSolver realizes stage Dynamics, which is not necessary. Stage Velocity should suffice, since this stage already contains all the information for the coriolis and gravitational forces. Therefore, the inverseDynamicsSolver can be made a lot faster.
See also my post on the OpenSim forum: https://simtk.org/forums/viewtopic.php?f=91&t=6292&view=unread&sid=c3815501f17b7ddc3653f1c146878bd7#unread