Camera child of a moving group, rotation matrix wrong

[DennisSmolek]

I have a camera rig object that follows other objects.
A common one would be a character controller.

As the character moves, multiple child Object3D's follow it.

I put cameras into these groups, making them automatically follow and move along with the character object.

When I want the camera to be controllable, i pass it to CameraControls with: this.cameraControls.camera = this.activeCamera

To keep the camera pointed the right way, every frame I update the target of CameraControls to the position of the character.

This works great!

Until I try to rotate the camera...

Because the camera is in a local-space, the transform matrix when rotating is TOTALLY off. rotating also dollys, and when dollying, it rotates.

When I move the camera into world-space, the movement works as expected.

Is there a flag to let camera-controls know it's in a local space?

Is there a property (like focal point) that I need to adjust to compensate for the local space transform?