There are two latency controls: the one in the camera settings, and the one in the Quest tab
If your real hands are taking too long to catch up with the virtual controllers, you'll want to increase the Camera Latency.
If your virtual sabers are taking too long to catch up with your real hands, you'll want to increase the Quest Latency. This compensates for the Quest running behind the camera.