While I am not 100% sure about the root cause, I believe this problem is caused by inversions due to negative determinants.
Try to treat negative determinants of the rotation matrix as a switch for the winding.
Try to treat negative determinants of the rotation matrix as a switch for the winding.