One option is to put the object in a container and make the camera the child of the container (and a sibling of the object) and have the container control movement and rotation of the object, so long as the container doesn't rotate, the camera wont either.
Option 2 is to use a smooth follow script or similar to make the camera track the position of the object without using parenting. You can do this with vector3 lerp and a minimum distance parameter to have the camera follow the object from a certain distance