It will depend on your location and the animal.
The rules for how close to get to an animal aren't just to prevent bothering the animal but also to help ensure the safety of the people. An animal may charge if you get too close and feels threatened. In the case of bears and other predators, the animal has to be dispatched if it attacks a person. Preventing these situations is ideal for all parties involved.
As for the drone, it will depend on where you are. As mentioned in the comments, drones are prohibited in national parks and monuments. States will also have different rules. So those rules and regulations are a good start.
Beyond those considerations, I would say trying to not disturb animals with the drone would be a good guideline. If the animal flees from the drone, don't chase it. Be respectful and minimize your impact. Individual animals are going to react differently so just maintain your distance and try to avoid disturbing them too much.