I always made sure to delete the possibility for ovh to get in over ssh, they add a string in the keyring for all ovh kernels which allows them to login as root at any moment.
You are allowed to delete this, but you are not eligible for support anymore, but i never needed that anyway so its a nice way to block access to the server to prevent them from eavesdropping on what your doing.