look in /etc/ssh/sshd_config, i think you need to disable password authorisation and enable one of the other methods. but best you read up on it first, make sure you don't lock yourself out