Your public key needs to be on the server in
~/.ssh/authorized_keys2
In your home directory, your private key should be found at
~/.ssh/id_rsa
and your public key at id_rsa.pub
If your public key is not in the server's authorized_keys2 file, you should try opening it in a text editor, copying and pasting the text in the id_rsa.pub file at the end of it (if it is not empty), and saving it back to the server in the same place.
Alternatively, you could just use SFTP to transfer files with Cyberduck.
It uses the same secure protocol as public private key auth, it just means you need to save your password in the Apple Keychain.
PKA is very geeky, so unless you're really comfortable with command lines and stuff, you're probably better off ignoring it.

I am not Ryan Seacrest, and I do not approve this message.