Zend certified PHP/Magento developer

How secure is it to open my ssh port on my server if it uses public private key encryption?

I have multiple Linux servers and would like to be able to access them from the outside using my public IP address, these servers are secured with public private key encryption and can not be authenticated with a password. Is it ok to open private port 22 to some random obscure public port or does this cause to much of a security issue.