How to setup two different configurations for two listenaddresses in sshd

I have the following setup: a Linux machine has two IP addresses (primary and secondary) on the same network. One receives requests NAT forwarded from outside by the router/modem, the other one receives requests from the internal network only.

I would like sshd to listen from both addresses, but allow password authentication only from the internal-facing IP.

Is there any way to do that with a single ssh daemon or do I need two, each listening to a different IP?