Authentication failure – system joined to Active Directory with sssd does not seem to be invoking pam_sss

Logins for domain users with su, cockpit, and ssh all show failures as if the passwords are incorrect.

$ su myuser
Password: 
su: Authentication failure

It’s got a (should be) identical twin system where everything is working as expected. Versions of required packages are the same as well as OS version. sssd, krb5, pam (system-auth,passwd-auth) files are all the same.

I set debug_level = 10 in the sssd, pam, and domain sections of sssd.conf.

/var/log/secure will show failures for ssh logins with:

sshd[367807]: error: PAM: Authentication failure for myuser from myIP

which seems like it’s not calling pam_sss at all. This is what it looks like on the working system with a wrong password:

pam_sss(sshd:auth): received for user myuser: 7 (Authentication failure)

ssh keys lookup from the directory is configured the same on both systems, but this is what I get testing on the problem one:

$ /usr/bin/sss_ssh_authorizedkeys myuser
Error looking up public keys

There are no sssd_pam.log or krb5_child.log files generated in /var/log/sssd/, and I haven’t seen anything notable in sssd_my.domain.log

There is nothing configured in /etc/security/access.conf.

The domain is joined properly as far as I can tell. The following returns ‘online’ and the correct domain controllers:

$ sudo sssctl domain-status my.domain
Online status: Online

Active servers:
AD Global Catalog: primarydc.my.domain
AD Domain Controller: primarydc.my.domain

Discovered AD Global Catalog servers:
- primarydc.my.domain
- seconddc.my.domain
- thirddc.my.domain

Discovered AD Domain Controller servers:
- primarydc.my.domain
- seconddc.my.domain
- thirddc.my.domain
$ getent passwd myuser
myuser:*:uid:gid:My User:/home/myuser:/bin/bash

I’ve tested both disabling selinux and the firewall, neither makes a difference.

It really seems like pam_sss.so is never called even though I can see it in the files in /etc/pam.d/