@vbabka hmm...
Check first taht you have both of these installed:
1. pcscd (system service check also systemctl)
2. scdaemon (user service)
I've never dared to change my bootstrap script. It's shitty and does stuff that is not probably necessary but seems to always do the trick:
GPG_AGENT_SOCKETS=(gpg-agent-ssh.socket
gpg-agent-browser.socket
gpg-agent-extra.socket
gpg-agent-ssh.socket
gpg-agent.socket)
systemctl --user disable --now "${GPG_AGENT_SOCKETS[@]}"
gpgconf --kill gpg-agent
sudo systemctl disable --now pcscd.socket
systemctl --user enable --now gpg-agent.socket gpg-agent-ssh.socket
sudo systemctl enable --now pcscd.socket