The YubiKey 4 and YubiKey NEO support the Personal Identity Verification (PIV) interface. This enables us to perform RSA or ECC sign/decrypt operations using a private key stored on the device through PKCS#11.
https://developers.yubico.com/PIV/Introduction/Certificate_slots.html