Microsoft Crypto API (CAPI) was first released with the Windows NT4 operating system in 1996. The OpenSSL project, which was originally a fork of SSLeay by Eric Young and Tim Hudson, was initiated in 1998 and has since become one of the most widely distributed cryptographic libraries available.
I recently required a Windows application using CAPI that can sign and verify files using the RSA digital signature algorithm, but It needed to read RSA keys and signatures generated by OpenSSL. The keys are stored using Abstract Syntax Notation One (ASN.1) and Privacy Enhanced Mail (PEM) format. The signatures are stored in binary using big-endian convention.
In this post, we’ll focus specifically on RSA key generation, the importation/exportation of RSA keys and the key management standards used to exchange keys in a platform independent manner.
I understand RSA is being phased out in favor of Elliptic Curve Cryptography (ECC) which may be discussed in a future post. I’m also aware ECC will eventually be phased out in favor of quantum resistant cryptography which is still under a lot of research and development, but that could be 10+ years away and RSA still offers good security margin, albeit with less efficiency.
Part 2 will focus specifically on generation and exchange of session keys over TCP for symmetric encryption, but the bulk of work needed to reach that stage is really within this post.
Key Management Standards
The 2 main issues developers appear to complain about for interoperability between OpenSSL and Microsoft Crypto API are:
- Signatures exported by OpenSSL functions use big-endian convention, Microsoft Crypto API uses little-endian
- Public and Private Keys exported by OpenSSL functions use ASN.1 structures, Microsoft CryptoAPI use their own structures or what are referred to as “Blobs”
However, both API support the following standards for public key management:
- PKCS #8: Private-Key Information Syntax Standard
- ASN.1 encoding rules: Specification of Basic Encoding Rules (BER), Canonical Encoding Rules (CER) and Distinguished Encoding Rules (DER)
We just need to import and export keys using these standard formats to successfully exchange keys between the 2 API.
Essentially, this encoding uses the base64 algorithm.
RSA Digital Signatures and RSA Key Exchange
When signing a file, we derive a cryptographic hash from its data. This hash is then encrypted using an RSA private key and modular exponentiation. The resulting ciphertext is called a signature. Verification of the signature involves decryption using an RSA public key and Modular Exponentiation.
When exchanging session keys, the client side will generate a value derived from a cryptographic pseudo-random number generator (CSPRNG). This value will be used as the symmetric encryption key. It’s then encrypted using an RSA public key and modular exponentiatation before being sent to a remote server. The server will perform RSA decryption using the private key to recover the same session key.
As stated already, OpenSSL exports signatures using the Big-Endian convention whereas Microsoft Crypto API uses Little-Endian.
High end servers and mainframes in the 80s and 90s used Big-Endian architectures like SPARC, MIPS and POWER. The legacy of this are many cryptography libraries using Big-Endian convention to store data on disk.
To accomodate this on Windows which predominantly runs on X86 architecture, we use the following piece of code to swap the order of bytes after signing and before verification.
Then we have no problem verifying signatures generated by OpenSSL.
RSA key context
The following structure is defined to hold RSA keys.
RSA Key Generation
CAPI uses 65537 as the public exponent in key generation so we need to use the same for OpenSSL.
Reading and writing PEM files
Before using the CAPI functions, we need to decode the PEM files into ASN.1 encoded structures. The CryptStringToBinary and CryptBinaryToString APIs can convert to and from PEM, however these were only made available since Windows XP and Windows 2003.
See PEM_read_file and PEM_write_file functions for more details.
Importing public and private keys
- Crypto API
For the Public key, decode the ASN.1 structure into a Public Key Info structure before importing to CAPI key object using CryptImportPublicKeyInfo API.
Private Key, decode the ASN.1 structure into a Private Key Info structure. Convert the PrivateKey value into a CAPI Private Key Blob before importing into a CAPI key object.
Unfortunately, there’s no CryptImportPrivateKeyInfo API, hence the extra call to CryptDecodeObjectEx.
OpenSSL offers a much simpler solution with a single API call for both private and public keys. We also don’t have to decode the PEM format before hand. Nice, eh?
- Crypto API
Since 2000, we can use CryptExportPKCS8 to export the private key.
Exporting the public key using CryptExportPublicKeyInfo before encoding with ASN.1.
Signing a file
- Crypto API
- Crypto API
RSA Tool Usage
- Key Generation
- Signing a file
- Verifying signature
The purpose of this post was to cover the main problems of key exchange between OpenSSL and Microsoft Crypto API.
For symmetric key exchange, so long as we use ASN.1 encoding for the exchange of public and private keys and remember that OpenSSL uses Big-Endian convention instead of Little-Endian by CAPI, there isn’t a significant problem.
In part 2, we’ll examine how to perform End-To-End encryption of network traffic between a windows machine using Crypto API and Linux using OpenSSL.
Source code for the RSA tool can be found here