QAT Engine: Hardware Crypto Acceleration with OpenSSL & Qdrant

Crypto operations make TLS connections slow. Most teams accept this cost. Intel QuickAssist Technology (QAT) removes much of it. QAT is hardware that does crypto work for the CPU. This post tells how I built, patched, and deployed the QAT Engine. The result is a 56x speedup for RSA-2048 signing.

The problem

Every TLS handshake needs a private key signing operation. For RSA-2048, this is a modular exponentiation (m^d mod n). One CPU core on a standard CPU does about 1,000–1,700 signs per second. Some services have thousands of concurrent connections. Qdrant, a vector database, is one example. For these services, signing is the bottleneck.

Intel QAT is a co-processor on the motherboard, next to the CPU. The CPU does not do the signing math. You send the numbers to the QAT hardware. The hardware computes the result and sends the answer back.

The QAT Engine code

The QAT Engine v2.0.0b repository has about 60,000 lines of C, documents, and patches. One codebase builds two outputs:

• qatengine.so — the OpenSSL Engine API (older, used by HAProxy and NGINX)
• qatprovider.so — the OpenSSL 3.x Provider API (newer, used by Rustls)

The build flag --enable-qat_provider selects the output. Both outputs use the same core C files to talk to the QAT hardware.

Build from source

I built the engine against the in-tree qatlib 24.02.0 of the system. The build failed:

./autogen.sh
./configure --with-qat_hw_dir=/usr
make -j$(nproc)

# Result: undefined symbol: icp_sal_AsymGetInflightRequests

The function icp_sal_AsymGetInflightRequests() manages congestion. It shows how many crypto operations are in progress on the hardware. Intel added it in qatlib 24.09.0+. The system had 24.02.0. The header file was present, but libqat.so did not contain the function.

The fix: weak symbol stubs

I added weak symbol fallbacks in e_qat.c:

#ifdef QAT_HW_INTREE
__attribute__((weak))
CpaStatus icp_sal_AsymGetInflightRequests(
    CpaInstanceHandle instanceHandle,
    Cpa32U *maxInflightRequests,
    Cpa32U *numInflightRequests)
{
    if (maxInflightRequests) *maxInflightRequests = 1;
    if (numInflightRequests) *numInflightRequests = 0;
    return CPA_STATUS_SUCCESS;
}
#endif

A weak symbol is a fallback that the linker can replace. If the real qatlib has these symbols at runtime, they replace my stubs. If it does not, my stubs report "no congestion". The engine then loads correctly. This is safe. It only turns off throttling that depends on congestion.

Qdrant and HAProxy

Qdrant does no cryptography itself. HAProxy runs in front of Qdrant as a TLS termination proxy. It ends the TLS connection and sends plain HTTP to Qdrant:

Client (HTTPS) → HAProxy :8443 → Qdrant :6333 (HTTP)
                     │
                     └── ssl-engine qatengine algo RSA
                         (offloads TLS signing to QAT HW)

I rebuilt HAProxy from source with USE_ENGINE=1. The system package did not include engine support. I also limited Qdrant to 8 CPUs with --cpus=8. Its Actix-rt framework panics on systems with 256 cores.

Rustls proof of concept

This test repeats the Intel benchmark. It uses patched Rust crates for asynchronous QAT offload. The call path is:

crypto_bench_async (Rust/Tokio binary)
    → rustls-openssl (patched crate)
        → qatprovider.so (OpenSSL 3.x Provider)
            → OpenSSL 3.5.0 (built from source)
                → qatlib (VFIO)
                    → QAT 4xxx Hardware

For this path, I built OpenSSL 3.5.0 from source. I rebuilt the QAT code as a Provider, not an Engine. I also applied two patches from the rustls-poc directory:

• rustls.patch — adds asynchronous, non-blocking sign support to rustls v0.23.33
• rustls-openssl.patch — adds the async-jobs feature, a batched worker loop, and the benchmark binary

Device setup

The system had 4 QAT Gen4 devices on 2 NUMA nodes. Only 2 devices were set up for crypto (sym;asym). The other 2 were set up for data compression (dc). I set all 4 devices to sym;asym through sysfs. This doubled the crypto capacity:

# Bring down, reconfigure, bring up
echo down > /sys/bus/pci/devices/XXXX/qat/state
echo asym > /sys/bus/pci/devices/XXXX/qat/cfg_services
echo up   > /sys/bus/pci/devices/XXXX/qat/state

In-tree and out-of-tree drivers

You must know the two driver types before you choose one:

• In-tree (qatlib) — comes with the Linux kernel. It uses VFIO to give userspace access through Virtual Functions. The qatmgr program manages it. It needs IOMMU to be on. It uses a shared memory allocator, which causes contention with many threads.
• Out-of-tree (OOT) — you download it from Intel. It uses UIO and USDM with thread-specific DMA allocators, so there is no contention. In PF mode, it needs IOMMU to be off.

I tried the OOT driver because of its thread-specific USDM allocator. It gave this error: Cannot use PF with IOMMU enabled and SVM off. I could not turn off the IOMMU without a reboot. So I used the in-tree driver.

Final results

Algorithm         | Software (Ring) | QAT HW        | Speedup
RSA-2048 Sign     | 1,035 ops/s     | 58,629 ops/s   | 56.6x
ECDSA P-384 Sign  | 896 ops/s       | 57,881 ops/s   | 64.6x

vs Intel Chart Reference:
RSA-2048:   58,629 → 84% of ~70,000 target
ECDSA P-384: 57,881 → 118% of ~49,000 (exceeded!)

Key files

• e_qat.c — the engine entry point; the only file I changed (weak symbol stubs)
• qat_hw_rsa.c — RSA sign and verify through the QAT hardware CPA API
• qat_hw_ec.c — ECDSA sign and verify, and ECDH key agreement
• qat_hw_polling.c — a background thread that polls for finished QAT operations
• qat_prov_init.c — the Provider entry point for OpenSSL 3.x
• configure.ac — the Autoconf check for an in-tree or OOT driver

Lessons learned

• Weak symbols help C code work with different library versions.
• The Engine API and the Provider API are different. The Engine API is older but common. The Provider API is newer and needs OpenSSL 3.x.
• IOMMU settings limit the OOT driver in PF mode. Plan for this in the infrastructure.
• Sysfs device settings reset at reboot. A production system needs a systemd unit or a udev rule.
• The in-tree driver scales badly with many threads because of allocator contention. Two threads were slower than one thread for RSA and ECDSA.