Skip to content

Commit

Permalink
QATlib: 24.02.0 release
Browse files Browse the repository at this point in the history
Changes from 23.11.0 to 24.02.0:

* Added Heartbeat support.
* Added support for QAT GEN 5 devices, including support for a range of crypto wireless algorithms.
* RAS - Device error reset and recovery handling.
* Bug fixes (See Resolved section in README.md)

Signed-off-by: Firos Sadarul <[email protected]>
  • Loading branch information
sfiros authored and fionatrahe committed Feb 20, 2024
1 parent abe15d7 commit ec81762
Show file tree
Hide file tree
Showing 98 changed files with 4,002 additions and 666 deletions.
41 changes: 32 additions & 9 deletions INSTALL
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Reference
=========


- Intel® Communications Chipset 4xxx Series Software for Linux*
- Intel® Communications Chipset 4xxx/4xxxx Series Software for Linux*
Getting Started Guide

===============================================================================
Expand Down Expand Up @@ -70,13 +70,15 @@ Check System Prerequisites
4xxx : Use "lspci -d 8086:4940" to check Physical Function (PF)
devices are present.
401xx : Use "lspci -d 8086:4942" to check PF devices are present.
420xx : Use "lspci -d 8086:4946" to check PF devices are present.
Note: Later, after "systemctl start qat" or "make install" steps, the
corresponding Virtual Function devices will also be visible and bound
to the vfio-pci driver.
4xxx : Use "lspci -d 8086:4941" to check VF devices have been created.
401xx : Use "lspci -d 8086:4943" to check VF devices have been created.
420xx : Use "lspci -d 8086:4947" to check VF devices have been created.
* firmware must be available
Check that these files exist:
For 4xxx or 401xx devices check that these files exist:
/lib/firmware/qat_4xxx.bin or /lib/firmware/qat_4xxx.bin.xz
/lib/firmware/qat_4xxx_mmp.bin or /lib/firmware/qat_4xxx_mmp.bin.xz
If not, download the firmware images from linux-firmware and copy them
Expand All @@ -85,17 +87,23 @@ Check System Prerequisites
wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/qat_4xxx_mmp.bin
sudo mv qat_4xxx.bin /lib/firmware
sudo mv qat_4xxx_mmp.bin /lib/firmware
On updating these files run "sudo dracut --force" to update initramfs.
For 420xx, these firmware files are needed:
/lib/firmware/qat_420xx.bin
/lib/firmware/qat_420xx_mmp.bin
If not present please contact [email protected]
On updating any firmware files run "sudo dracut --force" to update
initramfs.
* kernel driver must be running
Use "lsmod | grep qat" to check that these kernel modules are running:
intel_qat
qat_4xxx
qat_4xxx or qat_420xx
They should load by default if using any of the following:
* A recent Linux kernel (see https://intel.github.io/quickassist/qatlib/requirements.html#kernel-firmware-requirements )
* Fedora 36+
* Fedora 36+ (39+ for 420xx)
* RHEL 8.4+ (for compression use 9.0+)
* each PF device must be bound to the 4xxx driver
Use "ls /sys/bus/pci/drivers/4xxx/" to show the BDFs of each bound PF
* each PF device must be bound to the 4xxx or 420xx driver
Use "cd /sys/bus/pci/drivers; ls 4xxx; ls 420xx" to show the BDFs of
each bound PF
* BIOS settings
Intel VT-d and SR-IOV must be enabled in the platform BIOS.
Consult your platform guide on how to do this.
Expand Down Expand Up @@ -367,7 +375,9 @@ Compilation and installation - detailed instructions

In Managed mode, qatmgr manages allocation of the VF resources to
each process on process startup.
The following parameters can be set in /etc/sysconfig/qat:
A sample config file can be found in quickassist/utilities/service/qat.
it can be copied to /etc/sysconfig/qat and used to set the following
parameters:

POLICY
Indicates how many VFs will be assigned to each process.
Expand Down Expand Up @@ -572,7 +582,7 @@ Compilation and installation - detailed instructions
continue
fi

if [ "$did" != "0x4941" ] && [ "$did" != "0x4943" ]; then
if [ "$did" != "0x4941" ] && [ "$did" != "0x4943" ] && [ "$did" != "0x4947" ]; then
continue
fi

Expand Down Expand Up @@ -661,6 +671,9 @@ Full list of Configuration options
--enable-dc-error-simulation
Enables Data Compression Error Simulation.

--enable-hb-error-simulation
Enables Heartbeat Error Simulation.

--enable-legacy-lib-names
Enables legacy names for libraries.

Expand Down Expand Up @@ -719,10 +732,12 @@ Common issues
bound to qat_4xxx:
sudo lspci -vvd:4940 | grep "Kernel driver in use".
sudo lspci -vvd:4942 | grep "Kernel driver in use"
sudo lspci -vvd:4946 | grep "Kernel driver in use"
upgrade to a recent Linux Kernel.
(2) No VFs available. Check VFs are available and bound to vfio-pci
sudo lspci -vvd:4941 | grep "Kernel driver in use"
sudo lspci -vvd:4943 | grep "Kernel driver in use"
sudo lspci -vvd:4947 | grep "Kernel driver in use"

Issue: On running ./autogen.sh following warning appears:
aclocal: warning: couldn't open directory 'm4': No such file or dir...
Expand Down Expand Up @@ -786,6 +801,14 @@ Common issues
extra 's'. Sorry if it causes confusion.
Fix: sudo make samples-uninstall

Issue: on Ubuntu on make install, this error may be seen:
“Failed to start qat.service: Unit qat.service not found.”
Likely cause: The qat.service file is not installed by Ubuntu in the
expected directory
Fix: Explicitly configure the systemd path to where Ubuntu has installed
the qat.service, e.g.
./configure --enable-service systemdsystemunitdir=/lib/systemd/system/

===============================================================================


Expand Down
5 changes: 5 additions & 0 deletions Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -384,6 +384,11 @@ ICP_DC_ERROR_SIMULATION = 1
COMMON_FLAGS += -DICP_DC_ERROR_SIMULATION
endif

if ICP_HB_ERROR_SIMULATION_AC
ICP_HB_FAIL_SIM = 1
COMMON_FLAGS += -DICP_HB_FAIL_SIM
endif


if USE_CCODE_CRC
COMMON_FLAGS += -DUSE_CCODE_CRC
Expand Down
53 changes: 41 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,13 @@

| Date | Doc Revision | Version | Details |
|----------|:-------------:|------:|:------|
| February 2024 | 012 | 24.02 | - Added Heartbeat support. <br> - Added support for QAT GEN 5 devices, including support for a range of crypto wireless algorithms. <br> - RAS - Device error reset and recovery handling. <br> - Bug Fixes. See [Resolved Issues](#resolved-issues). |
| November 2023 | 011 | 23.11 | - Support DC NS (NoSession) APIs. <br> - Support DC compressBound APIs. <br> - Support Symmetric Crypto SM3 & SM4. <br> - Support Asymmetric Crypto SM2. <br> - Bug Fixes. See [Resolved Issues](#resolved-issues). |
| August 2023 | 010 | 23.08 | - Removal of following insecure algorithms: Diffie-Hellman and Elliptic curves less than 256-bits. <br> - Additional configuration profiles, including sym which facilitates improved symmetric crypto performance. <br> - DC Chaining (Hash then compress) <br> - Bug Fixes. See [Resolved Issues](#resolved-issues). |
| August 2023 | 010 | 23.08 | - Removal of following insecure algorithms: Diffie-Hellman and Elliptic curves less than 256-bits. <br> - Additional configuration profiles, including sym which facilitates improved symmetric crypto performance. <br> - DC Chaining (Hash then compress) <br> - Bug Fixes. See [Resolved Issues](#resolved-issues). <br> - The shared object version is changed from 3->4. |
| February 2023 | 009 | 23.02 | - Added configuration option --enable-legacy-algorithms to use these insecure crypto algorithms and disabled them by default (AES-ECB, SHA-1, SHA2-224, SHA3-224, RSA512/1024/1536, DSA)<br>- Refactored code in quickassist/utilities/libusdm_drv<br>- Bugfixes<br>- Updated documentation with configuration and tuning information |
| November 2022 | 008 | 22.07.2 | - Changed from yasm to nasm for assembly compilation<br> - Added configuration option to use C implementation of soft CRC implementation instead of asm<br>- Added support for pkg-config<br>- Added missing lock around accesses to some global data in qatmgr |
| October 2022 | 007 | 22.07.1 | - Fix for QATE-86605 |
| July 2022 | 006 | 22.07 | - Added support for lz4/lz4s compression algorithms<br>- Added support for Compression End-to-end (E2E) integrity check<br>- Added support for PKE generic point multiply<br>- Updated QAT APIs<br>- Enabled CPM2.0b<br>- Split rpm package |
| July 2022 | 006 | 22.07 | - Added support for lz4/lz4s compression algorithms<br>- Added support for Compression End-to-end (E2E) integrity check<br>- Added support for PKE generic point multiply<br>- Updated QAT APIs (as a result the shared object version changed from 2->3). <br>- Enabled CPM2.0b<br>- Split rpm package |
| November 2021 | 005 | 21.11 | - Added qatlib-tests rpm package<br>- Added option to configure script to skip building sample code |
| August 2021 | 004 | 21.08 | - Added support for deflate compression - Compress and Verify (CnV) and Compress and Verify and Recover (CnVnR)<br>- Added Physical Function to Virtual Function (PFVF) communication support |
| May 2021 | 003 | 21.05 | - Added support for AES-CCM 192/265<br>- Added support for SHA3-224/384/512 (no partials support)<br>- Added support for ChaCha20-Poly1305<br>- Added support for PKE 8K (RSA, DH, ModExp, ModInv)<br>- Fixed device enumeration on different nodes<br>- Fixed pci_vfio_set_command for 32 bit builds |
Expand Down Expand Up @@ -61,7 +62,7 @@ The following services are available in qatlib via the QuickAssist API:
SM4-CBC, SM4-CTR)
* Message digest/hash ([SHA1](#insecure-algorithms), SHA2 ([224](#insecure-algorithms)/256/384/512),
SHA3 ([224](#insecure-algorithms)/256/384/512) (no partials support), SM3) and
authentication (AES-CBC-MAC, AES-XCBC-MAC)
authentication (AES-CBC-MAC, AES-XCBC-MAC, AES-CMAC-128)
* Algorithm chaining (one cipher and one hash in a single operation)
* Authenticated encryption (CCM-128 (no partials support),
GCM (128/192/256) (no partials support), GMAC (no partials support)
Expand Down Expand Up @@ -90,6 +91,9 @@ The following services are available in qatlib via the QuickAssist API:
* DC NS (No Session) APIs
* Compression Chaining (Deflate only)
* Hash then compress
* Wireless Algorithms (supported on QAT GEN 5 devices)
* Ciphers (SNOW3G-UEA2, ZUC-128, ZUC-256, AES-F8)
* Message digest/hash (SNOW3G-UIA2, ZUC-128, ZUC-256) and authentication (AES-CMAC-128, AES-CMAC-192, AES-CMAC-256)

This package includes:
* libqat: user space library for QAT devices exposed via the vfio kernel driver
Expand Down Expand Up @@ -120,7 +124,8 @@ To enable these algorithms, use the following configuration option:
Please refer to [INSTALL](INSTALL) for details on installing the library.

## Supported Devices
* 4xxx (QAT gen 4 devices)
* 4xxx (QAT GEN 4 devices)
* 420xx (QAT GEN 5 devices)

Earlier generations of QAT devices (e.g. c62x, dh895xxcc, etc.) are not
supported.
Expand Down Expand Up @@ -187,7 +192,7 @@ where: \<Component\> is one of the following:
| QATE-41707 | [CY - Incorrect digest returned when performing a plain hash operation on input data of size 4GB or larger.](#qate-41707) |
| QATE-76073 | [GEN - If PF device configuration is modified without restarting qatmgr, undefined behavior may occur.](#qate-76073) |
| QATE-76698 | [GEN - Multi-process applications running in guest will fail when running with default Policy settings.](#qate-76698) |
| QATE-94369 | [GEN - SELinux Preventing QAT Service Startup.](#qate-94369) |
| QATE-98551 | [GEN - On a multi-socket platform, there can be a performance degradation on the remote sockets.](#qate-98551) |

## QATE-3241
| Title | CY - cpaCySymPerformOp when used with parameter checking may reveal the amount of padding. |
Expand Down Expand Up @@ -230,22 +235,24 @@ where: \<Component\> is one of the following:
| Affected OS | Linux |
| Driver/Module | CPM-IA - General |

## QATE-94369
| Title | GEN - SELinux Preventing QAT Service Startup |
## QATE-98551
| Title | GEN - On a multi-socket platform, there can be a performance degradation on the remote sockets. |
|----------|:-------------
| Reference # | QATE-94369 |
| Description | The qat service fails to start due to SELinux preventing the qat_init.sh script and qatmgr from accessing resources. The issue occurs when the system is running with SELinux enabled, causing insufficient permissions for the qat_init.sh script and qatmgr to function correctly. |
| Implication | This issue affects the proper functioning of the qat service on systems with SELinux enabled, potentially preventing QAT virtual functions (VFs) from functioning. |
| Resolution | None available. |
| Reference # | QATE-98551 |
| Description | On a multi-socket platform, there can be a performance degradation on remote sockets. This can arise when either the threads are not affinitised to the core on the socket the device is on and/or the memory is not allocated on the appropriate NUMA node. |
| Implication | Performance on socket 0 is as expected, but does not scale proportionally on remote sockets. |
| Resolution | This will be fixed in a future release. In the meantime, applications on a multi-socket platform should configure threads using a QAT VF device on a remote socket to be affinitised to the core on that remote socket. Then the memory allocations are more likely to be done on the remote socket, with minimal performance impact. |
| Affected OS | Linux |
| Driver/Module | QAT Linux Upstream - User |
| Driver/Module | CPM-IA - General |

## Resolved Issues
Resolved issues relating to the Intel® QAT software are described
in this section.

| Issue ID | Description |
|-------------|------------|
| QATE-97977 | [DC - 'Unable to get the physical address of Data Integrity buffer' error may be observed when using user-provided address translation functions.](#qate-97977) |
| QATE-94369 | [GEN - SELinux Preventing QAT Service Startup.](#qate-94369) |
| QATE-94286 | [GEN - Compression services not detected when crypto-capable VFs are added to VM.](#qate-94286) |
| QATE-95905 | [GEN - Fix build when building outside of main directory, issue #56](#qate-95905) |
| QATE-93844 | [DC - cpaDcLZ4SCompressBound is not returning correct value, which could lead to a buffer overflow.](#qate-93844)
Expand All @@ -255,6 +262,28 @@ in this section.
| QATE-76846 | [GEN - Forking and re-initializing use-cases do not work](#qate-76846) |
| QATE-12241 | [CY - TLS1.2 with secret key lengths greater than 64 are not supported.](#qate-12241) |

## QATE-97977
| Title | DC - 'Unable to get the physical address of Data Integrity buffer' error may be observed when using user-provided address translation functions. |
|----------|:-------------
| Reference # | QATE-97977 |
| Description | When using Integrity CRC feature (integrityCrcCheck in CpaDcOpData) and also user provided address translation functions (cpaDcSetAddressTranslation) the above error may be observed. |
| Implication | Compression request operations may fail in this scenario. |
| Resolution | Fixed in 24.02 |
| Affected OS | Linux |
| Driver/Module | CPM-IA - Data Compression |



## QATE-94369
| Title | GEN - SELinux Preventing QAT Service Startup |
|----------|:-------------
| Reference # | QATE-94369 |
| Description | The qat service fails to start due to SELinux preventing the qat_init.sh script and qatmgr from accessing resources. The issue occurs when the system is running with SELinux enabled, causing insufficient permissions for the qat_init.sh script and qatmgr to function correctly. |
| Implication | This issue affects the proper functioning of the qat service on systems with SELinux enabled, potentially preventing QAT virtual functions (VFs) from functioning. |
| Resolution | The fix is not in the scope of qatlib. Instead there are three possible methods to handle this issue: <br> 1) Update selinux-policy as seen in https://github.com/fedora-selinux/selinux-policy/pull/1992 <br>2) Disable SElinux <br>3) Update mode to SElinux mode to permissive using following commands: <br>semanage permissive -a qatlib_exec_t<br>semanage permissive -a qatlib_t<br>The audit warnings may be generated, but qatlib will be allowed access to vfio devices. |
| Affected OS | Linux |
| Driver/Module | CPM-IA - General |

## QATE-94286
| Title | GEN - Compression services not detected when crypto-capable VFs are also added to VM. |
|----------|:-------------
Expand Down
15 changes: 14 additions & 1 deletion Samples.am
Original file line number Diff line number Diff line change
Expand Up @@ -358,11 +358,22 @@ ec_montedwds_sample_CFLAGS = -I$(srcdir)/quickassist/lookaside/access_layer/src/
$(COMMON_FLAGS)
ec_montedwds_sample_LDADD = $(COMMON_SAMPLE_LDFLAGS) libcpa_sample_code_s.la

noinst_PROGRAMS += zuc_sample
zuc_sample_SOURCES = \
quickassist/lookaside/access_layer/src/sample_code/functional/common/cpa_sample_utils.c \
quickassist/lookaside/access_layer/src/sample_code/functional/sym/zuc_sample/cpa_zuc_sample.c \
quickassist/lookaside/access_layer/src/sample_code/functional/sym/zuc_sample/cpa_zuc_sample_user.c
zuc_sample_CFLAGS = -I$(srcdir)/quickassist/lookaside/access_layer/src/sample_code/functional/include \
$(COMMON_SAMPLE_INCLUDES) \
$(COMMON_SAMPLE_CFLAGS) \
$(COMMON_FLAGS)
zuc_sample_LDADD = $(COMMON_SAMPLE_LDFLAGS) libcpa_sample_code_s.la

samples: $(lib_LTLIBRARIES) cpa_sample_code dc_dp_sample dc_stateless_sample \
dc_stateless_multi_op_sample algchaining_sample ccm_sample \
cipher_sample gcm_sample hash_file_sample hash_sample ipsec_sample \
ssl_sample sym_dp_sample dh_sample prime_sample hkdf_sample \
ec_montedwds_sample eddsa_sample chaining_sample
ec_montedwds_sample eddsa_sample chaining_sample zuc_sample

samples-install: samples
@install -D -m 755 $(srcdir)/.libs/cpa_sample_code $(DESTDIR)$(bindir)/cpa_sample_code
Expand All @@ -384,6 +395,7 @@ samples-install: samples
@install -D -m 755 $(srcdir)/.libs/prime_sample $(DESTDIR)$(bindir)/prime_sample
@install -D -m 755 $(srcdir)/.libs/hkdf_sample $(DESTDIR)$(bindir)/hkdf_sample
@install -D -m 755 $(srcdir)/.libs/ec_montedwds_sample $(DESTDIR)$(bindir)/ec_montedwds_sample
@install -D -m 755 $(srcdir)/.libs/zuc_sample $(DESTDIR)$(bindir)/zuc_sample
@install -D -m 644 $(srcdir)/quickassist/lookaside/access_layer/src/sample_code/performance/compression/calgary $(DESTDIR)$(datadir)/qat/calgary
@install -D -m 644 $(srcdir)/quickassist/lookaside/access_layer/src/sample_code/performance/compression/calgary32 $(DESTDIR)$(datadir)/qat/calgary32
@install -D -m 644 $(srcdir)/quickassist/lookaside/access_layer/src/sample_code/performance/compression/canterbury $(DESTDIR)$(datadir)/qat/canterbury
Expand Down Expand Up @@ -413,6 +425,7 @@ samples-uninstall:
@rm -rf $(DESTDIR)$(bindir)/prime_sample
@rm -rf $(DESTDIR)$(bindir)/hkdf_sample
@rm -rf $(DESTDIR)$(bindir)/ec_montedwds_sample
@rm -rf $(DESTDIR)$(bindir)/zuc_sample
@rm -rf $(DESTDIR)$(datadir)/qat/calgary
@rm -rf $(DESTDIR)$(datadir)/qat/calgary32
@rm -rf $(DESTDIR)$(datadir)/qat/canterbury
Expand Down
11 changes: 9 additions & 2 deletions configure.ac
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# Process this file with autoconf to produce a configure script.

AC_PREREQ([2.69])
AC_INIT([qatlib], [23.11.0], [[email protected]])
AC_INIT([qatlib], [24.02.0], [[email protected]])
AM_INIT_AUTOMAKE([-Wall -Werror -Wno-portability foreign subdir-objects tar-pax])

AM_SILENT_RULES([yes])
Expand All @@ -11,7 +11,7 @@ AC_USE_SYSTEM_EXTENSIONS
AC_CONFIG_MACRO_DIRS([m4])
AC_CONFIG_HEADERS([config.h])

AC_SUBST([LIBQAT_VERSION], [5:0:1])
AC_SUBST([LIBQAT_VERSION], [6:0:2])
AC_SUBST([LIBUSDM_VERSION], [1:0:1])

# Checks for programs.
Expand Down Expand Up @@ -220,6 +220,13 @@ AC_ARG_ENABLE(dc-error-simulation,
)
AM_CONDITIONAL([ICP_DC_ERROR_SIMULATION_AC], [test x$dc_error_simulation = xtrue])

# ICP_HB_ERROR_SIMULATION
AC_ARG_ENABLE(hb-error-simulation,
AS_HELP_STRING([--enable-hb-error-simulation], [Enables Heartbeat Error Simulation.]),
[hb_error_simulation=true], [hb_error_simulation=false]
)
AM_CONDITIONAL([ICP_HB_ERROR_SIMULATION_AC], [test x$hb_error_simulation = xtrue])

AC_ARG_ENABLE(legacy-lib-names,
AS_HELP_STRING([--enable-legacy-lib-names], [Enables legacy names for libraries.]),
[
Expand Down
Loading

0 comments on commit ec81762

Please sign in to comment.