Intel® QuickAssist Technology SRIOV PF Driver Production Package for VMware ESXi 7.0.0 - RELEASE NOTES =============================================================================== This software enables SRIOV for the Intel® QuickAssist Technology (QAT) on VMware ESXi 7.0.0. SRIOV enables the creation of Virtual Functions from a single Intel® QAT device to support acceleration for multiple virtual machines. This document describes how to install Intel® QAT Physical Function (PF) driver on VMware ESXi hypervisor and how to attach Virtual Functions (VFs) to guests. For more details on how to use Intel® QAT from a Virtual Machine, please refer to the documents mentioned in the Documentation section. Note that this README assumes that the reader is familiar with virtualization technologies and has some level of familiarity with the VMware ESXi hypervisor. This README does not explain how to install VMware ESXi, how to install a virtual machine and how to administrate ESXi using VSphere Client. For more details on these topics please refer to VMware's documentation. License ======= Refer to license.txt in this package for Intel® software license agreement before using this software. Details/Limitations of this Release =================================== * This software release is intended for platforms that contain: * Intel® C62x Chipset * Intel® QAT Adapter 8960/8970 (formerly known as “Lewis Hill”) * This package contains a build of Intel® QAT driver compatible with ESXi 7.0.0. * This release is validated with Intel® QAT VF driver: qat1.7.L.4.9.0-00008 and is expected to work with newer versions. * ESXi limitation: the maximum number of Virtual Functions that can be mapped to a single VM is 16. ESXi will not allow a VM to start with more than 16 VFs. Documentation ============= Associated software and collateral can be found on the open source website: https://01.org/intel-quickassist-technology Additional documentation includes: * Using Intel® Virtualization Technology (Intel® VT) with Intel® QuickAssist Technology Application Note * Intel® QuickAssist Technology Software Release Notes * Intel® QuickAssist Technology Software for Linux Getting Started Guide * Intel® QuickAssist Technology Software for Linux Software Programmer's Guide * Intel® QuickAssist Technology API Programmer's Guide * Intel® QuickAssist Technology Cryptographic API Reference Manual * Intel® QuickAssist Technology Data Compression API Reference Manual Installing Intel® QAT VMware Driver ================================ 1) Open an SSH connection to the target ESXi host (ESXi shell and SSH needs to be enabled to perform this operation) 2) Extract the package (assuming it has been copied to /tmp folder) > cd /tmp > tar -xzf QAT17_SRIOV.VMW.700.1.1.0-*.tar.gz 3) Set host acceptance level to "PartnerSupported": > esxcli software acceptance set --level PartnerSupported 4) There are 2 ways to install the driver: a. From component via the following command (recommended approach): > esxcli software component apply -d \ /tmp/quickassist/DRIVER/ESXi7.0/icp-qat-pf-drv_1.1.0.7-15843807_16673147.zip b. From the VIB package extracted from component: > unzip /tmp/quickassist/DRIVER/ESXi7.0/icp-qat-pf-drv_1.1.0.7-15843807_16673147.zip > esxcli software vib install --maintenance-mode -v \ tmp/vib20/icp-qat-pf/INT_bootbank_icp-qat-pf_1.1.0.7-1OEM.700.1.0.15843807.vib 5) Reboot the system > reboot 6) If the QAT driver has been loaded without errors you should see the icp_qat_pf module showing in the list of system modules: > esxcfg-module -l | grep icp_qat_pf icp_qat_pf 3 1972 7) Now the QAT VFs are enabled in the system. You can verify this by running the lspci command. For example: > lspci -vn | grep 37c9 0000:85:01.0 Class 0b40: 8086:37c9 [PF_0.133.0_VF_0] 0000:85:01.1 Class 0b40: 8086:37c9 [PF_0.133.0_VF_1] 0000:85:01.2 Class 0b40: 8086:37c9 [PF_0.133.0_VF_2] 0000:85:01.3 Class 0b40: 8086:37c9 [PF_0.133.0_VF_3] 0000:85:01.4 Class 0b40: 8086:37c9 [PF_0.133.0_VF_4] 0000:85:01.5 Class 0b40: 8086:37c9 [PF_0.133.0_VF_5] 0000:85:01.6 Class 0b40: 8086:37c9 [PF_0.133.0_VF_6] 0000:85:01.7 Class 0b40: 8086:37c9 [PF_0.133.0_VF_7] 0000:85:02.0 Class 0b40: 8086:37c9 [PF_0.133.0_VF_8] 0000:85:02.1 Class 0b40: 8086:37c9 [PF_0.133.0_VF_9] 0000:85:02.2 Class 0b40: 8086:37c9 [PF_0.133.0_VF_10] 0000:85:02.3 Class 0b40: 8086:37c9 [PF_0.133.0_VF_11] 0000:85:02.4 Class 0b40: 8086:37c9 [PF_0.133.0_VF_12] 0000:85:02.5 Class 0b40: 8086:37c9 [PF_0.133.0_VF_13] 0000:85:02.6 Class 0b40: 8086:37c9 [PF_0.133.0_VF_14] 0000:85:02.7 Class 0b40: 8086:37c9 [PF_0.133.0_VF_15] At this point the QAT VFs can be attached to a guest Virtual Machine (see section Pass-through the PCI Device in this README). Pass-through the PCI Device =========================== 1) Connect to the target ESXi host via Web UI. 2) In the left pane, click on Virtual Machines. 4) In the center pane, click on the desired Virtual Machine. Note: Ensure that the VM is powered off. 5) Click on the “Edit” button to edit the virtual machines settings. A pop-up window with the VM settings will appear. 6) Click on “Add other Device” and select “PCI device”. The new PCI device will be added. By default it selects the first QAT VF if there is no other passthru-capable devices. To select specific VF, click the drop-down list. The BDFs listed here will match with the output of the “lspci -vn | grep 37c9” command. Additional VFs can be added by repeating this step. 7) Click "Save". Now you have one or more VFs attached to your guest. Installing Intel® QuickAssist Technology Software on the Guest ================================================================ For instructions on how to install the VF driver on a Linux guest please refer to 'Using Intel® Virtualization Technology (Intel® VT) with Intel® QuickAssist Technology Application Note'. Uninstalling the QAT VMware Driver ================================== 1) Open an SSH connection to the target ESXi host 2) Actual process depends on the way how the driver was installed on system: a. If it was installed as component (recommended) - run next command to remove driver component: > esxcli software component remove -n icp-qat-pf-drv b. If it was installed as VIB package – run next command to remove VIB: > esxcli software vib remove -n icp-qat-pf --maintenance-mode 3) Reboot system to complete removal > reboot Recover from fatal errors ========================= In the event of persistent device error state that cannot be recovered by software, it is recommended to manually reload the PF driver on the ESXi host or reset the host itself. The driver will reset and recover the PF device during driver reloading. Steps to reload the driver: 1) Power off all the VMs that are using QAT hardware. 2) Execute next commands to reload the PF driver and devmgr: > esxcfg-module -u icp_qat_pf > kill -HUP $(cat /var/run/vmware/vmkdevmgr.pid)