Differences

This shows you the differences between two versions of the page.

--- ubuntu:gpu:gpu_pass-through:configuring_the_host [2023/06/25 11:32] – created peter
+++ ubuntu:gpu:gpu_pass-through:configuring_the_host [2023/06/25 11:32] (current) – peter
@@ Line 1: / Line 1: @@
 ====== Ubuntu - GPU - GPU pass-through - Configuring the host ======
+===== Verify the host environment supports GPU pass-through =====
+TODO:
+----
+===== Enable IOMMU =====
+**IOMMU** is usually disabled by default.
+  * It needs to be enable at boot time in the **/etc/default/grub** configuration file.
+For AMD-based hosts:
+<file bash /etc/default/grub>
+GRUB_CMDLINE_LINUX="iommu=pt amd_iommu=on rd.driver.pre=vfio-pci"
+</file>
+For Intel-based hosts:
+<file bash /etc/default/grub>
+GRUB_CMDLINE_LINUX="intel_iommu=on iommu=pt rd.driver.pre=vfio-pci"
+</file>
+----
+===== Re-generate the main GRUB 2 configuration file =====
+The GRUB 2 configuration file is **/boot/grub2/grub.cfg**.
+<code bash>
+sudo grub2-mkconfig -o /boot/grub2/grub.cfg
+</code>
+----
+===== Reboot =====
+----
+===== Verify that IOMMU is enabled =====
+<code bash>
+sudo dmesg | grep -e DMAR -e IOMMU
+</code>
+returns:
+<code bash>
+[    2.172228] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
+[    2.176588] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
+[    2.176924] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
+[    3.875165] AMD-Vi: AMD IOMMUv2 loaded and initialized
+</code>
+----
+===== Blacklist the Nouveau driver =====
+Prevent the host OS from loading the built-in nouveau driver.
+Create the file **/etc/modprobe.d/60-blacklist-nouveau.conf**:
+<file bash /etc/modprobe.d/60-blacklist-nouveau.conf>
+blacklist nouveau
+</file>
+----
+===== Configure VFIO and isolate the GPU used for pass-through =====
+Find the card vendor and model IDs.
+<code bash>
+lspci -k | grep VGA
+</code>
+returns:
+<code bash>
+c:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 31 [Radeon RX 7900 XT/7900 XTX] (rev c8)
+</code>
+<WRAP info>
+**NOTE:**  This shows the Bus number is 0c:00.0.
+</WRAP>
+----
+===== Get the device ID =====
+Using the Bus number identified earlier.
+<code bash>
+lspci -nn | grep 0c:00.0
+</code>
+returns:
+<code bash>
+c:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 31 [Radeon RX 7900 XT/7900 XTX] [1002:744c] (rev c8)
+</code>
+<WRAP info>
+**NOTE:**  This shows the ID is 1002:744c.
+</WRAP>
+----
+===== Create a module VFIO file =====
+Create the file **/etc/modprobe.d/vfio.conf**:
+<file bash /etc/modprobe.d/vfio.conf>
+options vfio-pci ids=1002:744c
+</file>
+<WRAP info>
+**NOTE:**  Verify that the card does not need an extra **ids=** parameter.
+  * For some cards, the audio device must be specified too, so that ID must also be added to the list, otherwise the card will be unusable.
+</WRAP>
+----
+===== Load the VFIO driver =====
+There are various ways you can load the VFIO driver:
+==== Including the driver in the initrd file ====
+Create the file **/etc/dracut.conf.d/gpu-passthrough.conf**:
+<file>
+    add_drivers+=" vfio vfio_iommu_type1 vfio_pci vfio_virqfd"
+</file>
+<WRAP info>
+**NOTE:**  Mind the leading whitespace.
+</WRAP>
+Re-generate the initrd file:
+<code bash>
+sudo dracut --force /boot/initrd $(uname -r)
+</code>
+----
+==== Adding the driver to the list of auto-loaded modules ====
+Create the file **/etc/modules-load.d/vfio-pci.conf**:
+<file bash /etc/modules-load.d/vfio-pci.conf>
+vfio
+vfio_iommu_type1
+vfio_pci
+kvm
+kvm_intel
+</file>
+----
+==== Load the driver manually ====
+To load the driver manually at runtime:
+<code bash>
+sudo modprobe vfio-pci
+</code>
+----
+===== Disable MSR for Microsoft Windows guests =====
+For Microsoft Windows guests, it is recommended to disable MSR (model-specific register) to avoid the guest crashing.
+Create the file **/etc/modprobe.d/kvm.conf**:
+<file bash /etc/modprobe.d/kvm.conf>
+options kvm ignore_msrs=1
+</file>
+----
+===== Install UEFI firmware =====
+For proper GPU Pass-Through functionality, the host needs to boot using UEFI firmware.
+  * Not using a legacy-style BIOS boot sequence.
+Install the **qemu-ovmf** package if not already installed:
+<code bash>
+sudo apt install qemu-ovmf
+</code>
+----
+===== Reboot the host machine =====
+For most of the changes in the above steps to take effect, the host machine needs to be rebooted.
+<code bash>
+sudo shutdown -r now
+</code>
+----