Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Ubuntu - GPU - GPU pass-through - Configuring the host ======

===== Verify the host environment supports GPU pass-through =====

TODO:


----

===== Enable IOMMU =====

**IOMMU** is usually disabled by default.

  * It needs to be enable at boot time in the **/etc/default/grub** configuration file.

For AMD-based hosts:

<file bash /etc/default/grub>
GRUB_CMDLINE_LINUX="iommu=pt amd_iommu=on rd.driver.pre=vfio-pci"
</file>

For Intel-based hosts:

<file bash /etc/default/grub>
GRUB_CMDLINE_LINUX="intel_iommu=on iommu=pt rd.driver.pre=vfio-pci"
</file>

----

===== Re-generate the main GRUB 2 configuration file =====

The GRUB 2 configuration file is **/boot/grub2/grub.cfg**.

<code bash>
sudo grub2-mkconfig -o /boot/grub2/grub.cfg
</code>

----

===== Reboot =====


----

===== Verify that IOMMU is enabled =====

<code bash>
sudo dmesg | grep -e DMAR -e IOMMU
</code>

returns:

<code bash>
[    2.172228] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[    2.176588] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[    2.176924] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
[    3.875165] AMD-Vi: AMD IOMMUv2 loaded and initialized
</code>

----

===== Blacklist the Nouveau driver =====
  
Prevent the host OS from loading the built-in nouveau driver.

Create the file **/etc/modprobe.d/60-blacklist-nouveau.conf**:

<file bash /etc/modprobe.d/60-blacklist-nouveau.conf>
blacklist nouveau
</file>

----

===== Configure VFIO and isolate the GPU used for pass-through =====

Find the card vendor and model IDs.

<code bash>
lspci -k | grep VGA
</code>

returns:

<code bash>
0c:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 31 [Radeon RX 7900 XT/7900 XTX] (rev c8)
</code>

<WRAP info>
**NOTE:**  This shows the Bus number is 0c:00.0.
</WRAP>


----

===== Get the device ID =====

Using the Bus number identified earlier.

<code bash>
lspci -nn | grep 0c:00.0
</code>

returns:

<code bash>
0c:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 31 [Radeon RX 7900 XT/7900 XTX] [1002:744c] (rev c8)
</code>

<WRAP info>
**NOTE:**  This shows the ID is 1002:744c.
</WRAP>

----

===== Create a module VFIO file =====

Create the file **/etc/modprobe.d/vfio.conf**:

<file bash /etc/modprobe.d/vfio.conf>
options vfio-pci ids=1002:744c
</file>

<WRAP info>
**NOTE:**  Verify that the card does not need an extra **ids=** parameter.

  * For some cards, the audio device must be specified too, so that ID must also be added to the list, otherwise the card will be unusable.
</WRAP>

----

===== Load the VFIO driver =====

There are various ways you can load the VFIO driver:

==== Including the driver in the initrd file ====

Create the file **/etc/dracut.conf.d/gpu-passthrough.conf**:

<file>
    add_drivers+=" vfio vfio_iommu_type1 vfio_pci vfio_virqfd"
</file>

<WRAP info>
**NOTE:**  Mind the leading whitespace.
</WRAP>

Re-generate the initrd file:

<code bash>
sudo dracut --force /boot/initrd $(uname -r)
</code>

----

==== Adding the driver to the list of auto-loaded modules ====

Create the file **/etc/modules-load.d/vfio-pci.conf**:

<file bash /etc/modules-load.d/vfio-pci.conf>
vfio
vfio_iommu_type1
vfio_pci
kvm
kvm_intel
</file>

----

==== Load the driver manually ====

To load the driver manually at runtime:

<code bash>
sudo modprobe vfio-pci
</code>

----

===== Disable MSR for Microsoft Windows guests =====

For Microsoft Windows guests, it is recommended to disable MSR (model-specific register) to avoid the guest crashing.

Create the file **/etc/modprobe.d/kvm.conf**:

<file bash /etc/modprobe.d/kvm.conf>
options kvm ignore_msrs=1
</file>

----

===== Install UEFI firmware =====

For proper GPU Pass-Through functionality, the host needs to boot using UEFI firmware.

  * Not using a legacy-style BIOS boot sequence.

Install the **qemu-ovmf** package if not already installed:

<code bash>
sudo apt install qemu-ovmf
</code>

----

===== Reboot the host machine =====

For most of the changes in the above steps to take effect, the host machine needs to be rebooted.

<code bash>
sudo shutdown -r now
</code>

----