Proxmox - GPU Passthrough - Configure Proxmox

Update Grub

SSH directly into the Proxmox server.

Edit /etc/default/grub file.

Change the line:

GRUB_CMDLINE_LINUX_DEFAULT="quiet"

For an Intel CPU

/etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"

For an AMD CPU

/etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on"

NOTE: Additional commands may be needed as well in this line, if the passthrough ends up failing.

For example:

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction nofb nomodeset video=vesafb:off,efifb:off"

Disabling the Framebuffer: video=vesafb:off,efifb:off.
ACS Override for IOMMU groups: pcie_acs_override=downstream,multifunction.

When editing is finished run this command:

update-grub

VFIO Modules

Add a few VFIO (Virtual Function I/O) modules to your Proxmox system.

Add the following to the /etc/modules file:

/etc/modules

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

Save and exit.

IOMMU interrupt remapping

Run the following commands:

echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf
echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf

Blacklist Drivers

To prevent the Proxmox host system utilizing the GPU(s), some drivers need to be blacklisted.

echo "blacklist radeon" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidia" >> /etc/modprobe.d/blacklist.conf

Adding GPU to VFIO

Determine the GPU

lspci -v

returns:

01:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1070] (rev a1) (prog-if 00 [VGA controller])
01:00.1 Audio device: NVIDIA Corporation GP104 High Definition Audio Controller (rev a1)

NOTE: Look for the line(s) that show the video card.

Make note of the first set of numbers (e.g. 01:00.0 and 01:00.1).

Determine the GPU Vendor IDs

Run the command below.

NOTE: Replace 01:00 with whatever number was next to your GPU when you ran the previous command:

lspci -n -s 01:00

returns:

01:00.0 0000: 10de:1b81 (rev a1)
01:00.1 0000: 10de:10f0 (rev a1)

NOTE: This should output the GPU Vendor IDs, usually one ID for the GPU and one ID for the Audio bus.

Make a note of the vendor id codes: 10de:1b81 and 10de:10f0.

Add the GPU Vendor IDs to the VFIO

echo "options vfio-pci ids=10de:1b81,10de:10f0 disable_vga=1"> /etc/modprobe.d/vfio.conf

Update images

update-initramfs -u

NOTE: initramfs is a tiny version of the OS that gets loaded by the bootloader, right after the kernel.

It lives in RAM, and it provides just enough tools and instructions to tell the kernel how to set up the real filesystem, mount the HDD read/write, and start loading all the system services.
It includes the stub of init, PID #1.
If your initramfs is broken, your boot fails.

update-initramfs is a script that updates initramfs to work with a new kernel.

Updating the initramfs prevents some modules from being loaded.

Restart

sudo reboot

NOTE: The Proxmox host should be ready to passthrough GPUs!

Wiki

Table of Contents