PVE Enabling IOMMU for Hardware Passthrough and Troubleshooting Passthrough Errors

,

I. Introduction  

What is Hardware Passthrough?  

VT-d, DirectPath I/O, allows a virtual machine to access physical PCI functions on the platform through the I/O Memory Management Unit, commonly known as virtualization passthrough. Simply put, it allows the host to hand over the control of certain hardware resources directly to the virtual machine. The virtual machine can use the hardware exclusively through passthrough, and the host can no longer use that hardware. The utilization efficiency is almost equivalent to physically connecting the hardware to the virtual machine’s motherboard expansion slot. The main practical purpose is to avoid the performance degradation caused by the virtualization platform’s software layer conversion.

Typical use cases include assigning a physical network card directly to a virtual machine to achieve network performance similar to a physical machine with a physical network card. A more noticeable scenario is passing through a disk controller for exclusive use by a virtual machine, significantly improving the disk performance, which is often a bottleneck for virtual machines.

When adding a PCI device in the PVE system for hardware passthrough, you may see the following prompt:  

No IOMMU detected, please activate it. See Documentation for further information.

This is because, by default, Proxmox VE only supports the passthrough of hard drives and CPU models. For other PCI hardware, such as network cards or integrated graphics cards, you need to enable the IOMMU grouping feature.

Before enabling IOMMU in the Proxmox VE system to achieve hardware passthrough, ensure that your CPU supports VT-D technology.

Requirements for enabling passthrough:  

The CPU must support VT-D, and the motherboard must have VT-D support enabled.

II. Checking if the CPU Supports VT-D  

1. Visit the Intel official website or the AMD official website, and search for the corresponding processor model (e.g., i7-7700) to check if the CPU supports VT-D technology.

Enabling IOMMU  

Intel CPU  

For Intel CPUs, add `intel_iommu=on` as follows:

1. In the Shell, enter the command:  

   nano /etc/default/grub

   Example:  

   root@pve:~# nano /etc/default/grub  

2. Find the line:  

   GRUB_CMDLINE_LINUX_DEFAULT=”quiet”  

   Modify it to:  

   GRUB_CMDLINE_LINUX_DEFAULT=”quiet intel_iommu=on”  

3. After editing, use the shortcut Ctrl + O, press Enter to save the file, and use Ctrl + X to exit the editor.

4. Run the following command to save changes and update GRUB:  

   root@pve:~# update-grub

5. After the update, restart the PVE system:  

   root@pve:~# reboot

6. After rebooting, run the following command to verify:  

   dmesg | grep -e DMAR -e IOMMU

If there is no output, there is a problem. If you see output, it means the process was successful, and you can proceed to add hardware passthrough for the virtual machine.

AMD CPU  

For AMD CPUs, add `amd_iommu=on` as follows:

1. In the Shell, enter the command:  

   nano /etc/default/grub

   Example:  

   root@pve:~# nano /etc/default/grub  

2. Find the line:  

   GRUB_CMDLINE_LINUX_DEFAULT=”quiet”  

   Modify it to:   

   GRUB_CMDLINE_LINUX_DEFAULT=”quiet amd_iommu=on”  

3. After editing, use the shortcut Ctrl + O, press Enter to save the file, and use Ctrl + X to exit the editor.

4. Run the following command to save changes and update GRUB:  

   root@pve:~# update-grub

5. After the update, restart the PVE system:  

   root@pve:~# reboot

6. After rebooting, run the following command to verify:  

   dmesg | grep -e DMAR -e IOMMU

If there is no output, there is a problem. If you see output, it means the process was successful, and you can proceed to add hardware passthrough for the virtual machine.

Adding Virtualization Drivers and Loading the VFIO System Module  

This step is only necessary if you want to enable IOMMU translation. Enable the relevant kernel modules to improve the performance of unused PCIe devices in the virtual machine.

Modify the /etc/modules file:  

root@pve:~# nano /etc/modules

Add the following lines:   

vfio  

vfio_iommu_type1  

vfio_pci  

vfio_virqfd  

If the passthrough configuration is still invalid after following this method, double-check that your CPU supports VT-D technology.

Note:  

When performing passthrough for a virtual machine, disable the Autostart option. If a passthrough error occurs, simply restart the physical machine, and the virtual machine will not autostart, avoiding conflicts that could prevent the host from booting.

III. Network Card Passthrough Error and Solution for Loss of Connection Without Reinstalling PVE  

Tip: On a device with four network ports, only three PCI devices can be passed through to a virtual machine. The management port defaults to the one with the lowest value.

Solution:  

1. Shut down the virtual machine to prevent it from autostarting after the PVE system restarts.

2. Reboot the PVE system. Once successfully rebooted, access the web interface and remove the network card passthrough from the virtual machine.

Steps:  

– Connect a monitor and keyboard to the PVE server.  

– Log in as the administrator.

1. Check the virtual machine:  

   qm list

2. Stop the virtual machine:  

   qm stop 105

3. Disable the virtual machine autostart:  

   qm set 105 -onboot 0

4. Reboot the server:  

   reboot

After the reboot, use your browser to access the PVE management interface and remove the network card from the virtual machine’s passthrough configuration.

This completes the process of enabling IOMMU and addressing hardware passthrough issues in Proxmox VE.


Leave a Reply

Your email address will not be published. Required fields are marked *