9.6. PCI passthrough

Oracle VM VirtualBox

9.6. PCI passthrough

When running on Linux hosts, with a recent enough kernel (at least version 2.6.31) experimental host PCI devices passthrough is available.[42]

Note

The PCI passthrough module is shipped as a VirtualBox extension package, which must be installed separately. See Section 1.5, “Installing VirtualBox and extension packs” for more information.

Essentially this feature allows to directly use physical PCI devices on the host by the guest even if host doesn't have drivers for this particular device. Both, regular PCI and some PCI Express cards, are supported. AGP and certain PCI Express cards are not supported at the moment if they rely on GART (Graphics Address Remapping Table) unit programming for texture management as it does rather non-trivial operations with pages remapping interfering with IOMMU. This limitation may be lifted in future releases.

To be fully functional, PCI passthrough support in VirtualBox depends upon an IOMMU hardware unit which is not yet too widely available. If the device uses bus mastering (i.e. it performs DMA to the OS memory on its own), then an IOMMU is required, otherwise such DMA transactions may write to the wrong physical memory address as the device DMA engine is programmed using a device-specific protocol to perform memory transactions. The IOMMU functions as translation unit mapping physical memory access requests from the device using knowledge of the guest physical address to host physical addresses translation rules.

Intel's solution for IOMMU is marketed as "Intel Virtualization Technology for Directed I/O" (VT-d), and AMD's one is called AMD-Vi. So please check if your motherboard datasheet has appropriate technology. Even if your hardware doesn't have a IOMMU, certain PCI cards may work (such as serial PCI adapters), but the guest will show a warning on boot and the VM execution will terminate if the guest driver will attempt to enable card bus mastering.

It is very common that the BIOS or the host OS disables the IOMMU by default. So before any attempt to use it please make sure that

  1. Your motherboard has an IOMMU unit.

  2. Your CPU supports the IOMMU.

  3. The IOMMU is enabled in the BIOS.

  4. The VM must run with VT-x/AMD-V and nested paging enabled.

  5. Your Linux kernel was compiled with IOMMU support (including DMA remapping, see CONFIG_DMAR kernel compilation option). The PCI stub driver (CONFIG_PCI_STUB) is required as well.

  6. Your Linux kernel recognizes and uses the IOMMU unit (intel_iommu=on boot option could be needed). Search for DMAR and PCI-DMA in kernel boot log.

Once you made sure that the host kernel supports the IOMMU, the next step is to select the PCI card and attach it to the guest. To figure out the list of available PCI devices, use the lspci command. The output will look like this:

01:00.0 VGA compatible controller: ATI Technologies Inc Cedar PRO [Radeon HD 5450]
01:00.1 Audio device: ATI Technologies Inc Manhattan HDMI Audio [Mobility Radeon HD 5000 Series]
02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit
        Ethernet controller (rev 03)
03:00.0 SATA controller: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (rev 03)
03:00.1 IDE interface: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (rev 03)
06:00.0 VGA compatible controller: nVidia Corporation G86 [GeForce 8500 GT] (rev a1)

The first column is a PCI address (in format bus:device.function). This address could be used to identify the device for further operations. For example, to attach a PCI network controller on the system listed above to the second PCI bus in the guest, as device 5, function 0, use the following command:

VBoxManage modifyvm "VM name" --pciattach 02:00.0@01:05.0

To detach same device, use

VBoxManage modifyvm "VM name" --pcidetach 02:00.0

Please note that both host and guest could freely assign a different PCI address to the card attached during runtime, so those addresses only apply to the address of the card at the moment of attachment (host), and during BIOS PCI init (guest).

If the virtual machine has a PCI device attached, certain limitations apply:

  1. Only PCI cards with non-shared interrupts (such as using MSI on host) are supported at the moment.

  2. No guest state can be reliably saved/restored (as the internal state of the PCI card could not be retrieved).

  3. Teleportation (live migration) doesn't work (for the same reason).

  4. No lazy physical memory allocation. The host will preallocate the whole RAM required for the VM on startup (as we cannot catch physical hardware accesses to the physical memory).


[42] Experimental support for PCI passthrough was introduced with VirtualBox 4.1.