jcs.org / qemu
qemu with hax to log dma reads & writes jcs.org/2018/11/12/vfio
fork atom

pci/bus: let it has higher migration priority

In the past, we prioritized IOMMU migration so that we have such a
priority order:

IOMMU > PCI Devices

When migrating a guest with both vIOMMU and a pcie-root-port, we'll
always migrate vIOMMU first, since pci buses will be seen to have the
same priority of general PCI devices.

That's problematic.

The thing is that PCI bus number information is stored in the root port,
and that is needed by vIOMMU during post_load(), e.g., to figure out
context entry for a device. If we don't have correct bus numbers for
devices, we won't be able to recover device state of the DMAR memory
regions, and things will be messed up.

So let's boost the PCIe root ports to be even with higher priority:

PCIe Root Port > IOMMU > PCI Devices

A smoke test shows that this patch fixes bug 1538953.

Also, apply this rule to all the PCI bus/bridge devices: ioh3420,
xio3130_downstream, xio3130_upstream, pcie_pci_bridge, pci-pci bridge,
i82801b11.

I noted that we set pcie_pci_bridge_dev_vmstate twice. Clean that up
together.

CC: Alex Williamson <alex.williamson@redhat.com>
CC: Marcel Apfelbaum <marcel@redhat.com>
CC: Michael S. Tsirkin <mst@redhat.com>
CC: Dr. David Alan Gilbert <dgilbert@redhat.com>
CC: Juan Quintela <quintela@redhat.com>
CC: Laurent Vivier <lvivier@redhat.com>
Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1538953
Reported-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Marcel Apfelbaum <marcel@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

authored by

Peter Xu and committed by
Michael S. Tsirkin 9d6b9db1 ed247f40

+8 -1
+1
hw/pci-bridge/gen_pcie_root_port.c
··· 101 101 102 102 static const VMStateDescription vmstate_rp_dev = { 103 103 .name = "pcie-root-port", 104 + .priority = MIG_PRI_PCI_BUS, 104 105 .version_id = 1, 105 106 .minimum_version_id = 1, 106 107 .post_load = pcie_cap_slot_post_load,
+1
hw/pci-bridge/i82801b11.c
··· 80 80 81 81 static const VMStateDescription i82801b11_bridge_dev_vmstate = { 82 82 .name = "i82801b11_bridge", 83 + .priority = MIG_PRI_PCI_BUS, 83 84 .fields = (VMStateField[]) { 84 85 VMSTATE_PCI_DEVICE(parent_obj, PCIBridge), 85 86 VMSTATE_END_OF_LIST()
+1
hw/pci-bridge/ioh3420.c
··· 83 83 84 84 static const VMStateDescription vmstate_ioh3420 = { 85 85 .name = "ioh-3240-express-root-port", 86 + .priority = MIG_PRI_PCI_BUS, 86 87 .version_id = 1, 87 88 .minimum_version_id = 1, 88 89 .post_load = pcie_cap_slot_post_load,
+1
hw/pci-bridge/pci_bridge_dev.c
··· 174 174 175 175 static const VMStateDescription pci_bridge_dev_vmstate = { 176 176 .name = "pci_bridge", 177 + .priority = MIG_PRI_PCI_BUS, 177 178 .fields = (VMStateField[]) { 178 179 VMSTATE_PCI_DEVICE(parent_obj, PCIBridge), 179 180 SHPC_VMSTATE(shpc, PCIDevice, pci_device_shpc_present),
+1 -1
hw/pci-bridge/pcie_pci_bridge.c
··· 129 129 130 130 static const VMStateDescription pcie_pci_bridge_dev_vmstate = { 131 131 .name = TYPE_PCIE_PCI_BRIDGE_DEV, 132 + .priority = MIG_PRI_PCI_BUS, 132 133 .fields = (VMStateField[]) { 133 134 VMSTATE_PCI_DEVICE(parent_obj, PCIBridge), 134 135 SHPC_VMSTATE(shpc, PCIDevice, NULL), ··· 178 179 k->config_write = pcie_pci_bridge_write_config; 179 180 dc->vmsd = &pcie_pci_bridge_dev_vmstate; 180 181 dc->props = pcie_pci_bridge_dev_properties; 181 - dc->vmsd = &pcie_pci_bridge_dev_vmstate; 182 182 dc->reset = &pcie_pci_bridge_reset; 183 183 set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories); 184 184 hc->plug = pcie_pci_bridge_hotplug_cb;
+1
hw/pci-bridge/xio3130_downstream.c
··· 161 161 162 162 static const VMStateDescription vmstate_xio3130_downstream = { 163 163 .name = "xio3130-express-downstream-port", 164 + .priority = MIG_PRI_PCI_BUS, 164 165 .version_id = 1, 165 166 .minimum_version_id = 1, 166 167 .post_load = pcie_cap_slot_post_load,
+1
hw/pci-bridge/xio3130_upstream.c
··· 133 133 134 134 static const VMStateDescription vmstate_xio3130_upstream = { 135 135 .name = "xio3130-express-upstream-port", 136 + .priority = MIG_PRI_PCI_BUS, 136 137 .version_id = 1, 137 138 .minimum_version_id = 1, 138 139 .fields = (VMStateField[]) {
+1
include/migration/vmstate.h
··· 148 148 typedef enum { 149 149 MIG_PRI_DEFAULT = 0, 150 150 MIG_PRI_IOMMU, /* Must happen before PCI devices */ 151 + MIG_PRI_PCI_BUS, /* Must happen before IOMMU */ 151 152 MIG_PRI_GICV3_ITS, /* Must happen before PCI devices */ 152 153 MIG_PRI_GICV3, /* Must happen before the ITS */ 153 154 MIG_PRI_MAX,