#27 - Re: [Xen-devel] xend vs xl with pci=['<bdf'] wherein the '<bdf>' are not owned by pciback or pcistub will still launch.

Owner: Ian Campbell <Ian.Campbell@citrix.com>

Date: Tue Jan 7 11:30:02 2014

Last Update: Tue Jan 7 11:30:02 2014

Severity: normal

Affects:

State: Open

[ Retrieve as mbox ]


From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
To: xen-devel@lists.xenproject.org
Subject: [Xen-devel] xend vs xl with pci=['<bdf'] wherein the '<bdf>' are not owned by pciback or pcistub will still launch.
Date: Mon, 6 Jan 2014 12:57:13 -0500
Message-ID: <20140106175713.GB28636@phenom.dumpdata.com>

[ Reply to this message; Retrieve Raw Message; Archives: gmane, marc.info ]

In Xend, if I had a pci entry in the guest config and the 
PCI device was not assigned to xen-pciback or pci-stub it
would refuse to launch the guest.

Not so with 'xl'. It will complain but still launch:

-bash-4.1# cd drivers/pciback/
-bash-4.1# ls
0000:01:00.0  0000:03:08.1  0000:03:0a.0  0000:03:0b.1       irq_handlers  new_slot    remove_id    uevent
0000:01:00.1  0000:03:09.0  0000:03:0a.1  bind               module        permissive  remove_slot  unbind
0000:03:08.0  0000:03:09.1  0000:03:0b.0  irq_handler_state  new_id        quirks      slots
-bash-4.1# echo "0000:03:0b.0" > unbind
-bash-4.1# echo "0000:03:0b.1" > unbind
-bash-4.1# xl create /mnt/lab/security/security.cfg  
Parsing config from /mnt/lab/security/security.cfg
libxl: error: libxl_pci.c:1055:libxl__device_pci_add: PCI device 0:3:b.0 is not assignable
libxl: error: libxl_pci.c:1055:libxl__device_pci_add: PCI device 0:3:b.1 is not assignable
-bash-4.1# xl list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0  2047     4     r-----      14.7
security                                     1  1023     1     -b----       8.0
-bash-4.1# 
-bash-4.1# cat /mnt/lab/security/security.cfg |grep -v \#
device_model_version="qemu-xen-traditional"
builder="hvm"
memory = 1024
name = "security"
vcpus=1
vif = [ 'mac=00:0F:4B:00:00:84,bridge=switch' ]
disk = [ 'phy:/dev/sda,xvda,w' ]
pci= ['0000:03:08.0', '000:03:08.1', '0000:03:09.0', '0000:03:09.1', '0000:03:0a.0', '0000:03:0a.1', '0000:03:0b.0', '0000:03:0b.1']
vnc=1
vnclisten='0.0.0.0'
vncunused=1
serial="pty"


And naturally when shutting/destroying the guest it will say:
-bash-4.1# xl destroy 1
libxl: error: libxl_pci.c:1265:do_pci_remove: xc_deassign_device failed: No such device
libxl: error: libxl_pci.c:1265:do_pci_remove: xc_deassign_device failed: No such device

(XEN) [2014-01-06 17:54:39] deassign 0000:03:0b.0 from dom1 failed (-19)
(XEN) [2014-01-06 17:54:39] deassign 0000:03:0b.1 from dom1 failed (-19)

because it tries to de-allocate them even though they were not
part of the guest.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

From: Ian Campbell <Ian.Campbell@citrix.com>
To: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: xen-devel@lists.xenproject.org
Subject: Re: [Xen-devel] xend vs xl with pci=['<bdf'] wherein the '<bdf>' are not owned by pciback or pcistub will still launch.
Date: Tue, 7 Jan 2014 11:22:58 +0000
Message-ID: <1389093778.31766.131.camel@kazak.uk.xensource.com>

[ Reply to this message; Retrieve Raw Message; Archives: gmane, marc.info ]

create ^
thanks

On Mon, 2014-01-06 at 12:57 -0500, Konrad Rzeszutek Wilk wrote:
> In Xend, if I had a pci entry in the guest config and the 
> PCI device was not assigned to xen-pciback or pci-stub it
> would refuse to launch the guest.
> 
> Not so with 'xl'. It will complain but still launch:

It looks like domcreate_attach_pci() is ignoring the result of
libxl__device_pci_add(). It appears to have always done so.

I suppose there is an argument that there are usecases where starting
the domain at all even without the full set of devices is better than
not starting it at all, but I think I agree that the default should be
to fail if some devices are not available.

Is this a blocker for you for 4.4 or can it wait for 4.5?

> 
> -bash-4.1# cd drivers/pciback/
> -bash-4.1# ls
> 0000:01:00.0  0000:03:08.1  0000:03:0a.0  0000:03:0b.1       irq_handlers  new_slot    remove_id    uevent
> 0000:01:00.1  0000:03:09.0  0000:03:0a.1  bind               module        permissive  remove_slot  unbind
> 0000:03:08.0  0000:03:09.1  0000:03:0b.0  irq_handler_state  new_id        quirks      slots
> -bash-4.1# echo "0000:03:0b.0" > unbind
> -bash-4.1# echo "0000:03:0b.1" > unbind
> -bash-4.1# xl create /mnt/lab/security/security.cfg  
> Parsing config from /mnt/lab/security/security.cfg
> libxl: error: libxl_pci.c:1055:libxl__device_pci_add: PCI device 0:3:b.0 is not assignable
> libxl: error: libxl_pci.c:1055:libxl__device_pci_add: PCI device 0:3:b.1 is not assignable
> -bash-4.1# xl list
> Name                                        ID   Mem VCPUs      State   Time(s)
> Domain-0                                     0  2047     4     r-----      14.7
> security                                     1  1023     1     -b----       8.0
> -bash-4.1# 
> -bash-4.1# cat /mnt/lab/security/security.cfg |grep -v \#
> device_model_version="qemu-xen-traditional"
> builder="hvm"
> memory = 1024
> name = "security"
> vcpus=1
> vif = [ 'mac=00:0F:4B:00:00:84,bridge=switch' ]
> disk = [ 'phy:/dev/sda,xvda,w' ]
> pci= ['0000:03:08.0', '000:03:08.1', '0000:03:09.0', '0000:03:09.1', '0000:03:0a.0', '0000:03:0a.1', '0000:03:0b.0', '0000:03:0b.1']
> vnc=1
> vnclisten='0.0.0.0'
> vncunused=1
> serial="pty"
> 
> 
> And naturally when shutting/destroying the guest it will say:
> -bash-4.1# xl destroy 1
> libxl: error: libxl_pci.c:1265:do_pci_remove: xc_deassign_device failed: No such device
> libxl: error: libxl_pci.c:1265:do_pci_remove: xc_deassign_device failed: No such device
> 
> (XEN) [2014-01-06 17:54:39] deassign 0000:03:0b.0 from dom1 failed (-19)
> (XEN) [2014-01-06 17:54:39] deassign 0000:03:0b.1 from dom1 failed (-19)
> 
> because it tries to de-allocate them even though they were not
> part of the guest.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
To: Ian Campbell <Ian.Campbell@citrix.com>
Cc: xen-devel@lists.xenproject.org
Subject: Re: [Xen-devel] xend vs xl with pci=['<bdf'] wherein the '<bdf>' are not owned by pciback or pcistub will still launch.
Date: Tue, 7 Jan 2014 09:29:13 -0500
Message-ID: <20140107142913.GD3588@phenom.dumpdata.com>

[ Reply to this message; Retrieve Raw Message; Archives: gmane, marc.info ]

On Tue, Jan 07, 2014 at 11:22:58AM +0000, Ian Campbell wrote:
> create ^
> thanks
> 
> On Mon, 2014-01-06 at 12:57 -0500, Konrad Rzeszutek Wilk wrote:
> > In Xend, if I had a pci entry in the guest config and the 
> > PCI device was not assigned to xen-pciback or pci-stub it
> > would refuse to launch the guest.
> > 
> > Not so with 'xl'. It will complain but still launch:
> 
> It looks like domcreate_attach_pci() is ignoring the result of
> libxl__device_pci_add(). It appears to have always done so.
> 
> I suppose there is an argument that there are usecases where starting
> the domain at all even without the full set of devices is better than
> not starting it at all, but I think I agree that the default should be
> to fail if some devices are not available.

<nods> The guest wasn't too happy about some of them missing :-)

> 
> Is this a blocker for you for 4.4 or can it wait for 4.5?

Not a blocker. It can wait, just want to make sure we don't forget.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

From: Sander Eikelenboom <linux@eikelenboom.it>
To: Ian Campbell <Ian.Campbell@citrix.com>
Cc: xen-devel <xen-devel@lists.xenproject.org>
Subject: [Xen-devel] xen-unstable pci passthrough: bug in accounting assigned pci devices when assignment has failed
Date: Fri, 21 Feb 2014 15:34:00 +0100
Message-ID: <974010162.20140221153400@eikelenboom.it>

[ Reply to this message; Retrieve Raw Message; Archives: gmane, marc.info ]

Hi Ian,

It was decided that the bug that domain creation does not fail on non assignable pci devices was deferred to 4.5.
(and it wouldn't prevent this bug anyhow when doing pci hotplug with xl pci-attach)

But there seems to be a bug in the error path:

root@creanuc:~# xl pci-assignable-list
0000:02:00.0

Now when i boot a VM with  pci=['00:19.0'] in it's config file ... which is not assignable:

root@creanuc:~# xl create /etc/xen/domU/router.hvm
Parsing config from /etc/xen/domU/router.hvm
libxl: error: libxl_pci.c:1060:libxl__device_pci_add: PCI device 0:0:19.0 is not assignable

That looks ok ... and the pci device is not visible / accessible in the guest ...  but it seems the entry is still in xenstore nevertheless:

root@creanuc:~# xl pci-list router
Vdev Device
00.0 0000:00:19.0

--
Sander


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

From: Ian Campbell <Ian.Campbell@citrix.com>
To: Sander Eikelenboom <linux@eikelenboom.it>
Cc: xen-devel <xen-devel@lists.xenproject.org>
Subject: Re: [Xen-devel] xen-unstable pci passthrough: bug in accounting assigned pci devices when assignment has failed
Date: Mon, 24 Feb 2014 10:49:37 +0000
Message-ID: <1393238977.16570.37.camel@kazak.uk.xensource.com>

[ Reply to this message; Retrieve Raw Message; Archives: gmane, marc.info ]

graft 27 ^
thanks

Adding this info to the bug for the benefit of whoever ends up looking
at it. Thanks.

Ian.

On Fri, 2014-02-21 at 15:34 +0100, Sander Eikelenboom wrote:
> Hi Ian,
> 
> It was decided that the bug that domain creation does not fail on non assignable pci devices was deferred to 4.5.
> (and it wouldn't prevent this bug anyhow when doing pci hotplug with xl pci-attach)
> 
> But there seems to be a bug in the error path:
> 
> root@creanuc:~# xl pci-assignable-list
> 0000:02:00.0
> 
> Now when i boot a VM with  pci=['00:19.0'] in it's config file ... which is not assignable:
> 
> root@creanuc:~# xl create /etc/xen/domU/router.hvm
> Parsing config from /etc/xen/domU/router.hvm
> libxl: error: libxl_pci.c:1060:libxl__device_pci_add: PCI device 0:0:19.0 is not assignable
> 
> That looks ok ... and the pci device is not visible / accessible in the guest ...  but it seems the entry is still in xenstore nevertheless:
> 
> root@creanuc:~# xl pci-list router
> Vdev Device
> 00.0 0000:00:19.0
> 
> --
> Sander
> 



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel