From xen-devel-bounces@lists.xen.org Tue Nov 05 07:17:42 2013 Received: (at maildrop) by bugs.xenproject.org; 5 Nov 2013 07:17:42 +0000 Received: from lists.xen.org ([50.57.142.19]) by bugs.xenproject.org with esmtp (Exim 4.80) (envelope-from ) id 1VdatS-00079F-Fa for xen-devel-maildrop-Eithu9ie@bugs.xenproject.org; Tue, 05 Nov 2013 07:17:42 +0000 Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1Vdapa-0002gZ-Sf; Tue, 05 Nov 2013 07:13:42 +0000 Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1VdOu3-0000sO-Rp for xen-devel@lists.xen.org; Mon, 04 Nov 2013 18:29:32 +0000 Received: from [193.109.254.147:36632] by server-5.bemta-14.messagelabs.com id 35/9C-13525-987E7725; Mon, 04 Nov 2013 18:29:29 +0000 X-Env-Sender: saurabh.globe@gmail.com X-Msg-Ref: server-15.tower-27.messagelabs.com!1383589767!576869!1 X-Originating-IP: [209.85.220.172] X-SpamReason: No, hits=0.9 required=7.0 tests=BODY_RANDOM_LONG, HTML_50_60, HTML_MESSAGE, ML_RADAR_SPEW_LINKS_14, RCVD_BY_IP, spamassassin: X-StarScan-Received: X-StarScan-Version: 6.9.13; banners=-,-,- X-VirusChecked: Checked Received: (qmail 7712 invoked from network); 4 Nov 2013 18:29:28 -0000 Received: from mail-vc0-f172.google.com (HELO mail-vc0-f172.google.com) (209.85.220.172) by server-15.tower-27.messagelabs.com with RC4-SHA encrypted SMTP; 4 Nov 2013 18:29:28 -0000 Received: by mail-vc0-f172.google.com with SMTP id ks9so4869184vcb.17 for ; Mon, 04 Nov 2013 10:29:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:cc:content-type; bh=nYmq3vO5vE0VNkeAeOB0E4VUdl292dSCYW9WMtwiJvs=; b=BoEdjCcRRHZwBftskO2qvNqkFNsbQN2aQaOHJcMv+XYEQxC2QClSFVQ7q5JXY2lVEL JeJv0O6eWrMtwp/q8JsGOZTiRevF20KTBZiRqt1L2i24f5jyA21fIExAPOxhFlXk2W0Q O7ic1np1SeLxwHHqTOh6+dXc6UbBmxkYhQoq2TU1kA+8PDzQ//5kkX56dvbRuNFm11yc IbjcZgIVcEjjLdKj3hQu2J5r5RXjXhjrTyDxW0slI1/i2+at7g6S/kVZKSfiLPnOZOjd 61PXW/d4+83NE598JHAGEHyq2Op6m4ujSUrtnlCyFSKzpiEqrSK+orniYw+MpnbsY+DN i7yg== MIME-Version: 1.0 X-Received: by 10.220.184.70 with SMTP id cj6mr2563164vcb.23.1383589766836; Mon, 04 Nov 2013 10:29:26 -0800 (PST) Received: by 10.58.97.197 with HTTP; Mon, 4 Nov 2013 10:29:26 -0800 (PST) Date: Mon, 4 Nov 2013 10:29:26 -0800 Message-ID: From: Saurabh Mishra To: Ian Campbell X-Mailman-Approved-At: Tue, 05 Nov 2013 07:13:41 +0000 Cc: xen-devel@lists.xen.org Subject: [Xen-devel] xl does not support specifying virtual function for passthrough device X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1795693590646766360==" Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org --===============1795693590646766360== Content-Type: multipart/alternative; boundary=089e0141a44091375804ea5e185e --089e0141a44091375804ea5e185e Content-Type: text/plain; charset=ISO-8859-1 >FYI qemu-upstream xen passhthrough currently also doesn't support specifying a request > for a certain dev/function nr. > And it is vaguely related to the lacking ability to passthrough multifunction devices as > multifunction devices. (the xm + qemu_trad require you to specify the virtual dev/func in > this case, and additional some checks in libxl can't handle the multifunction mask and bail > out early) > Sander But it seems to work in XM toolchain. For example :- pci = [ '0000:08:00.1=0,2=1,3=2,4=3@0a' ] *ironpass3:~/sles11_sp2_gm # xm pci-list-assignable-devices* 0000:08:00.1 0000:08:00.2 0000:08:00.3 0000:08:00.4 *ironpass3:~/sles11_sp2_gm # xm create sles11_sp2_gm.xenconfig* Using config file "./sles11_sp2_gm.xenconfig". Started domain sles11_sp2_gm (id=5) *ironpass3:~/sles11_sp2_gm # uname -a* Linux ironpass3 3.0.93-0.8-xen #1 SMP Tue Aug 27 08:44:18 UTC 2013 (70ed288) x86_64 x86_64 x86_64 GNU/Linux ironpass3:~/sles11_sp2_gm # ironpass3:~/sles11_sp2_gm # *ironpass3:~/sles11_sp2_gm # grep pci sles11_sp2_gm.xenconfig* #pci = [ '0000:08:00.1=0@0a', '0000:08:00.2=0@0b', '0000:08:00.3=0@0c', '0000:08:00.4=0@0d' ] *pci = [ '0000:08:00.1=0,2=1,3=2,4=3@0a' ]* xen_platform_pci=0 ironpass3:~/sles11_sp2_gm # ironpass3:~/sles11_sp2_gm # ironpass3:~/sles11_sp2_gm # lspci -nn -s 08:00 08:00.0 Co-processor [0b40]: Intel Corporation Device [8086:0434] (rev 10) *08:00.1 Ethernet controller [0200]: Intel Corporation DH8900CC Series Gigabit Network Connection [8086:0438] (rev 10)* *08:00.2 Ethernet controller [0200]: Intel Corporation DH8900CC Series Gigabit Network Connection [8086:0438] (rev 10)* *08:00.3 Ethernet controller [0200]: Intel Corporation DH8900CC Series Gigabit Network Connection [8086:0438] (rev 10)* *08:00.4 Ethernet controller [0200]: Intel Corporation DH8900CC Series Gigabit Network Connection [8086:0438] (rev 10)* 08:00.5 Co-processor [0b40]: Intel Corporation Device [8086:043e] (rev 10) ironpass3:~/sles11_sp2_gm # ironpass3:~/sles11_sp2_gm # xm console sles11_sp2_gm linux-34o4:~ # lspci -nn 00:00.0 Host bridge [0600]: Intel Corporation 440FX - 82441FX PMC [Natoma] [8086:1237] (rev 02) 00:01.0 ISA bridge [0601]: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] [8086:7000] 00:01.1 IDE interface [0101]: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] [8086:7010] 00:01.3 Bridge [0680]: Intel Corporation 82371AB/EB/MB PIIX4 ACPI [8086:7113] (rev 01) 00:02.0 VGA compatible controller [0300]: Cirrus Logic GD 5446 [1013:00b8] *00:0a.0 Ethernet controller [0200]: Intel Corporation DH8900CC Series Gigabit Network Connection [8086:0438] (rev 10)* *00:0a.1 Ethernet controller [0200]: Intel Corporation DH8900CC Series Gigabit Network Connection [8086:0438] (rev 10)* *00:0a.2 Ethernet controller [0200]: Intel Corporation DH8900CC Series Gigabit Network Connection [8086:0438] (rev 10)* *00:0a.3 Ethernet controller [0200]: Intel Corporation DH8900CC Series Gigabit Network Connection [8086:0438] (rev 10)* linux-34o4:~ # >Do you happen to know (or can you easily do the experiment to find out) >what happens with '0000:07:11.6@1a' on xm? Does the guest see function 0 >or function 6? If it is 0 then I think we can safely say xl will not >change, but if it is 6 we'll have to have a think... >Ian. Each PCI- device have to have function 0. So in the above case it gets 0. However, we may specify '0000:07:11.6=0@1a' and ''0000:07:11.8=1@1a' and XM does handle it as expected. However, xl does not allow us to specify device function in the guest. Is there a way specifying virtual function slot can become a parameter? After all, xm cfg file is supposed to be 100% compatible with 'xl'. In Xen 4.2.2_06 version, is 'xl' preferred way to instantiate VMs? We currently use 'xm' but wanted to move to 'xl' because it has better support for NUMA alignment and memory backing. Thanks, /Saurabh On Mon, Nov 4, 2013 at 7:45 AM, Ian Campbell wrote: > create ^ > title it xl does not support specifying virtual function for passthrough > device > thanks > > On Thu, 2013-10-31 at 14:29 -0700, Saurabh Mishra wrote: > > In XM toolchain, we use to put pci = [ '0000:07:11.6=0@1a' ] but looks > > like in XL toolchain there is no way to specify function in the guest. > > The code is a bit confusing, but that does appear to be the case. libxl > seems to have some degree of support but I can't see any code in xl (or > the libxlu helper library function which parses the PCI options) that > would support the =N@M syntax. > > Given the lack of xl support there's a good chance that the libxl > support is relatively untested. > > > Kindly let me know. > > > > > > XM - pci = [ '0000:07:11.6=0@1a' ] > > XL - pci = [ '0000:07:11.6@1a' ] > > > > > > In the guest VM we see that PCI device is getting 0 function in both > > the cases however, it may not work the same way in the future. > > Do you happen to know (or can you easily do the experiment to find out) > what happens with '0000:07:11.6@1a' on xm? Does the guest see function 0 > or function 6? If it is 0 then I think we can safely say xl will not > change, but if it is 6 we'll have to have a think... > > Ian. > > > --089e0141a44091375804ea5e185e Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
>FYI qemu-upstream xen passhthrough currently also doesn't su= pport specifying a request
> for=A0a certain dev/function nr.
> And it is = vaguely related to the lacking ability to passthrough multifunction devices= as
> multifunction devices.=A0(the xm + qemu_trad require you to specify the = virtual dev/func in
> this = case, an= d additional some checks in libxl can't handle=A0the multifunction mask and bai= l
> out e= arly)
> Sander

But it seems to work in XM toolchain. For example :-

pci = =3D [ '0000:08:00.1=3D0,2=3D1,3=3D2,4=3D3@0a' ]


ironpass3:~/sles11_sp2_gm # xm pci-list-assignable-devices

0000:08:00.1

0000:08:00.2

0000:08:00.3

0000:08:00.4

=A0

ironpass3:~/= sles11_sp2_gm # xm create sles11_sp2_gm.xenconfig

Using config fi= le "./sles11_sp2_gm.xenconfig".

Started domain = sles11_sp2_gm (id=3D5)

=A0

ironpass3:~/= sles11_sp2_gm # uname -a

Linux ironpass3= 3.0.93-0.8-xen #1 SMP Tue Aug 27 08:44:18 UTC 2013 (70ed288) x86_64 x86_64 x86_64 GNU/Linux

ironpass3:~/sle= s11_sp2_gm #

ironpass3:~/sle= s11_sp2_gm #

ironpass3:~/= sles11_sp2_gm # grep pci sles11_sp2_gm.xenconfig

#pci =3D [ '= ;0000:08:00.1=3D0@0a', '0000:08:00.2=3D0@0b', '0000:08:00.3=3D0@0c', '0000:08:= 00.4=3D0@0d' ]

pci =3D [ '0000:08:00.1=3D0,2=3D1,3=3D2,4=3D3@0a' ]

xen_platform_pc= i=3D0

ironpass3:~/sle= s11_sp2_gm #

ironpass3:~/sle= s11_sp2_gm #

ironpass3:~/sle= s11_sp2_gm # lspci -nn -s 08:00

08:00.0 Co-proc= essor [0b40]: Intel Corporation Device [8086:0434] (rev 10)

08:00.1 Ethe= rnet controller [0200]: Intel Corporation DH8900CC Series Gigabit Network Connection [8086:0438] (rev 10)

08:00.2 Ethe= rnet controller [0200]: Intel Corporation DH8900CC Series Gigabit Network Connection [8086:0438] (rev 10)

08:00.3 Ethe= rnet controller [0200]: Intel Corporation DH8900CC Series Gigabit Network Connection [8086:0438] (rev 10)

08:00.4 Ethe= rnet controller [0200]: Intel Corporation DH8900CC Series Gigabit Network Connection [8086:0438] (rev 10)

08:00.5 Co-proc= essor [0b40]: Intel Corporation Device [8086:043e] (rev 10)

ironpass3:~/sle= s11_sp2_gm #

ironpass3:~/sle= s11_sp2_gm # xm console sles11_sp2_gm

=A0

linux-34o4:~ # = lspci -nn

00:00.0 Host br= idge [0600]: Intel Corporation 440FX - 82441FX PMC [Natoma] [8086:1237] (rev 02)<= span style=3D"color:black">

00:01.0 ISA bri= dge [0601]: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] [8086:7000]

00:01.1 IDE int= erface [0101]: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] [8086:7010]

00:01.3 Bridge = [0680]: Intel Corporation 82371AB/EB/MB PIIX4 ACPI [8086:7113] (rev 01)

00:02.0 VGA com= patible controller [0300]: Cirrus Logic GD 5446 [1013:00b8]

00:0a.0 Ethe= rnet controller [0200]: Intel Corporation DH8900CC Series Gigabit Network Connection [8086:0438] (rev 10)

00:0a.1 Ethe= rnet controller [0200]: Intel Corporation DH8900CC Series Gigabit Network Connection [8086:0438] (rev 10)

00:0a.2 Ethe= rnet controller [0200]: Intel Corporation DH8900CC Series Gigabit Network Connection [8086:0438] (rev 10)

00:0a.3 Ethe= rnet controller [0200]: Intel Corporation DH8900CC Series Gigabit Network Connection [8086:0438] (rev 10)

linux-34o4:~ #<= /span>


>Do you happen to know (or can you eas= ily do the experiment to find out)
>what happens with '0000:07:11= .6@1a' on xm? Does the guest see function 0
>or function 6? If it= is 0 then I think we can safely say xl will not
>change, but if it is 6 we'll have to have a think...
>Ian.

Each PCI- device hav= e to have function 0. So in the above case it gets 0.

However, = =A0we may specify '0000:07:11.6=3D0@1a' and ''0000:07:11.8= =3D1@1a' and XM does handle it as expected.

However, xl does not allow us to specify device = function in the guest.

Is there a way specifying virtual function slot can become= a parameter? After all, xm cfg file is supposed to be 100% compatible with= 'xl'.

In Xen 4.2.= 2_06 version, is 'xl' preferred way to instantiate VMs? We currentl= y use 'xm' but wanted to move to 'xl' because it has better= support for NUMA alignment and memory backing.

Thanks,
/Saurabh

<= div class=3D"gmail_quote">On Mon, Nov 4, 2013 at 7:45 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
create ^
title it xl does not support specifying virtual function for passthrough de= vice
thanks

On Thu, 2013-10-31 at 14:29 -0700, Saurabh Mishra wrote:
> In XM toolchain, we use to put pci =3D [ '0000:07:11.6=3D0@1a'= ] but looks
> like in XL toolchain there is no way to specify function in the guest.=

The code is a bit confusing, but that does appear to be the case. lib= xl
seems to have some degree of support but I can't see any code in xl (or=
the libxlu helper library function which parses the PCI options) that
would support the =3DN@M syntax.

Given the lack of xl support there's a good chance that the libxl
support is relatively untested.

> Kindly let me know.
>
>
> XM =A0- pci =3D [ '0000:07:11.6=3D0@1a' ]
> XL =A0 - pci =3D [ '0000:07:11.6@1a' ]
>
>
> In the guest VM we see that PCI device is getting 0 function in both > the cases however, it may not work the same way in the future.

Do you happen to know (or can you easily do the experiment to find ou= t)
what happens with '0000:07:11.6@1a' on xm? Does the guest see funct= ion 0
or function 6? If it is 0 then I think we can safely say xl will not
change, but if it is 6 we'll have to have a think...

Ian.



--089e0141a44091375804ea5e185e-- --===============1795693590646766360== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============1795693590646766360==--