From xen-devel-bounces@lists.xen.org Mon Mar 30 13:18:57 2015 Received: (at maildrop) by bugs.xenproject.org; 30 Mar 2015 12:18:57 +0000 Received: from lists.xen.org ([50.57.142.19]) by bugs.xenproject.org with esmtp (Exim 4.80) (envelope-from ) id 1YcYef-0007mm-HH for xen-devel-maildrop-Eithu9ie@bugs.xenproject.org; Mon, 30 Mar 2015 13:18:57 +0100 Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1YcYdK-0007Tx-Mb; Mon, 30 Mar 2015 12:17:34 +0000 Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1YcYdJ-0007Sj-6n for xen-devel@lists.xen.org; Mon, 30 Mar 2015 12:17:33 +0000 Received: from [193.109.254.147] by server-14.bemta-14.messagelabs.com id ED/A0-31676-CDE39155; Mon, 30 Mar 2015 12:17:32 +0000 X-Env-Sender: julien.grall@linaro.org X-Msg-Ref: server-3.tower-27.messagelabs.com!1427717851!14149761!1 X-Originating-IP: [209.85.212.173] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 6.13.6; banners=-,-,- X-VirusChecked: Checked Received: (qmail 20622 invoked from network); 30 Mar 2015 12:17:31 -0000 Received: from mail-wi0-f173.google.com (HELO mail-wi0-f173.google.com) (209.85.212.173) by server-3.tower-27.messagelabs.com with RC4-SHA encrypted SMTP; 30 Mar 2015 12:17:31 -0000 Received: by wicne17 with SMTP id ne17so28929769wic.0 for ; Mon, 30 Mar 2015 05:17:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=oJqptF6ntvAsezk7Dl1CXDQYAHQBtv2CfZpFf7/uL6M=; b=m2IrZjNL678mXthO8wLqFSqq0fAF8LvK/i54pjyx1VKQG9oLf5ikFcLHZiGppjDDEw 8BKdgvLtN4AN2RtAe6JIjDGpR0GQHcmIs8SMqRL0fGmxoZV4G0D+Ej9Gvl+gWuVfdQcZ 60k16tBKni01RiECRrwapHrT/CA/8OzbGz5JswKj47IAF+R2Ga/XOieFWXquq7IeyxqY Wk3+lGUmK+8MJfUhqXr2YRplpZinbdXK8Wkt878lxaYKzTbrq7H5R/nAI7NMRG7xTknq MuaNL418DvEN0rhyZX4whJxJQ9AU3IwAtBvGg2BFKKE4IhjIlLRs2k2AcjprtQlst4C8 3lVA== X-Gm-Message-State: ALoCoQnGe804tI+HNF4idFVuFHs9+EbcybQOm3k8id8qpULbFFzAephJG5z+/kEnDeOjagT+ZhSV X-Received: by 10.180.105.136 with SMTP id gm8mr22462054wib.13.1427717851367; Mon, 30 Mar 2015 05:17:31 -0700 (PDT) Received: from [10.80.2.139] ([185.25.64.249]) by mx.google.com with ESMTPSA id pa4sm15512155wjb.11.2015.03.30.05.17.30 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 30 Mar 2015 05:17:30 -0700 (PDT) Message-ID: <55193EC0.70207@linaro.org> Date: Mon, 30 Mar 2015 13:17:04 +0100 From: Julien Grall User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Icedove/31.3.0 MIME-Version: 1.0 To: Ian Campbell References: <1427466798.13935.158.camel@citrix.com> <1427466824-31967-8-git-send-email-ian.campbell@citrix.com> <5515870C.1080702@linaro.org> <1427475912.13935.192.camel@citrix.com> In-Reply-To: <1427475912.13935.192.camel@citrix.com> Cc: stefano.stabellini@eu.citrix.com, tim@xen.org, xen-devel@lists.xen.org Subject: Re: [Xen-devel] [PATCH v4 08/15] xen: arm: don't pretend to handle cache maintenance by set/way X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org Hi Ian, On 27/03/15 17:05, Ian Campbell wrote: > On Fri, 2015-03-27 at 16:36 +0000, Julien Grall wrote: >> Hi Ian, >> >> On 27/03/15 14:33, Ian Campbell wrote: >>> We set HCR_EL2.TSW but only (sort of) handle 32-bit access to DCCISW >>> but not the other two registers, nor any 64-bit access. Add handlers >>> for all of these. >> >> We don't set HCR_EL2.TSW so DCCISW is not trapped. > > I was completely sure we did, but I was wrong. > >>> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h >>> index c2dcb66..cf3d6cc 100644 >>> --- a/xen/include/public/arch-arm.h >>> +++ b/xen/include/public/arch-arm.h >>> @@ -161,6 +161,11 @@ >>> * >>> * - The device tree Xen compatible node is fully described under Linux >>> * at Documentation/devicetree/bindings/arm/xen.txt. >>> + * >>> + * - Cache maintenaince operations by set/way ("dc isw|cisw|csw" and > > I've just spotted a typo here, which I have fixed in my tree. > >>> + * the equivalent cp15 registers) are not available when running >>> + * under Xen and will result in an undefined instruction exception >>> + * delivered to the guest. >>> */ >> >> set/way operations is used by Linux ARM32 in order to flush all the >> cache. Injecting an undefined instruction would make guest unusable. > > Yes, that's a shame. AIUI operations by set/way aren't really very > useful under virt. See ARM v7 B1.14.4. > > AIUI2 the reasons Linux does those flushes are due to bootloader's > dirtying of cachelines, which we take great care to avoid in our domain > builder, so I _think_ they probably don't need this under Xen, but have > no easy way to know that at the point they do them. Perhaps it would be > needed for a bootloader run under Xen to do this, I think that would > come under "things to fix when paravirtualaising a bootloader" Technically we already support PV bootloader such as UEFI. > > So I think we have a few options: > > 1. Continue to not trap set/way operations. Guests will be able to > flush the entire host cache by set/way. I don't think this is a > good idea to keep allowing as a general principal. > 2. Trap set/way operations and do one of: > 1. Inject #undef > 2. Silently ignore > 3. Ignore with a debug level print of some sort > 4. Try to do some sort of useful operation. > > I don't think #1 is a very good idea, and we've essentially ruled out > #2.1 (essentially this patch + enable the trap) here I think. > > I've absolutely no idea what #2.4 might be. > > So I think we are down to trap and ignore either with or without > logging. > > Given the caveats with s/w under a hypervisor knowing about it would be > nice. The logging would be a few dozen (nr_sets*nr_ways) on each guest > boot, so would have to be a debug level log, but it wouldn't be terribly > spammy. > > On the other hand, there is no way for a kernel to know it can not > bother with these, so those log messages will always be there and any > problematic uses won't be noticeable anyway. > > So I am probably leaning towards #2.2 > > The KVM approach appears to be to flush the entire guest RAM space on > the first s/w operation and set HCR_EL2.TVM and then ignore all > subsequent s/w ops until caches are enabled, at which point they disable > HCR_EL2.TVM and go back to normal until the next s/w op. This catches > the actual expected use of s/w which is when enabling/disabling caches. > > Something similar might work for us too actually. Maybe we could go with > #2.2 in the short term and plan to do the more complex thing later? I'm ok with the #2.2 as a temporary solution. Although I think it should be properly fixed for Xen 4.6. Would it be worth to open a bug on the tracker and mark as a blocker? Regards, -- Julien Grall _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel