From xen-devel-bounces@lists.xen.org Thu Feb 18 17:29:18 2016 Received: (at maildrop) by bugs.xenproject.org; 18 Feb 2016 17:29:18 +0000 Received: from lists.xenproject.org ([50.57.142.19] helo=lists.xen.org) by bugs.xenproject.org with esmtp (Exim 4.80) (envelope-from ) id 1aWSOD-00083z-Ve for xen-devel-maildrop-Eithu9ie@bugs.xenproject.org; Thu, 18 Feb 2016 17:29:17 +0000 Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aWSLx-000489-Mt; Thu, 18 Feb 2016 17:26:57 +0000 Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aWSLw-000481-G0 for xen-devel@lists.xen.org; Thu, 18 Feb 2016 17:26:56 +0000 Received: from [193.109.254.147] by server-9.bemta-14.messagelabs.com id DC/1B-13475-FDEF5C65; Thu, 18 Feb 2016 17:26:55 +0000 X-Env-Sender: dunlapg@gmail.com X-Msg-Ref: server-16.tower-27.messagelabs.com!1455816413!24891325!1 X-Originating-IP: [209.85.214.179] X-SpamReason: No, hits=0.8 required=7.0 tests=BODY_RANDOM_LONG, RCVD_BY_IP X-StarScan-Received: X-StarScan-Version: 7.35.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 60608 invoked from network); 18 Feb 2016 17:26:54 -0000 Received: from mail-ob0-f179.google.com (HELO mail-ob0-f179.google.com) (209.85.214.179) by server-16.tower-27.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 18 Feb 2016 17:26:54 -0000 Received: by mail-ob0-f179.google.com with SMTP id gc3so75813518obb.3 for ; Thu, 18 Feb 2016 09:26:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=WKG2vr9R/RIIkwcHr7JZ+cBa4ZsQUpEQEQr+VbYH5fY=; b=UHS5Aw4baC4bmj/nvY65xX8Fj7/EPmJlXLznmRGaVpKqbx1G29icsUts9849/Xkeab uqB+cHVRXWGXpMdAJwzYarXyGws+MnJ2MSbfrxZfIGKEJ1x6vvGyT5eP7lHjuTZm1MSn aoJODqLN9hzpHw11OcbwPF2boRRqHvsc2cZMfmWM/EN9p7HJSzmzZGfGXDJ63McHlGFh s8g2LHpWmcLFZMdHafAxdXHJ3LfiFHfwBO0rziD3iBNlchGdOQe53OGH3MTjyHHNdEmG +uXD5niPYcyKLKH9dFuQWLqB8wjjQdynMtE5xTMbdSVf5mde0JUUjfourA1LV8/VRyng 3qKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=WKG2vr9R/RIIkwcHr7JZ+cBa4ZsQUpEQEQr+VbYH5fY=; b=JsatTaBrFfa1CDQ5E/hvCwv4TfnOYZRzdggdj92XXabaQewXSi7hGUNPYEKxc3hLve +K1O3VxekmR+Y5bRruXRWBJqvQCTgWOBt+CR3Q+xFcnGgWzYZv/hTumfAeibARDz12xe PJBifaPry9KG0naVBbjmc7HqlLnPyIn0ZUTCeJMrH2L8DrsOTEdJAc2VfV4zmQgWRR/9 9ZYOLdJQmaoZOOgDXFoRPaAB6SyPAIUqE8tey4EmGsR4SASa9yXwDCB5DeGp/p+j/7na nvdC7irscuIdWC5x7Zey4rrQ2lE8kYWP5GGgQfJ/yGl0nnWDSshMvIIlO8KFrDTDo545 oojg== X-Gm-Message-State: AG10YOT/nAYBBL+2HNO3BH0vCGdNqMGTEHiE3kX/Vair3w2hq5NJ0RPfR3dZqIfvlFRzzWC5BZFFt0maCEXhrw== MIME-Version: 1.0 X-Received: by 10.182.200.136 with SMTP id js8mr7418925obc.11.1455816412915; Thu, 18 Feb 2016 09:26:52 -0800 (PST) Received: by 10.202.205.140 with HTTP; Thu, 18 Feb 2016 09:26:52 -0800 (PST) In-Reply-To: <1447932195.5647.46.camel@citrix.com> References: <564CC43B.1000904@ainfosec.com> <1447924858.5647.15.camel@citrix.com> <564DAA8D.5060305@citrix.com> <1447932195.5647.46.camel@citrix.com> Date: Thu, 18 Feb 2016 17:26:52 +0000 X-Google-Sender-Auth: 3PakpGgBp5DAaMwWYtTPExw6M3E Message-ID: From: George Dunlap To: Ian Campbell Cc: Andrew Cooper , Jonathan Ludlam , Martin Osterloh , Rob Hoes , "xen-devel@lists.xen.org" Subject: Re: [Xen-devel] Current LibXL Status X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org On Thu, Nov 19, 2015 at 11:23 AM, Ian Campbell wrote: > create ! > title it libxl exit() on ENOMEM incompatible with gc'd languages > thanks > > On Thu, 2015-11-19 at 10:55 +0000, Andrew Cooper wrote: >> On 19/11/15 09:20, Ian Campbell wrote: >> > On Wed, 2015-11-18 at 18:32 +0000, Martin Osterloh wrote: >> > >> > > I wanted to inquire about the current state of LibXL and in >> > > particular >> > > if there are any issues with using it in long-running processes. >> > It is currently being used by libvirtd which I think has shaken out >> > most of >> > the issues with that environment. >> > >> > There are certain to be other bugs, but nothing show-stopping. [snip] >> Languages such as OCaml use -ENOMEM as a hint to run the garbage >> collector some more. I expect Haskell is the same. So the implication you've made here, is that if libxl were to return -ENOMEM to ocaml, that it would run the garbage collector and free up some heap space, so that malloc() could succeed and the libxl operation could proceed. In preparation for writing up a descripion for an "outreach project" (Outreachy/OPW/GSoC), I went and had a chat with some of the guys on the xapi team (cc'd) about what such a thing might look like. According to them, the ocaml garbage collector never frees memory. It grows its own internal heap as necessary, but it never reduces the size of its heap. So no -ENOMEM call or OOM callback can make a failed malloc succeed. One might then ask if libxl could simply allocate memory *from the ocaml heap* itself. It turns out that is also not tenable: Data in the ocaml heap is stored in a heavily coded format. (For example integers are stored as (n*2+1), so that pointers can all be even and non-pointers can all be odd.) The only thing they said might be improved is: 1. To know that libxl would never call exit() for any other reason (which it seems is true) 2. To have a callback in OOM conditions. It's unlikely the process as a whole could do anything but exit, but the ocaml runtime itself might still have internal heap available, which would allow it to exit more gracefully. I think if Haskell or any library *is* capable of integrating with the garbage collector, we'll have to wait until someone who understands the language actually writes bindings and can ask for something they know to be useful for that language. -George _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel