From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) by finch.gentoo.org (Postfix) with ESMTP id 71B0E138CBD for ; Thu, 12 Mar 2015 01:27:55 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 52D9CE089D; Thu, 12 Mar 2015 01:27:54 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id B432EE0893 for ; Thu, 12 Mar 2015 01:27:53 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1YVrug-0006Xs-12 for gentoo-portage-dev@lists.gentoo.org; Thu, 12 Mar 2015 02:27:50 +0100 Received: from ip68-231-22-224.ph.ph.cox.net ([68.231.22.224]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 12 Mar 2015 02:27:49 +0100 Received: from 1i5t5.duncan by ip68-231-22-224.ph.ph.cox.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 12 Mar 2015 02:27:49 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: gentoo-portage-dev@lists.gentoo.org From: Duncan <1i5t5.duncan@cox.net> Subject: [gentoo-portage-dev] Re: running ebuild in src tree Date: Thu, 12 Mar 2015 01:27:44 +0000 (UTC) Message-ID: References: <1426089808.31989.42.camel@transmode.se> <55008A9F.1060705@gentoo.org> <1426100187.31989.54.camel@transmode.se> <5500916E.5040203@gentoo.org> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-portage-dev@lists.gentoo.org Reply-to: gentoo-portage-dev@lists.gentoo.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: ip68-231-22-224.ph.ph.cox.net User-Agent: Pan/0.140 (Chocolate Salty Balls; GIT 10ca3f5) X-Archives-Salt: 57cbaf2c-0b26-4d1a-a942-a2b29b8ec789 X-Archives-Hash: 07ab4d66939b17e098a72fa2e041630c Zac Medico posted on Wed, 11 Mar 2015 12:03:10 -0700 as excerpted: > On 03/11/2015 11:56 AM, Joakim Tjernlund wrote: >> On Wed, 2015-03-11 at 11:34 -0700, Zac Medico wrote: >>> On 03/11/2015 09:03 AM, Joakim Tjernlund wrote: >>>> When developing code it would be really nice if one could run your >>>> ebuild on that src tree as is(no fetch, unpack etc.) >>> >>> The existing convention is to create an ebuild with version 9999 and >>> use one of the live vcs eclasses such as git-r3 to pull the live >>> sources in the src_unpack function. In a future EAPI, we plan to add >>> some features related to this [1]. >> >> I think you misunderstand, [1] is not what I want to do(I think): >> >> Got my src working copy and made a few modds, not commitet yet. >> Now I just want build/test etc. before committing and to do that I just >> run mytree/overlay/dev-util/myapp/myapp.ebuild compile and voila, my >> code is built which I already have in mytree. > > Well, you can create a -9999 ebuild that copies your sources from > $directory to $WORKDIR. Maybe use an environment to configure whether it > pulls from a local directory or a vcs repository. @ Joakim T: FWIW, a commonly recommended user-level portage optimization is to point $PORTAGE_TMPDIR at a tmpfs mount. As long as you have sufficient memory, that lets all building take place in the tmpfs and thus in memory, eliminating many read-accesses and most/all write accesses to permanent storage during the build and (fake-)install phases. In addition to speeding up emerge itself, this reduces build-time I/O, which often becomes the bottleneck on which other processes may be waiting as well, thus allowing other processes more efficient access to permanent storage while emerge is ongoing. Between this and setting PORTAGE_NICENESS=20, emerge is /much/ better behaved during builds, interrupting other processes much less and thus letting you carry on with other things while emerge is doing its thing, with far less interruption. =:^) For instance, here I have /tmp as a tmpfs mount (with /var/tmp being a bind-mount of the same tmpfs), and in make.conf, have the line: PORTAGE_TMPDIR=/tmp Emerge then creates /tmp/portage, and within it, creates the usual cat/ pkg/ build trees, etc, as it emerges various packages. Obviously, your sources in permanent storage are going to be cache-copied into memory as you do the build anyway, and pointing PORTAGE_TMPDIR at tmpfs then becomes a copy to (tmpfs) memory only. While that doesn't technically eliminate the copies (since the read into tmpfs will cache and you'll have the tmpfs copy as well), it DOES mean most of the work beyond the initial read into memory will be memory-only, so you DO eliminate the permanent-storage copies. Is that sufficiently close to what you were looking to do? Beyond that, as Zac suggests, just have the ebuild grab the sources from wherever you put them as your src_unpack, as at that point it'll be a copy to tmpfs, and thus take essentially the same time (or even less since it'll avoid the build-time writes to permanent storage) as doing the in-place build directly. Plus, creating a tmpfs mount if necessary, and setting PORTAGE_TMPDIR, is easy, and you'll dramatically speed-up normal builds as well. =:^) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman