[gentoo-dev] Experiment: confcache patch for portage-2.0.50-r10

[gentoo-dev] Experiment: confcache patch for portage-2.0.50-r10

[Stuart Herbert]

[-- Attachment #1: Type: text/plain, Size: 1092 bytes --]

Hi,

GNU autoconf is a bottleneck for compiling packages - especially on 
multi-processor boxes.  It supports the idea of a cache, but provides no 
tools for maintaining the cache at all.

I've put together an experimental patch for Portage 2.0.50-r10, which 
maintains a cache for configure to reuse.  You can find it here:

 http://dev.gentoo.org/~stuart/confcache/

Once you've patched and re-installed Portage, to activate the cache, make sure 
you have both 'sandbox' and 'confcache' set in FEATURES in /etc/make.conf.  
This feature only helps ebuilds which call 'econf'.

I'd be interested in getting some feedback on this patch, as well as any 
suggested improvements.

Best regards,
Stu
-- 
Stuart Herbert                                              stuart@gentoo.org
Gentoo Developer                                       http://www.gentoo.org/
                                                   http://stu.gnqs.org/diary/

GnuPG key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Experiment: confcache patch for portage-2.0.50-r10

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 1153 bytes --]

On Thursday 09 September 2004 23:12, Stuart Herbert wrote:
> Hi,
>
> GNU autoconf is a bottleneck for compiling packages - especially on
> multi-processor boxes.  It supports the idea of a cache, but provides
> no tools for maintaining the cache at all.
>
> I've put together an experimental patch for Portage 2.0.50-r10, which
> maintains a cache for configure to reuse.  You can find it here:
>
>  http://dev.gentoo.org/~stuart/confcache/
>
> Once you've patched and re-installed Portage, to activate the cache,
> make sure you have both 'sandbox' and 'confcache' set in FEATURES in
> /etc/make.conf. This feature only helps ebuilds which call 'econf'.
>
> I'd be interested in getting some feedback on this patch, as well as
> any suggested improvements.

Great. One other thing is that many configure functions also support some 
kind of presets where the answers can be provided (site configuration) in 
`PREFIX/etc/config.site'. This is not a cache but a way to set defaults. 
I think we could also look into that.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Experiment: confcache patch for portage-2.0.50-r10

[Stuart Herbert]

[-- Attachment #1: Type: text/plain, Size: 817 bytes --]

On Friday 10 September 2004 09:06, Paul de Vrieze wrote:
> Great. One other thing is that many configure functions also support some
> kind of presets where the answers can be provided (site configuration) in
> `PREFIX/etc/config.site'. This is not a cache but a way to set defaults.
> I think we could also look into that.
>
> Paul

Sounds like a good idea.  I'll leave that as an exercise for the reader ;-)

Best regards,
Stu
-- 
Stuart Herbert                                              stuart@gentoo.org
Gentoo Developer                                       http://www.gentoo.org/
                                                   http://stu.gnqs.org/diary/

GnuPG key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Experiment: confcache patch for portage-2.0.50-r10

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 679 bytes --]

On Thursday 09 September 2004 23:12, Stuart Herbert wrote:
> Once you've patched and re-installed Portage, to activate the cache,
> make sure you have both 'sandbox' and 'confcache' set in FEATURES in
> /etc/make.conf. This feature only helps ebuilds which call 'econf'.
>
> I'd be interested in getting some feedback on this patch, as well as
> any suggested improvements.

One question I have is how cache invalidation is handled? Will the cache 
not think until the end of time that e.g. kde is installed 
in /usr/kde/3.2 while we allready went on to 4.0?

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Experiment: confcache patch for portage-2.0.50-r10

[Stuart Herbert]

[-- Attachment #1: Type: text/plain, Size: 911 bytes --]

On Friday 10 September 2004 09:32, Paul de Vrieze wrote:
> One question I have is how cache invalidation is handled? Will the cache
> not think until the end of time that e.g. kde is installed
> in /usr/kde/3.2 while we allready went on to 4.0?
>
> Paul

My patch uses the sandbox to see which files GNU configure uses.  I maintain a 
list of those files, and their checksums.  At the start of econf, I check the 
list of checksums.  If any of them have changed, I assume the entire cache is 
invalid.

Best regards,
Stu
-- 
Stuart Herbert                                              stuart@gentoo.org
Gentoo Developer                                       http://www.gentoo.org/
                                                   http://stu.gnqs.org/diary/

GnuPG key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Experiment: confcache patch for portage-2.0.50-r10

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 708 bytes --]

On Friday 10 September 2004 19:42, Stuart Herbert wrote:
> On Friday 10 September 2004 09:32, Paul de Vrieze wrote:
> > One question I have is how cache invalidation is handled? Will the cache
> > not think until the end of time that e.g. kde is installed
> > in /usr/kde/3.2 while we allready went on to 4.0?
> >
> > Paul
>
> My patch uses the sandbox to see which files GNU configure uses.  I
> maintain a list of those files, and their checksums.  At the start of
> econf, I check the list of checksums.  If any of them have changed, I
> assume the entire cache is invalid.

Great,

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]