[gentoo-user] Symlinking /usr/portage/distfiles

[gentoo-user] Symlinking /usr/portage/distfiles

[Vlad Dogaru]

Hello,

My distfiles is getting quite big and I was thinking of symlinking it
to another partition (just as a temporary solution until I find the
time to re-partition my hard drive). I know I could just delete what I
don't use, but I hope to keep them until the planned reinstall of
Gentoo, so that I can use at least part of the 1.1 GiB (my bandwith is
limited).

Can symlinking /usr/portage/distfiles (and even /usr/tmp/portage) to
directories on another ext2 partition hurt Portage? Are there any
common pitfalls to this procedure? Are the rights on these directories
preserved at the next mount or do I also have to edit fstab?

Thanks,
Vlad

-- 
How's my English? How about my Netiquette?
Do mail me if something is wrong with my behaviour. Thank you.
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Mick]

[-- Attachment #1: Type: text/plain, Size: 969 bytes --]

On Saturday 27 January 2007 13:16, Vlad Dogaru wrote:
> Hello,
>
> My distfiles is getting quite big and I was thinking of symlinking it
> to another partition (just as a temporary solution until I find the
> time to re-partition my hard drive). I know I could just delete what I
> don't use, but I hope to keep them until the planned reinstall of
> Gentoo, so that I can use at least part of the 1.1 GiB (my bandwith is
> limited).
>
> Can symlinking /usr/portage/distfiles (and even /usr/tmp/portage) to
> directories on another ext2 partition hurt Portage? Are there any
> common pitfalls to this procedure? Are the rights on these directories
> preserved at the next mount or do I also have to edit fstab?

Do you know that you can use eclean to remove obsolete/older versions of 
source files from your /usr/portage/distfiles directory?  Alternatively, you 
can just select and rm some of these manually to save some space. 
-- 
Regards,
Mick

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Dale]

Vlad Dogaru wrote:
> Hello,
>
> My distfiles is getting quite big and I was thinking of symlinking it
> to another partition (just as a temporary solution until I find the
> time to re-partition my hard drive). I know I could just delete what I
> don't use, but I hope to keep them until the planned reinstall of
> Gentoo, so that I can use at least part of the 1.1 GiB (my bandwith is
> limited).
>
> Can symlinking /usr/portage/distfiles (and even /usr/tmp/portage) to
> directories on another ext2 partition hurt Portage? Are there any
> common pitfalls to this procedure? Are the rights on these directories
> preserved at the next mount or do I also have to edit fstab?
>
> Thanks,
> Vlad
>

You can change the path to distfiles in make.conf and then move it there.

You do know about eclean I assume?  It will remove some of the tarballs
that are no longer needed and give you some space back. 

Hope that helps.

Dale

:-)  :-)  :-)

-- 
www.myspace.com/dalek1967

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Vlad Dogaru]

On 1/27/07, Dale <dalek@exceedtech.net> wrote:
> Vlad Dogaru wrote:
> > Hello,
> >
> > My distfiles is getting quite big and I was thinking of symlinking it
> > to another partition (just as a temporary solution until I find the
> > time to re-partition my hard drive). I know I could just delete what I
> > don't use, but I hope to keep them until the planned reinstall of
> > Gentoo, so that I can use at least part of the 1.1 GiB (my bandwith is
> > limited).
> >
> > Can symlinking /usr/portage/distfiles (and even /usr/tmp/portage) to
> > directories on another ext2 partition hurt Portage? Are there any
> > common pitfalls to this procedure? Are the rights on these directories
> > preserved at the next mount or do I also have to edit fstab?
> >
> > Thanks,
> > Vlad
> >
>
> You can change the path to distfiles in make.conf and then move it there.
>
> You do know about eclean I assume?  It will remove some of the tarballs
> that are no longer needed and give you some space back.

Hi everyone,

I had no idea about these settings in make.conf or about eclean. I
apologise for not having read the proverbial manual thoroughly enough.
One question though: is there a reason why PORTAGE_TMPDIR does not
default to /tmp?

Cheers,
Vlad

-- 
How's my English? How about my Netiquette?
Do mail me if something is wrong with my behaviour. Thank you.
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Jürgen Geuter]

[-- Attachment #1: Type: text/plain, Size: 1001 bytes --]

On Sat, 2007-01-27 at 18:40 +0200, Vlad Dogaru wrote:

Hossa.

> I had no idea about these settings in make.conf or about eclean. I
> apologise for not having read the proverbial manual thoroughly enough.
> One question though: is there a reason why PORTAGE_TMPDIR does not
> default to /tmp?

Sometimes an ebuild needs to run scripts from the package's tarball for
installation or something like that. Many people have an extra partition
for /tmp that is mounted noexec to give people less opportunity to mess
around with the system (for example build weird binaries for local root
exploits).

Apart from that it's often useful to have all ebuild-related stuff in
one place (/tmp/ is often messy as hell so having a "special tmp" for
building sounds like a good idea, especially if things go wrong and you
need to check why).


Jürgen Geuter
-- 
ICQ #81510866 - http://the-gay-bar.com - MSN thanatos@gambit.ath.cx
        "Entia non sunt multiplicanda praeter necessitatem"

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Jeffrey Rollin]

On Saturday 27 January 2007 18:14, Jürgen Geuter wrote:
> On Sat, 2007-01-27 at 18:40 +0200, Vlad Dogaru wrote:
>
> > One question though: is there a reason why PORTAGE_TMPDIR does not
> > default to /tmp?
>
> 
>Many people have an extra partition
> for /tmp that is mounted noexec to give people less opportunity to mess
> around with the system (for example build weird binaries for local root
> exploits).
>
> Apart from that it's often useful to have all ebuild-related stuff in
> one place

I would add that since /tmp is often cleaned on boot-up, /var/tmp is 
considered a "less temporary" place than /tmp. For example, if you hose 
your /opt/foo directory, then assuming you have an appropriate version 
of /foo in /var/tmp/portage, when you re-emerge "foo" it will skip steps that 
don't need to be done  again (because they have already been completed and 
left results in /var/tmp/portage).

Jeff Rollin

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Vlad Dogaru]

On 1/27/07, Jeffrey Rollin <jeff.rollin@gmail.com> wrote:
> On Saturday 27 January 2007 18:14, Jürgen Geuter wrote:
> > On Sat, 2007-01-27 at 18:40 +0200, Vlad Dogaru wrote:
> >
> > > One question though: is there a reason why PORTAGE_TMPDIR does not
> > > default to /tmp?
> >
> >
> >Many people have an extra partition
> > for /tmp that is mounted noexec to give people less opportunity to mess
> > around with the system (for example build weird binaries for local root
> > exploits).
> >
> > Apart from that it's often useful to have all ebuild-related stuff in
> > one place
>
> I would add that since /tmp is often cleaned on boot-up, /var/tmp is
> considered a "less temporary" place than /tmp. For example, if you hose
> your /opt/foo directory, then assuming you have an appropriate version
> of /foo in /var/tmp/portage, when you re-emerge "foo" it will skip steps that
> don't need to be done  again (because they have already been completed and
> left results in /var/tmp/portage).

Thanks to everyone for all your help and for the clarification.

Have a nice day,
Vlad

-- 
How's my English? How about my Netiquette?
Do mail me if something is wrong with my behaviour. Thank you.

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 835 bytes --]

On Sat, 27 Jan 2007 19:05:18 +0000, Jeffrey Rollin wrote:

> I would add that since /tmp is often cleaned on boot-up, /var/tmp is 
> considered a "less temporary" place than /tmp. For example, if you hose 
> your /opt/foo directory, then assuming you have an appropriate version 
> of /foo in /var/tmp/portage, when you re-emerge "foo" it will skip
> steps that don't need to be done  again (because they have already been
> completed and left results in /var/tmp/portage).

Only if you emerge with FEATURES="keepwork", otherwise emerge clears out
the temporary files after completing an emerge and before starting a
new one.

another reason for using /var/tmp is that /tmp is often too small,
especially if using tmpfs.


-- 
Neil Bothwick

Despite the cost of living, have you noticed how it remains so popular?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Alan McKinnon]

On Saturday 27 January 2007 18:40, Vlad Dogaru wrote:
> One question though: is there a reason why PORTAGE_TMPDIR does not
> default to /tmp?

The real nature of /tmp isn't adequate for portage, that's why it uses a 
different one. If memory serves, the FHS defines /tmp as a temporary 
place to store files, and the continued existence of the file after a 
process has finished is not guaranteed. In other words, if there are no 
existing locks on a file, it's up for summary deletion. This could be 
fatal in a big compile - imagine if some cleaner process nuked a binary 
compiled 4 hours ago in an openoffice compile....

But the best reason is that some compiles are HUGE. Openoffice can take 
up all of 5G with everything enabled, and as /tmp is often a tmpfs, 
it's highly unlikely most users will have enough space on /tmp to 
emerge it. 

alan
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Albert Hopkins]

On Mon, 2007-01-29 at 09:38 +0200, Alan McKinnon wrote:
> On Saturday 27 January 2007 18:40, Vlad Dogaru wrote:
> > One question though: is there a reason why PORTAGE_TMPDIR does not
> > default to /tmp?

I've been running PORTAGE_TMPDIR in /tmp for at least a couple of years
without any issues (actually /var/tmp/portage, but /var/tmp symlinks
to /tmp on most of my systems).

> The real nature of /tmp isn't adequate for portage, that's why it uses a 
> different one. If memory serves, the FHS defines /tmp as a temporary 
> place to store files, and the continued existence of the file after a 
> process has finished is not guaranteed. In other words, if there are no 
> existing locks on a file, it's up for summary deletion. This could be 
> fatal in a big compile - imagine if some cleaner process nuked a binary 
> compiled 4 hours ago in an openoffice compile....

I'm not sure if your memory is correct, but I've always been told
"never put anything in /tmp that you want to survive a reboot".  But
still using your def I suppose that process would be 'emerge' which, on
the default config, deletes the files before it finishes anyway.

Most cleaners have sane mtime/atime parameters that they don't interfere
with merges.  The the default Gentoo tmpwatch config for /tmp is 168
(336 hrs for /var/tmp/portage). I've never had an emerge take 168 hours.
If you do, you can adjust that parameter. I do also have DISTDIR
pointing to /var/portage/distfiles and I have a different policy for
tmpwatch for that.

> But the best reason is that some compiles are HUGE. Openoffice can take 
> up all of 5G with everything enabled, and as /tmp is often a tmpfs, 
> it's highly unlikely most users will have enough space on /tmp to 
> emerge it. 

Not that that's ever been a problem for me but you can always
temporarily divert it when compiling "HUGE" jobs.

    # PORTAGE_TMPDIR=/var/scratch/portage emerge openoffice

IMO it's more than worth the convenience/performance of running it
in /tmp than not.  As I've said I've been doing it for a long while and
I'd don't remember ever having files "disappear" or running out of space
on /tmp.  

But if you want to discuss FHS let's talk about how /usr/portage doesn't
belong in /usr ;-)

-m

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Bo Ørsted Andresen]

[-- Attachment #1: Type: text/plain, Size: 619 bytes --]

On Monday 29 January 2007 08:38:08 Alan McKinnon wrote:
> If memory serves, the FHS defines /tmp as a temporary
> place to store files, and the continued existence of the file after a
> process has finished is not guaranteed.

Gentoo does not and never did follow FHS. Really /var/tmp is just a default 
value.

> In other words, if there are no 
> existing locks on a file, it's up for summary deletion.

Plenty of things break if you just delete arbitrary files in /tmp while 
programs using it are running. I don't know what kind of locks you expect are 
being used all over /tmp..

-- 
Bo Andresen

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 1376 bytes --]

On Mon, 29 Jan 2007 21:12:22 +0200, Alan McKinnon wrote:

> I don't trust my memory either so I looked it up. The most recent copy 
> of FHS I have is 2.2:
> 
> "The /tmp directory must be made available for programs that require 
> temporary files. 
> "Programs must not assume that any files or directories in /tmp are 
> preserved between invocations of the program."
> 
> It says nothing about reboots, that is a common mis-interpretation of 
> the standard.

But

> Why not just keep it as /var/tmp? Defined as:
> 
> "The /var/tmp directory is made available for programs that require 
> temporary files or directories that are preserved between system 
> reboots. Therefore, data stored in /var/tmp is more persistent than 
> data in /tmp.

So it does say that /tmp can't be relied upon to survive reboots, but
not in the definition of /tmp :(

AIUI FHS is for binary distros, so doesn't apply to Gentoo anyway.

> Portage shouldn't even begin to start thinking about belonging 
> in /usr :-). That's why I have:
> 
> nazgul ~ # cat /etc/make.conf | grep PORTDIR
> PORTDIR="/var/portage"

Or mount /usr/portage on its own filesystem. I have it mounted on a
sparse file as per
http://gentoo-wiki.com/TIP_Speeding_up_portage#MultiPurpose_Trick


-- 
Neil Bothwick

"A hundred years of forgetting and it all comes rushing back..."

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Bo Ørsted Andresen]

[-- Attachment #1: Type: text/plain, Size: 915 bytes --]

On Monday 29 January 2007 20:12:22 Alan McKinnon wrote:
> Why not just keep it as /var/tmp? Defined as:
>
> "The /var/tmp directory is made available for programs that require
> temporary files or directories that are preserved between system
> reboots. Therefore, data stored in /var/tmp is more persistent than
> data in /tmp.
> "Files and directories located in /var/tmp must not be deleted when the
> system is booted. Although data stored in /var/tmp is typically deleted
> in a site-specific manner, it is recommended that deletions occur at a
> less frequent interval than /tmp."
>
> Strictly per the standard, /var/tmp is the correct place for emerge temp
> files and /tmp is incorrect. Not that it matters on your box with your
> symlink (which is totally standard-compliant btw)

Why would PORTAGE_TMPDIR be required to or in any way benefit from surviving 
reboots?

-- 
Bo Andresen

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Albert Hopkins]

I think you confused my message.  When I said "I've always been told..."
I didn't mean I was told it was part of the standard, I mean it is
common knowledge, common sense, rule-of-thumb, best practice --
whatever. Yes there is FHS but I don't consider it the Bible.  most
distros break FHS in some way anyhow... I mean let's get a little
realistic here.  We're talking about temporary files, not /etc/passwd...

My main point was not to point out theory (FHS) but practice.  Over two
years of use shows that it is perfectly fine to run portage in /tmp
(with tmp on tmpfs) and, if you take a second to think about it, it does
make sense that that would be a viable alternative.  You mentioned
exceptions like OpenOffice and I suggested a workaround.  As always
YMMV.


-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 321 bytes --]

On Tue, 30 Jan 2007 13:22:07 +0100, Bo Ørsted Andresen wrote:

> Why would PORTAGE_TMPDIR be required to or in any way benefit from
> surviving reboots?

Ask that when you've had a power failure ten hours into an OOo emerge :-O


-- 
Neil Bothwick

If at first you don't suceed, try the switch marked "Power"

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Uwe Thiem]

On 30 January 2007 15:52, Alan McKinnon wrote:
> On Tuesday 30 January 2007 15:22, Bo Ørsted Andresen wrote:

> > Anyway if you know
> > how to do that you certainly know how to avoid that /tmp gets wiped
> > during reboot too (which it doesn't unless you make it so). And OOo
> > only takes 5½ hours to compile.. :p
>
> Hah, so my machine isn't so bad. 4 hours 57 minutes 34 seconds with
> everything enabled except linguas (english only) and dev stuff.
>
> 4 hours 2 seconds with gnome, kde and all other fluff out of USE. It's
> enough to make a fellow wanna consider openoffice-bin...

What are the specs of your box?

Uwe

-- 
A fast and easy generator of fractals for KDE:
http://www.SysEx.com.na/iwy-1.0.tar.bz2
Proof of concept of a TSP solver for KDE:
http://www.SysEx.com.na/epat-0.1.tar.bz2
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 532 bytes --]

On Tue, 30 Jan 2007 14:22:10 +0100, Bo Ørsted Andresen wrote:

> > Ask that when you've had a power failure ten hours into an OOo
> > emerge :-O  
> 
> So you actually used FEATURES=keepwork for that?

I tend to use "ebuild /pah/to/ebuild package" followed by "emerge -K
package".

> And OOo only takes 5½ hours to compile.. :p

Not on my 1GHz G4 iBook, for which there are no binary packages
available. It takes around 15 hours :(


-- 
Neil Bothwick

Orcs aren't all that bad... if you have plenty of ketchup.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Anthony E. Caudel]

Neil Bothwick wrote:

snip

>> And OOo only takes 5½ hours to compile.. :p
> 
> Not on my 1GHz G4 iBook, for which there are no binary packages
> available. It takes around 15 hours :(
> 
> 
So when are the Openoffice people going to break it into separate 
packages (Write, Calc, etc.) like KDE did?  This would get rid of that 
nonsense of 15 hrs for a single package build.

-- 
Those who would give up essential Liberty, to purchase a little temporary
Safety, deserve neither Liberty nor Safety.
    -- Benjamin Franklin
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Mick]

[-- Attachment #1: Type: text/plain, Size: 912 bytes --]

On Tuesday 30 January 2007 14:35, Neil Bothwick wrote:
> On Tue, 30 Jan 2007 14:22:10 +0100, Bo Ørsted Andresen wrote:
> > > Ask that when you've had a power failure ten hours into an OOo
> > > emerge :-O
> >
> > So you actually used FEATURES=keepwork for that?
>
> I tend to use "ebuild /pah/to/ebuild package" followed by "emerge -K
> package".
>
> > And OOo only takes 5½ hours to compile.. :p
>
> Not on my 1GHz G4 iBook, for which there are no binary packages
> available. It takes around 15 hours :(

Ha, ha!  :)

     Sat Mar 18 21:22:50 2006 >>> app-office/openoffice-2.0.1-r1
       merge time: 23 hours, 30 minutes and 58 seconds.

As if that's not bad enough I remember a couple of years ago I tried it 3 
times in a row, with different fixes each time (some bug wouldn't let it 
complete the emerge).  I must have been at it for the best part of a week.
-- 
Regards,
Mick

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 923 bytes --]

On Tue, 30 Jan 2007 19:10:45 +0000, Mick wrote:

> > Not on my 1GHz G4 iBook, for which there are no binary packages
> > available. It takes around 15 hours :(  
> 
> Ha, ha!  :)
> 
>      Sat Mar 18 21:22:50 2006 >>> app-office/openoffice-2.0.1-r1
>        merge time: 23 hours, 30 minutes and 58 seconds.
> 
> As if that's not bad enough I remember a couple of years ago I tried it
> 3 times in a row, with different fixes each time (some bug wouldn't let
> it complete the emerge).  I must have been at it for the best part of a
> week.

I know how you felt. there was a problem that caused the OOo install
script to fail, after 15 hours of compilation. That took quite a few
runs to get sorted. The day after I got it to install, a new version came
out with the same problem :(


-- 
Neil Bothwick

Windows 98, the most installed system in the world, I know, I've done it
5 or 6 times myself.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Albert Hopkins]

Ok, just to prove it could be done (and because I was bored).  I
compiled openoffice entirely in /tmp which is tmpfs in about 5:07.

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 305 bytes --]

On Tue, 30 Jan 2007 14:18:26 -0600, Albert Hopkins wrote:

> Ok, just to prove it could be done (and because I was bored).  I
> compiled openoffice entirely in /tmp which is tmpfs in about 5:07.

That's fine if you have 8GB of RAM...


-- 
Neil Bothwick

IRQs? We don't need no stinking IRQs!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Steve Dibb]

Neil Bothwick wrote:
> On Tue, 30 Jan 2007 14:18:26 -0600, Albert Hopkins wrote:
> 
>> Ok, just to prove it could be done (and because I was bored).  I
>> compiled openoffice entirely in /tmp which is tmpfs in about 5:07.
> 
> That's fine if you have 8GB of RAM...
> 
> 

Not necessarily.  tmpfs will start to use the harddrive when it runs out of 
memory, that being one if its nice handy dandy features.

Steve
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 742 bytes --]

On Tue, 30 Jan 2007 17:45:26 -0700, Steve Dibb wrote:

> Not necessarily.  tmpfs will start to use the harddrive when it runs
> out of memory, that being one if its nice handy dandy features.

Really? The lat time I tried putting /tmp on tmpfs on this box, I had
problems when VMware tried to save 512MB files there. I use it on my
laptop though.

I'll give it another try, perhaps things have changed since I last used
it, although Documentation/filesystems/tmpfs.txt warns of the dangers of
setting the size too high. Thinking about it, my problems may have been
caused by the default size being to low.

Even so, you'd need a huge swap partition to build OOo in tmpfs.


-- 
Neil Bothwick

Pentium is a risk processor

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 937 bytes --]

On Tuesday 30 January 2007 19:22, Neil Bothwick <neil@digimed.co.uk> wrote 
about 'Re: [gentoo-user] Symlinking /usr/portage/distfiles':
> On Tue, 30 Jan 2007 17:45:26 -0700, Steve Dibb wrote:
> > Not necessarily.  tmpfs will start to use the harddrive when it runs
> > out of memory, that being one if its nice handy dandy features.
>
> Really?

Yes, tmpfs is backed by virtual memory not real memory so it will use your 
swap if needed.  However, by default, each tmpfs mount is limited to 1/2 
the physical RAM.

In any case, it seems questionable to put yourself in a position where 
tmpfs is hitting the HD often; it's not tuned for such craziness and 
probably performs worse than other filesystems in that case.

-- 
Boyd Stephen Smith Jr.             ,= ,-_-. =. 
bss03@volumehost.net              ((_/)o o(\_))
ICQ: 514984 YM/AIM: DaTwinkDaddy   `-'(. .)`-' 
http://iguanasuicide.org/              \_/     

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Alan McKinnon]

[-- Attachment #1: Type: text/plain, Size: 1485 bytes --]

On Tuesday 30 January 2007 16:06, Uwe Thiem wrote:
> On 30 January 2007 15:52, Alan McKinnon wrote:
> > On Tuesday 30 January 2007 15:22, Bo Ørsted Andresen wrote:
> > > Anyway if you know
> > > how to do that you certainly know how to avoid that /tmp gets
> > > wiped during reboot too (which it doesn't unless you make it so).
> > > And OOo only takes 5½ hours to compile.. :p
> >
> > Hah, so my machine isn't so bad. 4 hours 57 minutes 34 seconds with
> > everything enabled except linguas (english only) and dev stuff.
> >
> > 4 hours 2 seconds with gnome, kde and all other fluff out of USE.
> > It's enough to make a fellow wanna consider openoffice-bin...
>
> What are the specs of your box?

Dell Latitude D810
2GHz Centrino
2GB Ram
80G SATA
2.6.19-suspend2-r1

But, OOo is a well known resource hog that really stresses a machine when compiling, so I don't think it makes a useful measure of anything. And KDE-meta isn't much better these days either. Yesterdays sync brought in 3.5.6 and 3 or 4 other bits and pieces, which I started at 1am this morning. It's just finished now at 1pm - 12 hours!

But having said that, I've noticed that this kernel gives really slow disk IO which I haven't managed to track down. It feels less than half the speed I got on 2.6.18.*, and my three year old desktop with a similar world runs 'emerge -avuNDt world' twice as quick.

I'm almost ready to give up on .19 and go back to .18 till .20 comes out.

alan


[-- Attachment #2: Type: text/html, Size: 1913 bytes --]

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Alan McKinnon]

On Tuesday 30 January 2007 18:26, Anthony E. Caudel wrote:
> Neil Bothwick wrote:
>
> snip
>
> >> And OOo only takes 5½ hours to compile.. :p
> >
> > Not on my 1GHz G4 iBook, for which there are no binary packages
> > available. It takes around 15 hours :(
>
> So when are the Openoffice people going to break it into separate
> packages (Write, Calc, etc.) like KDE did?  This would get rid of
> that nonsense of 15 hrs for a single package build.

haha, good joke, nice one, you just made my day.

Oh wait, you mean you're serious? Erm, well, I once did have a peek into 
the OOo makefiles and what I saw there was .... scary. If no-one has 
had the courage so far to separate out the packages, I really wouldn't 
hold it against them.

alan


-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Uwe Thiem]

On 31 January 2007 13:02, Alan McKinnon wrote:
> On Tuesday 30 January 2007 16:06, Uwe Thiem wrote:

> > What are the specs of your box?
>
> Dell Latitude D810
> 2GHz Centrino
> 2GB Ram
> 80G SATA
> 2.6.19-suspend2-r1

Odd. My 2.8GHz Pentium 4 takes *far* longer to compile OO, something close to 
10h, though I haven't really timed it.

>
> But, OOo is a well known resource hog that really stresses a machine when
> compiling, so I don't think it makes a useful measure of anything. And
> KDE-meta isn't much better these days either. Yesterdays sync brought in
> 3.5.6 and 3 or 4 other bits and pieces, which I started at 1am this
> morning. It's just finished now at 1pm - 12 hours!

KDE is in so far better as it doesn't forbit parallel compiling - as OO does. 
So I can use distcc and let all my boxes contribute. That brings the compile 
time of KDE down a lot. Unfortunately, that isn't possible with OO.
 
>
> But having said that, I've noticed that this kernel gives really slow disk
> IO which I haven't managed to track down. It feels less than half the speed
> I got on 2.6.18.*, and my three year old desktop with a similar world runs
> 'emerge -avuNDt world' twice as quick.

I am still with 2.6.18. Too many problems with 19.

Uwe

-- 
A fast and easy generator of fractals for KDE:
http://www.SysEx.com.na/iwy-1.0.tar.bz2
Proof of concept of a TSP solver for KDE:
http://www.SysEx.com.na/epat-0.1.tar.bz2
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Dan Farrell]

On Wed, 31 Jan 2007 13:16:01 +0200
Alan McKinnon <alan@linuxholdings.co.za> wrote:

> > > available. It takes around 15 hours :(

distcc + crossdev = ; )

im not sure, but i bet you can maybe build G4 code on another box.  
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Bo Ørsted Andresen]

[-- Attachment #1: Type: text/plain, Size: 466 bytes --]

On Wednesday 31 January 2007 13:22:36 Uwe Thiem wrote:
> KDE is in so far better as it doesn't forbit parallel compiling - as OO
> does. So I can use distcc and let all my boxes contribute. That brings the
> compile time of KDE down a lot. Unfortunately, that isn't possible with OO.

Actually it isn't impossible with OOo. It just isn't reliable enough to be 
enabled by default yet. `export WANT_MP=true` will enable parallel compiling.

-- 
Bo Andresen

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Alan McKinnon]

On Wednesday 31 January 2007 14:22, Uwe Thiem wrote:
> On 31 January 2007 13:02, Alan McKinnon wrote:
> > On Tuesday 30 January 2007 16:06, Uwe Thiem wrote:
> > > What are the specs of your box?
> >
> > Dell Latitude D810
> > 2GHz Centrino
> > 2GB Ram
> > 80G SATA
> > 2.6.19-suspend2-r1
>
> Odd. My 2.8GHz Pentium 4 takes *far* longer to compile OO, something
> close to 10h, though I haven't really timed it.

Intuition tells me that on my machine emerging OOo must do a lot of 
stuff my machine is fast at, and relatively little that it's slow at. I 
know for a fact it's got a nippy cpu and lots of ram, but disk IO is 
much slower than it ought to be.

Also, my times come from genlop, and I may well have moved between home 
and office networks, and I can't guarantee that both networks have ntp 
servers exactly synced

> > But, OOo is a well known resource hog that really stresses a
> > machine when compiling, so I don't think it makes a useful measure
> > of anything. And KDE-meta isn't much better these days either.
> > Yesterdays sync brought in 3.5.6 and 3 or 4 other bits and pieces,
> > which I started at 1am this morning. It's just finished now at 1pm
> > - 12 hours!
>
> KDE is in so far better as it doesn't forbit parallel compiling - as
> OO does. So I can use distcc and let all my boxes contribute. That
> brings the compile time of KDE down a lot. Unfortunately, that isn't
> possible with OO.

kde-meta also runs ./configure something like 300 times :-) which is 
very disk intensive and my machine sucks at that

alan
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Alan McKinnon]

On Wednesday 31 January 2007 14:34, Bo Ørsted Andresen wrote:
> On Wednesday 31 January 2007 13:22:36 Uwe Thiem wrote:
> > KDE is in so far better as it doesn't forbit parallel compiling -
> > as OO does. So I can use distcc and let all my boxes contribute.
> > That brings the compile time of KDE down a lot. Unfortunately, that
> > isn't possible with OO.
>
> Actually it isn't impossible with OOo. It just isn't reliable enough
> to be enabled by default yet. `export WANT_MP=true` will enable
> parallel compiling.

Purely for interest's sake, is it known what exactly in OOo is causing 
these problems?

alan


-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Bo Ørsted Andresen]

[-- Attachment #1.1: Type: text/plain, Size: 720 bytes --]

On Wednesday 31 January 2007 13:22:36 Uwe Thiem wrote:
> > > What are the specs of your box?
> >
> > Dell Latitude D810
> > 2GHz Centrino
[SNIP]
> Odd. My 2.8GHz Pentium 4 takes *far* longer to compile OO, something close
> to 10h, though I haven't really timed it.

Heh, I completely missed this in my previous mail. OOo 2.1.0 does take less time to compile than OOo 2.0.4 which takes less time than 2.0.2... (yes, it actually is improving :). My provided time (and likely Alans as well) was for 2.1.0. Furthermore Pentium 4 is a joke (it performs horribly). A 2 GHz (Dothan I presume) Pentium-M should be faster than a 2,8 GHz Pentium 4. My timing is for an 1,6 GHz (Banias) Pentium-M btw.

-- 
Bo Andresen

[-- Attachment #1.2: Type: text/html, Size: 957 bytes --]

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Anthony E. Caudel]

Alan McKinnon wrote:
> On Tuesday 30 January 2007 18:26, Anthony E. Caudel wrote:
>> Neil Bothwick wrote:
>>
>> snip
>>
>>>> And OOo only takes 5½ hours to compile.. :p
>>> Not on my 1GHz G4 iBook, for which there are no binary packages
>>> available. It takes around 15 hours :(
>> So when are the Openoffice people going to break it into separate
>> packages (Write, Calc, etc.) like KDE did?  This would get rid of
>> that nonsense of 15 hrs for a single package build.
> 
> haha, good joke, nice one, you just made my day.
> 
> Oh wait, you mean you're serious? Erm, well, I once did have a peek into 
> the OOo makefiles and what I saw there was .... scary. If no-one has 
> had the courage so far to separate out the packages, I really wouldn't 
> hold it against them.
> 
> alan
> 
> 

LOL!  Hadn't realized it was that bad.  Wonder who the culprit is?  The 
original German group that wrote it or Sun when they got their hands on it.

Tony

-- 
Those who would give up essential Liberty, to purchase a little temporary
Safety, deserve neither Liberty nor Safety.
    -- Benjamin Franklin
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Alan McKinnon]

On Wednesday 31 January 2007, Bo Ørsted Andresen wrote:
> Furthermore Pentium 4 is a joke (it performs horribly). A 2 GHz
> (Dothan I presume) Pentium-M should be faster than a 2,8 GHz Pentium
> 4. My timing is for an 1,6 GHz (Banias) Pentium-M btw.

This sounds odd, but I'm not a cpu expert so can't really comment. Care 
to elaborate on why the P4 performs so horribly?

alan


-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 1158 bytes --]

On Wednesday 31 January 2007 09:58, Alan McKinnon 
<alan@linuxholdings.co.za> wrote about 'Re: [gentoo-user] 
Symlinking /usr/portage/distfiles':
> On Wednesday 31 January 2007, Bo Ørsted Andresen wrote:
> > Furthermore Pentium 4 is a joke (it performs horribly). A 2 GHz
> > (Dothan I presume) Pentium-M should be faster than a 2,8 GHz Pentium
> > 4. My timing is for an 1,6 GHz (Banias) Pentium-M btw.
>
> This sounds odd, but I'm not a cpu expert so can't really comment. Care
> to elaborate on why the P4 performs so horribly?

The instruction pipeline is very long, the CPU <-> RAM bandwith is quite 
small, and the pipeline has to be emptied any time the branch predictor is 
wrong.  While the pipeline fills, the CPU works but no results are 
visible.

Hz has never been a complete trump of other issues affecting CPU 
performance, but is always a factor to consider.  (Among CPUs that are 
otherwise identical, higher Hz wins.)

-- 
Boyd Stephen Smith Jr.             ,= ,-_-. =. 
bss03@volumehost.net              ((_/)o o(\_))
ICQ: 514984 YM/AIM: DaTwinkDaddy   `-'(. .)`-' 
http://iguanasuicide.org/              \_/     

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Mark Kirkwood]

Boyd Stephen Smith Jr. wrote:
> On Wednesday 31 January 2007 09:58, Alan McKinnon 
> <alan@linuxholdings.co.za> wrote about 'Re: [gentoo-user] 
> Symlinking /usr/portage/distfiles':
>> On Wednesday 31 January 2007, Bo Ørsted Andresen wrote:
>>> Furthermore Pentium 4 is a joke (it performs horribly). A 2 GHz
>>> (Dothan I presume) Pentium-M should be faster than a 2,8 GHz Pentium
>>> 4. My timing is for an 1,6 GHz (Banias) Pentium-M btw.
>> This sounds odd, but I'm not a cpu expert so can't really comment. Care
>> to elaborate on why the P4 performs so horribly?
> 
> The instruction pipeline is very long, the CPU <-> RAM bandwith is quite 
> small, and the pipeline has to be emptied any time the branch predictor is 
> wrong.  While the pipeline fills, the CPU works but no results are 
> visible.
> 
> Hz has never been a complete trump of other issues affecting CPU 
> performance, but is always a factor to consider.  (Among CPUs that are 
> otherwise identical, higher Hz wins.)
> 

Also Pentium-M has a lower latency L2 cache than P-4. With respect to 
pipeline lengths I was curious to see what they actually were: P-4 has 
20 stages, P-M has.. err... < 20 stages (Intel won't say exactly!).

I found this an interesting read for those of you interested in this:

http://www.anandtech.com/cpuchipsets/showdoc.aspx?i=2342&p=1

Cheers

Mark
-- 
gentoo-user@gentoo.org mailing list

RE: [gentoo-user] Symlinking /usr/portage/distfiles

[Nelson, David (ED, PAR&D)]

> -----Original Message-----
> From: Mark Kirkwood [mailto:markir@paradise.net.nz]
> Sent: 31 January 2007 23:49
> To: gentoo-user@lists.gentoo.org
> Subject: Re: [gentoo-user] Symlinking /usr/portage/distfiles
> 
> 
> Boyd Stephen Smith Jr. wrote:
> > On Wednesday 31 January 2007 09:58, Alan McKinnon 
> > <alan@linuxholdings.co.za> wrote about 'Re: [gentoo-user] 
> > Symlinking /usr/portage/distfiles':
> >> On Wednesday 31 January 2007, Bo Ørsted Andresen wrote:
> >>> Furthermore Pentium 4 is a joke (it performs horribly). A 2 GHz
> >>> (Dothan I presume) Pentium-M should be faster than a 2,8 
> GHz Pentium
> >>> 4. My timing is for an 1,6 GHz (Banias) Pentium-M btw.
> >> This sounds odd, but I'm not a cpu expert so can't really 
> comment. Care
> >> to elaborate on why the P4 performs so horribly?
> > 
> > The instruction pipeline is very long, the CPU <-> RAM 
> bandwith is quite 
> > small, and the pipeline has to be emptied any time the 
> branch predictor is 
> > wrong.  While the pipeline fills, the CPU works but no results are 
> > visible.
> > 
> > Hz has never been a complete trump of other issues affecting CPU 
> > performance, but is always a factor to consider.  (Among 
> CPUs that are 
> > otherwise identical, higher Hz wins.)
> > 
> 
> Also Pentium-M has a lower latency L2 cache than P-4. With respect to 
> pipeline lengths I was curious to see what they actually 
> were: P-4 has 
> 20 stages, P-M has.. err... < 20 stages (Intel won't say exactly!).
> 
> I found this an interesting read for those of you interested in this:
> 
> http://www.anandtech.com/cpuchipsets/showdoc.aspx?i=2342&p=1
> 
> Cheers
> 
> Mark


At the risk of pulling this topic a little more off-topic - the P-M vs P-4 is an interesting case of a Pentium 3 chipset with a die shrink outperforming a P-4.

The Intel Core (2) Solo/Duo CPUs are based on the Pentium M as well. Netburst is pretty much dead afaik.

--
djn

Disclaimer: I represent no-one else in my emails to this list. Use any advice given at your own risk.

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Alan McKinnon]

On Thursday 01 February 2007, Nelson, David (ED, PAR&D) wrote:
> > Also Pentium-M has a lower latency L2 cache than P-4. With respect
> > to pipeline lengths I was curious to see what they actually
> > were: P-4 has
> > 20 stages, P-M has.. err... < 20 stages (Intel won't say exactly!).
> >
> > I found this an interesting read for those of you interested in
> > this:
> >
> > http://www.anandtech.com/cpuchipsets/showdoc.aspx?i=2342&p=1
> >
> > Cheers
> >
> > Mark
>
> At the risk of pulling this topic a little more off-topic - the P-M
> vs P-4 is an interesting case of a Pentium 3 chipset with a die
> shrink outperforming a P-4.
>
> The Intel Core (2) Solo/Duo CPUs are based on the Pentium M as well.
> Netburst is pretty much dead afaik.

Thanks for everyone's replies. I now know, 6 months later, exactly what 
cpu I have :-)

I've been finding over the last 10 years or so that if I don't keep up 
with new cpu developments, it takes ages to get familiar with the 
terminology and current products again. I think it's called "the price 
of rapid technology advances"

alan


-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 713 bytes --]

On Wed, 31 Jan 2007 06:25:52 -0600, Dan Farrell wrote:

> > > > available. It takes around 15 hours :(  
> 
> distcc + crossdev = ; )
> 
> im not sure, but i bet you can maybe build G4 code on another box.  

It's possible, but the OOo build disables multiple processing unless you
set WANT_MP=1, so distcc would have to do everything on the other box.
Cross-compiling a package on the other box may be a simpler solution.
Sleeping during the compile is even easier :)

Although compiling OOo takes a long time, it's not as though there is any
urgency about it. The machine, and the old version of OOo, is still
usable during compilation.


-- 
Neil Bothwick

This tagline SHAREWARE. Send .

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Ralf Stephan]

> > 2GHz Centrino
> > 2GB Ram
> > 80G SATA
> > 2.6.19-suspend2-r1
> 
> Odd. My 2.8GHz Pentium 4 takes *far* longer to compile OO, something close to 
> 10h, though I haven't really timed it.

Memory is essential for compiling, so a guess would be that you
have less than 1 GB RAM. Maybe even 1GB is not enough.


ralf

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Symlinking /usr/portage/distfiles

[Dan Farrell]

On Thu, 1 Feb 2007 19:43:00 +0100
Ralf Stephan <ralf@ark.in-berlin.de> wrote:

> > > 2GHz Centrino
> > > 2GB Ram
> > > 80G SATA
> > > 2.6.19-suspend2-r1
> > 
> > Odd. My 2.8GHz Pentium 4 takes *far* longer to compile OO,
> > something close to 10h, though I haven't really timed it.

frankly, in my experience pentium 4s are absolutely horrendous
processors.  They're just very, very slow.  Their clock speed is great
but ... i don't know.  My compusa-tech friend assures me that it's the
'quad-pumped' architecture that makes my p-4 celeron 2.4 perform about
as well as a pentium III.  I have'nt done any benchmarks either,
though.

> Memory is essential for compiling, so a guess would be that you
> have less than 1 GB RAM. Maybe even 1GB is not enough.
> 
> 
> ralf
> 
as long as you don't have -pipe in your cflags, i don't think more than
512 megs is essential for compiling. In fact, i don't think even that is
essential.  -pipe puts all temp files in ram.  Without -pipe, the files
are stored on disk (/var/tmp/, i  think, for emerges) and therefore you
don't need a lot of memory.  Of course, linux caches extremely
aggressively so if the ram's there, it'll be used.  
-- 
gentoo-user@gentoo.org mailing list