[gentoo-user] Size of portage tree

[gentoo-user] Size of portage tree

[Harry Putnam]

I've noticed /usr/portage is standing at a little over 2 gigs in
size.  Is this about normal?

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Size of portage tree

[glumtail]

You can alway rm /usr/portage/distfiles/
Those files can be downloaded again when emerge.

On 9/28/05, Harry Putnam <reader@newsguy.com> wrote:
> I've noticed /usr/portage is standing at a little over 2 gigs in
> size.  Is this about normal?
>
> --
> gentoo-user@gentoo.org mailing list
>
>

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Size of portage tree

[Michael Kjorling]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 2005-09-28 07:30 -0500, reader@newsguy.com wrote:
> I've noticed /usr/portage is standing at a little over 2 gigs in
> size.  Is this about normal?

Don't forget that portage keeps all downloaded source code in
/usr/portage/distfiles. The actual portage tree totals something like
100 MB plus losses due to inode/block size.

My workstation's "portage" directory is about 2.4 GB.

- -- 
Michael Kjörling, michael@kjorling.com - http://michael.kjorling.com/
* ASCII Ribbon Campaign: Against HTML Mail, Proprietary Attachments *
* ..... No bird soars too high if he soars with his own wings ..... *
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDOpF0dY+HSb3praYRAtdSAJ9BttgwBNN37AAcQlaYnsIP7RJxIQCdEzVb
llqzj1EiuuIAMc949zh+dAw=
=Gn2k
-----END PGP SIGNATURE-----
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Size of portage tree

[Ryan Viljoen]

[-- Attachment #1: Type: text/plain, Size: 1198 bytes --]

If you do a:
du -hs /usr/portage/distfiles
and a
du -hs /usr/portage

You will see that the majority of the space is taken up by the distfiles.
This is where emerge stores all the packages that it downloads when
installing them on your system. To decrease the size of the directory you
can go through it and delete older versions of packages such as:

I have:
zlib-1.2.1.tar.bz2
zlib-1.2.3.tar.bz2

So you could delete zlib-1.2.1.tar.bz2 since there is a new version. I think
there are some scripts out there that do such things but I havent bothered
to find them. I wouldnt suggest deleting all the files since if you want to
reinstall or add new packages they may require files that have been already
downloaded ie: save you the time and bandwidth of redownloading them.

Cheers
Rav


On 9/28/05, Harry Putnam <reader@newsguy.com> wrote:
>
> I've noticed /usr/portage is standing at a little over 2 gigs in
> size. Is this about normal?
>
> --
> gentoo-user@gentoo.org mailing list
>
>


--
"When you say "I wrote a program that crashed Windows", people just stare at
you blankly and say "Hey, I got those with the system, for free". - Linus
Torvalds, 1995

[-- Attachment #2: Type: text/html, Size: 1633 bytes --]

Re: [gentoo-user] Size of portage tree

[Ryan Viljoen]

[-- Attachment #1: Type: text/plain, Size: 764 bytes --]

Sorry forgot to add, yeah its pretty much normal, my results:
du -hs /usr/portage/distfiles = 1.9Gb
du -hs /usr/portage/distfiles/portage = 2.4Gb

Cheers
Rav


> On 9/28/05, Harry Putnam <reader@newsguy.com> wrote:
> >
> > I've noticed /usr/portage is standing at a little over 2 gigs in
> > size. Is this about normal?
> >
> > --
> > gentoo-user@gentoo.org mailing list
> >
> >
>
>
> --
> "When you say "I wrote a program that crashed Windows", people just stare
> at you blankly and say "Hey, I got those with the system, for free". - Linus
> Torvalds, 1995




--
"When you say "I wrote a program that crashed Windows", people just stare at
you blankly and say "Hey, I got those with the system, for free". - Linus
Torvalds, 1995

[-- Attachment #2: Type: text/html, Size: 1620 bytes --]

Re: [gentoo-user] Size of portage tree

[Paweł Madej]

Ryan Viljoen wrote:
> If you do a:
> du -hs /usr/portage/distfiles
> and a
> du -hs /usr/portage
> 
> You will see that the majority of the space is taken up by the 
> distfiles. This is where emerge stores all the packages that it 
> downloads when installing them on your system. To decrease the size of 
> the directory you can go through it and delete older versions of 
> packages such as:
> 
> I have:
> zlib-1.2.1.tar.bz2
> zlib-1.2.3.tar.bz2
> 
> So you could delete zlib-1.2.1.tar.bz2 since there is a new version. I 
> think there are some scripts out there that do such things but I havent 
> bothered to find them. I wouldnt suggest deleting all the files since if 
> you want to reinstall or add new packages they may require files that 
> have been already downloaded ie: save you the time and bandwidth of 
> redownloading them.

if you use getdelta to fetch updates you can set

REMOVE_OLD=true

in /etc/deltup/getdelta.rc and it will delete old file as above 
zlib-1.2.1.tar.bz2 after successfull make of new version from patches.

More about getdelta on [1]

[1] http://gentoo-wiki.com/HOWTO_Install_Deltup

Greets
Paweł
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Size of portage tree

[Jason Stubbs]

On Wednesday 28 September 2005 21:49, Michael Kjorling wrote:
> Don't forget that portage keeps all downloaded source code in
> /usr/portage/distfiles. The actual portage tree totals something like
> 100 MB plus losses due to inode/block size.

A couple of years ago it not much more than 100MB, but nowadays...

/mnt/archive/gentoo $ du -sh *
750M    distfiles
516M    rsync

--
Jason Stubbs
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Size of portage tree

[Vladimir Mikhailichenko]

And my /usr/portage/distfiles = 15Gb :)

> I've noticed /usr/portage is standing at a little over 2 gigs in
> size.  Is this about normal?


-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Size of portage tree

[Vladimir Mikhailichenko]

And my /usr/portage/distfiles = 15Gb :)

> I've noticed /usr/portage is standing at a little over 2 gigs in
> size.  Is this about normal?


-- 
gentoo-user@gentoo.org mailing list

[gentoo-user] Re: Size of portage tree

[Harry Putnam]

> On 9/28/05, Harry Putnam <reader@newsguy.com> wrote:
>> I've noticed /usr/portage is standing at a little over 2 gigs in
>> size.  Is this about normal?

glumtail <glumtail@gmail.com> writes:

> You can alway rm /usr/portage/distfiles/
> Those files can be downloaded again when emerge.
>

Yup, that turned out tobe 1.5 gigs of it...

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Size of portage tree

[Kurt Guenther]

Harry Putnam wrote:

>I've noticed /usr/portage is standing at a little over 2 gigs in
>size.  Is this about normal?
>
>  
>

Just add a crontab entry to delete files older then, say, 30 days in 
/usr/portage/distfiles.  I've noticed on ~x86 that there is often a 
number of portage changes on the same release for the first few days, so 
doing this daily is too much, but 15-30 days is about right.

--Kurt


-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Size of portage tree

[Etaoin Shrdlu]

On Wednesday 28 September 2005 14:43, glumtail wrote:

> You can alway rm /usr/portage/distfiles/
> Those files can be downloaded again when emerge.

Also, the block size of the file system in which /usr/portage lives can 
make a big difference. 
Try a clean /usr/portage on an ext2/3 filesystem vs. a /usr/portage on 
reiserfs and you'll see what I mean.
-- 
gentoo-user@gentoo.org mailing list

[gentoo-user] Re: Size of portage tree

[Harry Putnam]

Etaoin Shrdlu <shrdlu@unlimitedmail.org> writes:

> On Wednesday 28 September 2005 14:43, glumtail wrote:
>
>> You can alway rm /usr/portage/distfiles/
>> Those files can be downloaded again when emerge.
>
> Also, the block size of the file system in which /usr/portage lives can 
> make a big difference. 
> Try a clean /usr/portage on an ext2/3 filesystem vs. a /usr/portage on 
> reiserfs and you'll see what I mean.

I am using reiserfs but only on trial basis.  I've noticed what
appears to be quite a large increase in time needed for fs intensive
things like du or rm -rf as compared to ext3 but I've done no real
comparison testing.

Have you noticed that too?

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Re: Size of portage tree

[Hemmann, Volker Armin]

On Wednesday 28 September 2005 15:55, Harry Putnam wrote:
> Etaoin Shrdlu <shrdlu@unlimitedmail.org> writes:
> > On Wednesday 28 September 2005 14:43, glumtail wrote:
> >> You can alway rm /usr/portage/distfiles/
> >> Those files can be downloaded again when emerge.
> >
> > Also, the block size of the file system in which /usr/portage lives can
> > make a big difference.
> > Try a clean /usr/portage on an ext2/3 filesystem vs. a /usr/portage on
> > reiserfs and you'll see what I mean.
>
> I am using reiserfs but only on trial basis.  I've noticed what
> appears to be quite a large increase in time needed for fs intensive
> things like du or rm -rf as compared to ext3 but I've done no real
> comparison testing.
>
> Have you noticed that too?

no, but I noticed, that reiserfs needs much less space with small files (like 
portage tree) than ext2/3.
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Re: Size of portage tree

[José Pablo Ezequiel Fernández]

On Wednesday 28 September 2005 12:21, Hemmann, Volker Armin wrote:
> no, but I noticed, that reiserfs needs much less space with small files
> (like portage tree) than ext2/3.
Any numbers you can post ?
-- 
José Pablo Ezequiel Fernández

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Re: Size of portage tree

[Rumen Yotov]

[-- Attachment #1: Type: text/plain, Size: 585 bytes --]

On Wed, 28 Sep 2005 14:29:17 -0300
Jos__ Pablo Ezequiel Fern__ndez <pablo.fernandez@reliable.com.ar> wrote:

> On Wednesday 28 September 2005 12:21, Hemmann, Volker Armin wrote:
> > no, but I noticed, that reiserfs needs much less space with small
> > files (like portage tree) than ext2/3.
> Any numbers you can post ?
Hi,
Some time ago there was such 'subject' with some data to confirm it.
Now: "#du -h --exclude=packages --exclude=distfiles /var/portage/"
Result=434M. This is on reiserfs-3.6 with tail packing ON.
Note: my portage directory is in /var not /usr
Rumen

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Re: Size of portage tree

[Bastian Balthazar Bux]

Rumen Yotov wrote:
> On Wed, 28 Sep 2005 14:29:17 -0300
> Jos__ Pablo Ezequiel Fern__ndez <pablo.fernandez@reliable.com.ar> wrote:
> 
> 
>>On Wednesday 28 September 2005 12:21, Hemmann, Volker Armin wrote:
>>
>>>no, but I noticed, that reiserfs needs much less space with small
>>>files (like portage tree) than ext2/3.
>>
>>Any numbers you can post ?
> 
> Hi,
> Some time ago there was such 'subject' with some data to confirm it.
> Now: "#du -h --exclude=packages --exclude=distfiles /var/portage/"
> Result=434M. This is on reiserfs-3.6 with tail packing ON.
> Note: my portage directory is in /var not /usr
> Rumen

confirmed:
reiserfs = 434M
ext3     = 516M

having reiserfs = 100M :

100 -- 118.89
-- 
gentoo-user@gentoo.org mailing list

Reiserfs speed (Was: Re: [gentoo-user] Re: Size of portage tree)

[Richard Fish]

Harry Putnam wrote:

>I am using reiserfs but only on trial basis.  I've noticed what
>appears to be quite a large increase in time needed for fs intensive
>things like du or rm -rf as compared to ext3 but I've done no real
>comparison testing.
>
>Have you noticed that too?
>  
>

This is normal, and it's a feature.  Reiserfs uses hash values to speed 
the lookup of single files, and as a result the readdir() system call in 
reiserfs (which is what find, rm -rf, and du use to walk a directory 
tree) returns file names in order of their hash value, which probably 
does not match the order of the files on disk.  On the other hand, ext3 
readdir() returns files in inode order.  This means the disk will 
typically have to do more seeking for these operations on reiserfs than 
ext3, which returns file names in inode order.  Actually, you can see 
similar performance differences between ext3 filesystems formatted with 
"-O dir_index" and those without.

You can 'fix' this by tar'ing, reformatting, and restoring the 
filesystem, which will have the effect of ordering files on disk 
according to their hash value. 

Cheers,
-Richard
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Re: Size of portage tree

[A. Khattri]

On Wed, 28 Sep 2005, Rumen Yotov wrote:

> Note: my portage directory is in /var not /usr

Why?


-- 

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Re: Size of portage tree

[Hemmann, Volker Armin]

On Wednesday 28 September 2005 19:29, José Pablo Ezequiel Fernández wrote:
> On Wednesday 28 September 2005 12:21, Hemmann, Volker Armin wrote:
> > no, but I noticed, that reiserfs needs much less space with small files
> > (like portage tree) than ext2/3.
>
> Any numbers you can post ?
> --
> José Pablo Ezequiel Fernández


not real numbers, but a very convincing experience:
some yoears ago, when I was a slackware user, I had a 10gig harddisk, that was 
pretty full - mostly small to medium sized files.
I switched from ext2 to reiserfs and freed 2gb.
Yep, that is true. Instead of 9gb with ext2, my installation only needed 7gb 
with reiserfs.

Since that time, I am a reiserfs-only user.
Harddisks are big - but never big enough.
 Although they get cheaper per megabyte, but they grow so fast, that they are 
almost  always a little bit too expensive to just buy another one ;)

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Re: Size of portage tree

[Rumen Yotov]

[-- Attachment #1: Type: text/plain, Size: 724 bytes --]

On Wed, 28 Sep 2005 14:48:39 -0400 (EDT)
"A. Khattri" <ajai@bway.net> wrote:

> On Wed, 28 Sep 2005, Rumen Yotov wrote:
> 
> > Note: my portage directory is in /var not /usr
> 
> Why?
> 
> 
Hi,
Nothing special, just it seemed more logical to me ;).
To leave /usr only for apps/libs etc.
That's maybe not 100% truth cause i use (in another partiton) a
hardened setup. So think it's easier to manage permissions/access-rights
(MAC-model) if at least /portage isn't in /usr.
Till now never used this config though.
/var is IMHO more appropriate for resources.
PS: another one - glad i use separate /boot as it serves me to hold
now two,later three different kernels (gentoo-sources,RSBAC,Xen-to go).
Rumen

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Size of portage tree

[Nick Rout]

see here for a tip on cleaning it out

http://clug.net.nz/index.php/GentooTips


On Wed, 28 Sep 2005 16:15:22 +0300
Vladimir Mikhailichenko wrote:

> And my /usr/portage/distfiles = 15Gb :)
> 
> > I've noticed /usr/portage is standing at a little over 2 gigs in
> > size.  Is this about normal?
> 
> 
> -- 
> gentoo-user@gentoo.org mailing list

-- 
Nick Rout <nick@rout.co.nz>

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Re: Size of portage tree

[Bryan Whitehead]

> no, but I noticed, that reiserfs needs much less space with small files 
> (like portage tree) than ext2/3.

The only problem with this "solution" is you are then stuck using 
reiserfs...

</fsflamewar> :D

-- 
Bryan Whitehead
Email:driver@megahappy.net
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Re: Size of portage tree

[Hemmann, Volker Armin]

On Thursday 29 September 2005 00:32, Bryan Whitehead wrote:
> > no, but I noticed, that reiserfs needs much less space with small files
> > (like portage tree) than ext2/3.
>
> The only problem with this "solution" is you are then stuck using
> reiserfs...
>
> </fsflamewar> :D

better than stuck with ext3 ;)
http://bugzilla.kernel.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=NEW&bug_status=OPEN&bug_status=ASSIGNED&bug_status=REOPENED&field0-0-0=product&type0-0-0=substring&value0-0-0=ext3&field0-0-1=component&type0-0-1=substring&value0-0-1=ext3&field0-0-2=short_desc&type0-0-2=substring&value0-0-2=ext3&field0-0-3=status_whiteboard&type0-0-3=substring&value0-0-3=ext3

they are ALL buggy - choose your poison ;)

I have choosen reiser, because space is important for me - and I have a nice 
tape-drive, which makes backup/restore very easy... but to be honest, I never 
had reiserfs-bugs.. only hardware errors...
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Re: Size of portage tree

[Bryan Whitehead]

I've lost 5 filesystems on reiser... So I won't touch it anymore. :P :(

I'm all XFS and so far have not many problems (with over 100 machines in 
production). My favorite part about XFS is snapshotting and a working dump 
command for mounted filesystems...

On Thu, 29 Sep 2005, Hemmann, Volker Armin wrote:

> On Thursday 29 September 2005 00:32, Bryan Whitehead wrote:
>>> no, but I noticed, that reiserfs needs much less space with small files
>>> (like portage tree) than ext2/3.
>>
>> The only problem with this "solution" is you are then stuck using
>> reiserfs...
>>
>> </fsflamewar> :D
>
> better than stuck with ext3 ;)
> http://bugzilla.kernel.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=NEW&bug_status=OPEN&bug_status=ASSIGNED&bug_status=REOPENED&field0-0-0=product&type0-0-0=substring&value0-0-0=ext3&field0-0-1=component&type0-0-1=substring&value0-0-1=ext3&field0-0-2=short_desc&type0-0-2=substring&value0-0-2=ext3&field0-0-3=status_whiteboard&type0-0-3=substring&value0-0-3=ext3
>
> they are ALL buggy - choose your poison ;)
>
> I have choosen reiser, because space is important for me - and I have a nice
> tape-drive, which makes backup/restore very easy... but to be honest, I never
> had reiserfs-bugs.. only hardware errors...
>

-- 
Bryan Whitehead
Email:driver@megahappy.net
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Size of portage tree

[Hans-Werner Hilse]

Hi,

On Wed, 28 Sep 2005 15:52:17 +0200
Etaoin Shrdlu <shrdlu@unlimitedmail.org> wrote:

> Also, the block size of the file system in which /usr/portage lives can 
> make a big difference. 

True. See below...

> Try a clean /usr/portage on an ext2/3 filesystem vs. a /usr/portage on 
> reiserfs and you'll see what I mean.

FS type isn't about blocksize (well, maybe about blocksize
constraints). Reiserfs supports putting more than one file in a single
block, that's why using Reiser makes a difference, too.

But reg. blocksize:

$ du -sh --exclude=packages --exclude=distfiles /usr/portage/
138M    /usr/portage/

Neat, eh? That's because of this:

$ xfs_info /usr/portage/
[...]
data     =                       bsize=512    blocks=706792, imaxpct=25
         =                       sunit=0      swidth=0 blks, unwritten=1
[...]

As you can see, a blocksize of 512 bytes is enough to keep portage
small. I've dedicated a small partition to the portage tree because of
this.

-hwh
-- 
gentoo-user@gentoo.org mailing list

[gentoo-user] Re: Reiserfs speed

[Harry Putnam]

Richard Fish <bigfish@asmallpond.org> writes:

> Harry Putnam wrote:
>
>>I am using reiserfs but only on trial basis.  I've noticed what
>>appears to be quite a large increase in time needed for fs intensive
>>things like du or rm -rf as compared to ext3 but I've done no real
>>comparison testing.
>>
>>Have you noticed that too?
>>  
>>
>
> This is normal, and it's a feature.  Reiserfs uses hash values to speed 

[...]

Thanks for the interesting (snipped) info.

> You can 'fix' this by tar'ing, reformatting, and restoring the 
> filesystem, which will have the effect of ordering files on disk 
> according to their hash value. 

I was thinking more along the line of fixing it by reverting to ext3.

I'm not a power user or even a very knowledgable user.  But am
something of a long time user, and since `96, I've used ext2 or 3
exclusively. I don't recall a single incident of losing or corrupting
files that was attributable to ext[23].

I'd heard reiserfs was `better' but apparently the things its better
at aren't things I use or notice.

Is it possible to revert to ext3 from single boot mode or mounted from
a live cd  without a lot of hassles?

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Re: Reiserfs speed

[Fernando Meira]

[-- Attachment #1: Type: text/plain, Size: 943 bytes --]

On 9/29/05, Harry Putnam <reader@newsguy.com> wrote:
>
> I'm not a power user or even a very knowledgable user. But am
> something of a long time user, and since `96, I've used ext2 or 3
> exclusively. I don't recall a single incident of losing or corrupting
> files that was attributable to ext[23].
>
> I'd heard reiserfs was `better' but apparently the things its better
> at aren't things I use or notice.


Do you need speed?
What about space free? reiserfs is quite good! I experimented a ~1G
reduction from having the same data in a ext3 partition to a reiserfs!!

Is it possible to revert to ext3 from single boot mode or mounted from
> a live cd without a lot of hassles?


You can create a tarball of your current system, put it in temporary place
(other partition, removable media, etc), change the partition type from
reiserfs to ext3 and then deploy your system in the brand new partition.

HTH,
Fernando

[-- Attachment #2: Type: text/html, Size: 1419 bytes --]

[gentoo-user] Re: Reiserfs speed

[Harry Putnam]

Fernando Meira <fmeira@gmail.com> writes:

> Do you need speed?
> What about space free? reiserfs is quite good! I experimented a ~1G
> reduction from having the same data in a ext3 partition to a
> reiserfs!!

No, not anymore than the next guy I guess but have grown tired of the
hefty waits when doing fs intensive things like du.  Space isn't
really a big problem here.

> You can create a tarball of your current system, put it in temporary place
> (other partition, removable media, etc), change the partition type from
> reiserfs to ext3 and then deploy your system in the brand new partition.

Well, yeah.  I now how to do something like that.  What I'm asking is
if there is some simpler way.  Over time I've ended up with a somewhat
complicated file system.  Having run out of space or changed existing
dir to partitions or whatever. So it would be a bit more complicated
than what you described.

-- 
gentoo-user@gentoo.org mailing list