[gentoo-dev] What happend to portage speed?

[gentoo-dev] What happend to portage speed?

[John Nilsson]

I have a 486DX2 66MHz 16MB server running as NAT,DNS,DNSCACHE,MTA and 
that sort of things.

A couple of months ago a "emerge rsync" would take about 20min to an 
hour to complete. But this has changed... last time I did "emerge sync" 
it was working for 20+ hours.
What on earth happened?
Is it worth it?

-John


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] What happend to portage speed?

[Jon Portnoy]

On Sun, Aug 31, 2003 at 08:22:39AM +0200, John Nilsson wrote:
> I have a 486DX2 66MHz 16MB server running as NAT,DNS,DNSCACHE,MTA and 
> that sort of things.
> 
> A couple of months ago a "emerge rsync" would take about 20min to an 
> hour to complete. But this has changed... last time I did "emerge sync" 
> it was working for 20+ hours.
> What on earth happened?
> Is it worth it?
> 

The portage tree has expanded, for one thing.

You also should make sure you're getting the metadata directory.

-- 
Jon Portnoy
avenj/irc.freenode.net

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] What happend to portage speed?

[Dejan Nikic]

I noticed that some of the servers have been disconecting, I took me a
while to on one some of my machines (they are much faster than what you
have), but it looks like people are syncing like crazy now days :). 
Maybe one of those "Sync once a day" reminders is in order again.

On Sat, 2003-08-30 at 23:27, Jon Portnoy wrote:
> On Sun, Aug 31, 2003 at 08:22:39AM +0200, John Nilsson wrote:
> > I have a 486DX2 66MHz 16MB server running as NAT,DNS,DNSCACHE,MTA and 
> > that sort of things.
> > 
> > A couple of months ago a "emerge rsync" would take about 20min to an 
> > hour to complete. But this has changed... last time I did "emerge sync" 
> > it was working for 20+ hours.
> > What on earth happened?
> > Is it worth it?
> > 
> 
> The portage tree has expanded, for one thing.
> 
> You also should make sure you're getting the metadata directory.


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] What happend to portage speed?

[John Nilsson]

No, no. The download is quick. But the calculation of deptree and that 
kind of things that takes so much time.

Also what is this metadata directory? (Sorry for not RTFM).

-John

Dejan Nikic wrote:

> I noticed that some of the servers have been disconecting, I took me a
> while to on one some of my machines (they are much faster than what you
> have), but it looks like people are syncing like crazy now days :). 
> Maybe one of those "Sync once a day" reminders is in order again.
> 
> On Sat, 2003-08-30 at 23:27, Jon Portnoy wrote:
> 
>>On Sun, Aug 31, 2003 at 08:22:39AM +0200, John Nilsson wrote:
>>
>>>I have a 486DX2 66MHz 16MB server running as NAT,DNS,DNSCACHE,MTA and 
>>>that sort of things.
>>>
>>>A couple of months ago a "emerge rsync" would take about 20min to an 
>>>hour to complete. But this has changed... last time I did "emerge sync" 
>>>it was working for 20+ hours.
>>>What on earth happened?
>>>Is it worth it?
>>>
>>
>>The portage tree has expanded, for one thing.
>>
>>You also should make sure you're getting the metadata directory.
> 
> 
> 
> --
> gentoo-dev@gentoo.org mailing list
> 



--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] What happend to portage speed?

[leio]

On Sun, 31 Aug 2003, John Nilsson wrote:

> No, no. The download is quick. But the calculation of deptree and that 
> kind of things that takes so much time.
> 
> Also what is this metadata directory? (Sorry for not RTFM).
> 
> -John
> 

After every rsync it rebuilds the cache... at least by default.
That's what takes the most time with an emerge sync on my 166MHz computer.
It was discussed a while ago that the cache has changed maybe by 5%, but 
it does it all again. Parses some things, writes lots of files to cache, 
etc. There were ideas how to make it quick again, but unfortunately either 
no-one has brought it to reality or into the official portage.
If that's not your speed problem, disregard this e-mail, but I believe 
this the most time demanding operation with emerge sync as of now.

Mart Raudsepp


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] What happend to portage speed?

[John Nilsson]

Actually it seems most time is spent after each "...done" statement.

-John

söndagen den 31 augusti 2003 kl 16.16 skrev leio@hercules.dustbite.org:

> On Sun, 31 Aug 2003, John Nilsson wrote:
>
>> No, no. The download is quick. But the calculation of deptree and that
>> kind of things that takes so much time.
>>
>> Also what is this metadata directory? (Sorry for not RTFM).
>>
>> -John
>>
>
> After every rsync it rebuilds the cache... at least by default.
> That's what takes the most time with an emerge sync on my 166MHz 
> computer.
> It was discussed a while ago that the cache has changed maybe by 5%, 
> but
> it does it all again. Parses some things, writes lots of files to 
> cache,
> etc. There were ideas how to make it quick again, but unfortunately 
> either
> no-one has brought it to reality or into the official portage.
> If that's not your speed problem, disregard this e-mail, but I believe
> this the most time demanding operation with emerge sync as of now.
>
> Mart Raudsepp
>
>
> --
> gentoo-dev@gentoo.org mailing list
>


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] What happend to portage speed?

[Riyad Kalla]

Maybe the code to print out that "done" statement is unoptimized?

a la:

for(int i = 0; i < 10000000000000; i++);
printf("...done");

:)

John Nilsson wrote:
> Actually it seems most time is spent after each "...done" statement.
> 
> -John
> 
> söndagen den 31 augusti 2003 kl 16.16 skrev leio@hercules.dustbite.org:
> 
>> On Sun, 31 Aug 2003, John Nilsson wrote:
>>
>>> No, no. The download is quick. But the calculation of deptree and that
>>> kind of things that takes so much time.
>>>
>>> Also what is this metadata directory? (Sorry for not RTFM).
>>>
>>> -John
>>>
>>
>> After every rsync it rebuilds the cache... at least by default.
>> That's what takes the most time with an emerge sync on my 166MHz 
>> computer.
>> It was discussed a while ago that the cache has changed maybe by 5%, but
>> it does it all again. Parses some things, writes lots of files to cache,
>> etc. There were ideas how to make it quick again, but unfortunately 
>> either
>> no-one has brought it to reality or into the official portage.
>> If that's not your speed problem, disregard this e-mail, but I believe
>> this the most time demanding operation with emerge sync as of now.
>>
>> Mart Raudsepp
>>
>>
>> -- 
>> gentoo-dev@gentoo.org mailing list
>>
> 
> 
> -- 
> gentoo-dev@gentoo.org mailing list
> 


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] What happend to portage speed?

[George Shapovalov]

On Sunday 31 August 2003 07:23, John Nilsson wrote:
> Actually it seems most time is spent after each "...done" statement.
Hm, do you by chance have a lot of binary packages you created in packages/All 
*and* you have fixpackages in your FEATURES in make.conf?

George



--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] What happend to portage speed?

[John Nilsson]

No

George Shapovalov wrote:

> On Sunday 31 August 2003 07:23, John Nilsson wrote:
> 
>>Actually it seems most time is spent after each "...done" statement.
> 
> Hm, do you by chance have a lot of binary packages you created in packages/All 
> *and* you have fixpackages in your FEATURES in make.conf?
> 
> George
> 
> 
> 
> --
> gentoo-dev@gentoo.org mailing list
> 



--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] What happend to portage speed?

[foser]

Portage just has gotten slower over time with the added functionality,
especially removing a lot of files takes ages for me. I suppose extra
checks and all that take time.

- foser


On Sun, 2003-08-31 at 20:35, John Nilsson wrote:
> No
> 
> George Shapovalov wrote:
> 
> > On Sunday 31 August 2003 07:23, John Nilsson wrote:
> > 
> >>Actually it seems most time is spent after each "...done" statement.
> > 
> > Hm, do you by chance have a lot of binary packages you created in packages/All 
> > *and* you have fixpackages in your FEATURES in make.conf?
> > 
> > George
> > 
> > 
> > 
> > --
> > gentoo-dev@gentoo.org mailing list
> > 
> 
> 
> 
> --
> gentoo-dev@gentoo.org mailing list
> 


--
gentoo-dev@gentoo.org mailing list