* Re: [gentoo-dev] Re: emerge suggestions
2004-09-08 23:46 ` Heiko Wundram
@ 2004-09-09 0:03 ` Ned Ludd
2004-09-09 1:01 ` Heiko Wundram
2004-09-09 13:16 ` Chris Gianelloni
2004-09-09 0:19 ` Thomas de Grenier de Latour
2004-09-09 13:13 ` Chris Gianelloni
2 siblings, 2 replies; 16+ messages in thread
From: Ned Ludd @ 2004-09-09 0:03 UTC (permalink / raw
To: Heiko Wundram; +Cc: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 1763 bytes --]
On Wed, 2004-09-08 at 19:46, Heiko Wundram wrote:
> Am Mittwoch, 8. September 2004 18:43 schrieb Chris White:
> > How are you going to effectively measure the times?
>
> IIRC, there once was a proposal to do this using bash-units. Each product in
> the tree gets assigned a bash unit, which is a floating point number >0 which
> measures how long compilation takes relative to compiling some certain
> version of bash.
>
> Now, all that needs to be done is to measure package compilation and merging
> time, divide by the number of bash units this package has, and you get an
> estimate on the time for a bash unit on this computer. The more packages you
> merge, the finer this number will become by simply averaging it out. Of
> course, this does not take into account changing the LDFLAGS (which should
> make up for the biggest part of different merge times), or CFLAGS (which
> might also change timing by varying optimization levels and swap
> requirement). But, anyway, these numbers don't change anything about the
> underlying unit, which should be to a large extent platform and machine
> independent.
To complex and still wont account for various USE= & FEATURES= flags,
gcc versions and loads on said server that's building XYZ. It's
therefore sorta a waste of all of our time and bandwidth to continue
this thread when we already know we are going to all come to the same
conclusion in the end that said functionality will never be accurate.
>
> I don't know when this proposal came up, I read about it on some forum, some
> time ago.
>
> Heiko.
>
> --
> gentoo-dev@gentoo.org mailing list
--
Ned Ludd <solar@gentoo.org>
Gentoo (hardened,security,infrastructure,embedded,toolchain) Developer
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [gentoo-dev] Re: emerge suggestions
2004-09-09 0:03 ` Ned Ludd
@ 2004-09-09 1:01 ` Heiko Wundram
2004-09-09 13:16 ` Chris Gianelloni
1 sibling, 0 replies; 16+ messages in thread
From: Heiko Wundram @ 2004-09-09 1:01 UTC (permalink / raw
To: gentoo-dev
Am Donnerstag, 9. September 2004 02:03 schrieb Ned Ludd:
> To complex and still wont account for various USE= & FEATURES= flags,
> gcc versions and loads on said server that's building XYZ. It's
> therefore sorta a waste of all of our time and bandwidth to continue
> this thread when we already know we are going to all come to the same
> conclusion in the end that said functionality will never be accurate.
The part about USE flags certainly is true, but still, I guess it wouldn't be
all that hard to factor out the amount of bash units each use flag gives to
the total amount of bash units for the package (simple, compile package
without X, compile with X, subtract first from second, you have the amount
that X adds to the package). And, calculating the bash units a program takes
to compile only needs to be done once, so I don't think this is all the big
hassle that you make out of it.
What, IMO, makes results rather unpredictable is the problem of using a lowest
common denominator machine to create the measurements for bash units, as the
main thing that has to be taken into account is not the speed of the
processor, but rather the memory usage of the compile, as compilation starts
to drop sharply in speed when the compiler has to swap (I've experienced this
more than once with PyQt, which takes longer to compile on an Athlon XP 2400+
with 256MB of RAM than on an Athlon 1300 with 1024MB of RAM, with otherwise
identical systems from the software side of view).
Anyway, I just wanted to tell the op that there were proposals to calculate
the time, I didn't want to start a discussion about the methods... And I
basically don't care if portage tells me how long a package takes to merge.
Heiko.
--
gentoo-dev@gentoo.org mailing list
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [gentoo-dev] Re: emerge suggestions
2004-09-09 0:03 ` Ned Ludd
2004-09-09 1:01 ` Heiko Wundram
@ 2004-09-09 13:16 ` Chris Gianelloni
2004-09-09 16:32 ` Danny van Dyk
1 sibling, 1 reply; 16+ messages in thread
From: Chris Gianelloni @ 2004-09-09 13:16 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 909 bytes --]
On Wed, 2004-09-08 at 20:03, Ned Ludd wrote:
> To complex and still wont account for various USE= & FEATURES= flags,
> gcc versions and loads on said server that's building XYZ. It's
> therefore sorta a waste of all of our time and bandwidth to continue
> this thread when we already know we are going to all come to the same
> conclusion in the end that said functionality will never be accurate.
I don't think the point was ever to be accurate, but to give a rough
idea. If OpenOffice.org is 1047 bash units, and bash takes 1.0 minutes
on your machine, then you know *about* how long OpenOffice.org will take
to compile. I don't think anyone is concerned with exact numbers so
much as ballpark figures to give them a reasonable estimate of needed
time.
--
Chris Gianelloni
Release Engineering - Operations/QA Manager
Games - Developer
Gentoo Linux
Is your power animal a penguin?
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [gentoo-dev] Re: emerge suggestions
2004-09-09 13:16 ` Chris Gianelloni
@ 2004-09-09 16:32 ` Danny van Dyk
2004-09-10 9:29 ` Marcus D. Hanwell
0 siblings, 1 reply; 16+ messages in thread
From: Danny van Dyk @ 2004-09-09 16:32 UTC (permalink / raw
To: gentoo-dev
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
| I don't think the point was ever to be accurate, but to give a rough
| idea. If OpenOffice.org is 1047 bash units, and bash takes 1.0 minutes
| on your machine, then you know *about* how long OpenOffice.org will take
| to compile. I don't think anyone is concerned with exact numbers so
| much as ballpark figures to give them a reasonable estimate of needed
| time.
|
I don't share your optimism. I already see bug reports about wrong ETAs
... :-/
- --
Danny van Dyk
Gentoo/AMD64 Developer
kugelfang@gentoo.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFBQIWsaVNL8NrtU6IRAvAaAJ0bvAgtGActc0e+OVgqUNJbvmb1RwCfQbmj
jTuggfceFM21bYOnlwcVaD0=
=C2Jy
-----END PGP SIGNATURE-----
--
gentoo-dev@gentoo.org mailing list
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [gentoo-dev] Re: emerge suggestions
2004-09-09 16:32 ` Danny van Dyk
@ 2004-09-10 9:29 ` Marcus D. Hanwell
0 siblings, 0 replies; 16+ messages in thread
From: Marcus D. Hanwell @ 2004-09-10 9:29 UTC (permalink / raw
To: gentoo-dev
Danny van Dyk wrote:
> | I don't think the point was ever to be accurate, but to give a rough
> | idea. If OpenOffice.org is 1047 bash units, and bash takes 1.0 minutes
> | on your machine, then you know *about* how long OpenOffice.org will take
> | to compile. I don't think anyone is concerned with exact numbers so
> | much as ballpark figures to give them a reasonable estimate of needed
> | time.
> |
> I don't share your optimism. I already see bug reports about wrong ETAs
> ... :-/
Surely these can be dismissed as WONTFIX, and move on. There will always
be some users who expect everything to be perfect. Round up to the
nearest minute, output Estimated/Approximate/Expected emerge time: 12
minutes. This would be a great feature, and most users will not care if
it takes 15 minutes when it said 12 - it's just a lot better to have a
reasonable idea of emerge times.
If it was within 10 - 20% most of the time that is still useful to most,
and I think it will do better than that most of the time. Even if we
could make an exact estimate of emerge time issues such as other
processes using processor time etc will always throw these estimates off.
--
gentoo-dev@gentoo.org mailing list
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [gentoo-dev] Re: emerge suggestions
2004-09-08 23:46 ` Heiko Wundram
2004-09-09 0:03 ` Ned Ludd
@ 2004-09-09 0:19 ` Thomas de Grenier de Latour
2004-09-09 1:01 ` Daniel Goller
2004-09-09 13:13 ` Chris Gianelloni
2 siblings, 1 reply; 16+ messages in thread
From: Thomas de Grenier de Latour @ 2004-09-09 0:19 UTC (permalink / raw
To: gentoo-dev
On Thu, 9 Sep 2004 01:46:45 +0200
Heiko Wundram <heikowu@ceosg.de> wrote:
> I don't know when this proposal came up, I read about it on some
> forum, some time ago.
That's also an idea that I've seen several times (here and in
forums). I've found this in archives:
http://article.gmane.org/gmane.linux.gentoo.devel/10528
There are in this thread some valid points (USE flags mainly) that
show how difficult and time consuming this idea would be to
actually implement.
--
TGL.
--
gentoo-dev@gentoo.org mailing list
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [gentoo-dev] Re: emerge suggestions
2004-09-09 0:19 ` Thomas de Grenier de Latour
@ 2004-09-09 1:01 ` Daniel Goller
2004-09-08 20:10 ` Chris White
0 siblings, 1 reply; 16+ messages in thread
From: Daniel Goller @ 2004-09-09 1:01 UTC (permalink / raw
To: Thomas de Grenier de Latour; +Cc: gentoo-dev
rather than --pretend, why not use simply 'genlop -t gcc' and see how
much time gcc TOOK to compile
s/gcc/any_package/
isn't that and the pipe with --pretend more than enough info already?
Thomas de Grenier de Latour wrote:
>On Thu, 9 Sep 2004 01:46:45 +0200
>Heiko Wundram <heikowu@ceosg.de> wrote:
>
>
>
>>I don't know when this proposal came up, I read about it on some
>>forum, some time ago.
>>
>>
>
>That's also an idea that I've seen several times (here and in
>forums). I've found this in archives:
>http://article.gmane.org/gmane.linux.gentoo.devel/10528
>There are in this thread some valid points (USE flags mainly) that
>show how difficult and time consuming this idea would be to
>actually implement.
>
>
>
--
gentoo-dev@gentoo.org mailing list
^ permalink raw reply [flat|nested] 16+ messages in thread
* [gentoo-dev] Re: emerge suggestions
2004-09-09 1:01 ` Daniel Goller
@ 2004-09-08 20:10 ` Chris White
2004-09-09 5:06 ` Alin Nastac
0 siblings, 1 reply; 16+ messages in thread
From: Chris White @ 2004-09-08 20:10 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 1059 bytes --]
I think we're failing to see the point. As solar had already stated,
there is no accuracy in this program. The core fundamentals of said
program deals with information processing, or taking raw data (bash
units?) and processing them to format information usable to someome.
While some programs are fairly usable in estimates, it still lacks one
of the core fundamentals of information processing, which is acuracy.
Without this, the program would simply spew random data with no regards
to any sort of useful correlation. In THEORY, it's a good idea, but the
implementation would not go through due to the amount of variable
fluxations in the data presented. Genlops usefullness in data processing
lies in the non-variable data that is provided to it (log files
containing times of ALREADY merged packages, something that cannot be
changed since the action has already occured). I hope this clears things
up. My 4 cents(r).
--
Chris White <chriswhite@gentoo.org>
------------------------
Sound | Video | Security
ChrisWhite @ irc.freenode.net
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [gentoo-dev] Re: emerge suggestions
2004-09-08 23:46 ` Heiko Wundram
2004-09-09 0:03 ` Ned Ludd
2004-09-09 0:19 ` Thomas de Grenier de Latour
@ 2004-09-09 13:13 ` Chris Gianelloni
2 siblings, 0 replies; 16+ messages in thread
From: Chris Gianelloni @ 2004-09-09 13:13 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 1862 bytes --]
On Wed, 2004-09-08 at 19:46, Heiko Wundram wrote:
> Am Mittwoch, 8. September 2004 18:43 schrieb Chris White:
> > How are you going to effectively measure the times?
>
> IIRC, there once was a proposal to do this using bash-units. Each product in
> the tree gets assigned a bash unit, which is a floating point number >0 which
> measures how long compilation takes relative to compiling some certain
> version of bash.
Let's assume we'll just use bash-2.05 (or 3.0, doesn't matter).
> Now, all that needs to be done is to measure package compilation and merging
> time, divide by the number of bash units this package has, and you get an
> estimate on the time for a bash unit on this computer. The more packages you
> merge, the finer this number will become by simply averaging it out. Of
> course, this does not take into account changing the LDFLAGS (which should
> make up for the biggest part of different merge times), or CFLAGS (which
> might also change timing by varying optimization levels and swap
> requirement). But, anyway, these numbers don't change anything about the
> underlying unit, which should be to a large extent platform and machine
> independent.
That's pretty much accurate. Another thing that could be done is there
could be a way to "calibrate" the system... essentially, performing a
bash build using the current {C,CXX,LD}FLAGS to get an accurate time.
This measurement could then be used by portage in giving time estimates.
> I don't know when this proposal came up, I read about it on some forum, some
> time ago.
I know that I mentioned it a while back, but it had been said before
that, so I'm not taking credit (or blame ;p) for it.
--
Chris Gianelloni
Release Engineering - Operations/QA Manager
Games - Developer
Gentoo Linux
Is your power animal a penguin?
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread