[gentoo-dev] Last rites: www-client/chromium-bin

[gentoo-dev] Last rites: www-client/chromium-bin

["Paweł Hajdan, Jr."]

[-- Attachment #1: Type: text/plain, Size: 646 bytes --]

# Pawel Hajdan jr <phajdan.jr@gentoo.org> (04 Mar 2011)
# Masked for removal in 90 days.
# Multiple hard to fix and time consuming maintenance problems:
#  - history of bugs (bug #304527, bug #335101, bug #347175,
#    bug #349249, bug #356609)
#  - upstream's binary builds cannot be used on Gentoo
#    as easily as other -bin packages (ubuntu-specific
#    library names that require compatibility symlinks,
#    bundling libraries, mirror/redistribution policy, and so on)
#  - dependencies for the -bin package are harder to manage;
#    often we have source compatibility, but not binary compatibility
www-client/chromium-bin


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: [gentoo-dev] Last rites: www-client/chromium-bin

[Tomáš Chvátal]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dne 4.3.2011 10:35, "Paweł Hajdan, Jr." napsal(a):
> # Pawel Hajdan jr <phajdan.jr@gentoo.org> (04 Mar 2011)
> # Masked for removal in 90 days.
> # Multiple hard to fix and time consuming maintenance problems:
> #  - history of bugs (bug #304527, bug #335101, bug #347175,
> #    bug #349249, bug #356609)
> #  - upstream's binary builds cannot be used on Gentoo
> #    as easily as other -bin packages (ubuntu-specific
> #    library names that require compatibility symlinks,
> #    bundling libraries, mirror/redistribution policy, and so on)
> #  - dependencies for the -bin package are harder to manage;
> #    often we have source compatibility, but not binary compatibility
> www-client/chromium-bin
> 
Well I can't afford to compile it for 3 hours. Could you at least make
it compile against system webkit?
That would save so much time i would not complain :)

Tom
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk1xdn8ACgkQHB6c3gNBRYfVAACggDqC1IBOLzMfiHBclXf3e9qH
wXQAn0AISAPglT2ZBr5KfTEcSQ+rh3Nk
=p81E
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Last rites: www-client/chromium-bin

[Alex Alexander]

[-- Attachment #1: Type: text/plain, Size: 1291 bytes --]

On Sat, Mar 05, 2011 at 12:32:15AM +0100, Tomáš Chvátal wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Dne 4.3.2011 10:35, "Paweł Hajdan, Jr." napsal(a):
> > # Pawel Hajdan jr <phajdan.jr@gentoo.org> (04 Mar 2011)
> > # Masked for removal in 90 days.
> > # Multiple hard to fix and time consuming maintenance problems:
> > #  - history of bugs (bug #304527, bug #335101, bug #347175,
> > #    bug #349249, bug #356609)
> > #  - upstream's binary builds cannot be used on Gentoo
> > #    as easily as other -bin packages (ubuntu-specific
> > #    library names that require compatibility symlinks,
> > #    bundling libraries, mirror/redistribution policy, and so on)
> > #  - dependencies for the -bin package are harder to manage;
> > #    often we have source compatibility, but not binary compatibility
> > www-client/chromium-bin
> > 
> Well I can't afford to compile it for 3 hours. Could you at least make
> it compile against system webkit?
> That would save so much time i would not complain :)

What system webkit? Chromium has its own version of webkit :)

Anyway, compilation on a modern system shouldn't take more than an
hour. ~15-20 minutes on a quad i5.
-- 
Alex Alexander | wired
+ Gentoo Linux Developer
++ www.linuxized.com

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Last rites: www-client/chromium-bin

[Tomáš Chvátal]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dne 5.3.2011 00:46, Alex Alexander napsal(a):
> 
> What system webkit? Chromium has its own version of webkit :)
For this lovely thing it should be webkit-gtk
> 
> Anyway, compilation on a modern system shouldn't take more than an
> hour. ~15-20 minutes on a quad i5.
Do I look like I *censored* money? How am I supposed to build such system :D
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk1xevsACgkQHB6c3gNBRYfIfACeOfqKMRHaqq8idg0acABT6x3+
DkEAniY5pfmKqS7T9U4jsd1dJTwx+3Up
=lYQC
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Last rites: www-client/chromium-bin

[Alex Alexander]

[-- Attachment #1: Type: text/plain, Size: 784 bytes --]

On Sat, Mar 05, 2011 at 12:51:23AM +0100, Tomáš Chvátal wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Dne 5.3.2011 00:46, Alex Alexander napsal(a):
> > 
> > What system webkit? Chromium has its own version of webkit :)
> For this lovely thing it should be webkit-gtk
> > 
> > Anyway, compilation on a modern system shouldn't take more than an
> > hour. ~15-20 minutes on a quad i5.
> Do I look like I *censored* money? How am I supposed to build such system :D

heh. then you should stick to stable releases and compile less
frequently, the binary package is too painful to maintain [also known as
a demotivator].

I can also give you a binpkg from one of my chroots :P
-- 
Alex Alexander | wired
+ Gentoo Linux Developer
++ www.linuxized.com

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Last rites: www-client/chromium-bin

["Paweł Hajdan, Jr."]

[-- Attachment #1: Type: text/plain, Size: 1313 bytes --]

On 3/5/11 12:51 AM, Tomáš Chvátal wrote:
> Dne 5.3.2011 00:46, Alex Alexander napsal(a):
>> Chromium has its own version of webkit :)
> For this lovely thing it should be webkit-gtk

Actually, Alex is right. Chromium has its own port of WebKit, different
from Qt and GTK ports. Reasons for that include sandboxing, process
separation, and little rendering differences (for example using the skia
library).

This might be possible to change in the future (a separate library to
possibly avoid recompilations), but for now I'm not aware of any effort
to do that (it's really non-trivial).

On 3/5/11 3:41 AM, Rich Freeman wrote:
> Still, the build has gotten faster with time as the excellent g.o
> chromium team slowly strips out bundled libs.  If you want a real
> eye-opener do a du -s * in the source tree for chromium and see where
> all the code is.

Thanks! Anyway, I guess that the WebKit part of the browser still takes
the majority of the compile time.

On 3/5/11 12:58 AM, Alex Alexander wrote:
> I can also give you a binpkg from one of my chroots :P

It sounds like a possible option. We could then advertise those binpkgs
on the project page, or make them semi-official. If you or someone else
is interested in doing that, I second that effort.

Paweł Hajdan, Jr.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

[gentoo-dev] Re: Last rites: www-client/chromium-bin

[Duncan]

Paweł Hajdan, Jr. posted on Sat, 05 Mar 2011 10:23:34 +0100 as excerpted:

> On 3/5/11 12:58 AM, Alex Alexander wrote:
>> I can also give you a binpkg from one of my chroots :P
> 
> It sounds like a possible option. We could then advertise those binpkgs
> on the project page, or make them semi-official. If you or someone else
> is interested in doing that, I second that effort.

What about handling chromium-bin the same way amd64 handles grub-static?  
They create a standard binpkg of the normal grub ebuild (using 
standardized USE flags, of course), using that as the source tarball for 
the grub-static ebuild, which then simply ebuild-scripts the unpack and 
install of the binpkg tarball.

In theory, the ebuild could even grab and merge the appropriate binpkg 
based on arch, allowing the ebuild to be keyworded for more archs than the 
usual binary-only ebuild tends to be, altho I'm not sure that'd work in 
practice as I'm unsure of the effects on the metadata cache and whether 
that would be allowed or not.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: Last rites: www-client/chromium-bin

["Paweł Hajdan, Jr."]

[-- Attachment #1: Type: text/plain, Size: 1076 bytes --]

On 3/5/11 11:05 AM, Duncan wrote:
> What about handling chromium-bin the same way amd64 handles grub-static?  
> They create a standard binpkg of the normal grub ebuild (using 
> standardized USE flags, of course), using that as the source tarball for 
> the grub-static ebuild, which then simply ebuild-scripts the unpack and 
> install of the binpkg tarball.

That's the chromium-bin, really. The difference is that chromium has
more deps and takes more time to compile than grub. Also, it has much
more frequent releases, and almost every stable release is a security
update.

> In theory, the ebuild could even grab and merge the appropriate binpkg 
> based on arch, allowing the ebuild to be keyworded for more archs than the 
> usual binary-only ebuild tends to be, altho I'm not sure that'd work in 
> practice as I'm unsure of the effects on the metadata cache and whether 
> that would be allowed or not.

Yeah, I can understand how it could work technically. The only issue is
to make the version bump one automated step.

Paweł Hajdan, Jr.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: [gentoo-dev] Re: Last rites: www-client/chromium-bin

[James Cloos]

>>>>> "PH" == Paweł Hajdan, <phajdan.jr@gentoo.org> writes:

PH> That's the chromium-bin, really. The difference is that chromium has
PH> more deps and takes more time to compile than grub. Also, it has much
PH> more frequent releases, and almost every stable release is a security
PH> update.

And every one of those chromium releases is another 200 Meg download.

Or is that yet another?

Still yet another?

-JimC  (who still prefers the src)
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: [gentoo-dev] Re: Last rites: www-client/chromium-bin

[Mike Gilbert]

On Thu, Mar 10, 2011 at 3:04 PM, James Cloos <cloos@jhcloos.com> wrote:
>>>>>> "PH" == Paweł Hajdan, <phajdan.jr@gentoo.org> writes:
>
> PH> That's the chromium-bin, really. The difference is that chromium has
> PH> more deps and takes more time to compile than grub. Also, it has much
> PH> more frequent releases, and almost every stable release is a security
> PH> update.
>
> And every one of those chromium releases is another 200 Meg download.
>
> Or is that yet another?
>
> Still yet another?

Chromium tarballs are actually around 140 MB. It would be interesting
to see if we can trim that tarball down.

For comparison, Firefox weighs in at about 50 MB.

Re: [gentoo-dev] Re: Last rites: www-client/chromium-bin

["Paweł Hajdan, Jr."]

[-- Attachment #1: Type: text/plain, Size: 424 bytes --]

On 3/10/11 9:33 PM, Mike Gilbert wrote:
> Chromium tarballs are actually around 140 MB. It would be interesting
> to see if we can trim that tarball down.

Oh yes, we can. I guess the biggest problem is testing, but we can
certainly remove more from the tarball.

If anyone's interested, it's src/tools/export_tarball/export_tarball.py
in the Chromium source tree. I can review patches. :-D

Paweł Hajdan, Jr.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: [gentoo-dev] Re: Last rites: www-client/chromium-bin

[James Cloos]

>>>>> "MG" == Mike Gilbert <floppymaster@gmail.com> writes:

MG> Chromium tarballs are actually around 140 MB. It would be
MG> interesting to see if we can trim that tarball down.

Woops.  Misremembered.  It is qt that is over 200 MB.

On the plus side it is going down.  Chromium 5 was ~160 MB.

MG> For comparison, Firefox weighs in at about 50 MB.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: [gentoo-dev] Last rites: www-client/chromium-bin

[Antoni Grzymala]

[-- Attachment #1: Type: text/plain, Size: 1376 bytes --]

Alex Alexander dixit (2011-03-05, 01:46):

> On Sat, Mar 05, 2011 at 12:32:15AM +0100, Tomáš Chvátal wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> > 
> > Dne 4.3.2011 10:35, "Paweł Hajdan, Jr." napsal(a):
> > > # Pawel Hajdan jr <phajdan.jr@gentoo.org> (04 Mar 2011)
> > > # Masked for removal in 90 days.
> > > # Multiple hard to fix and time consuming maintenance problems:
> > > #  - history of bugs (bug #304527, bug #335101, bug #347175,
> > > #    bug #349249, bug #356609)
> > > #  - upstream's binary builds cannot be used on Gentoo
> > > #    as easily as other -bin packages (ubuntu-specific
> > > #    library names that require compatibility symlinks,
> > > #    bundling libraries, mirror/redistribution policy, and so on)
> > > #  - dependencies for the -bin package are harder to manage;
> > > #    often we have source compatibility, but not binary compatibility
> > > www-client/chromium-bin
> > > 
> > Well I can't afford to compile it for 3 hours. Could you at least make
> > it compile against system webkit?
> > That would save so much time i would not complain :)
> 
> What system webkit? Chromium has its own version of webkit :)
> 
> Anyway, compilation on a modern system shouldn't take more than an
> hour. ~15-20 minutes on a quad i5.

Takes ~32 minutes on my i7 (dual core @2.67 ghz).

-- 
[a]

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Last rites: www-client/chromium-bin

[Rich Freeman]

On Fri, Mar 4, 2011 at 6:46 PM, Alex Alexander <wired@gentoo.org> wrote:
> Anyway, compilation on a modern system shouldn't take more than an
> hour. ~15-20 minutes on a quad i5.

Clearly your definition of modern doesn't include my server...  :)
Just checked and the last build clocked in at 192 minutes.  I need to
make sure I have /var/tmp/portage symlinked back to a non-tmpfs
location whenever I build it or else the system pretty-much dies from
a lack of RAM.

For kicks I tried to do better with distcc and EC2.  That worked great
until it started running a bunch of python scripts in the makefile -
at -j15 or whatever I had it set to.  Distcc really needs a solution
for the fact that you can't pick a single optimum value for -j when
gcc is only part of the build.  Sure, the EC2 latency isn't great, but
you can parallelize as much as you want, and for me the bigger benefit
is not sucking down half a gig of RAM.

Still, the build has gotten faster with time as the excellent g.o
chromium team slowly strips out bundled libs.  If you want a real
eye-opener do a du -s * in the source tree for chromium and see where
all the code is.

Re: [gentoo-dev] Last rites: www-client/chromium-bin

[dev-random]

On Fri, Mar 04, 2011 at 09:41:30PM -0500, Rich Freeman wrote:
<...>
> For kicks I tried to do better with distcc and EC2.  That worked great
> until it started running a bunch of python scripts in the makefile -
> at -j15 or whatever I had it set to.  Distcc really needs a solution
> for the fact that you can't pick a single optimum value for -j when
> gcc is only part of the build.
<...>

from 'man make':
       -l [load], --load-average[=load]
          Specifies that no new jobs (commands) should be  started if
	  there are others jobs running and the load average is at
	  least load (a floating-point number). With no argument,
	  removes a previous load limit.

Re: [gentoo-dev] Last rites: www-client/chromium-bin

[Jeroen Roovers]

On Fri, 4 Mar 2011 21:41:30 -0500
Rich Freeman <rich0@gentoo.org> wrote:

> and for me the bigger benefit is not sucking down half a gig of RAM.

Oh, that's exactly what is catching more and more users out: Hey I got
8 cores so I can run MAKEOPTS=-j8! What happens? With only 6GB RAM, and
each of those eight cc1plus jobs picking more than half a gigabyte,
jobs get OOM killed. Please volunteer on bug-wranglers@. :)



     jer

[gentoo-dev] Re: Last rites: www-client/chromium-bin

[Nikos Chantziaras]

On 03/05/2011 04:41 AM, Rich Freeman wrote:
> On Fri, Mar 4, 2011 at 6:46 PM, Alex Alexander<wired@gentoo.org>  wrote:
>> Anyway, compilation on a modern system shouldn't take more than an
>> hour. ~15-20 minutes on a quad i5.
>
> Clearly your definition of modern doesn't include my server...  :)
> Just checked and the last build clocked in at 192 minutes.  I need to
> make sure I have /var/tmp/portage symlinked back to a non-tmpfs
> location whenever I build it or else the system pretty-much dies from
> a lack of RAM.

Then I'd say you have your tmpfs mount options configured wrongly :-) 
You should specify a maximum amount of RAM it should use.  When it fill 
up, it then uses swap.

[gentoo-dev] Re: Last rites: www-client/chromium-bin

[Nikos Chantziaras]

On 03/05/2011 12:00 PM, Nikos Chantziaras wrote:
> On 03/05/2011 04:41 AM, Rich Freeman wrote:
>> On Fri, Mar 4, 2011 at 6:46 PM, Alex Alexander<wired@gentoo.org> wrote:
>>> Anyway, compilation on a modern system shouldn't take more than an
>>> hour. ~15-20 minutes on a quad i5.
>>
>> Clearly your definition of modern doesn't include my server... :)
>> Just checked and the last build clocked in at 192 minutes. I need to
>> make sure I have /var/tmp/portage symlinked back to a non-tmpfs
>> location whenever I build it or else the system pretty-much dies from
>> a lack of RAM.
>
> Then I'd say you have your tmpfs mount options configured wrongly :-)
> You should specify a maximum amount of RAM it should use. When it fill
> up, it then uses swap.

I take that back.  It seems you can't do that with tmpfs :-(

Re: [gentoo-dev] Re: Last rites: www-client/chromium-bin

[Dale]

Nikos Chantziaras wrote:
> On 03/05/2011 12:00 PM, Nikos Chantziaras wrote:
>> On 03/05/2011 04:41 AM, Rich Freeman wrote:
>>> On Fri, Mar 4, 2011 at 6:46 PM, Alex Alexander<wired@gentoo.org> wrote:
>>>> Anyway, compilation on a modern system shouldn't take more than an
>>>> hour. ~15-20 minutes on a quad i5.
>>>
>>> Clearly your definition of modern doesn't include my server... :)
>>> Just checked and the last build clocked in at 192 minutes. I need to
>>> make sure I have /var/tmp/portage symlinked back to a non-tmpfs
>>> location whenever I build it or else the system pretty-much dies from
>>> a lack of RAM.
>>
>> Then I'd say you have your tmpfs mount options configured wrongly :-)
>> You should specify a maximum amount of RAM it should use. When it fill
>> up, it then uses swap.
>
> I take that back.  It seems you can't do that with tmpfs :-(
>
>

It seems you correct the first time.

http://en.wikipedia.org/wiki/Tmpfs

I found the same examples in other paces as well.  One is in the mount 
man page.

Dale

:-)  :-)

Re: [gentoo-dev] Re: Last rites: www-client/chromium-bin

[Rich Freeman]

On Sat, Mar 5, 2011 at 5:21 AM, Dale <rdalek1967@gmail.com> wrote:
> It seems you correct the first time.
>
> http://en.wikipedia.org/wiki/Tmpfs
>
> I found the same examples in other paces as well.  One is in the mount man
> page.

While this is drifting off-topic this is not the case.  You can limit
the size of a tmpfs, but you can't limit the amount of physical RAM
that it uses (other than its size limit).  At least, not using
anything in either of those sources.

When building chromium limiting the tmpfs size isn't a very practical
solution... At least, not until that contrib directory doesn't have
half a gig of stuff already on my system BEFORE I compile it.

Re: [gentoo-dev] Re: Last rites: www-client/chromium-bin

[Rich Freeman]

On Sat, Mar 5, 2011 at 5:00 AM, Nikos Chantziaras <realnc@arcor.de> wrote:
> On 03/05/2011 04:41 AM, Rich Freeman wrote:
>> I need to
>> make sure I have /var/tmp/portage symlinked back to a non-tmpfs
>> location whenever I build it or else the system pretty-much dies from
>> a lack of RAM.
>
> Then I'd say you have your tmpfs mount options configured wrongly :-) You
> should specify a maximum amount of RAM it should use.  When it fill up, it
> then uses swap.
>

So, your later redaction aside, I should say that the kernel devs have
their swap behavior wrong in any case.

Think about it - if you write to a disk-based filesystem every byte
that you write ends up on disk, subject to cache (with little
discretion - all data needs to be written out within 30 seconds or
so).

If you have very little RAM free the tmpfs should really be no worse
since there too every byte that gets written gets swapped to disk.
However, the kernel has complete discretion about when this happens,
whether this happens at all, and it gets to write it in a partition
optimized for this purpose without regard for long-term
preservation/etc.  Yet, for whatever reason all this swapping seems
slower.

The likely reason is that the kernel isn't that good at figuring out
what to swap and what not to swap.  It might be prioritizing keeping
some object files around in RAM on a tmpfs while trying to swap in/out
some other process that keeps getting woken up or whatever.

Better swap behavior should let the tmpfs perform better than
ext4/etc, since it gives the kernel more discretion about what to
write out and when.