[gentoo-amd64] Emerging package as both 64 and 32 bit

[gentoo-amd64] Emerging package as both 64 and 32 bit

[Boky Boky]

Hi all.

I'm sure this question must have been answered before, but I am slowly
giving up on finding the solution after months of browsing.


Anyways, is it possible to emerge the same package in parallel? As
32-bit and 64-bit version?

Something in the lines of:
ABI="amd64,x86" emerge libxml2

or:
emerge libxml2
ABI="x86" emerge
--some-option-to-ignore-already-installed-package-and-not-remove-it
libxml2


Namely, I am trying to compile openoffice (using ABI="x86") and some GTK themes.

openoffice depends one some packages (libxml2, libz, ...) that need
therefore need to exist in 32-bit and I'd like to have firefox (...and
other 32bit apps) themed not just with themes that come with emul-*
packages.


Thank you for your replies,
Boky
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Emerging package as both 64 and 32 bit

[Indarios]

[-- Attachment #1: Type: text/plain, Size: 1059 bytes --]

No, you can not run 32 bit code in a 64 bit environment. You have to use
emulated software, check out net-www/nspluginwrapper to get plugins working
for Firefox

On 12/16/06, Boky Boky <verynotbad@gmail.com> wrote:
>
> Hi all.
>
> I'm sure this question must have been answered before, but I am slowly
> giving up on finding the solution after months of browsing.
>
>
> Anyways, is it possible to emerge the same package in parallel? As
> 32-bit and 64-bit version?
>
> Something in the lines of:
> ABI="amd64,x86" emerge libxml2
>
> or:
> emerge libxml2
> ABI="x86" emerge
> --some-option-to-ignore-already-installed-package-and-not-remove-it
> libxml2
>
>
> Namely, I am trying to compile openoffice (using ABI="x86") and some GTK
> themes.
>
> openoffice depends one some packages (libxml2, libz, ...) that need
> therefore need to exist in 32-bit and I'd like to have firefox (...and
> other 32bit apps) themed not just with themes that come with emul-*
> packages.
>
>
> Thank you for your replies,
> Boky
> --
> gentoo-amd64@gentoo.org mailing list
>
>

[-- Attachment #2: Type: text/html, Size: 1453 bytes --]

Re: [gentoo-amd64] Emerging package as both 64 and 32 bit

[Mike Doty]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Boky Boky wrote:
> Hi all.
> 
> I'm sure this question must have been answered before, but I am slowly
> giving up on finding the solution after months of browsing.
> 
> 
> Anyways, is it possible to emerge the same package in parallel? As
> 32-bit and 64-bit version?
> 
> Something in the lines of:
> ABI="amd64,x86" emerge libxml2
> 
> or:
> emerge libxml2
> ABI="x86" emerge
> --some-option-to-ignore-already-installed-package-and-not-remove-it
> libxml2
> 
> 
> Namely, I am trying to compile openoffice (using ABI="x86") and some GTK
> themes.
> 
> openoffice depends one some packages (libxml2, libz, ...) that need
> therefore need to exist in 32-bit and I'd like to have firefox (...and
> other 32bit apps) themed not just with themes that come with emul-*
> packages.
> 
> 
> Thank you for your replies,
> Boky
This is a limitation of portage, one which everyone on the amd64 team
wishes would go away.  the best you could do now with portage would be
to make emul- packages for your deps or install manually.  Expect a lot
of effort and mess either way you choose.

- --
=======================================================
Mike Doty                      kingtaco -at- gentoo.org
Gentoo/AMD64 Strategic Lead
Gentoo Council
Gentoo Developer Relations
Gentoo Recruitment Lead
Gentoo Infrastructure
GPG: E1A5 1C9C 93FE F430 C1D6  F2AF 806B A2E4 19F4 AE05
=======================================================
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iQCVAwUBRYQ3tIBrouQZ9K4FAQKZuAQAunAS3iqzFTJfAbs+J8JXp3xAjDVM20G/
d22lPVoH6MOpEamm5pTl/sUNL/1w9/XskIjKcq7GwKffwtDdYk6d+LlIH9x1iKSM
cIb+er2tRL2L6YGiqMIcLf4Bo4Fq9tYLTM4n4v1fHB2XipE5YZFPJkR3QvE9whyO
1gySV2893nE=
=X1Y0
-----END PGP SIGNATURE-----
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Emerging package as both 64 and 32 bit

[Duncan]

"Boky Boky" <verynotbad@gmail.com> posted
e8666cf90612160745k19aa5622ueb3c7e84295285bc@mail.gmail.com, excerpted
below, on  Sat, 16 Dec 2006 16:45:24 +0100:

> Anyways, is it possible to emerge the same package in parallel? As
> 32-bit and 64-bit version?
> 
> Something in the lines of:
> ABI="amd64,x86" emerge libxml2
> 
> or:
> emerge libxml2
> ABI="x86" emerge
> --some-option-to-ignore-already-installed-package-and-not-remove-it
> libxml2

> Namely, I am trying to compile openoffice (using ABI="x86") and some GTK
> themes.
> 
> openoffice depends one some packages (libxml2, libz, ...) that need
> therefore need to exist in 32-bit and I'd like to have firefox (...and
> other 32bit apps) themed not just with themes that come with emul-*
> packages.

You can do what you are trying to do, but it's more work than it might
first appear.

The problem is that portage isn't multi-ABI aware.  As far as it is
concerned, if a dependency is merged for one or the other, it's merged,
period.  If you merge a second ABI, it will erase what it knows of the
first.

There are as a result very specific cautions against trying to merge both
ABIs using the same portage database.  It can and likely WILL screw up
your system BIG TIME, because portage won't have the foggiest which end is
up or down in terms of what's installed for what bitness, once you start
to mix them.

As I said, however, there IS a way to do it -- actually two, but in
general only one solution of the two will fit your needs.

The first way works best if you only have a few second-ABI things you wish
to install, using the emul- or -bin stuff for the rest.  Basically, this
method consists of installing everything "extra" that you need for the
second ABI manually -- without using portage directly.  You get the
tarball, unpack it, verify that you have the dependencies it needs in the
desired ABI (you either have them as -emul/-bin or you already did
this same process with them), apply any desired patches, configure, build,
and install, all manually. You can look at the ebuild script to see if it
does anything special and do the same things manually if you want the same
thing that the ebuild /would/ give you, but you don't do it with portage
so it's not in the portage database and therefore can't screw it up.  The
CFLAGS for 32-bit would include -m32, and when you run configure, you'd
supply the command line options to point it at the 32-bit libraries and
etc.

The second way works best if you have more than a handful of things to
build, because it's a bigger hassle to setup initially, but all the usual
stuff is automated after that.  Basically, you setup a 32-bit chroot that
has most of a 32-bit system -- leaving out stuff like the kernel and
system services of course because you have them in 64-bit mode already,
but starting from a regular x86 stage-whatever and building all the
supporting libraries and other dependencies as 32-bit, using a second
chrooted 32-bit only version of portage, with its own database (you can
point it at the same portage tree, of course), so it's not going to
overwrite your main system 64-bit database.

There's a HOWTO for doing the 32-bit chroot.  I'll link it below.  This is
covered there but sketching it out briefly...  You can make use of the
mount --bind option to remount filesystems or portions of filesystems
(directories) so they are seen in both the original location and in the
chroot.  It's often convenient to mount /home and /tmp, and your portage
tree, this way, for instance, while keeping stuff like /usr separate, so
merges in the chroot have no chance of overwriting your 64-bit stuff
outside the chroot.  Except for maintaining the chroot, for which you'll
always want to go into the chroot to avoid the possibility of overwriting
your main system, it's possible to add the 32-bit paths and library paths
(for ld) to your main paths and run both 32-bit and 64-bit stuff from your
main system outside the chroot.  If you have the same executables merged
in both 32-bit and 64-bit, the order of your path will naturally determine
which one is called if you don't specify the one you want by specifying
the path with the command.

Of course, with the 32-bit chroot, you are building most of a 32-bit
system anyway, so it's not hard (simpler but more building to do, so
take your pick) to go all the way if desired, building the 32-bit kernel
and services, after which you can add a 32-bit boot option to your
grub/lilo boot menu and dual boot, if desired. =8^)

Official Gentoo/amd64 32-bit chroot HOWTO:
http://www.gentoo.org/proj/en/base/amd64/howtos/chroot.xml

For a variation on the 32-bit chroot that involves creating it, then
using it to build your own 32-bit emul- packages that you can merge on
your 64-bit system without screwing up its database, check here.  (This is
NOT official Gentoo, just something a Gentoo/amd64 user put up.  I'm not
sure if it's fully upto date, but the idea will remain the same even if
you have to tweak the specifics a bit.)  It's worth a read, anyway, just
to see how it can be done, whether you choose to do it yourself or not.

Unofficial (User's) Gentoo/amd64 Creating your own 32-bit emul packages
HOWTO: http://www.andyjeffries.co.uk/32bit-ebuild-amd64.html

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Emerging package as both 64 and 32 bit

[Boky]

Duncan <1i5t5.duncan <at> cox.net> writes:

> 
> "Boky" <verynotbad <at> gmail.com> posted
> e8666cf90612160745k19aa5622ueb3c7e84295285bc <at> mail.gmail.com, excerpted
> below, on  Sat, 16 Dec 2006 16:45:24 +0100:
> 
> > Anyways, is it possible to emerge the same package in parallel? As
> > 32-bit and 64-bit version?
> > 
> > Something in the lines of:
> > ABI="amd64,x86" emerge libxml2
> > 
> > or:
> > emerge libxml2
> > ABI="x86" emerge
> > --some-option-to-ignore-already-installed-package-and-not-remove-it
> > libxml2
> 
> 
> For a variation on the 32-bit chroot that involves creating it, then
> using it to build your own 32-bit emul- packages that you can merge on
> your 64-bit system without screwing up its database, check here.  (This is
> NOT official Gentoo, just something a Gentoo/amd64 user put up.  I'm not
> sure if it's fully upto date, but the idea will remain the same even if
> you have to tweak the specifics a bit.)  It's worth a read, anyway, just
> to see how it can be done, whether you choose to do it yourself or not.
> 
> Unofficial (User's) Gentoo/amd64 Creating your own 32-bit emul packages
> HOWTO: http://www.andyjeffries.co.uk/32bit-ebuild-amd64.html
> 

Thank you Duncan.

This is exacly what I was looking for. I have already created a 32-bit chroot,
but wasn't very keen on the idea of issuing 10+ commands (mount -o bind...) to
start openoffice or firefox.

I was experimenting with "emerge -b" but sadly, everything gets installed into
/lib so it's not usable on the main root.

I will give the mentioned link a shoot and tell you how it goes.


Cheers,
B

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Emerging package as both 64 and 32 bit

[Kevin F. Quinn]

[-- Attachment #1: Type: text/plain, Size: 560 bytes --]

On Sat, 16 Dec 2006 18:17:43 +0000 (UTC)
Duncan <1i5t5.duncan@cox.net> wrote:

> As I said, however, there IS a way to do it -- actually two, but in
> general only one solution of the two will fit your needs.

Actually there's three - the third is kanaka's patches to portage to
get it to build all ABIs.  This solves the multilib dependency problem
in a stroke (by eliminating it) - it does mean your build times go up,
of course.

http://bugs.gentoo.org/show_bug.cgi?id=145737
http://dev.gentoo.org/~kanaka/auto-multilib/

-- 
Kevin F. Quinn

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[gentoo-amd64] Re: Emerging package as both 64 and 32 bit

[Duncan]

"Kevin F. Quinn" <kevquinn@gentoo.org> posted
20061216210848.5c36a70a@c1358217.kevquinn.com, excerpted below, on  Sat,
16 Dec 2006 21:08:48 +0100:

> On Sat, 16 Dec 2006 18:17:43 +0000 (UTC) Duncan <1i5t5.duncan@cox.net>
> wrote:
> 
>> As I said, however, there IS a way to do it -- actually two, but in
>> general only one solution of the two will fit your needs.
> 
> Actually there's three - the third is kanaka's patches to portage to get
> it to build all ABIs.  This solves the multilib dependency problem in a
> stroke (by eliminating it) - it does mean your build times go up, of
> course.
> 
> http://bugs.gentoo.org/show_bug.cgi?id=145737
> http://dev.gentoo.org/~kanaka/auto-multilib/

Very cool, thanks!  I'd not see that.

(Hmm... I've been looking forward to upgrading to Opteron 285s, from my
current 242s.  That'd give me some compile-time speed.  I could decide to
do this tho and take it right back away. =8^(  Actually probably not as I
don't do closed source and most open source stuff compiles in 64-bit
native, so there's little reason to do 32-bit at all, here, except to keep
from having to use the grub binary package.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Emerging package as both 64 and 32 bit

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 444 bytes --]

On Sat, 16 Dec 2006 20:41:16 +0000 (UTC), Duncan wrote:

> there's little reason to do 32-bit at all, here, except to keep
> from having to use the grub binary package.

What's wrong with the GRUB source package? I remember using grub-static
when I first went 64 bit, but haven't used it for ages. The only 32 bit
stuff I use here is firefox-bin and vmware.


-- 
Neil Bothwick

I just took an IQ test. The results were negative.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] Re: Emerging package as both 64 and 32 bit

[Simon Stelling]

Neil Bothwick wrote:
> What's wrong with the GRUB source package? I remember using grub-static
> when I first went 64 bit, but haven't used it for ages. The only 32 bit
> stuff I use here is firefox-bin and vmware.

No problem, but it's 32bit.

-- 
Kind Regards,

Simon Stelling
Gentoo/AMD64 Developer
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Emerging package as both 64 and 32 bit

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 217 bytes --]

On Mon, 18 Dec 2006 12:22:24 +0200, Simon Stelling wrote:

> No problem, but it's 32bit.

Well, you live and learn - thanks :)


-- 
Neil Bothwick

Due to inflation, all clouds will now be lined with zinc.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[gentoo-amd64] Re: Emerging package as both 64 and 32 bit

[Duncan]

Simon Stelling <blubb@gentoo.org> posted 45866BE0.20905@gentoo.org,
excerpted below, on  Mon, 18 Dec 2006 12:22:24 +0200:

> Neil Bothwick wrote:
>> What's wrong with the GRUB source package? I remember using grub-static
>> when I first went 64 bit, but haven't used it for ages. The only 32 bit
>> stuff I use here is firefox-bin and vmware.
> 
> No problem, but it's 32bit.

Indeed...  for backward compatibility, amd64/x86_64 boots in 32-bit mode. 
Actually, I /believe/ it boots in 16-bit real mode, just like an x86, then
switches to 32-bit or 64-bit when the appropriate command is given, but
AFAIK the difference between compiling 16-bit and 32-bit code is simply a
few compile-time switches, so it uses a standard 32-bit toolchain.

My point, however, was that since everything else I run is 64-bit, if I
didn't need the 32-bit tools to compile grub -- or if I was willing to
settle for the pre-compiled grub-static -- I could save myself a *LOT* of
extra work by simply using the no-multilib subprofile, thus saving myself
all that time compiling the 32-bit side of glibc and gcc in particular.

One of these days maybe I'll probably just do it, unmerging grub, merging
grub-static, switching to the no-multilib subprofile and remerging
glibc/binutils/gcc (they may have to be remerged in a particular order,
which I don't know at this point).  However, I've been thinking that for
awhile and haven't done it yet, and I'll be upgrading my pair of Opteron
242s to dual-core 285s pretty soon here, making it that much less
necessary since compiles will be rather faster then, so who knows?

Hmm... thinking about it as I write this, something new occurred to me.
There's a good probability I could compile grub independent of my system's
portage, using a LiveCD (either Gentoo or other), and could therefore go
no-multilib without losing my self-compiled grub, if I decided to.  I'll
have to think on it a bit more.  OTOH, simply using grub-static would be
far less hassle for what amounts to the same thing, since using a gcc I
didn't build myself would leave outside influences on the produced grub
anyway.  Still, part of what was holding me up was the "just in case"
factor, for other 32-bit software as well, and now that I realize I could
compile it from a liveCD 32-bit environment or the like if necessary, that
pretty much does away with /that/ particular excuse.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Emerging package as both 64 and 32 bit

[Etaoin Shrdlu]

On Monday 18 December 2006 13:39, Duncan wrote:

> My point, however, was that since everything else I run is 64-bit, if
> I didn't need the 32-bit tools to compile grub -- or if I was willing
> to settle for the pre-compiled grub-static -- I could save myself a
> *LOT* of extra work by simply using the no-multilib subprofile, thus
> saving myself all that time compiling the 32-bit side of glibc and gcc
> in particular.

I believe that lilo can be built in a 64-bit only environment 
(no-multilib). 
I use the no-multilib subprofile, and a few days ago, I found out that 
lilo was not masked (ie, "emerge -a lilo" did prompt me to install the 
package), so I suppose that it should work even in my 64-bit, 
no-multilib system. (However, I did not actually merge it, since 
grub-static still works fine here, so I can't be 100% sure).
Looking at the changelog, it seems that my suppositions are correct:

06 Jan 2006; Olivier Crête <tester@gentoo.org> lilo-22.7.ebuild:
  Stable on amd64

So, it's been stable for almost a year now.

Of course, this is not meant to be the start of another grub vs. lilo 
flamewar, but rather just a FYI.

-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Emerging package as both 64 and 32 bit

[Duncan]

Etaoin Shrdlu <shrdlu@unlimitedmail.org> posted
200612181510.08865.shrdlu@unlimitedmail.org, excerpted below, on  Mon, 18
Dec 2006 15:10:08 +0100:

> On Monday 18 December 2006 13:39, Duncan wrote:
> 
>> My point, however, was that since everything else I run is 64-bit, if
>> I didn't need the 32-bit tools to compile grub -- or if I was willing
>> to settle for the pre-compiled grub-static -- I could save myself a
>> *LOT* of extra work by simply using the no-multilib subprofile, thus
>> saving myself all that time compiling the 32-bit side of glibc and gcc
>> in particular.
> 
> I believe that lilo can be built in a 64-bit only environment 
> (no-multilib). 
> I use the no-multilib subprofile, and a few days ago, I found out that 
> lilo was not masked (ie, "emerge -a lilo" did prompt me to install the 
> package), so I suppose that it should work even in my 64-bit, 
> no-multilib system. (However, I did not actually merge it, since 
> grub-static still works fine here, so I can't be 100% sure).
> Looking at the changelog, it seems that my suppositions are correct:
> 
> 06 Jan 2006; Olivier Crête <tester@gentoo.org> lilo-22.7.ebuild:
>   Stable on amd64
> 
> So, it's been stable for almost a year now.
> 
> Of course, this is not meant to be the start of another grub vs. lilo 
> flamewar, but rather just a FYI.

No flamewar here as I had already learned LILO (but not GRUB) when I
switched to Gentoo, and simply copied over and continued to use the
Mandrake executable (which was at the time from the exact same RPM they
used on i586) for awhile, then was running the precompiled binary from the
LILO homepage for awhile, then the Gentoo/amd64 version when it finally
worked, before I finally decided to get with the program and learn GRUB,
since everybody said it was more flexible -- which it turned out to be. =8^)

However, there's a bit more to the 64-bit LILO case than it first appears,
and than you mention above.  I'll admit it's a bit beyond my understanding
as well, and you'd probably have to get it from the Gentoo devs
responsible for working out the 64-bit stuff on it, but from what I could
make out...

LILO is actually two different pieces, the part that runs when you invoke
it to install a new boot sector config, and the part that actually builds,
that actually runs at boot time.

As I understand it, formerly, the part invoked to do the build and install
of the boot sector bit was a regular 32-bit executable.  Now, it's a
64-bit executable, that includes a 32-bit "microcore" (for lack of a
better word).

That was based on the remarks I read back when the 64-bit version was
introduced.  Now, part of what I don't quite understand is how that all
fits together and whether the "microcore" is a dependency or actually
compiled as part of the LILO compile and install process, or whether it's
similar to the binary blob at the core of the NVidia video drivers (which
I won't run as that blob isn't free software) and the LILO compile process
simply builds a wrapper around that core.  Actually, that was another
reason I eventually switched to GRUB -- I wasn't particularly comfortable
with my lack of understanding of the issues surrounding this microcore and
whether it was a binary blob (which I don't like to and generally won't
run, just as I won't run the NVidia binary blob) or whether it required
the 32-bit parts of GCC, or whether it was "cross-compiled" as it were by
the LILO build-process, or what.  Since GRUB was rumored to be more
flexible anyway, and was more the Gentoo way, I decided it was better to
spend my time learning it than trying to trace down and figure out all the
ins and outs of the LILO thing to my satisfaction, so that's exactly what
I did, and indeed, I AM rather happier with GRUB, now that I actually took
the time to learn how to work with it.

FWIW, now I'm looking at LinuxBIOS, which I've already verified IS known
to work on my mobo.  I'd be rather more comfortable running it, as
entirely freedomware code, than the ordinary vendor supplied proprietary
BIOS I'm running now.  However, that's a big step, and I've got a lot more
research to do and possibly some hardware and tools to buy so I can keep
my existing BIOS as-is while I experiment, before I'll be comfortable or
ready to take that step.  As I understand it now, I can use GRUB as the
payload (making the grub first stage what amounts to a second stage after
the LinuxBIOS), or one of the two options that normally come with
LinuxBIOS, or even load a shrunken Linux kernel with just enough
functionality to read the main kernel off the disk and kexec into it. 
What happens to all the stuff like memory timing options I can configure
in the current proprietary BIOS?  What defaults does it choose and is
there a way to configure them if I don't like those defaults?  It's those
types of questions among others I still have to research -- or get the
hardware to allow me to safely experiment and find out on my own, before
I'm comfortable actually switching to LinuxBIOS.  Still, it might be a
year or two, but I'll probably do it eventually, thus ridding my system of
yet one more proprietary software (firmware) bit.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Emerging package as both 64 and 32 bit

[Etaoin Shrdlu]

On Tuesday 19 December 2006 16:01, Duncan wrote:

> Now, part of what I don't quite understand is how that 
> all fits together and whether the "microcore" is a dependency or
> actually compiled as part of the LILO compile and install process, or
> whether it's similar to the binary blob at the core of the NVidia
> video drivers (which I won't run as that blob isn't free software) and
> the LILO compile process simply builds a wrapper around that core. 

I'm not sure about the "microcore" thing: you could be right, since I 
noticed that many files which used to be in /boot now are gone, and they 
have possibly been linked into the lilo executable. In 
particular, /etc/lilo.conf.example mentions that now boot.b is linked 
directly into the /sbin/lilo binary. So, the LILO executable (the 
program the user runs after installing a new kernel image) might very 
well be a sort of hybrid.
Regarding the binary blob, however, I think your fears are not justified: 
there are no binary files inside the LILO source tarball, so everything 
must be built from sources one way or another. (Compare this with, for 
example, the nvidia-drivers, which instead contains lots of binary 
executable and object files - it's actually a .run file, a sort of mix 
between a shell script and a .tgz archive, which in turn contains the 
binary blobs, most notably the infamous nv-kernel.o).

After a brief look at the LILO Makefile, my understanding is that the 
parts which make up what you call the "microcore" (eg, the mbr, the 
various stages of the loader and other bootloader pieces) are built from 
assembly (.S) source files, using gcc -E, as86 and ld86 (the last two 
are from the bin86 package, which LILO depends upon).

So, while it might be difficult to understand all the internal trickery 
employed by the LILO executable behind the scenes when it runs, it 
certainly can't be likened to nvidia-like binary blobs, since here 
everything is built from sources, and certainly qualifies (IMHO at 
least) as "free software" (portage lists the LILO license as BSD GPL-2).

FWIW, here's my understanding of what /sbin/lilo does:

- if not already done, install suitable mbr and first and second stage 
loaders (probably reading them from somewhere inside the "microcore" and 
writing the raw disk sectors in place); note that this is similar to 
what grub-install (or the equivalent command in the GRUB interactive 
shell) does;

- create or update the so-called "map file", which lists where the 
various parts of the kernel are located on the disk, so that at boot 
time the loader can find them, load them into memory and finally run the 
kernel. The map is needed because, as you probably know already, the 
LILO loader is not able to read a file system, so it needs to have a 
lower level physical map of the disk sectors where the kernel file 
resides. The GRUB loader, OTOH, is able to understand and read from the 
most popular filesystems, so it does not need any map and its loader can 
literally read the kernel file from /boot (or wherever it is) and thus 
autonomously do all the work at boot time.

I hope the above is (mostly) correct and that helps a bit.

> I decided it was better to spend
> my time learning it than trying to trace down and figure out all the
> ins and outs of the LILO thing to my satisfaction, so that's exactly
> what I did, and indeed, I AM rather happier with GRUB, now that I
> actually took the time to learn how to work with it.

I was once a LILO-only user (mostly because GRUB was not popular when I 
first started using linux, so I simply stayed with LILO), but I must 
admit that I'm slowly migrating to GRUB. The great feature is (among 
other things) the ability to fully edit the command line before booting, 
which gives you more control than LILO over the boot process.

> FWIW, now I'm looking at LinuxBIOS, which I've already verified IS
> known to work on my mobo. 
>[cut]
> As I understand it now, I can use GRUB as the payload (making the grub
> first stage what amounts to a second stage after the LinuxBIOS), or
> one of the two options that normally come with LinuxBIOS, or even load
> a shrunken Linux kernel with just enough functionality to read the
> main kernel off the disk and kexec into it.
> [cut]
> Still, it might be a year or two, but I'll probably do it eventually,
> thus ridding my system of yet one more proprietary software (firmware)
> bit. 

This sounds really interesting and promising. I wonder how they can do "3 
seconds from power-on to Linux console". Even if the LinuxBIOS loads 
straightly a kernel as the payload, the kernel still takes more than 3 
seconds to initialize...this alone would be a good enough reason to do 
the switch! I hope you eventually succeed in the task.
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Emerging package as both 64 and 32 bit

[Duncan]

Etaoin Shrdlu <shrdlu@unlimitedmail.org> posted
200612202249.56315.shrdlu@unlimitedmail.org, excerpted below, on  Wed, 20
Dec 2006 22:49:56 +0100:

Duncan wrote...

>> I decided it was better to spend my time learning [GRUB] than trying to
>> trace down and figure out all the ins and outs of the LILO thing to my
>> satisfaction, so that's exactly what I did, and indeed, I AM rather
>> happier with GRUB, now that I actually took the time to learn how to
>> work with it.
> 
> I was once a LILO-only user (mostly because GRUB was not popular when I
> first started using linux, so I simply stayed with LILO), but I must
> admit that I'm slowly migrating to GRUB. The great feature is (among
> other things) the ability to fully edit the command line before booting,
> which gives you more control than LILO over the boot process.

Exactly.  That's the big bonus of GRUB and the reason more distributions
are defaulting to it.

Contrastingly, one of the weaknesses of GRUB for quite awhile was that it
didn't have a failsafe method to do an unattented/remote boot.  That is,
with LILO, one could remotely install a new kernel on their colocated
server and set it as a  once-only boot-to that would default to a known
good kernel on subsequent boots, if the first boot failed.  Combine that
with a hardware watchdog card to reboot if it didn't get its timer reset
periodically, and a new kernel that failed to boot didn't require onsite
intervention.  GRUB had nothing at the time like that, and would require
onsite intervention, so LILO remained the preferred choice for remotely
administered systems.  I'm not into that end of things so I'm poorly
informed on GRUB's current capacities in the area, but I understand that's
no longer the case, that it can do single-shot then return to a standard
default boot, just as can LILO, now.  Thus, there's now little reason
other than older sysadmin familiarity with it (and the odd corner case
hardware issue with one or the other) to keep LILO over GRUB.

>> FWIW, now I'm looking at LinuxBIOS
> 
> This sounds really interesting and promising. I wonder how they can do
> "3 seconds from power-on to Linux console". Even if the LinuxBIOS loads
> straightly a kernel as the payload, the kernel still takes more than 3
> seconds to initialize...this alone would be a good enough reason to do
> the switch! I hope you eventually succeed in the task.

Well, one BIOS-specific but entirely practical reason I'm looking into
LinuxBIOS ATM is that the current proprietary BIOS on my main system has
some frustrating issues... Namely, despite ranking the boot order to put
the hard drive first, and turning off scan of the floppy (and DVD, tho
fortunately it's a bit faster), the BIOS still INSISTS on activating and
scanning both for media before activating the hard drive boot.  The only
way to avoid the issue appears to be to deactivate the offending hardware
entirely in the BIOS, thereby not allowing it to be used post-boot,
either.  Older BIOS versions didn't have this issue but had other issues,
including lacking the ability to limit memory timings.  As long time list
regulars will recall, I had a terrible time with (generic, yes, I
know) memory that simply wasn't stable at its rated speeds, and only
worked well once I upgraded the BIOS and got the ability to limit memory
clock speeds.

Thus, a large fraction of my reboot time is frustratingly going to this
totally unnecessary floppy and DVD-drive scan, and the rest of the
hardware, kernel, and general system init process, don't seem so long in
comparison, particularly when they are doing obviously useful things as I
can see from dmesg and the like. If there was only a way to disable the
stupid removable media scans properly...

Of course, I'm not a BIOS programmer by a LONG shot, so just having the
BIOS sources to tinker with wouldn't help me much, but having an open
choice is certainly appealing.  While at this stage there may be such
frustrating issues with LinuxBIOS as well, if it were to get to even a
decent fraction of the current Linux acceptance level, there'd be enough
momentum behind it to have a good chance of dealing with this sort of
issue on the majority of supported hardware.

As for the 3-second claim, what they are doing is taking a drastically
slimmed down kernel, customized for the particular hardware such that it's
not necessary to do much of the standard kernel init and hardware scan
process.  Combine that with the fact that they avoid the current situation
where the BIOS scans and inits much of the hardware, only to have the
kernel redo much of the same stuff, and the effect cutting out that
duplicated scanning has on the boot time, and a 3-second kernel init time
isn't so far fetched.

If I understand the way they do it, one of the tricks they use is to take
hardware configuration information from the running system and from the
config files at build time, and actually build not only a slimmed down
kernel that doesn't scan for hardware that's known not to be there, but
they actually pre-init part of the kernel at build-time, so it doesn't
have to do that scanning at all, it's simply loaded straight from
FLASH-ROM already partially initialized and ready to go.

Something else they mention on the site... they had originally predicted
BIOS ROM sizes to increase faster than they have, and expected 8-32 Mbit
(so 1-4 MByte) BIOS image sizes by now.  With that, they could have fit an
entire compressed main kernel image complete with initramfs in the BIOS
image.  Instead, 4-8 Mbit (half to 1 MB) BIOS capacities are now the norm,
and a boot directly to full-size kernel remains impractical.  Thus, they
use multi-stage approach, with a second in-BIOS stage consisting of either
a larger bootloader that reads off of disk or network, or an extremely
stripped down kernel (other second stages such as the OpenFirmware BIOS
that Macs use are also available), then making use of the still fairly
new Linux kernel kexec feature to allow the stripped down kernel to load
the full kernel off of disk and kexec into it.

My big question at this time, one I didn't find an answer to in my
initial round of preliminary research, is what happens to all the existing
proprietary BIOS functions such as memory configuration, and BIOS level
enabling/disabling of various on-board hardware.  I /did/ see that there's
a provision for cutting out the various firmware mini-drivers (such as the
common VideoBIOS, NetBIOS, and RAID firmwares, subpackaged into
proprietary BIOSs) and including the binary blob images in LinuxBIOS as
well (yes, the binary blobs are galling, but it's a further step in the
right direction from a full proprietary BIOS), so that's addressed, but
the functionality such as memory config normally provided by the BIOS
itself (AFAIK)?  I'm not sure I'm ready to forego the ability to
BIOS-disable the various on-board hardware, and BIOS-configure stuff such
as memory parameters.  From my limited look around, I /think/ a fair
amount of that is configurable via text based config files at
BIOS-build-time, which is reasonable, but ONLY if one has the hardware to
supply a known-working backup BIOS option should one screw up their config
while experimenting.  (Some mobos include such a backup BIOS as a feature,
making bricking a machine due to botched flash upgrade a threat for
others to worry about. My present one does not.)

The LinuxBIOS site /does/ point to sources for all the necessary BIOS
switcher hardware and extra BIOS chips, but the one link I clicked on was
either stale or required Flash (as in the browser plugin not the memory
type) or the like to load, and I didn't investigate that aspect further,
so I've yet no idea what the cost would be.  Still, the site implies it's
rather more reasonable than I had thought based on the limited outdated
information I'd run across earlier, but really, that's to be expected
given the rise in popularity and scale and dropping of cost of flash
memory technology over the last half-decade or so.  The implied cost on
the site is less than $100 (US) now, which is significantly less than the
$500 or more I had thought, based on old information, which at least makes
it reasonably economically accessible to those with a non-professional
interest in it, unlike the $500 price-point, and it sounded like a $20-50
investment may do it.

Like I said tho, I've got significantly more research to do before I'm
actually downloading and running BIOS-builders.  It's definitely a
mid-term interest, not something I'll be doing before new year's day.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Emerging package as both 64 and 32 bit

[Mike Doty]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Duncan wrote:
> Simon Stelling <blubb@gentoo.org> posted 45866BE0.20905@gentoo.org,
> excerpted below, on  Mon, 18 Dec 2006 12:22:24 +0200:
> 
>> Neil Bothwick wrote:
>>> What's wrong with the GRUB source package? I remember using grub-static
>>> when I first went 64 bit, but haven't used it for ages. The only 32 bit
>>> stuff I use here is firefox-bin and vmware.
>> No problem, but it's 32bit.
> 
> Indeed...  for backward compatibility, amd64/x86_64 boots in 32-bit mode. 
> Actually, I /believe/ it boots in 16-bit real mode, just like an x86, then
> switches to 32-bit or 64-bit when the appropriate command is given, but
> AFAIK the difference between compiling 16-bit and 32-bit code is simply a
> few compile-time switches, so it uses a standard 32-bit toolchain.
You're referring to real mode(16 bit).  the BIOS will load the
bootloader in real mode, the bootloader will switch to protected mode(32
bit) and if you have the right kernel, it will switch to extended
mode(64 bit)

- --
=======================================================
Mike Doty                      kingtaco -at- gentoo.org
Gentoo/AMD64 Strategic Lead
Gentoo Council
Gentoo Developer Relations
Gentoo Recruitment Lead
Gentoo Infrastructure
GPG: E1A5 1C9C 93FE F430 C1D6  F2AF 806B A2E4 19F4 AE05
=======================================================
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iQCVAwUBRYuC74BrouQZ9K4FAQIy3QQA1gToVVmEzCbPnVDAQYDH6LhNf7x/zZ4v
OQBDCW5ipthADqpwIiyb3Xo7ka8ADSm3A0Oggyfr8Az87ZZFJupETIuiTjaYLudo
9Il0VqKAdb0URawyX7o4LKlmlfz83Hf5QuXkaATh7g8XH+Ia9vuSdAPOsQFf7/1L
Q2NO2XaLVw4=
=x3VG
-----END PGP SIGNATURE-----
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Emerging package as both 64 and 32 bit

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 1456 bytes --]

On Friday 22 December 2006 01:02, Mike Doty <kingtaco@gentoo.org> wrote 
about 'Re: [gentoo-amd64]  Re: Emerging package as both 64 and 32 bit':
> Duncan wrote:
> > Simon Stelling <blubb@gentoo.org> posted 45866BE0.20905@gentoo.org,
> >
> > excerpted below, on  Mon, 18 Dec 2006 12:22:24 +0200:
> >> Neil Bothwick wrote:
> >>> What's wrong with the GRUB source package?
> >> No problem, but it's 32bit.
> > Indeed...  for backward compatibility, amd64/x86_64 boots in 32-bit
> > mode. Actually, I /believe/ it boots in 16-bit real mode, just like an
> > x86, [...] but AFAIK the difference between compiling 16-bit and 32-bit
> > code is simply a few compile-time switches, so it uses a standard
> > 32-bit toolchain.
> You're referring to real mode(16 bit).  the BIOS will load the
> bootloader in real mode, the bootloader will switch to protected mode(32
> bit) and if you have the right kernel, it will switch to extended
> mode(64 bit)

Also, it's technically possible for the bootloader to switch into extended 
mode before loading the kernel, but not done (in GRUB, at least) for solid 
technical reasons.  (IIRC, there's some memory mappings that have to be 
set up before entering extended mode.)

-- 
"If there's one thing we've established over the years,
it's that the vast majority of our users don't have the slightest
clue what's best for them in terms of package stability."
-- Gentoo Developer Ciaran McCreesh

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

[gentoo-amd64] Re: Emerging package as both 64 and 32 bit

[Duncan]

"Boyd Stephen Smith Jr." <bss03@volumehost.net> posted
200612220854.59448.bss03@volumehost.net, excerpted below, on  Fri, 22 Dec
2006 08:54:59 -0600:

> Also, it's technically possible for the bootloader to switch into extended 
> mode before loading the kernel, but not done (in GRUB, at least) for solid 
> technical reasons.  (IIRC, there's some memory mappings that have to be 
> set up before entering extended mode.)

That's something I've actually always wondered, so it's interesting to see
a bit on it.  Is there anything not /too/ technical* but somewhat
more detailed you could point me to?  

*I'm not a C/C++/ASM coder, but understand the terminology well enough to
find many LKML discussions interesting, for instance, and I'd say I follow
about half of the LWN kernel page articles, enough to have found myself
recently correcting (I think/hope) some inaccuracies in a Linux.com article
by Joe Barr, for instance, see the article and comments following here, my
comment should be obvious, given its length <g>:
http://enterprise.linux.com/article.pl?sid=06/12/14/1910259

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Emerging package as both 64 and 32 bit

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 1136 bytes --]

On Friday 22 December 2006 15:20, Duncan <1i5t5.duncan@cox.net> wrote 
about '[gentoo-amd64]  Re: Emerging package as both 64 and 32 bit':
> "Boyd Stephen Smith Jr." <bss03@volumehost.net> posted
> 200612220854.59448.bss03@volumehost.net, excerpted below, on  Fri, 22
> Dec
>
> 2006 08:54:59 -0600:
> > Also, it's technically possible for the bootloader to switch into
> > extended mode before loading the kernel, but not done (in GRUB, at
> > least) for solid technical reasons.  (IIRC, there's some memory
> > mappings that have to be set up before entering extended mode.)
>
> That's something I've actually always wondered, so it's interesting to
> see a bit on it.  Is there anything not /too/ technical* but somewhat
> more detailed you could point me to?

I don't have anything bookmarked, but I'll look around after I get back 
from work and see if I can dig up any more references.

-- 
"If there's one thing we've established over the years,
it's that the vast majority of our users don't have the slightest
clue what's best for them in terms of package stability."
-- Gentoo Developer Ciaran McCreesh

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]