[gentoo-user] ...recreating exactly the same applications on a new harddisc?

[gentoo-user] ...recreating exactly the same applications on a new harddisc?

[tuxic]

Hi,

I am currently preparing a new harddisc as home for my new Gentoo
system.

Is it possible to recreate exactlu the same pool of
applications/programs/libraries etc..., which my current
system have - in one go?

That is: Copy <something> from the current system into
the chroot environment, fire up emerge, go to bed and
tommorow morning the new system ready...?

Does this <something> exists and is it reasonable to do
it this way?

Thanks for any hint in advance!
Stay healthy!
Cheers,
Meino

[gentoo-user] Re: ...recreating exactly the same applications on a new harddisc?

[Ian Zimmerman]

On 2020-04-04 19:34, tuxic@posteo.de wrote:

> Is it possible to recreate exactlu the same pool of
> applications/programs/libraries etc..., which my current
> system have - in one go?

You don't say if you want exactly the same _versions_ of everything.

If you don't need that, wouldn't just transferring the world file be
enough?

If you do, maybe you can pin the versions in the world file, though
I have never tried that.

-- 
Ian

Re: [gentoo-user] Re: ...recreating exactly the same applications on a new harddisc?

[tuxic]

On 04/04 10:59, Ian Zimmerman wrote:
> On 2020-04-04 19:34, tuxic@posteo.de wrote:
> 
> > Is it possible to recreate exactlu the same pool of
> > applications/programs/libraries etc..., which my current
> > system have - in one go?
> 
> You don't say if you want exactly the same _versions_ of everything.
> 
> If you don't need that, wouldn't just transferring the world file be
> enough?
> 
> If you do, maybe you can pin the versions in the world file, though
> I have never tried that.
> 
> -- 
> Ian
> 

Hi Ian,

no, I dont' need the same versions...

And interesting you have the same question as I:

Wouldn't transferring the world file be enough?

Is it?

Cheers!
Meino

[gentoo-user] Re: ...recreating exactly the same applications on a new harddisc?

[Ian Zimmerman]

On 2020-04-04 20:03, tuxic@posteo.de wrote:

> Wouldn't transferring the world file be enough?
> 
> Is it?

As far as I know, for having the same packages pulled in, this
is all that matters (plus possibly @world_sets as Neil mentions).

Of course you have all sorts of other configuration to replicate,
including portage's.  That's what git is for :-P.

-- 
Ian

Re: [gentoo-user] Re: ...recreating exactly the same applications on a new harddisc?

[Jack]

On 4/5/20 1:41 PM, Ian Zimmerman wrote:
> On 2020-04-04 20:03, tuxic@posteo.de wrote:
>
>> Wouldn't transferring the world file be enough?
>>
>> Is it?
> As far as I know, for having the same packages pulled in, this
> is all that matters (plus possibly @world_sets as Neil mentions).
>
> Of course you have all sorts of other configuration to replicate,
> including portage's.  That's what git is for :-P.

I suspect those portage config files become even more important if you 
have any unstable (~) packages installed.  If a package has no stable 
versions in the tree, then I don't think it will get installed at all, 
without keywording it.  Even more true if you have a masked package 
installed.

Jack

Re: [gentoo-user] Re: ...recreating exactly the same applications on a new harddisc?

[Jack]

On 4/5/20 1:41 PM, Ian Zimmerman wrote:
>> On 2020-04-04 20:03, tuxic@posteo.de wrote:
>>
>>> Wouldn't transferring the world file be enough?
>>>
>>> Is it?
>> As far as I know, for having the same packages pulled in, this
>> is all that matters (plus possibly @world_sets as Neil mentions).
>>
>> Of course you have all sorts of other configuration to replicate,
>> including portage's.  That's what git is for :-P.
I suspect those portage config files become even more important if you 
have any unstable (~) packages installed.  If a package has no stable 
versions in the tree, then I don't think it will get installed at all, 
without keywording it.  Even more true if you have a masked package 
installed.

Jack

Re: [gentoo-user] Re: ...recreating exactly the same applications on a new harddisc?

[Neil Bothwick]

On Sat, 4 Apr 2020 10:59:31 -0700, Ian Zimmerman wrote:

> > Is it possible to recreate exactlu the same pool of
> > applications/programs/libraries etc..., which my current
> > system have - in one go?  
> 
> You don't say if you want exactly the same _versions_ of everything.
> 
> If you don't need that, wouldn't just transferring the world file be
> enough?

You may need to copy world_sets as well, if you have any portage sets
installed.


-- 
Neil Bothwick

He who laughs last thinks slowest!

Re: [gentoo-user] Re: ...recreating exactly the same applications on a new harddisc?

[Peter Humphrey]

On Saturday, 4 April 2020 22:26:16 BST Neil Bothwick wrote:
> On Sat, 4 Apr 2020 10:59:31 -0700, Ian Zimmerman wrote:
> > > Is it possible to recreate exactlu the same pool of
> > > applications/programs/libraries etc..., which my current
> > > system have - in one go?
> > 
> > You don't say if you want exactly the same _versions_ of everything.
> > 
> > If you don't need that, wouldn't just transferring the world file be
> > enough?
> 
> You may need to copy world_sets as well, if you have any portage sets
> installed.

I'd also copy much of /etc and /etc/conf.d as well, especially /etc/portage.

-- 
Regards,
Peter.

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Mark Knecht]

[-- Attachment #1: Type: text/plain, Size: 1208 bytes --]

Your world file should do that for the Gentoo stuff, with limitations. It
assumes you have nothing on the system that was created outside of normal
portage/emerge. It would probably duplicate the latest kernel tree but
wouldn't build it, and wouldn't copy old kernels that aren't in portage if
you still have them on the system. It isn't going to get virtual
environment, be they python or things like virtualbox if you use those.

I suspect you'll get a 'working' machine (I've done it) but you will still
have a lot of stuff to transfer by hand or from backups which you really
should do anyway.

HTH,
Mark

On Sat, Apr 4, 2020 at 10:35 AM <tuxic@posteo.de> wrote:

> Hi,
>
> I am currently preparing a new harddisc as home for my new Gentoo
> system.
>
> Is it possible to recreate exactlu the same pool of
> applications/programs/libraries etc..., which my current
> system have - in one go?
>
> That is: Copy <something> from the current system into
> the chroot environment, fire up emerge, go to bed and
> tommorow morning the new system ready...?
>
> Does this <something> exists and is it reasonable to do
> it this way?
>
> Thanks for any hint in advance!
> Stay healthy!
> Cheers,
> Meino
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 1660 bytes --]

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[tuxic]

Hi Mark,

thank you for answering my question! :)

(only to check whether I have understood correctlu)
Stuff I build outside of emerge/portage should not be in the
world-file...correct?

I will transfer the kernel related stuff from my current system to
the new root (currenly chrooted).

Ah! By the way: Recompiling all the stuff works while being chrooted,
right?
The command to use is: 
 emerge --update --newuse --deep --quiet @world

after syncing.


Correct?

Cheers!
Meino


On 04/04 11:05, Mark Knecht wrote:
> Your world file should do that for the Gentoo stuff, with limitations. It
> assumes you have nothing on the system that was created outside of normal
> portage/emerge. It would probably duplicate the latest kernel tree but
> wouldn't build it, and wouldn't copy old kernels that aren't in portage if
> you still have them on the system. It isn't going to get virtual
> environment, be they python or things like virtualbox if you use those.
> 
> I suspect you'll get a 'working' machine (I've done it) but you will still
> have a lot of stuff to transfer by hand or from backups which you really
> should do anyway.
> 
> HTH,
> Mark
> 
> On Sat, Apr 4, 2020 at 10:35 AM <tuxic@posteo.de> wrote:
> 
> > Hi,
> >
> > I am currently preparing a new harddisc as home for my new Gentoo
> > system.
> >
> > Is it possible to recreate exactlu the same pool of
> > applications/programs/libraries etc..., which my current
> > system have - in one go?
> >
> > That is: Copy <something> from the current system into
> > the chroot environment, fire up emerge, go to bed and
> > tommorow morning the new system ready...?
> >
> > Does this <something> exists and is it reasonable to do
> > it this way?
> >
> > Thanks for any hint in advance!
> > Stay healthy!
> > Cheers,
> > Meino
> >
> >
> >
> >

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Mark Knecht]

[-- Attachment #1: Type: text/plain, Size: 3254 bytes --]

Meino,
   I think as your question was posed, yes, copying the world file is
enough to get the applications into the chroot built and working. Your
command looks reasonable but I'll let someone who runs Gentoo these days
make suggestions for improvement on that. (I no longer run Gentoo much.
Just in a VM and I don't update it any more)

   As Ashley points out what you're going to wake up to in the morning (or
whenever it finishes) is a chroot with all the apps and no special
configuration. For instance, if you allow ssh logins to this machine then
to get that in the chroot that requires changes to the config file for ssh
which you'll still have to do. There are likely to be 20, 50, 100 little
issues like that that you'll be dealing with for days.

   If your goal is to have a chroot which is functionally identical to your
main system (with at least a new IP address and probably little or no X
access) then you can go this direction. However if you're wanting to get to
a newer/bigger/faster hard drive, or from a hard drive to an SSD or
something, isn't the more tested (non-Gentoo) way to do this just to clone
the drive using clonezilla or something like that? You boot a CD, tell it
to clone the HDD to the SDD and wait for it to finish?

Mark

On Sat, Apr 4, 2020 at 11:24 AM <tuxic@posteo.de> wrote:

> Hi Mark,
>
> thank you for answering my question! :)
>
> (only to check whether I have understood correctlu)
> Stuff I build outside of emerge/portage should not be in the
> world-file...correct?
>
> I will transfer the kernel related stuff from my current system to
> the new root (currenly chrooted).
>
> Ah! By the way: Recompiling all the stuff works while being chrooted,
> right?
> The command to use is:
>  emerge --update --newuse --deep --quiet @world
>
> after syncing.
>
>
> Correct?
>
> Cheers!
> Meino
>
>
> On 04/04 11:05, Mark Knecht wrote:
> > Your world file should do that for the Gentoo stuff, with limitations. It
> > assumes you have nothing on the system that was created outside of normal
> > portage/emerge. It would probably duplicate the latest kernel tree but
> > wouldn't build it, and wouldn't copy old kernels that aren't in portage
> if
> > you still have them on the system. It isn't going to get virtual
> > environment, be they python or things like virtualbox if you use those.
> >
> > I suspect you'll get a 'working' machine (I've done it) but you will
> still
> > have a lot of stuff to transfer by hand or from backups which you really
> > should do anyway.
> >
> > HTH,
> > Mark
> >
> > On Sat, Apr 4, 2020 at 10:35 AM <tuxic@posteo.de> wrote:
> >
> > > Hi,
> > >
> > > I am currently preparing a new harddisc as home for my new Gentoo
> > > system.
> > >
> > > Is it possible to recreate exactlu the same pool of
> > > applications/programs/libraries etc..., which my current
> > > system have - in one go?
> > >
> > > That is: Copy <something> from the current system into
> > > the chroot environment, fire up emerge, go to bed and
> > > tommorow morning the new system ready...?
> > >
> > > Does this <something> exists and is it reasonable to do
> > > it this way?
> > >
> > > Thanks for any hint in advance!
> > > Stay healthy!
> > > Cheers,
> > > Meino
> > >
> > >
> > >
> > >
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 4164 bytes --]

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Dale]

Mark Knecht wrote:
> Your world file should do that for the Gentoo stuff, with limitations.
> It assumes you have nothing on the system that was created outside of
> normal portage/emerge. It would probably duplicate the latest kernel
> tree but wouldn't build it, and wouldn't copy old kernels that aren't
> in portage if you still have them on the system. It isn't going to get
> virtual environment, be they python or things like virtualbox if you
> use those.
>
> I suspect you'll get a 'working' machine (I've done it) but you will
> still have a lot of stuff to transfer by hand or from backups which
> you really should do anyway.
>
> HTH,
> Mark
>

I recently tried this in a chroot starting with a stage3 tarball.  At
first, I tried unpacking the tarball, copying over /etc and the world
file.  I also copied over the binaries and tried using -k.  It was a
disaster.  I ran into hard blacks that I never was able to get around
not to mention emerge complaining about USE flags and such.  Then I
tried unpacking a tarball and just updating the tarball itself with no
changes on my part, not even the profile, it to ran into hard blocks
just not as many of them.

In the 2nd attempt, I think something was off in the tarball itself. 
When you unpack a tarball and try to sync and update it and it fails,
something is wrong somewhere.  It's not covered in the install handbook
either.  In theory, one should be able to unpack a tarball, copy over
/etc and the world file and do a emerge -e world.  If one copies over
the binaries from the old system, one could add a -k to speed up the
process, for most if not all packages.  Thing is, theory meets real
world real fast and it gets ugly. After multiple attempts, I ended up
coping my original OS over and that worked better and MUCH faster.

Way back in the day, I would boot a rescue disk of some type, mount both
drives and then copy everything over, excluding /home if it is on a
separate drive or any others that shouldn't be transferred.  Once that
is done, chroot in and install grub, the old original one not grub2. 
Once that is done, shutdown and remove old drive, plug new drive into
old port and then power up, crossing fingers and toes.  It worked first
time generally.  I have NOT done that with grub2.  It may work the same,
it may not.  Grub2 is a bit of a beast.

Theory, should work.  In my real world experience, it does not. Coping
tends to work if you do it all right.

Just my thoughts.

Dale

:-)  :-) 

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[John Covici]

On Sat, 04 Apr 2020 14:33:21 -0400,
Dale wrote:
> 
> Mark Knecht wrote:
> > Your world file should do that for the Gentoo stuff, with limitations.
> > It assumes you have nothing on the system that was created outside of
> > normal portage/emerge. It would probably duplicate the latest kernel
> > tree but wouldn't build it, and wouldn't copy old kernels that aren't
> > in portage if you still have them on the system. It isn't going to get
> > virtual environment, be they python or things like virtualbox if you
> > use those.
> >
> > I suspect you'll get a 'working' machine (I've done it) but you will
> > still have a lot of stuff to transfer by hand or from backups which
> > you really should do anyway.
> >
> > HTH,
> > Mark
> >
> 
> I recently tried this in a chroot starting with a stage3 tarball.  At
> first, I tried unpacking the tarball, copying over /etc and the world
> file.  I also copied over the binaries and tried using -k.  It was a
> disaster.  I ran into hard blacks that I never was able to get around
> not to mention emerge complaining about USE flags and such.  Then I
> tried unpacking a tarball and just updating the tarball itself with no
> changes on my part, not even the profile, it to ran into hard blocks
> just not as many of them.
> 
> In the 2nd attempt, I think something was off in the tarball itself. 
> When you unpack a tarball and try to sync and update it and it fails,
> something is wrong somewhere.  It's not covered in the install handbook
> either.  In theory, one should be able to unpack a tarball, copy over
> /etc and the world file and do a emerge -e world.  If one copies over
> the binaries from the old system, one could add a -k to speed up the
> process, for most if not all packages.  Thing is, theory meets real
> world real fast and it gets ugly. After multiple attempts, I ended up
> coping my original OS over and that worked better and MUCH faster.
> 
> Way back in the day, I would boot a rescue disk of some type, mount both
> drives and then copy everything over, excluding /home if it is on a
> separate drive or any others that shouldn't be transferred.  Once that
> is done, chroot in and install grub, the old original one not grub2. 
> Once that is done, shutdown and remove old drive, plug new drive into
> old port and then power up, crossing fingers and toes.  It worked first
> time generally.  I have NOT done that with grub2.  It may work the same,
> it may not.  Grub2 is a bit of a beast.
> 
> Theory, should work.  In my real world experience, it does not. Coping
> tends to work if you do it all right.
> 
> Just my thoughts.

I did something like  this a few months ago, I first got the tarball,
did an immediate update,  copied a lot of /etc, particularly
/etc/portage, but not some things like package.use, because I wanted
to let it redetect them, and did parts of the world file at a time,
not all at once, because I had some crud in the world file which I
wanted to be sure I got rid of.  Took a couple of weeks, but did work
and then I had a better system than I had before.

-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

         John Covici wb2una
         covici@ccs.covici.com

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Dale]

John Covici wrote:
> On Sat, 04 Apr 2020 14:33:21 -0400,
> Dale wrote:
>> Mark Knecht wrote:
>>> Your world file should do that for the Gentoo stuff, with limitations.
>>> It assumes you have nothing on the system that was created outside of
>>> normal portage/emerge. It would probably duplicate the latest kernel
>>> tree but wouldn't build it, and wouldn't copy old kernels that aren't
>>> in portage if you still have them on the system. It isn't going to get
>>> virtual environment, be they python or things like virtualbox if you
>>> use those.
>>>
>>> I suspect you'll get a 'working' machine (I've done it) but you will
>>> still have a lot of stuff to transfer by hand or from backups which
>>> you really should do anyway.
>>>
>>> HTH,
>>> Mark
>>>
>> I recently tried this in a chroot starting with a stage3 tarball.  At
>> first, I tried unpacking the tarball, copying over /etc and the world
>> file.  I also copied over the binaries and tried using -k.  It was a
>> disaster.  I ran into hard blacks that I never was able to get around
>> not to mention emerge complaining about USE flags and such.  Then I
>> tried unpacking a tarball and just updating the tarball itself with no
>> changes on my part, not even the profile, it to ran into hard blocks
>> just not as many of them.
>>
>> In the 2nd attempt, I think something was off in the tarball itself. 
>> When you unpack a tarball and try to sync and update it and it fails,
>> something is wrong somewhere.  It's not covered in the install handbook
>> either.  In theory, one should be able to unpack a tarball, copy over
>> /etc and the world file and do a emerge -e world.  If one copies over
>> the binaries from the old system, one could add a -k to speed up the
>> process, for most if not all packages.  Thing is, theory meets real
>> world real fast and it gets ugly. After multiple attempts, I ended up
>> coping my original OS over and that worked better and MUCH faster.
>>
>> Way back in the day, I would boot a rescue disk of some type, mount both
>> drives and then copy everything over, excluding /home if it is on a
>> separate drive or any others that shouldn't be transferred.  Once that
>> is done, chroot in and install grub, the old original one not grub2. 
>> Once that is done, shutdown and remove old drive, plug new drive into
>> old port and then power up, crossing fingers and toes.  It worked first
>> time generally.  I have NOT done that with grub2.  It may work the same,
>> it may not.  Grub2 is a bit of a beast.
>>
>> Theory, should work.  In my real world experience, it does not. Coping
>> tends to work if you do it all right.
>>
>> Just my thoughts.
> I did something like  this a few months ago, I first got the tarball,
> did an immediate update,  copied a lot of /etc, particularly
> /etc/portage, but not some things like package.use, because I wanted
> to let it redetect them, and did parts of the world file at a time,
> not all at once, because I had some crud in the world file which I
> wanted to be sure I got rid of.  Took a couple of weeks, but did work
> and then I had a better system than I had before.
>

Since I keep a clean world file, it wouldn't matter for me.  I have -1
as a default emerge option in make.conf which means I have to
intentionally add a package to the world file, either by hand or with
--select y as a option on the command line. Even if I got the tarball to
work, I'd still end up with the same system I got on my main install. 
It wouldn't matter any. 

What really got me tho, being unable to get a bare stage3 tarball to
update without running into hard blocks.  I thought that to be really
strange. 

Still, based on past experience, copying the system over is the
fastest.  No compiling or anything, just copying it over. 

Dale

:-)  :-)

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Ashley Dixon]

[-- Attachment #1: Type: text/plain, Size: 1182 bytes --]

On Sat, Apr 04, 2020 at 07:34:59PM +0200, tuxic@posteo.de wrote:
> Hi,
> 
> I am currently preparing a new harddisc as home for my new Gentoo
> system.
> 
> Is it possible to recreate exactlu the same pool of
> applications/programs/libraries etc..., which my current
> system have - in one go?
> 
> That is: Copy <something> from the current system into
> the chroot environment, fire up emerge, go to bed and
> tommorow morning the new system ready...?
> 
> Does this <something> exists and is it reasonable to do
> it this way?
> 
> Thanks for any hint in advance!
> Stay healthy!
> Cheers,
> Meino

Do you also want to copy configuration and data files, attaining a total
replica ? If so, copying the world file is a start, but then you'll have a
plethora of /etc and /var files through which to sift.

Perhaps a little more detailed context to your problem would allow for more
accurate/helpful recommendations ? I.e., are you looking for near-complete
duplication, or just a collection of familiar packages which happens to be on
similar hardware ?

-- 

Ashley Dixon
suugaku.co.uk

2A9A 4117
DA96 D18A
8A7B B0D2
A30E BF25
F290 A8AA


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[tuxic]

On 04/04 07:25, Ashley Dixon wrote:
> On Sat, Apr 04, 2020 at 07:34:59PM +0200, tuxic@posteo.de wrote:
> > Hi,
> > 
> > I am currently preparing a new harddisc as home for my new Gentoo
> > system.
> > 
> > Is it possible to recreate exactlu the same pool of
> > applications/programs/libraries etc..., which my current
> > system have - in one go?
> > 
> > That is: Copy <something> from the current system into
> > the chroot environment, fire up emerge, go to bed and
> > tommorow morning the new system ready...?
> > 
> > Does this <something> exists and is it reasonable to do
> > it this way?
> > 
> > Thanks for any hint in advance!
> > Stay healthy!
> > Cheers,
> > Meino
> 
> Do you also want to copy configuration and data files, attaining a total
> replica ? If so, copying the world file is a start, but then you'll have a
> plethora of /etc and /var files through which to sift.
> 
> Perhaps a little more detailed context to your problem would allow for more
> accurate/helpful recommendations ? I.e., are you looking for near-complete
> duplication, or just a collection of familiar packages which happens to be on
> similar hardware ?
> 
> -- 
> 
> Ashley Dixon
> suugaku.co.uk
> 
> 2A9A 4117
> DA96 D18A
> 8A7B B0D2
> A30E BF25
> F290 A8AA
> 

Hi Ashley,

ok...here it comes...the story of "System 9 outer space" <hhhrmrrm>:

My current system has two drawbacks: The harddisc has become way to small and I don't
want more than one harddisc in my PC.
My old PC is 12 years old...and it is - in relation to software of today 
(especially blender) - much too slow and the main memory is also not
of sufficient size.

Expanding of the old system is -- in respect to its age -- economically wise
not the correct decision...I think.

So I bought the parts of a new PC (again AMD), build a new PC, inserts
the harddisc of the old PC and booted the system. which works fine.

Now I have an old system on the harddisc whith some legacy structure
(I think), which I want to replace with """the same system""" --
freshly rebuild in a way that I can fire up one command before I go to
bed only to recognize next morning, that I forget to become root
beforehand... ;)

Jokes aside:
I want to try to recompile every Gentoo related stuff in the new system,
which was present in the old system (application-wise, and not
necessarily version-wise).

This gives me the chance to use a new set of cpuflags given by cpuid2cpuflags, too.
(by the way: This command show far less flags than diplayed via the
command 'lscpu'....is cpuid2cpuflags uptodate?)

For the configuration I will move a lot of stuff from the current
system to the new system. That's ok...

For the "partition and boot" scheme (not the correct words...sorry no
native speaker ahead....;) ) I thought of this:

One hardisc (3T) with the complete system including 256 GB root. The
harddisc has a GPT and has a grub bootloader also. This makes this
harddisc bootable as "standalone solution".

Additonally there is a M.2 NvME SSD
It is a mirror of the root partion with all directories, to which are often
written to (/var/tmp, /tmp,...) mounted on tmpfs.

The plan is to update (emerge ... ) the system with in a way, that
less as possible writes hits the SSD (for example by mounting certain
parts of the filesystem on tmpfs) and use the root on harddisc as 
backup.

The "real backup" will be a image copy of the harddisc to another
identical harddisc which I will create on a regular basis.

This way I always have a bootable system. The best backup is
worthless, if I don't have a system to read it....

One thing:
Would it possible to boot grub from harddisc, which in turn has
entries in the menu to boot either from harddisc or (as default) 
from SSD? I don't care about the 23.6573 ms it takes longer to
read grub stage 1 and 2 from harddisc instead off the SSD... ;)

Feeling still a bit paranoid when it comes to SSDs. I know, its
supersticous...but... ;)

Is this somehow reasonable...or...?

Cheers!
Meino

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Ashley Dixon]

[-- Attachment #1: Type: text/plain, Size: 1923 bytes --]

On Sat, Apr 04, 2020 at 09:05:09PM +0200, tuxic@posteo.de wrote:
> This gives me the chance to use a new set of cpuflags given by cpuid2cpuflags, too.
> (by the way: This command show far less flags than diplayed via the
> command 'lscpu'....is cpuid2cpuflags uptodate?)

I assume it's up-to-date; the last commit was made in late September of last
year. While lscpu reads from /proc/cpuinfo and lists _all_ reported flags,
`cpuid2cpuflags` only regards ones which are applicable to the CPU_FLAGS
variable in Gentoo, so its no surprise that less flags are shown by the latter.

> For the "partition and boot" scheme (not the correct words...sorry no
> native speaker ahead....;) ) I thought of this:

Don't worry, I understood you well. Aside from your marvellous usage of
ellipses ("..."), your communication in English is fine :)

> One thing:
> Would it possible to boot grub from harddisc, which in turn has
> entries in the menu to boot either from harddisc or (as default) 
> from SSD? I don't care about the 23.6573 ms it takes longer to
> read grub stage 1 and 2 from harddisc instead off the SSD... ;)

`grub` is capable of showing entries from multiple partition tables/block
devices, if that's what you mean ? Just emerge sys-boot/grub with the `mount`
USE flag.

> Feeling still a bit paranoid when it comes to SSDs. I know, its
> supersticous...but... ;)

Yeah, I know what you mean there. Solid-state is the way things are going, but I
always keep backups/mirrors on hard disk and tape drives (but the latter is only
feasible for huge volumes of data).

I'll have a more detailed look upon your question tomorrow when I'm not quite as
tired, but what you're trying to do is certainly reasonable, and shouldn't be a
huge amount of work.

Thanks for an interesting problem :)

-- 

Ashley Dixon
suugaku.co.uk

2A9A 4117
DA96 D18A
8A7B B0D2
A30E BF25
F290 A8AA


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Mark Knecht]

[-- Attachment #1: Type: text/plain, Size: 5601 bytes --]

On Sat, Apr 4, 2020 at 12:05 PM <tuxic@posteo.de> wrote:
>
> On 04/04 07:25, Ashley Dixon wrote:
> > On Sat, Apr 04, 2020 at 07:34:59PM +0200, tuxic@posteo.de wrote:
> > > Hi,
> > >
> > > I am currently preparing a new harddisc as home for my new Gentoo
> > > system.
> > >
> > > Is it possible to recreate exactlu the same pool of
> > > applications/programs/libraries etc..., which my current
> > > system have - in one go?
> > >
> > > That is: Copy <something> from the current system into
> > > the chroot environment, fire up emerge, go to bed and
> > > tommorow morning the new system ready...?
> > >
> > > Does this <something> exists and is it reasonable to do
> > > it this way?
> > >
> > > Thanks for any hint in advance!
> > > Stay healthy!
> > > Cheers,
> > > Meino
> >
> > Do you also want to copy configuration and data files, attaining a total
> > replica ? If so, copying the world file is a start, but then you'll
have a
> > plethora of /etc and /var files through which to sift.
> >
> > Perhaps a little more detailed context to your problem would allow for
more
> > accurate/helpful recommendations ? I.e., are you looking for
near-complete
> > duplication, or just a collection of familiar packages which happens to
be on
> > similar hardware ?
> >
> > --
> >
> > Ashley Dixon
> > suugaku.co.uk
> >
> > 2A9A 4117
> > DA96 D18A
> > 8A7B B0D2
> > A30E BF25
> > F290 A8AA
> >
>
> Hi Ashley,
>
> ok...here it comes...the story of "System 9 outer space" <hhhrmrrm>:
>
> My current system has two drawbacks: The harddisc has become way to small
and I don't
> want more than one harddisc in my PC.
> My old PC is 12 years old...and it is - in relation to software of today
> (especially blender) - much too slow and the main memory is also not
> of sufficient size.
>
> Expanding of the old system is -- in respect to its age -- economically
wise
> not the correct decision...I think.
>
> So I bought the parts of a new PC (again AMD), build a new PC, inserts
> the harddisc of the old PC and booted the system. which works fine.
>
> Now I have an old system on the harddisc whith some legacy structure
> (I think), which I want to replace with """the same system""" --
> freshly rebuild in a way that I can fire up one command before I go to
> bed only to recognize next morning, that I forget to become root
> beforehand... ;)
>
> Jokes aside:
> I want to try to recompile every Gentoo related stuff in the new system,
> which was present in the old system (application-wise, and not
> necessarily version-wise).
>
> This gives me the chance to use a new set of cpuflags given by
cpuid2cpuflags, too.
> (by the way: This command show far less flags than diplayed via the
> command 'lscpu'....is cpuid2cpuflags uptodate?)
>
> For the configuration I will move a lot of stuff from the current
> system to the new system. That's ok...
>
> For the "partition and boot" scheme (not the correct words...sorry no
> native speaker ahead....;) ) I thought of this:
>
> One hardisc (3T) with the complete system including 256 GB root. The
> harddisc has a GPT and has a grub bootloader also. This makes this
> harddisc bootable as "standalone solution".
>
> Additonally there is a M.2 NvME SSD
> It is a mirror of the root partion with all directories, to which are
often
> written to (/var/tmp, /tmp,...) mounted on tmpfs.
>
> The plan is to update (emerge ... ) the system with in a way, that
> less as possible writes hits the SSD (for example by mounting certain
> parts of the filesystem on tmpfs) and use the root on harddisc as
> backup.
>
> The "real backup" will be a image copy of the harddisc to another
> identical harddisc which I will create on a regular basis.
>
> This way I always have a bootable system. The best backup is
> worthless, if I don't have a system to read it....
>
> One thing:
> Would it possible to boot grub from harddisc, which in turn has
> entries in the menu to boot either from harddisc or (as default)
> from SSD? I don't care about the 23.6573 ms it takes longer to
> read grub stage 1 and 2 from harddisc instead off the SSD... ;)
>
> Feeling still a bit paranoid when it comes to SSDs. I know, its
> supersticous...but... ;)
>
> Is this somehow reasonable...or...?
>
> Cheers!
> Meino
>

Possibly I'm still misunderstanding. However your description here is
helpful.

Maybe you're approaching this the hard way? Why not build an absolutely
minimal Gentoo system on the new machine, using the M.2 or SDD in the new
machine, and then mount the old HDD in the chroot? Then you could just copy
up the world file and config stuff from the chroot into the new M.2
environment and do small rebuilds until you get done? You could do that in
small chunks each night and you'd always have the chroot HDD available.

While I understand the paranoia about the SDD failing it suggests lack of
adequate backups. Any disk can fail on you. If the SDD failure is due to
wear out then (short of infant mortality) that's sometime out in the
future.

You can certainly put the Gentoo work area on an HDD and save write cycles
on the SDD but them you're taking the worst part of Gentoo (all the compute
cycles wasted on building software) and putting them on the slowest part of
your new machine. To me that sounds painful.

I don't think you should worry about booting from an SDD or M.2. That's
completely a read operation. You just want portage/emerge work and log
files somewhere. I'd opt for a 2nd SDD for that. Once the code is built
your Gentoo machine isn't much different than my Kubuntu machine in terms
of how much read vs write there is.

Just my thoughts,
Mark

[-- Attachment #2: Type: text/html, Size: 6926 bytes --]

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[tuxic]

On 04/04 12:30, Mark Knecht wrote:
> On Sat, Apr 4, 2020 at 12:05 PM <tuxic@posteo.de> wrote:
> >
> > On 04/04 07:25, Ashley Dixon wrote:
> > > On Sat, Apr 04, 2020 at 07:34:59PM +0200, tuxic@posteo.de wrote:
> > > > Hi,
> > > >
> > > > I am currently preparing a new harddisc as home for my new Gentoo
> > > > system.
> > > >
> > > > Is it possible to recreate exactlu the same pool of
> > > > applications/programs/libraries etc..., which my current
> > > > system have - in one go?
> > > >
> > > > That is: Copy <something> from the current system into
> > > > the chroot environment, fire up emerge, go to bed and
> > > > tommorow morning the new system ready...?
> > > >
> > > > Does this <something> exists and is it reasonable to do
> > > > it this way?
> > > >
> > > > Thanks for any hint in advance!
> > > > Stay healthy!
> > > > Cheers,
> > > > Meino
> > >
> > > Do you also want to copy configuration and data files, attaining a total
> > > replica ? If so, copying the world file is a start, but then you'll
> have a
> > > plethora of /etc and /var files through which to sift.
> > >
> > > Perhaps a little more detailed context to your problem would allow for
> more
> > > accurate/helpful recommendations ? I.e., are you looking for
> near-complete
> > > duplication, or just a collection of familiar packages which happens to
> be on
> > > similar hardware ?
> > >
> > > --
> > >
> > > Ashley Dixon
> > > suugaku.co.uk
> > >
> > > 2A9A 4117
> > > DA96 D18A
> > > 8A7B B0D2
> > > A30E BF25
> > > F290 A8AA
> > >
> >
> > Hi Ashley,
> >
> > ok...here it comes...the story of "System 9 outer space" <hhhrmrrm>:
> >
> > My current system has two drawbacks: The harddisc has become way to small
> and I don't
> > want more than one harddisc in my PC.
> > My old PC is 12 years old...and it is - in relation to software of today
> > (especially blender) - much too slow and the main memory is also not
> > of sufficient size.
> >
> > Expanding of the old system is -- in respect to its age -- economically
> wise
> > not the correct decision...I think.
> >
> > So I bought the parts of a new PC (again AMD), build a new PC, inserts
> > the harddisc of the old PC and booted the system. which works fine.
> >
> > Now I have an old system on the harddisc whith some legacy structure
> > (I think), which I want to replace with """the same system""" --
> > freshly rebuild in a way that I can fire up one command before I go to
> > bed only to recognize next morning, that I forget to become root
> > beforehand... ;)
> >
> > Jokes aside:
> > I want to try to recompile every Gentoo related stuff in the new system,
> > which was present in the old system (application-wise, and not
> > necessarily version-wise).
> >
> > This gives me the chance to use a new set of cpuflags given by
> cpuid2cpuflags, too.
> > (by the way: This command show far less flags than diplayed via the
> > command 'lscpu'....is cpuid2cpuflags uptodate?)
> >
> > For the configuration I will move a lot of stuff from the current
> > system to the new system. That's ok...
> >
> > For the "partition and boot" scheme (not the correct words...sorry no
> > native speaker ahead....;) ) I thought of this:
> >
> > One hardisc (3T) with the complete system including 256 GB root. The
> > harddisc has a GPT and has a grub bootloader also. This makes this
> > harddisc bootable as "standalone solution".
> >
> > Additonally there is a M.2 NvME SSD
> > It is a mirror of the root partion with all directories, to which are
> often
> > written to (/var/tmp, /tmp,...) mounted on tmpfs.
> >
> > The plan is to update (emerge ... ) the system with in a way, that
> > less as possible writes hits the SSD (for example by mounting certain
> > parts of the filesystem on tmpfs) and use the root on harddisc as
> > backup.
> >
> > The "real backup" will be a image copy of the harddisc to another
> > identical harddisc which I will create on a regular basis.
> >
> > This way I always have a bootable system. The best backup is
> > worthless, if I don't have a system to read it....
> >
> > One thing:
> > Would it possible to boot grub from harddisc, which in turn has
> > entries in the menu to boot either from harddisc or (as default)
> > from SSD? I don't care about the 23.6573 ms it takes longer to
> > read grub stage 1 and 2 from harddisc instead off the SSD... ;)
> >
> > Feeling still a bit paranoid when it comes to SSDs. I know, its
> > supersticous...but... ;)
> >
> > Is this somehow reasonable...or...?
> >
> > Cheers!
> > Meino
> >
> 
> Possibly I'm still misunderstanding. However your description here is
> helpful.
> 
> Maybe you're approaching this the hard way? Why not build an absolutely
> minimal Gentoo system on the new machine, using the M.2 or SDD in the new
> machine, and then mount the old HDD in the chroot? Then you could just copy
> up the world file and config stuff from the chroot into the new M.2
> environment and do small rebuilds until you get done? You could do that in
> small chunks each night and you'd always have the chroot HDD available.
> 
> While I understand the paranoia about the SDD failing it suggests lack of
> adequate backups. Any disk can fail on you. If the SDD failure is due to
> wear out then (short of infant mortality) that's sometime out in the
> future.
> 
> You can certainly put the Gentoo work area on an HDD and save write cycles
> on the SDD but them you're taking the worst part of Gentoo (all the compute
> cycles wasted on building software) and putting them on the slowest part of
> your new machine. To me that sounds painful.
> 
> I don't think you should worry about booting from an SDD or M.2. That's
> completely a read operation. You just want portage/emerge work and log
> files somewhere. I'd opt for a 2nd SDD for that. Once the code is built
> your Gentoo machine isn't much different than my Kubuntu machine in terms
> of how much read vs write there is.
> 
> Just my thoughts,
> Mark


Hi,

ok...I am too tired to think any thinkful thought now...or so...
since I am up at 3:30 in the morning (no, I am not ill...;) 
I am down early...

As soon I have booted myself tomorrow I will be happy to post
again ...

One question completly offtopic...

Ashley, you wrote:
>  Don't worry, I understood you well. Aside from your marvellous usage of
>  ellipses ("..."), your communication in English is fine :)

Thank youfor your nice words! :)

I have some problems to understand, whether I understood...

In the german language the "'s are often used to express the
opposite of what is written in words. For example:

    What a "nice" weather it is...!

will say:

    For heavens sake, what the hell all this rain is coming
    from!!!???

another usage in german is to mark something, which is using
not quite the correct wording in the hope the reade will understand
nonetheless:

    This "rotating something" is cooling my CPU.

"rotating something" stands for "fan".

Is the English usage of the ellipses identical to that.
Otherwise I had screwed up my sentences beyond repair....

You wrote "Aside from your marvellous usage of "'s ... is fine"

Translated via dict.cc and translated back to english:

"Not takeing into account the marvellous usage of ellipses...is fine"

Long story short: Why something is fine, if something marvellous needs
NOT to be taken into account for that?

Don't get me wrong here, Ashley: This is BY NO MEANS any form of
critic. I am simply curious, whu I struggle to understand a context
which seems unlogical to me...which is not,,,but I don,t know why.

Ok...I really need sleep... :) :)

Stay healthy!
Read you tommorow! :)
Cheers!
Meino

[OT] Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Peter Humphrey]

On Saturday, 4 April 2020 20:59:56 BST tuxic@posteo.de wrote:

> I have some problems to understand, whether I understood...
> 
> In the german language the "'s are often used to express the
> opposite of what is written in words. For example:
> 
>     What a "nice" weather it is...!
> 
> will say:
> 
>     For heavens sake, what the hell all this rain is coming
>     from!!!???

It's the same in English, except that "weather" is an uncountable noun, so you 
can't have "a weather" - it's just "weather".

I'd like to put in a word about punctuation. In English it is not permissible 
to put a comma between the verb and its object*. It seems to be required in 
German, but it destroys the natural flow in English. Thus, your first sentence 
quoted above should not include a comma.

HTH.

*   Sometimes you'll see a pair of commas there, setting off a parenthetical 
expression, but by the nature of those, they don't really contribute to the 
sentence, merely slipping a by-the-way phrase because it fits.

-- 
Regards,
Peter.

Re: [OT] Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Peter Humphrey]

On Sunday, 5 April 2020 10:21:32 BST Peter Humphrey wrote:
> On Saturday, 4 April 2020 20:59:56 BST tuxic@posteo.de wrote:
> > I have some problems to understand, whether I understood...
> > 
> > In the german language the "'s are often used to express the
> > 
> > opposite of what is written in words. For example:
> >     What a "nice" weather it is...!
> > 
> > will say:
> >     For heavens sake, what the hell all this rain is coming
> >     from!!!???
> 
> It's the same in English, except that "weather" is an uncountable noun, so
> you can't have "a weather" - it's just "weather".
> 
> I'd like to put in a word about punctuation. In English it is not
> permissible to put a comma between the verb and its object*. It seems to be
> required in German, but it destroys the natural flow in English. Thus, your
> first sentence quoted above should not include a comma.
> 
> HTH.
> 
> *   Sometimes you'll see a pair of commas there, setting off a parenthetical
> expression, but by the nature of those, they don't really contribute to the
> sentence, merely slipping a by-the-way phrase because it fits.

"...slipping in..."

Even Homer nods.  :(

-- 
Regards,
Peter.

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Grant Taylor]

On 4/4/20 11:34 AM, tuxic@posteo.de wrote:
> Hi,

Hi,

> I am currently preparing a new harddisc as home for my new Gentoo 
> system.
> 
> Is it possible to recreate exactlu the same pool of 
> applications/programs/libraries etc..., which my current system have - 
> in one go?

Baring cosmic influences, I would expect so.

> That is: Copy <something> from the current system into the chroot 
> environment, fire up emerge, go to bed and tommorow morning the new 
> system ready...?
> 
> Does this <something> exists and is it reasonable to do it this way?
> 
> Thanks for any hint in advance!

I think that any given system is the product of it's various components. 
  Change any of those components, and you change the product.

I see the list of components as being at least:

  · world file
  · portage config (/etc/portage)
     · USEs
     · accepted keywords
     · accepted licenses
  · portage files (/usr/portage)
     · this significantly influences the version of packages that get 
installed, which is quite important
  · kernel
     · version
     · config

Copying these things across should get you a quite similar system.  I 
suspect you would be down to how different packages are configured.

But the world file is only one of many parts that make up the system.

I didn't include distfiles because theoretically, you can re-download 
files.  However, I've run into cases where I wasn't able to download 
something and had to transfer (part of) distfiles too.

If you're going to the trouble to keep a system this similar, why not 
simply copy the system from one drive / machine to another?



-- 
Grant. . . .
unix || die

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[tuxic]

On 04/04 03:56, Grant Taylor wrote:
> On 4/4/20 11:34 AM, tuxic@posteo.de wrote:
> > Hi,
> 
> Hi,
> 
> > I am currently preparing a new harddisc as home for my new Gentoo
> > system.
> > 
> > Is it possible to recreate exactlu the same pool of
> > applications/programs/libraries etc..., which my current system have -
> > in one go?
> 
> Baring cosmic influences, I would expect so.
> 
> > That is: Copy <something> from the current system into the chroot
> > environment, fire up emerge, go to bed and tommorow morning the new
> > system ready...?
> > 
> > Does this <something> exists and is it reasonable to do it this way?
> > 
> > Thanks for any hint in advance!
> 
> I think that any given system is the product of it's various components.
> Change any of those components, and you change the product.
> 
> I see the list of components as being at least:
> 
>  · world file
>  · portage config (/etc/portage)
>     · USEs
>     · accepted keywords
>     · accepted licenses
>  · portage files (/usr/portage)
>     · this significantly influences the version of packages that get
> installed, which is quite important
>  · kernel
>     · version
>     · config
> 
> Copying these things across should get you a quite similar system.  I
> suspect you would be down to how different packages are configured.
> 
> But the world file is only one of many parts that make up the system.
> 
> I didn't include distfiles because theoretically, you can re-download files.
> However, I've run into cases where I wasn't able to download something and
> had to transfer (part of) distfiles too.
> 
> If you're going to the trouble to keep a system this similar, why not simply
> copy the system from one drive / machine to another?
> 
> 
> 
> -- 
> Grant. . . .
> unix || die


Hi,

a new morning... :)

Being on the way to install/setup the base system (mostly getting
stage3 uptodate) I came accross kinda inconsistency -- or at least 
it looks like for me.

The system uses a 3T harddisc (and later a SSD) and therefore GPT.
GPT is the sister/brother of an U/EFI boot.

For that the documentation (AMD64 handbook):
https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Disks#Using_UEFI

says:
Default partitioning scheme
Throughout the remainder of the handbook, the following partitioning scheme will be used as a simple example layout:
Partition   Filesystem 	Size 	Description
/dev/sda1   (bootloader) 	2M 	BIOS boot partition
/dev/sda2   ext2 (or fat32 if UEFI is being used) 	128M 	Boot/EFI system partition
/dev/sda3   (swap) 	512M or higher 	Swap partition
/dev/sda4   ext4 	Rest of the disk 	Root partition

and later on at https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/System

FILE /etc/fstabA full /etc/fstab example

/dev/sda2   /boot        ext2    defaults,noatime     0 2
/dev/sda3   none         swap    sw                   0 0
/dev/sda4   /            ext4    noatime              0 1
  
/dev/cdrom  /mnt/cdrom   auto    noauto,user          0 0

Here /boot changes from fat32 to ext2. 

Since this is my first U/EFI system I am a little confused.

Currentlu it looks like the vmlinuz binaries will be installed on 
a FAT32 filesystem. Since the kernel can be launched from a ext4
filesystem I cannot see, why this have to be a FAT32 filesystem.

My plan (if this is possible), is to U/EFI-boot grub, from which
I can select the kernel in question as it has been on my old
system (MBR based).

My current partition table looks like (only relevant parts shown):

Number  Start   End     Size    File system     Name    Flags
 1      1049kB  3146kB  2097kB                  grub    bios_grub
 2      3146kB  137MB   134MB   fat32           boot    boot, esp
 3      137MB   674MB   537MB   linux-swap(v1)  swap
 4      674MB   269GB   268GB   ext4            root

What did I messed up here?

Cheers!
Meino

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Ashley Dixon]

[-- Attachment #1: Type: text/plain, Size: 1406 bytes --]

On Sun, Apr 05, 2020 at 10:17:41AM +0200, tuxic@posteo.de wrote:
> Here /boot changes from fat32 to ext2. 
> Since this is my first U/EFI system I am a little confused.

Good morning Meino,

For U.E.F.I., the boot partition (/dev/sda2) must be FAT32. In your fstab file,
this is written as (the last three arguments may vary for your system):

/dev/sda2 /boot vfat defaults,noatime 0 2
https://wiki.gentoo.org/wiki/Handbook:AMD64/Full/Installation#Using_UEFI

A conversation about the (sometimes-interchangeable) uses of "fat" vs. "vfat",
see https://docs.microsoft.com/en-us/previous-versions//cc750354(v=technet.10)

Regarding your question last night on the use of ellipses, unfortunately your
e-mail server (posteo.de) does not like my (domestic) I.P.\ address, and thus
assumed it spam when I tried to reply. I doubt the fine people of gentoo-user
will have much regard for a conversation about English linguistics. If you would
like to white-list my domain and I.P.\ (mail.suugaku.co.uk, suugaku.co.uk, and
90.193.124.236), I'll re-send in a few hours.

In short, I included an example of ellipses because as you are a non-native
English speaker, I did not want to cause undue confusion if you did not know the
meaning of the word 'ellipses'. That is all; no hidden meanings :)

-- 

Ashley Dixon
suugaku.co.uk

2A9A 4117
DA96 D18A
8A7B B0D2
A30E BF25
F290 A8AA


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 822 bytes --]

On Sun, 5 Apr 2020 10:28:50 +0100, Ashley Dixon wrote:

> > Here /boot changes from fat32 to ext2. 
> > Since this is my first U/EFI system I am a little confused.  
> 
> Good morning Meino,
> 
> For U.E.F.I., the boot partition (/dev/sda2) must be FAT32. In your
> fstab file, this is written as (the last three arguments may vary for
> your system):
> 
> /dev/sda2 /boot vfat defaults,noatime 0 2
> https://wiki.gentoo.org/wiki/Handbook:AMD64/Full/Installation#Using_UEFI

This isn't strictly true, the ESP must be vfat, but you can still have an
ext? /boot. However, it generally makes sense to have the ESP and /boot
as the same partition, that's how I generally do it. either way, the
handbook appears to need updating in this respect.


-- 
Neil Bothwick

Don't just do something, sit there!

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Ashley Dixon]

[-- Attachment #1: Type: text/plain, Size: 565 bytes --]

On Sun, Apr 05, 2020 at 01:52:52PM +0100, Neil Bothwick wrote:
> This isn't strictly true, the ESP must be vfat, but you can still have an
> ext? /boot. However, it generally makes sense to have the ESP and /boot
> as the same partition, that's how I generally do it. either way, the
> handbook appears to need updating in this respect.

Sorry, my mistake. I wonder, would there ever be any valid reason to have the
E.S.P.\ and /boot as different partitions ?

-- 

Ashley Dixon
suugaku.co.uk

2A9A 4117
DA96 D18A
8A7B B0D2
A30E BF25
F290 A8AA


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Michael]

[-- Attachment #1: Type: text/plain, Size: 1605 bytes --]

On Sunday, 5 April 2020 13:56:55 BST Ashley Dixon wrote:
> On Sun, Apr 05, 2020 at 01:52:52PM +0100, Neil Bothwick wrote:
> > This isn't strictly true, the ESP must be vfat, but you can still have an
> > ext? /boot. However, it generally makes sense to have the ESP and /boot
> > as the same partition, that's how I generally do it. either way, the
> > handbook appears to need updating in this respect.
> 
> Sorry, my mistake. I wonder, would there ever be any valid reason to have
> the E.S.P.\ and /boot as different partitions ?

Having two separate partitions for the same reason sounds superfluous, but 
there may be special reasons.

For example, you may want to have separate OS installations and each with 
their own standalone /boot partition.  When you uninstall one OS you won't 
have to fish into the EFI Boot partition for any entries specific to this OS.  
These days most OS tend to behave and install their boot manager/kernel images 
into separate subdirectories within the EFI Boot partition - but there are no 
guarantees.  What if rogue-OS decides to install its vmlinuz ontop of your own 
images, without even asking first?

Perhaps you could try chainloading various boot managers to try them out, so 
the EFI firmware could boot GRUB, which could chainload rEFInd, which could 
chainload C:\boot\bcd.exe, etc. After you finish with your experiment you 
could decide to remove a particular /boot partition with its boot manager 
altogether.  Tenuous, I know, but I'm trying to think for reasons to keep a 
separate /boot partition, without actually having a need for it myself.  ;-)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 645 bytes --]

On Sun, 5 Apr 2020 13:56:55 +0100, Ashley Dixon wrote:

> > This isn't strictly true, the ESP must be vfat, but you can still
> > have an ext? /boot. However, it generally makes sense to have the ESP
> > and /boot as the same partition, that's how I generally do it. either
> > way, the handbook appears to need updating in this respect.  
> 
> Sorry, my mistake. I wonder, would there ever be any valid reason to
> have the E.S.P.\ and /boot as different partitions ?

Possibly if you have multiple distros installed. That way you could
isolate each distro.


-- 
Neil Bothwick

Politically Incorrect -- and damn proud of it!

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Peter Humphrey]

On Sunday, 5 April 2020 13:56:55 BST Ashley Dixon wrote:

> I wonder, would there ever be any valid reason to have the E.S.P.\ and /boot
> as different partitions ?

I found I had to do so. I couldn't get Neil's prererred layout to work. I 
forget the details now, but I only managed to build a usable system by leaving 
a small unformatted partition before the VFAT /boot partition. The first 
partition is marked bios_grub and the second 'boot, esp'.

I did try later to combine the two, but I still couldn't get it to work. 
Perhaps the BIOS has something odd in it.

-- 
Regards,
Peter.

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 825 bytes --]

On Sun, 05 Apr 2020 19:08:24 +0100, Peter Humphrey wrote:

> > I wonder, would there ever be any valid reason to have the E.S.P.\
> > and /boot as different partitions ?  
> 
> I found I had to do so. I couldn't get Neil's prererred layout to work.
> I forget the details now, but I only managed to build a usable system
> by leaving a small unformatted partition before the VFAT /boot
> partition. The first partition is marked bios_grub and the second
> 'boot, esp'.
> 
> I did try later to combine the two, but I still couldn't get it to
> work. Perhaps the BIOS has something odd in it.

If you are using a BIOS system, you need the compatibility partition at
the start. If you are using UEFI it shouldn't be necessary.


-- 
Neil Bothwick

Tribble math: * + * = ***********************************

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Peter Humphrey]

On Sunday, 5 April 2020 20:39:47 BST Neil Bothwick wrote:
> On Sun, 05 Apr 2020 19:08:24 +0100, Peter Humphrey wrote:
> > > I wonder, would there ever be any valid reason to have the E.S.P.\
> > > and /boot as different partitions ?
> > 
> > I found I had to do so. I couldn't get Neil's prererred layout to work.
> > I forget the details now, but I only managed to build a usable system
> > by leaving a small unformatted partition before the VFAT /boot
> > partition. The first partition is marked bios_grub and the second
> > 'boot, esp'.
> > 
> > I did try later to combine the two, but I still couldn't get it to
> > work. Perhaps the BIOS has something odd in it.
> 
> If you are using a BIOS system, you need the compatibility partition at
> the start. If you are using UEFI it shouldn't be necessary.

No, I know, but it is - unless I'm doing something daft, of course.

-- 
Regards,
Peter.

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[antlists]

On 05/04/2020 13:52, Neil Bothwick wrote:
> This isn't strictly true, the ESP must be vfat, but you can still have an
> ext? /boot.

This isn't true at all - you've got the cart before the horse. The 
original (U)EFI spec comes from Sun, I believe, with no vfat in sight.

A standards-compliant factory-fresh Mac boots using UEFI with no vfat in 
sight.

A standards-compliant UEFI firmware MUST BE CAPABLE of booting from (a 
certain version of) vfat. So if you use a vfat partition the spec says 
it must work. It doesn't demand you use vfat, so Macs use HFS+ or 
whatever it is, and there is no reason why eg System7 couldn't write 
their own firmware to use ext2 or whatever.

Cheers,
Wol

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 663 bytes --]

On Mon, 6 Apr 2020 22:02:04 +0100, antlists wrote:

> > This isn't strictly true, the ESP must be vfat, but you can still
> > have an ext? /boot.  
> 
> This isn't true at all - you've got the cart before the horse. The 
> original (U)EFI spec comes from Sun, I believe, with no vfat in sight.
> 
> A standards-compliant factory-fresh Mac boots using UEFI with no vfat
> in sight.

That's true, but firmware on commodity PC motherboards can only be relied
upon to handle vfat. So while my use of "must" is a bit strong, it should
be vfat if you want to be sure it will boot on a PC.


-- 
Neil Bothwick

Top Oxymorons Number 2: Exact estimate

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Michael]

[-- Attachment #1: Type: text/plain, Size: 1161 bytes --]

On Monday, 6 April 2020 22:15:20 BST Neil Bothwick wrote:
> On Mon, 6 Apr 2020 22:02:04 +0100, antlists wrote:
> > > This isn't strictly true, the ESP must be vfat, but you can still
> > > have an ext? /boot.
> > 
> > This isn't true at all - you've got the cart before the horse. The
> > original (U)EFI spec comes from Sun, I believe, with no vfat in sight.
> > 
> > A standards-compliant factory-fresh Mac boots using UEFI with no vfat
> > in sight.
> 
> That's true, but firmware on commodity PC motherboards can only be relied
> upon to handle vfat. So while my use of "must" is a bit strong, it should
> be vfat if you want to be sure it will boot on a PC.

Perhaps older UEFI specifications allowed Mac-baked filesystems, or perhaps 
Apple were/are doing their own thing.  The current UEFI specification 
*requires* a FAT 12/16/32 filesystem type on an ESP partition to boot an OS 
image/bootloader from - see section '13.3 File System Format':

https://uefi.org/sites/default/files/resources/UEFI_Spec_2_8_final.pdf

I can't recall the ESP partition format last time I installed and dual-booted 
Linux on a MacBookPro, c.2014, but I'd wager it was VFAT.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[antlists]

On 07/04/2020 00:38, Michael wrote:
> Perhaps older UEFI specifications allowed Mac-baked filesystems, or perhaps
> Apple were/are doing their own thing.  The current UEFI specification
> *requires*  a FAT 12/16/32 filesystem type on an ESP partition to boot an OS
> image/bootloader from - see section '13.3 File System Format':

Reading the spec, it said "must *support*", not must *require*.

What I was told - by someone I see no reason to disbelieve - was that if 
a vendor wants to support a different filesystem *in addition*, provided 
it supports all the calls then there's no problem.

(Incidentally, if that's the final spec, I think I've spotted a mistake 
in it - it clearly doesn't actually mean what it says in at least one 
place ...)

Cheers,
Wol

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 1529 bytes --]

Am Dienstag, 7. April 2020, 01:38:01 CEST schrieb Michael:
> On Monday, 6 April 2020 22:15:20 BST Neil Bothwick wrote:
> > On Mon, 6 Apr 2020 22:02:04 +0100, antlists wrote:
> > > > This isn't strictly true, the ESP must be vfat, but you can still
> > > > have an ext? /boot.
> > >
> > > This isn't true at all - you've got the cart before the horse. The
> > > original (U)EFI spec comes from Sun, I believe, with no vfat in sight.
> > >
> > > A standards-compliant factory-fresh Mac boots using UEFI with no vfat
> > > in sight.
> >
> > That's true, but firmware on commodity PC motherboards can only be relied
> > upon to handle vfat. So while my use of "must" is a bit strong, it should
> > be vfat if you want to be sure it will boot on a PC.
>
> Perhaps older UEFI specifications allowed Mac-baked filesystems, or perhaps
> Apple were/are doing their own thing.  The current UEFI specification
> *requires* a FAT 12/16/32 filesystem type on an ESP partition to boot an OS
> image/bootloader from - see section '13.3 File System Format':
>
> https://uefi.org/sites/default/files/resources/UEFI_Spec_2_8_final.pdf
>
> I can't recall the ESP partition format last time I installed and
> dual-booted Linux on a MacBookPro, c.2014, but I'd wager it was VFAT.

I checked my 2007 Mac Mini and its ESP is vfat.  Of course, newer Macs or
other models might use something else.

--
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[William Kenworthy]

On 7/4/20 5:15 am, Neil Bothwick wrote:
> On Mon, 6 Apr 2020 22:02:04 +0100, antlists wrote:
>
>>> This isn't strictly true, the ESP must be vfat, but you can still
>>> have an ext? /boot.
>> This isn't true at all - you've got the cart before the horse. The
>> original (U)EFI spec comes from Sun, I believe, with no vfat in sight.
>>
>> A standards-compliant factory-fresh Mac boots using UEFI with no vfat
>> in sight.
> That's true, but firmware on commodity PC motherboards can only be relied
> upon to handle vfat. So while my use of "must" is a bit strong, it should
> be vfat if you want to be sure it will boot on a PC.
>
>
Years ago I installed refind to dual boot gentoo and Win10 on a surface 
pro4 tablet.  The recommendation then was / is a linux FS (btrfs in my 
case) and vfat (because MS will always be vfat - no choice!) for the EFI 
mounted on /boot/efi:

bunyip ~ # mount|grep nv
/dev/nvme0n1p6 on / type btrfs 
(rw,noatime,compress=lzo,ssd,discard,space_cache,subvolid=5,subvol=/)
/dev/nvme0n1p1 on /boot/efi type vfat 
(rw,noatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro)

bunyip ~ # tree /boot
/boot
├── amd-uc.img
├── early_ucode.cpio
└── efi
     └── EFI
         ├── Boot
         │   └── bootx64.efi
         ├── gentoo
         │   ├── initramfs-5.4.25-gentoo-x86_64.img
         │   ├── initramfs-5.4.26-gentoo-x86_64.img
         │   ├── initramfs-5.5.10-gentoo-x86_64.img
         │   ├── System.map-5.4.25-gentoo-x86_64
         │   ├── System.map-5.4.26-gentoo-x86_64
         │   ├── System.map-5.5.10-gentoo-x86_64
         │   ├── vmlinuz-5.4.25-gentoo-x86_64
         │   ├── vmlinuz-5.4.26-gentoo-x86_64
         │   └── vmlinuz-5.5.10-gentoo-x86_64
         ├── Microsoft
         │   ├── Boot
         │   │   ├── ar-SA
         │   │   │   ├── bootmgfw.efi.mui
         │   │   │   └── bootmgr.efi.mui
         │   │   ├── BCD
         │   │   ├── BCD.LOG
         │   │   ├── BCD.LOG1
         │   │   ├── BCD.LOG2
         │   │   ├── bg-BG
         │   │   │   ├── bootmgfw.efi.mui
         │   │   │   └── bootmgr.efi.mui
         │   │   ├── bootmgfw.efi
         │   │   ├── bootmgr.efi
         │   │   ├── BOOTSTAT.DAT
         │   │   ├── boot.stl
         │   │   ├── cs-CZ
...

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[Michael]

[-- Attachment #1: Type: text/plain, Size: 3995 bytes --]

Hi Meino,

On Sunday, 5 April 2020 09:17:41 BST tuxic@posteo.de wrote:

> Hi,
> 
> a new morning... :)
> 
> Being on the way to install/setup the base system (mostly getting
> stage3 uptodate) I came accross kinda inconsistency -- or at least
> it looks like for me.
> 
> The system uses a 3T harddisc (and later a SSD) and therefore GPT.
> GPT is the sister/brother of an U/EFI boot.
> 
> For that the documentation (AMD64 handbook):
> https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Disks#Using_UEFI
> 
> says:
> Default partitioning scheme
> Throughout the remainder of the handbook, the following partitioning scheme
> will be used as a simple example layout: 

I think the following table provided in the handbook was probably written some 
time ago, when the migration was happening between legacy BIOS and UEFI MoBos.  
Others may know more about the rationale behind the partitioning scheme given 
as an example in the handbook, but for what it's worth I share my 
understanding below:

> Partition   Filesystem 	Size       Description
> /dev/sda1   (bootloader) 	2M 	BIOS boot partition

The above is probably a manually created 'GPT protective MBR'.  This is now 
created as a default by modern partitioning tools, but not if you're using 
some old Knoppix CD you had burned in the early '00s.  Its purpose is to add 
some code in the first 1M of the disk (LBA 0) to signify to a legacy BIOS or 
OS the partition table is not bootable.  Old partitioning tools/OS' will show 
the disk as already partitioned with 'Unknown Partition Type', thus prompting 
the user not to mess up the partition table/partitions on this disk.  If you 
use a modern fdisk/gptdisk/parted, etc. you won't see this first partition 
mentioned above, but you will see the first partition starting at 1M, or on a 
512B sector size at 2048.


> /dev/sda2   ext2 (or fat32 if UEFI is being used)  128M  Boot/EFI system

This the partition the UEFI firmware will jump to and load bootable code from.  
The GRUB efi boot binary (grubx64.efi) and any other kernel/initrd efi code 
will be stored here.


> and later on at
> https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/System
> 
> FILE /etc/fstabA full /etc/fstab example
> 
> /dev/sda2   /boot        ext2    defaults,noatime     0 2
> /dev/sda3   none         swap    sw                   0 0
> /dev/sda4   /            ext4    noatime              0 1
> 
> /dev/cdrom  /mnt/cdrom   auto    noauto,user          0 0
> 
> Here /boot changes from fat32 to ext2.

I think this is because the handbook really needs to be updated.  It mixes old 
with new.  All UEFI MoBos require a VFAT partition to boot from.


> Since this is my first U/EFI system I am a little confused.
> 
> Currentlu it looks like the vmlinuz binaries will be installed on
> a FAT32 filesystem. Since the kernel can be launched from a ext4
> filesystem I cannot see, why this have to be a FAT32 filesystem.

You can chainload a vmlinuz binary stored on any other partition number and 
type from a boot manager (e.g. GRUB).  The boot manager will have to be stored 
on the VFAT EFI boot partition for the MoBo UEFI firmware to be able to boot 
it.  Future UEFI firmware may be able to boot from ext2, but AFAIK not 
presently.


> My plan (if this is possible), is to U/EFI-boot grub, from which
> I can select the kernel in question as it has been on my old
> system (MBR based).
> 
> My current partition table looks like (only relevant parts shown):
> 
> Number  Start   End     Size    File system     Name    Flags
>  1      1049kB  3146kB  2097kB                  grub    bios_grub
>  2      3146kB  137MB   134MB   fat32           boot    boot, esp
>  3      137MB   674MB   537MB   linux-swap(v1)  swap
>  4      674MB   269GB   268GB   ext4            root

Unless you intend to use the disk both on MBR and on UEFI MoBos 
interchangeably, I suggest you stick with the GPT partitioning scheme and a 
VFAT EFI Boot partition where GRUB and vmlinuz will be stored.

HTH.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] ...recreating exactly the same applications on a new harddisc?

[tuxic]

On 04/05 10:46, Michael wrote:
> Hi Meino,
> 
> On Sunday, 5 April 2020 09:17:41 BST tuxic@posteo.de wrote:
> 
> > Hi,
> > 
> > a new morning... :)
> > 
> > Being on the way to install/setup the base system (mostly getting
> > stage3 uptodate) I came accross kinda inconsistency -- or at least
> > it looks like for me.
> > 
> > The system uses a 3T harddisc (and later a SSD) and therefore GPT.
> > GPT is the sister/brother of an U/EFI boot.
> > 
> > For that the documentation (AMD64 handbook):
> > https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Disks#Using_UEFI
> > 
> > says:
> > Default partitioning scheme
> > Throughout the remainder of the handbook, the following partitioning scheme
> > will be used as a simple example layout: 
> 
> I think the following table provided in the handbook was probably written some 
> time ago, when the migration was happening between legacy BIOS and UEFI MoBos.  
> Others may know more about the rationale behind the partitioning scheme given 
> as an example in the handbook, but for what it's worth I share my 
> understanding below:
> 
> > Partition   Filesystem 	Size       Description
> > /dev/sda1   (bootloader) 	2M 	BIOS boot partition
> 
> The above is probably a manually created 'GPT protective MBR'.  This is now 
> created as a default by modern partitioning tools, but not if you're using 
> some old Knoppix CD you had burned in the early '00s.  Its purpose is to add 
> some code in the first 1M of the disk (LBA 0) to signify to a legacy BIOS or 
> OS the partition table is not bootable.  Old partitioning tools/OS' will show 
> the disk as already partitioned with 'Unknown Partition Type', thus prompting 
> the user not to mess up the partition table/partitions on this disk.  If you 
> use a modern fdisk/gptdisk/parted, etc. you won't see this first partition 
> mentioned above, but you will see the first partition starting at 1M, or on a 
> 512B sector size at 2048.
> 
> 
> > /dev/sda2   ext2 (or fat32 if UEFI is being used)  128M  Boot/EFI system
> 
> This the partition the UEFI firmware will jump to and load bootable code from.  
> The GRUB efi boot binary (grubx64.efi) and any other kernel/initrd efi code 
> will be stored here.
> 
> 
> > and later on at
> > https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/System
> > 
> > FILE /etc/fstabA full /etc/fstab example
> > 
> > /dev/sda2   /boot        ext2    defaults,noatime     0 2
> > /dev/sda3   none         swap    sw                   0 0
> > /dev/sda4   /            ext4    noatime              0 1
> > 
> > /dev/cdrom  /mnt/cdrom   auto    noauto,user          0 0
> > 
> > Here /boot changes from fat32 to ext2.
> 
> I think this is because the handbook really needs to be updated.  It mixes old 
> with new.  All UEFI MoBos require a VFAT partition to boot from.
> 
> 
> > Since this is my first U/EFI system I am a little confused.
> > 
> > Currentlu it looks like the vmlinuz binaries will be installed on
> > a FAT32 filesystem. Since the kernel can be launched from a ext4
> > filesystem I cannot see, why this have to be a FAT32 filesystem.
> 
> You can chainload a vmlinuz binary stored on any other partition number and 
> type from a boot manager (e.g. GRUB).  The boot manager will have to be stored 
> on the VFAT EFI boot partition for the MoBo UEFI firmware to be able to boot 
> it.  Future UEFI firmware may be able to boot from ext2, but AFAIK not 
> presently.
> 
> 
> > My plan (if this is possible), is to U/EFI-boot grub, from which
> > I can select the kernel in question as it has been on my old
> > system (MBR based).
> > 
> > My current partition table looks like (only relevant parts shown):
> > 
> > Number  Start   End     Size    File system     Name    Flags
> >  1      1049kB  3146kB  2097kB                  grub    bios_grub
> >  2      3146kB  137MB   134MB   fat32           boot    boot, esp
> >  3      137MB   674MB   537MB   linux-swap(v1)  swap
> >  4      674MB   269GB   268GB   ext4            root
> 
> Unless you intend to use the disk both on MBR and on UEFI MoBos 
> interchangeably, I suggest you stick with the GPT partitioning scheme and a 
> VFAT EFI Boot partition where GRUB and vmlinuz will be stored.
> 
> HTH.


Hi Michael,

Your posting helps me a LOT!!! :)

Thank you very much!!! :)

Cheers!
Meino