[gentoo-user] Install PreQualifying Matrix

[gentoo-user] Install PreQualifying Matrix

[James]

Hello,

So another day, another borked set of installs. This really got me thinking
as I have been munging around a multitude of gentoo(ish) installation
options. What I really would appreciate is some feedback on the Planning
Questions listed below, as to help folks organized their thoughts and
hardware details BEFORE actually performing an install or test-drive.


It has dawned on me that an expansive set of Planning Questions that should
be answered prior to installation might be one posible way for those new
to gentoo to decide critial issues before choosing from the possible install
semantics for their given set of choices (constraints).  As the matrix of
installation choices grows, so to can options for installation. If a given 
set of choices does not have a viable installation pathway, of is just not
recommended, then that should be clearly explained.


Later on, a web interface and (a similarly in functional) GUI that is
extensible for installation media also) can be developed to enhance the 
presentation of the decision processes.  A graphical decision matrix 
(row-column table initially). The handbook seems to be sort of moving
this direction anyway. A recent discussion of the dev list showed
encouragement for pointing gentoo-noobs to some of the gentoo derivative
distros for a quick install experience.


Many/most of these options exist, some atrophying currently, it's just not
straightforward for folks to discern the best route to their desired final
result. When new installation semantics [1]  mature, the installation matrix
can be modified to include those options as links.



        Install PreQualifying Matrx::QUESTIONS
        ....................................................................
        ....................................................................
        ....................................................................
        Live Testdrive options before installation(usb/cd/dvd)::

        Intended Usage (workstation/server/device/)

        Hardware or Vitual installation::

        PC mobo or tablet/embedded/device::

        Processor/Ram characteristics::

        MBR vs (u)EFI (type of mobo)::

        Single or Multi or RAID disk configuration::

        OpenRC or Systemd::  

        Grub1 vs Grub2 or other boot-semantics::

        File System type(s)::


James

[1] https://wiki.gentoo.org/wiki/Project:RelEng_GRS

Re: [gentoo-user] Install PreQualifying Matrix

[Dale]

James wrote:
> Hello,
>
> So another day, another borked set of installs. This really got me thinking
> as I have been munging around a multitude of gentoo(ish) installation
> options. What I really would appreciate is some feedback on the Planning
> Questions listed below, as to help folks organized their thoughts and
> hardware details BEFORE actually performing an install or test-drive.
>
>
> It has dawned on me that an expansive set of Planning Questions that should
> be answered prior to installation might be one posible way for those new
> to gentoo to decide critial issues before choosing from the possible install
> semantics for their given set of choices (constraints).  As the matrix of
> installation choices grows, so to can options for installation. If a given 
> set of choices does not have a viable installation pathway, of is just not
> recommended, then that should be clearly explained.
>
>
> Later on, a web interface and (a similarly in functional) GUI that is
> extensible for installation media also) can be developed to enhance the 
> presentation of the decision processes.  A graphical decision matrix 
> (row-column table initially). The handbook seems to be sort of moving
> this direction anyway. A recent discussion of the dev list showed
> encouragement for pointing gentoo-noobs to some of the gentoo derivative
> distros for a quick install experience.
>
>
> Many/most of these options exist, some atrophying currently, it's just not
> straightforward for folks to discern the best route to their desired final
> result. When new installation semantics [1]  mature, the installation matrix
> can be modified to include those options as links.
>
>
>
>         Install PreQualifying Matrx::QUESTIONS
>         ....................................................................
>         ....................................................................
>         ....................................................................
>         Live Testdrive options before installation(usb/cd/dvd)::
>
>         Intended Usage (workstation/server/device/)
>
>         Hardware or Vitual installation::
>
>         PC mobo or tablet/embedded/device::
>
>         Processor/Ram characteristics::
>
>         MBR vs (u)EFI (type of mobo)::
>
>         Single or Multi or RAID disk configuration::
>
>         OpenRC or Systemd::  
>
>         Grub1 vs Grub2 or other boot-semantics::
>
>         File System type(s)::
>
>
> James
>
> [1] https://wiki.gentoo.org/wiki/Project:RelEng_GRS
>
>
>
>


I can't help much but will give my 2 cents worth.   Grub, I switched to
grub2 and it works.  If you decide on grub1, keep in mind, I don't think
it is maintained anymore.  Some "new" stuff may not ever work with it
and you may end up switching whether you want to or not.  I think it is
Neil that uses some other boot loader that he is happy with.  Maybe he
will chime in on it's name.  I think it is a lot like Grub1 but
currently maintained. 

I used resiserfs for ages.  It sort of died a while back and at this
point, I'm not sure if anyone is maintaining it at all.  When I switched
from reiserfs, I went to ext4.  So far, I've had a drive to outright
start failing hardware wise with only a loss of the data that was on
that bad spot.  The rest was fine.  I have a UPS here so I can't really
say much about how it handles a power fail or being rudely removed.  All
I can say is this, I'm happy with it so far.  Others here like the
shiney new BTRFS.  I don't use it but maybe others can share their
experience.  I also use LVM and it just works, so far. 

I'm not sure what you want regarding Processor and Ram.  I have a AMD
Phenom II X4 955.  That's 4 cores running at 3.2GHz.  Libreoffice
usually takes around 2 hours to compile.  Firefox takes around 45
minutes.  I have 16Gbs and have portage's work directory on tmpfs.  I
hope to upgrade both CPU and ram at some point.  Hope to go to 8 cores
at 4GHz and to 32GBs of ram.  Hope.   Ka ching.  $$$$$$

I use openrc.  Not going to start a flame war so leaving it at that.  ;-) 

Hope other will also share and help give you ideas. 

Dale

:-)  :-)

[gentoo-user] Re: Install PreQualifying Matrix

[James]

Dale <rdalek1967 <at> gmail.com> writes:


> James wrote:

> >  What I really would appreciate is some feedback on the Planning
> > Questions listed below, as to help folks organized their thoughts and
> > hardware details BEFORE actually performing an install or test-drive.


> > Many/most of these options exist

> >         Install PreQualifying Matrx::QUESTIONS

> >         Live Testdrive options before installation(usb/cd/dvd)::

> >         Intended Usage (workstation/server/device/)

> >         Hardware or Vitual installation::

> >         PC mobo or tablet/embedded/device::

> >         Processor/Ram characteristics::

> >         MBR vs (u)EFI (type of mobo)::

> >         Single or Multi or RAID disk configuration::

> >         OpenRC or Systemd::  

> >         Grub1 vs Grub2 or other boot-semantics::

> >         File System type(s)::

> Hope other will also share and help give you ideas. 

> Dale


Hello Dale,

Acutally answering the question, with comments is a good 
idea.

But what I had in mind, that is much more pressing
is a list of additional questions, or
re-ordering the questions
or re-stating the quesions
or matrix logic on the causal relationships between 
these quesions and other questions
as to conclusion of valid install options
is more of what I had in mind.


Once that is reasoably vetted then
I would look for some statisical inferences
on the actual answers to these quetions

as well as valid install links
like sabayon for gentoo(ish) systemd
like calculate-linus for gentoo(ish)openrc
like pentoo for gentoo-penetration systems
like zentoo for gentoo CI systems
Like funtoo as an option install
like gentooliveUSB   for a gentoo + persistence experience.


I think this sort of approach will take some stress off of the
gentoo-user list and handbook whilst Blueness brings maturity
to his efforts; he alreayd has lilblue, tinhat and tor-ramdisk
gentoo installs, so he is one of those guys that can single-handedly
solve this crisis: should he put his fingers to the task.
MaffBlaster has been very quite of late.....

Blueness is a wonderful and collegial type of dev
and is currently seeking input on his 'alpha' ideas::

https://wiki.gentoo.org/wiki/Project:RelEng_GRS


THANKS!
James

Re: [gentoo-user] Install PreQualifying Matrix

[Jeremi Piotrowski]

On Mon, 17 Aug 2015, James wrote:
> So another day, another borked set of installs. 

You are aware that for every failed install that comes to this mailing
lists there are countless that go just fine, right?


Planning questions are an OK-ish idea, but I surely wouldn't link to
derivative distributions to answer them. We have appropriate wiki pages
for all options, those that are insufficient should be improved. These
could be linked to so that people know what to expect.

> What I really would appreciate is some feedback on the Planning
> Questions listed below, as to help folks organized their thoughts and
> hardware details BEFORE actually performing an install or test-drive.

It's always good to plan before doing something so *this* part of your
proposal I support.


> A recent discussion of the dev list showed
> encouragement for pointing gentoo-noobs to some of the gentoo derivative
> distros for a quick install experience.

Perhaps it would be enough to extend this page

https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/About

and under `Troubles` mention derivative distributions (by name) with a
_hint_ that their installers quickly lead to a working base system.


The decisions to be made during the installation are mostly orthogonal, so
I wouldn't try to break the current installation procedure which is for
the most part linear. 
A matrix implies some form of interaction between the options, which I
don't quite see.


> straightforward for folks to discern the best route to their desired final
> result. When new installation semantics [1]  mature, the installation matrix
> can be modified to include those options as links.
> 
>         Install PreQualifying Matrx::QUESTIONS
>         ....................................................................
>         ....................................................................
>         ....................................................................
>         Live Testdrive options before installation(usb/cd/dvd)::

Pretty much already covered by

https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Media

and should really be a pre-install thing which it currently already is. 

>         Intended Usage (workstation/server/device/)
>         Hardware or Vitual installation::
>         PC mobo or tablet/embedded/device::
>         Processor/Ram characteristics::

How are any of these relevant to the installation? For virtual
installations I would only add mention of the `make kvmconfig` option that
quickly pulls in qemu drivers. But the other things you mentioned don't
have corresponding choices that need to be made (during the install and
especially by newcomers). I'd remove them.

>         MBR vs (u)EFI (type of mobo)::
>         Single or Multi or RAID disk configuration::
>         File System type(s)::
>         Grub1 vs Grub2 or other boot-semantics::

To me these are the only real things that need to be thought about during
the install. MBR vs UEFI is well explained if you ask me.  Single/RAID and
filesystems are strongly connected but can be chosen freely independent of
the other two. Grub2 can boot pretty much anything and if you use the EFI
stub kernel on the ESP with initrd then that too can handle anything.  So
no dependencies here.

So I would ask these questions in this order, and this is actually the
order in which they show up in the handbook... which makes me wonder
whether there is really a need for this.

>         OpenRC or Systemd::  

More of a post install thing if you ask me but the handbook currently
links to the systemd article at just the right time.


If anything I would actually go for a simplification of the install
procedure, to something extremely low maintenence (for the handbook
authors ofc). An ext4 single disk install with grub2 (meh) that every one
can handle. 

Sure gentoo gives you choices but you have to be ready to handle them, so
perhaps the first install is not the right one for experimenting?


> as well as valid install links
> like sabayon for gentoo(ish) systemd
> like calculate-linus for gentoo(ish)openrc
> like pentoo for gentoo-penetration systems
> like zentoo for gentoo CI systems
> Like funtoo as an option install
> like gentooliveUSB   for a gentoo + persistence experience.

The goal should be to get people to come to gentoo-gentoo, not to go
elsewhere.


> gentoo installs, so he is one of those guys that can single-handedly
> solve this crisis:

I actually don't feel that there is any crisis. The only time I've ever
had problems with the install was when I decided to not follow the
handbook. Most people should just stick to the handbook and learn.
Experiment once they know what they're dealing with.

I think an einstein quote is relevant here:

    Everything should be made as simple as possible, but not simpler.

The current install procedure is pretty much as simple as can be, once you
think about it. 

Re: [gentoo-user] Install PreQualifying Matrix

[Rich Freeman]

On Mon, Aug 17, 2015 at 4:14 PM, Jeremi Piotrowski
<jeremi.piotrowski@gmail.com> wrote:
> If anything I would actually go for a simplification of the install
> procedure, to something extremely low maintenence (for the handbook
> authors ofc). An ext4 single disk install with grub2 (meh) that every one
> can handle.
>
> Sure gentoo gives you choices but you have to be ready to handle them, so
> perhaps the first install is not the right one for experimenting?

I'm not convinced this is the right approach.  The whole point of
Gentoo is that if offers users a LOT of choice.  We should of course
present reasonable defaults, but we shouldn't send them hunting for
alternative guides and trying to figure out what steps to substitute
where anytime they change things.  If they just want a default
configuration they might as well just use something like Sabayon or
Debian.

Besides, once you start demoting all the options you invite the
inevitable war over what the defaults should be.

-- 
Rich

[gentoo-user] Re: Install PreQualifying Matrix

[James]

Jeremi Piotrowski <jeremi.piotrowski <at> gmail.com> writes:

> Planning questions are an OK-ish idea, but I surely wouldn't link to
> derivative distributions to answer them. We have appropriate wiki pages
> for all options, those that are insufficient should be improved. These
> could be linked to so that people know what to expect.

I'm not suggesting that the handbook not be referenced or recommended.
I'm suggesting that we point to some sites for quick installs, including
gentoo livedvd. Others might be after a failed handbook install.
It's an idea, certainly not a mutiny.... But I do see that the handbook
being the face of the gentoo install experience is sub optimal. ymmv.
I think it should be an reference option for those ready for a deeper
learning (pedantic) experience. I do think there is room for a quickie
install semantic, if not many other install semantics other than the
handbook. Hence the idea of the Planning Matrix Questions before a
particular installation semantic is chosen by the  new user.


> > What I really would appreciate is some feedback on the Planning
> > Questions listed below, as to help folks organized their thoughts and
> > hardware details BEFORE actually performing an install or test-drive.

> It's always good to plan before doing something so *this* part of your
> proposal I support.

Yes the idea works with just keeping the status quo for installs (pain and
torture via the handbook) too. That's a minimal scope of what I have
in mind so hence I'm first shopping the idea here to give gentoo the first
shot. If not, I might just put up a neutral site and point to all the
gentoo derivative distros; and let folks choose as they like. I've been
on this list over 10 years now. I'm pretty convicted about this 
need to offer up a softer face to gentoo installs, one way or another.
Sites like distrowatch do Gentoo a great dis-service.


> > A recent discussion of the dev list showed
> > encouragement for pointing gentoo-noobs to some of the gentoo derivative
> > distros for a quick install experience.

> Perhaps it would be enough to extend this page
> https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/About

That seems like a page to read just before attempting a Handbook install.
However, you can take what ideas you like and make those mods as you like.
What I'm talking about is a set of questions that help a user
prepare the info and make critical decisions, like systemd vs openrc.

> and under `Troubles` mention derivative distributions (by name) with a
> _hint_ that their installers quickly lead to a working base system.

That sounds like a good idea; do you have rights to the wiki page?
Gonna post a bgo doc bug?

> The decisions to be made during the installation are mostly orthogonal, so
> I wouldn't try to break the current installation procedure which is for
> the most part linear. 

Many think the current install (via the handbook) is like kissing a sour
lemon on the first date. ymmv.

> A matrix implies some form of interaction between the options, which I
> don't quite see.

The proposal is for planning before the install occurs. It does suggest
that the handbook be only one of the possible pathways to a successful
installation of gentoo (or a limited number of gentoo derivatives). 

It can be a (3) column table with links to appropriate install semantics.
It's a thought looking for comments; not a hard pitch at all. I think
I have identified some excellent questions to pose to potential gentoo
install noobs, so they at least prepare for whatever  installation semantic
they choose to follow. If folks do not like the idea of pointing to other
distros with installer programs. OK. That can be something
informally suggested. I thought the link to calculate where its is clearly
explained how to covert a calculate linux install to a gentoo install, is a
valid idea and it first appeared in gentoo-dev. Many of the devs are aware
of the drudgery of installing gentoo via the handbook. Sure many folks
think that pain is necessary, but I do disagree, strongly. I never taught
like that in any of my labs or folks I have mentored over the decades. But
the 'hard ass' approach is a popular, legacy mentality  and many youthful as
well as older folks with experience just do not respond well to that sort of
speech, imho. That's what the combination of the handbook and many
responses in gentoo-user project, imho.

Calcuate linux keeping a page around where folks and easily see how to 
convert a calculate linux install to a gentoo install is very classy on
their part as they are interested in what is best for the user.


> > straightforward for folks to discern the best route to their desired
> > final result. When new installation semantics [1]  mature, the  
> > installation matrix can be modified to include those options as links.

> >         Install PreQualifying Matrx::QUESTIONS
> >         Live Testdrive options before installation(usb/cd/dvd)::

> Pretty much already covered by
> https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Media

Really ? USB ? Persistence ? Net or remote boots?  auto-Installer?

> >         Intended Usage (workstation/server/device/)
> >         Hardware or Vitual installation::
> >         PC mobo or tablet/embedded/device::
> >         Processor/Ram characteristics::

> How are any of these relevant to the installation? 

Profiles? Data needed for kernel configs?  Which set of instructions
to follow (vm vs hardware based)?

> For virtual
> installations I would only add mention of the `make kvmconfig` option that
> quickly pulls in qemu drivers. But the other things you mentioned don't
> have corresponding choices that need to be made (during the install and
> especially by newcomers). I'd remove them.

I never said every question brings forth useful information for every
possible install pathway. It's a collection of planning questions before
anyone at gentoo-user knows what's are the goals or expectations of the
person newly installing gentoo.


> >         MBR vs (u)EFI (type of mobo)::
> >         Single or Multi or RAID disk configuration::
> >         File System type(s)::
> >         Grub1 vs Grub2 or other boot-semantics::

> To me these are the only real things that need to be thought about during
> the install. MBR vs UEFI is well explained if you ask me.  

Maybe but has the person preparing to do the intalll congnicent of the choice?
> Single/RAID and filesystems are strongly connected but can be chosen 
> freely independent of the other two. 


> Grub2 can boot pretty much anything and if you use the EFI
> stub kernel on the ESP with initrd then that too can handle anything.  So
> no dependencies here.

Good questions for preparation of the install. The idea is to avoid sending
the  new install person on a reading lesson during the installation. Maybe
suggest the reading before the install commences.


> So I would ask these questions in this order, and this is actually the
> order in which they show up in the handbook... which makes me wonder
> whether there is really a need for this.

The point is that this should be discuss *before* a handbook install
is conducted, not during the install.... Looking back a documents one
has been previously introduced to, is usually more successful, too.

> >         OpenRC or Systemd::  

> More of a post install thing if you ask me but the handbook currently
> links to the systemd article at just the right time.

> If anything I would actually go for a simplification of the install
> procedure, to something extremely low maintenence (for the handbook
> authors ofc). An ext4 single disk install with grub2 (meh) that every one
> can handle. 

YES and script it up so the answers  to those aforementioned questions
can be parsed during a (semi) auto install..... HELL YES!

But still that would be just one of the pathways. Let's face it
the concept of a singular handbook install doc for all install
cases is dated at best. What about tablets?   Arm64. Cell phones
running linux. Vitural or Canotainer based installs, embedded just to name a
few. A planning/decision matrix can be linked to a multitude of install
choices, old and new (lilblue, tinhat, tor-ramdisk, pentoo, etc etc).
I'm quite certain the install handbook would be cross referenced,
frequently, kinda like a websters' dictionary is when doing homework.
To me, I am suggesting using the handbook as a reference document.

> Sure gentoo gives you choices but you have to be ready to handle them, so
> perhaps the first install is not the right one for experimenting?

Agreed. The first ride on Gentoo should be a live or autoinstall.
Even technically astute folks will appreciated that convenience, that
is folks that do not even need the handbook.....

> > as well as valid install links
> > like sabayon for gentoo(ish) systemd
> > like calculate-linus for gentoo(ish)openrc
> > like pentoo for gentoo-penetration systems
> > like zentoo for gentoo CI systems
> > Like funtoo as an option install
> > like gentooliveUSB   for a gentoo + persistence experience.

> The goal should be to get people to come to gentoo-gentoo, not to go
> elsewhere.

Really? If you read a bit there is a significant support for running off
many folks new to gentoo; as they are not worthy or some horse_feathers like
that.....


> I actually don't feel that there is any crisis. The only time I've ever
> had problems with the install was when I decided to not follow the
> handbook. Most people should just stick to the handbook and learn.
> Experiment once they know what they're dealing with.

I chatted up one of the failed installs today. Seems the guy has been
using linux and unix for quite some time He is disillusioned with debian +
systemd. He feels pretty insulted by the 'tone' of some of the responses; so
I would  politely disagree with you here.


> I think an einstein quote is relevant here:
>     Everything should be made as simple as possible, but not simpler.

I doubt Einstein could install gentoo. He was known for bumbling around
a bit. How would you feel if Albert failed on a gentoo handbook install?
Pretty crappy is my bet.

> The current install procedure is pretty much as simple as can be, 
> once you think about it. 

Bull Crap. I strongly disagree here. I think it's a sadistic ritual
for smoots needing self satisfaction and emancipation at the expense
of others. Many do not like the systemd landscape and are looking for
a lifeline. The guy I chatted up felt like he had his teeth kicked in,
and he not young either.

Again, I disagree with you. We use to have an installer. Most distros,
including most of the gentoo derivative distros have an installer
program, at least for basic installs. I certainly would appreciated 
one, and many others have expressed that such an auto-installer would be
keen. When somebody is ready for a pedantic beat down, the handbook can
emerge as victorious......

The handbook is fine, but an installer would be fantastic, imho. ymmv.

James

Re: [gentoo-user] Re: Install PreQualifying Matrix

[Dale]

James wrote:
> Dale <rdalek1967 <at> gmail.com> writes:
>
>
>> James wrote:
>>>  What I really would appreciate is some feedback on the Planning
>>> Questions listed below, as to help folks organized their thoughts and
>>> hardware details BEFORE actually performing an install or test-drive.
>
>>> Many/most of these options exist
>>>         Install PreQualifying Matrx::QUESTIONS
>>>         Live Testdrive options before installation(usb/cd/dvd)::
>>>         Intended Usage (workstation/server/device/)
>>>         Hardware or Vitual installation::
>>>         PC mobo or tablet/embedded/device::
>>>         Processor/Ram characteristics::
>>>         MBR vs (u)EFI (type of mobo)::
>>>         Single or Multi or RAID disk configuration::
>>>         OpenRC or Systemd::  
>>>         Grub1 vs Grub2 or other boot-semantics::
>>>         File System type(s)::
>> Hope other will also share and help give you ideas. 
>> Dale
>
> Hello Dale,
>
> Acutally answering the question, with comments is a good 
> idea.
>
> But what I had in mind, that is much more pressing
> is a list of additional questions, or
> re-ordering the questions
> or re-stating the quesions
> or matrix logic on the causal relationships between 
> these quesions and other questions
> as to conclusion of valid install options
> is more of what I had in mind.
>
>
> Once that is reasoably vetted then
> I would look for some statisical inferences
> on the actual answers to these quetions
>
> as well as valid install links
> like sabayon for gentoo(ish) systemd
> like calculate-linus for gentoo(ish)openrc
> like pentoo for gentoo-penetration systems
> like zentoo for gentoo CI systems
> Like funtoo as an option install
> like gentooliveUSB   for a gentoo + persistence experience.
>
>
> I think this sort of approach will take some stress off of the
> gentoo-user list and handbook whilst Blueness brings maturity
> to his efforts; he alreayd has lilblue, tinhat and tor-ramdisk
> gentoo installs, so he is one of those guys that can single-handedly
> solve this crisis: should he put his fingers to the task.
> MaffBlaster has been very quite of late.....
>
> Blueness is a wonderful and collegial type of dev
> and is currently seeking input on his 'alpha' ideas::
>
> https://wiki.gentoo.org/wiki/Project:RelEng_GRS
>
>
> THANKS!
> James
>
>

So this is to create a installer then?  Someone built a installer a long
time ago and it didn't work well.  Heck, I never could get the thing to
even complete the install and that was IF it would boot at all to even
start the process.  It would hang somewhere and then sit there doing
nothing.  After that, I found a installer to be useless and a waste of
time.  I wasn't alone on that point either.  Not long after that, the
installer project died.  The current handbook, it works. 

This is the issue as I see it.  A few people want a installer to make
Gentoo easier to install.  Well, why?  After you install Gentoo, you
have to update, maintain and maybe repair that install.  A installer
isn't going to do that unless you wait for a new version of the
installer and re-install/update sort of like windoze does.  Basically,
you are going to need what is learned during the install to
maintain/repair your system and that is just the start of it.  It's that
simple. 

Another issue with having a installer.  People install Gentoo with the
installer, if it works, and are basically completely clueless about
Gentoo and the effort it takes to run it.  I'd be surprised if even a
small percentage that used the last installer are still using Gentoo. 
People use the installer, find out that Gentoo isn't a point n click
distro, get pissed because they actually have to work at it and then
they switch to something else.  Does that benefit Gentoo?  Not likely. 

Gentoo can be a pain and most people don't want that because they don't
want to put any real effort into their OS.  When I install Linux for
someone else, I put some sort of Ubuntu or something that they can
handle.  Putting Gentoo on a system and expecting them to handle updates
would be . . . well . . . silly.  It would be a setup for failure.  If
someone wanted to run Gentoo on their puter, I'd sit with them while
they went through the install, with them doing the work and learning. 

Before I first installed Gentoo way back in 2003, I did my research.  I
researched my hardware, all sorts of options and read the handbook
several times.  It took me a few tries to get it right but I did.  I
don't recall asking anyone for help during that install process.  I just
followed the handbook and learned from the few mistakes I made.  Later
on, I learned how to customize things to suite my needs.  When I built
my new rig a few years ago, I sat down, figured out what I wanted to use
and adjusted the install process to suite that.  That effort was on me
not someone else.  If I want to use LVM, RAID, BTRFS or something else
that isn't included in the default install handbook, it's on me to
figure out where to insert that part of my install.  When a person has
used Linux for a while, they tend to learn about that sort of thing. 
Gentoo will pretty much make sure you do. 

While I could care less if someone creates a installer, I'm not going to
use it.  I also won't recommend that someone new to Gentoo use it
either.  If a person needs a installer to get a OS installed, they
really need something besides Gentoo.  I doubt they are going to be
happy in the end.  That doesn't even touch all the new users that are
clueless coming here and the forums with the same questions that would
be answered if they went through the manual install process.  I recall
that back then too. 

Just my opinion.

Dale

:-)  :-) 

[gentoo-user] Re: Install PreQualifying Matrix

[James]

Dale <rdalek1967 <at> gmail.com> writes:

> > Blueness is a wonderful and collegial type of dev
> > and is currently seeking input on his 'alpha' ideas::
> > https://wiki.gentoo.org/wiki/Project:RelEng_GRS


His work is progressing and there are (3) major versions just 
posted to gentoo-dev.


> So this is to create a installer then?  Someone built a installer a long
> time ago and it didn't work well.  Heck, I never could get the thing to
> even complete the install and that was IF it would boot at all to even
> start the process.  It would hang somewhere and then sit there doing
> nothing.  After that, I found a installer to be useless and a waste of
> time.  I wasn't alone on that point either.  Not long after that, the
> installer project died.  The current handbook, it works. 

Funny. I just recently took the old 2008 version and installed on old
vintage hardware and it worked like a charm. ymmv.


> This is the issue as I see it.  A few people want a installer to make
> Gentoo easier to install.  Well, why?  After you install Gentoo, you
> have to update, maintain and maybe repair that install.  A installer
> isn't going to do that unless you wait for a new version of the
> installer and re-install/update sort of like windoze does.  Basically,
> you are going to need what is learned during the install to
> maintain/repair your system and that is just the start of it.  It's that
> simple. 

I look at your argument here as "mono-dimensional" as there are a plethora
of 'gentoo' systems one can end up with now; a lot has changed.

Embedded, tablet, gentoo-cell phone, efi, mbr not to mention what the final
target is (server, security-appliance, terminal server, CI, vm or container
host etc etc). One install semantic does not fit all current nor future needs.

Besides, if I want to deploy 50 systems for a cluster, one at a time in
parallel what do you recommend? via handbook? The modern diversity of
hardware options has rendered the gentoo handbook, dysfunctional, at best,
imho. ymmv.



> Another issue with having an installer.  People install Gentoo with the
> installer, if it works, and are basically completely clueless about
> Gentoo and the effort it takes to run it.  I'd be surprised if even a
> small percentage that used the last installer are still using Gentoo. 

I am; that's at least one.

> People use the installer, find out that Gentoo isn't a point n click
> distro, get pissed because they actually have to work at it and then
> they switch to something else.  Does that benefit Gentoo?  Not likely. 

So we split off the install support to another group so the good-folks
on gentoo-user do not have to be bothered with these sort of
installer-folks. My bet is this *attitude* is bullshit and these problems,
with an automated install system will be quite manageable by the
gentoo-noob-community directly. ymmv. We'll see, won't we? Either way,
your participate will be optional; so don't stress out about it.


> Gentoo can be a pain and most people don't want that because they don't
> want to put any real effort into their OS.  When I install Linux for
> someone else, I put some sort of Ubuntu or something that they can
> handle.  Putting Gentoo on a system and expecting them to handle updates
> would be . . . well . . . silly.  It would be a setup for failure.  If
> someone wanted to run Gentoo on their puter, I'd sit with them while
> they went through the install, with them doing the work and learning. 

Dale, kids, old folk and such blue collar folks run gentoo. I know I have
set up probably hundreds of gentoo systems for folks over the years. Many
haver gone on to study computer science or EE in school, other keep busting
wrenches for a living. The mystic that gentoo is only for the compiler
genies of the world is absolutely bullshit, so get that out of your brain,
or at least stop spewing that venom as gospel. You have no statistical
proof, only one at a time experiences. YES some behave that way. But
countless others do not and have not behave that way in the past and
currently;  and they would appreciate a simple semi-automated install
pathway, if not many such options for unattended installs of gentoo.

To me, gentoo is an emancipation of one's ablity,  to both run and optimize
software on hardware or virtual; and I run into lots of folks, including
recent college grads that just love it. Gentoo is NOT DIFFICULT, once the
basic install is accomplished, in my experiences. A frustrating gentoo
install does not even come close to learing everything one needs to know
about gentoo to manipulated the gentoo system going forward. Nor is
it the only pathway to a happy gentoo install.

Embedded software developers that have little *nix experience readily take
to gentoo, because of it's sourcecode nature. There are many of those
folks being force into linux in the past and currently. Many of them are
older and some have lots of experience with assembler codes. All that
I have dealt with are bit agry that somebody did not tell them about
gentoo, a decade ago. ymmv.

> Before I first installed Gentoo way back in 2003, I did my research.  I
> researched my hardware, all sorts of options and read the handbook
> several times.  It took me a few tries to get it right but I did.  I
> don't recall asking anyone for help during that install process.  I just
> followed the handbook and learned from the few mistakes I made.  Later
> on, I learned how to customize things to suite my needs.  When I built
> my new rig a few years ago, I sat down, figured out what I wanted to use
> and adjusted the install process to suite that.  That effort was on me
> not someone else.  If I want to use LVM, RAID, BTRFS or something else
> that isn't included in the default install handbook, it's on me to
> figure out where to insert that part of my install.  When a person has
> used Linux for a while, they tend to learn about that sort of thing. 
> Gentoo will pretty much make sure you do. 

OK so that is a valid pathway, maybe 5%>? I worked under many source
code license restrictions in the late 80s and early 90s. I remember when
BSD* was the open source and linus was experimenting, at best.  It was the
unix license lawsuts that drove the open source development comunity from
*BSD to linus and his experiment (thank you AT&T, SCO, and Sun
microsystems.)I know tons of folks in that category, coming from a
university background. I was threated with a laysuit myself, personlly
be DEC executives, when we deleted Ultrix from a room full of Dec
unix workstations, and installed several different versions of X and BSD
on them. I had a meeting with them and walk them into other labs
where "my kids" had done the same thing to quit a few other unix machines
and they did not even believe it was possible. I've dealt with lots
of bullshit in my life; it only invigorates me. So, please, please, please
get on the train or get the hell out of my way. One way or another 
*GENTOO is going PRIME-TIME* !!!!!!!!!!!!!!!!!!!!!!!!!!!

I ran part of SURAnet's network in Florida. I hacked on proteon routers
before there was a cisco. I personally know dozens of folks that use
to have on cisco's codes, for free. WE wanted options other than proteon
as that company was run by a bunch of stiffs. I've seen large parts of 
cisco stack not only legally, but in the wild and at semiconductor companies
where they  build new product for companies like cisco. Much of cisco's tree
comes from semi-conductor vendors and it's crap code with
many problems, imho. Wanna Compare *nix experience? There are lots of folks,
like me, around gentoo. Most *never* give back because the eco_system is
bullshit and the heart of the bullshit is the install semantics, imho.
Obviously, others have different valid view and therefore gentoo is
extraordinarily multi-faceted with a plethora of pathway to gentoo-paradise.

So we differ on perspective. OK? Yours is vaild to you and is appreciated
by others. But, we are 180 degress polar opposites, just do you know. 


> While I could care less if someone creates a installer, I'm not going to
> use it.  I also won't recommend that someone new to Gentoo use it
> either.  If a person needs a installer to get a OS installed, they
> really need something besides Gentoo.  I doubt they are going to be
> happy in the end.  That doesn't even touch all the new users that are
> clueless coming here and the forums with the same questions that would
> be answered if they went through the manual install process.  I recall
> that back then too. 

The situation is not so monotonically linear as you envision.
Users need to learn, some degree of expertise before experiencing
the full range of gentoo-derived benefits. Do you think everybody
needs to become a mechanic before they drive a really nice/fast/custom
ride?  No is my belief. So Gentoo enables us to both pursue what we
believe in our own comfortable-pathways. I got to fly an airplane
(cessna 180) when I was 6 years old. NO license, no hassle, even my
mom was was shocked when she found out, I even landed that sucker,
on my own. I did have a season bush pilot with me. But it was his
idea. He encouraged me, He guided me. And that has made all the difference
in my life. Now days you could get your license pulled. But many of
us think there are too many nay-sayers and regulators. No wonder our
kids do not dream big any more. Me, I say HELL YES TO GENTOO for EVERYBODY!

> Just my opinion.
> Dale

Your opinion and that of others is greatly respected and appreciated,
despite the resolve of my mindset. But, you and others should not continue
on the 'brow-beating' pathway that the handbook is the only pathway
to gentoo paradise. You have zero proof that the handbook is even needed
at all by folks. Certainly those that code in C/C++ and shell have never
needed the handbook. Not all that visit/test gentoo are in need of
mentoring. Many/Most technically astute folks that I've interacted with
look at gentoo's lack of easy install as a filter to discourage usage.
The look at it and leave. The handbook does appeal to those in the middle
or with lesser skills willing to pursue the handbook way of learning.
But millions of folks have learn/used *nix without the gentoo handbook. BSD
is probably a better teaching platform anyway. I have never ran into anyone
with a little bit of BSD competence that could not install gentoo in a snap.
Most did not like a variety of things about gentoo and left. I think the
maturity of Gentoo, portage, and the overlay semantics (now on git) will
allow gentoo to provide a fresh prospective to *nix folks. Let's face it,
the systemd turmoil is dis-lodging tons of folks, many technically astute,
from their current linux distros. That has created a wonderful, time
limited, opportunity for gentoo to snag lots of technically astute folks
into our paradise. But, we must strike hard and fast; and several pathways
of easy/trivial installation, will allow those with expertise and otherwise
the opportunity to taste, just how sweet and powerful gentoo has become.
We're going to shock quite a few folks and were going to add some more 'top
notch' devs to our stable  of 'show ponies'. ymmv.

So there are millions, of pathways to *nix paradise. Please stop with this
ridiculous belief that the gentoo handbook is the only valid pathway.

Once we have an easy installer semantic again, you will see what 
I'm talking about; and you'll have the right (privilege) to not
help those that have use the installer and not the handbook.

OK?

Can you or the others wonderful folks at gentoo-user think of any more
(pseudo) issues or additional PreQualifying Pathway Selection Questions?

Here are (8) of the current *developmental  pathways*, should you
want to help::

[1] http://releases.freeharbor.net/
[2] http://bluemoon.freeharbor.net
[3] http://lilblue.freeharbor.net
[4] http://bluedragon.freeharbor.net
[5] https://wiki.gentoo.org/wiki/Project:RelEng_GRS
[6] http://bluemoon-tinderbox.freeharbor.net
[7] http://lilblue-tinderbox.freeharbor.net
[8] http://bluedragon-tinderbox.freeharbor.net



peace,
James

Re: [gentoo-user] Re: Install PreQualifying Matrix

[Rich Freeman]

On Fri, Aug 21, 2015 at 11:39 AM, James <wireless@tampabay.rr.com> wrote:
> Dale <rdalek1967 <at> gmail.com> writes:
>
> Besides, if I want to deploy 50 systems for a cluster, one at a time in
> parallel what do you recommend? via handbook? The modern diversity of
> hardware options has rendered the gentoo handbook, dysfunctional, at best,
> imho. ymmv.

I have mixed feelings on this one.

After reading some accounts in a completely different list I can see a
lot of the value of just being able to click a few buttons and have
gentoo running, and then having the luxury of tailoring it later.
This was the driver to drop the stage1 installs in favor of stage3 in
the first place.

Still, if I were actually deploying on a cluster I don't think any of
this is the way I'd probably do it.  On a cluster I'd be more
concerned with integration with a configuration management system.
I'd be thinking more of things like openstack and coreos for the
initial install, and then Gentoo is just something that goes on top
(or in the case of coreos, underneath).  It is a bit like sticking
your filesystem on top of lvm - it just makes things easier down the
road with almost zero cost.

-- 
Rich

[gentoo-user] Re: Install PreQualifying Matrix

[James]

Rich Freeman <rich0 <at> gentoo.org> writes:


> > Besides, if I want to deploy 50 systems for a cluster, one at a time in
> > parallel what do you recommend? via handbook? The modern diversity of
> > hardware options has rendered the gentoo handbook, dysfunctional, 
> > at best,  imho. ymmv.
> I have mixed feelings on this one.

YES. We all do. I just think the time has come for gentoo to offer
a variety of installation semantics. The hand book is valid. An installer
is valid. Using Ansible and such is valid. Clonezilla is valid [1].
Using  scp or dd HD to HD is valid. There are no limits to valid
pathways. We should just get on with 'diversity of gentoo installs' and
be done with it. If the handbook is 95% of the new installs, so be it.
MY prediction is with other viable options, the handbook we be actively
used as a reference, but we'll quickly experience an increase in usage 
of new install semantics. Pentoo is one well kept secret, as you know.[2]
zchaos is a titan, imho, and his work deserves accolades as well as exposure
in the greater gentoo community. I certainly appreciate pentoo
more and more every day.


> After reading some accounts in a completely different list I can see a
> lot of the value of just being able to click a few buttons and have
> gentoo running, and then having the luxury of tailoring it later.
> This was the driver to drop the stage1 installs in favor of stage3 in
> the first place.

We can all interpret the past via a variety of lenses. As always your
perspectives have a causal effect on my mind, so go easy with me? I do not
have all of this worked out, but, I am very passionate about this paradigm
shift.....

Bootstrapping, from a micro processor point of view, has a myriad of
semantics, all valid and millions of embedded products use bootstrapping
semantics mostly uniquely created by the coders of those individual
products. Semiconductor companies usually provide the stub code, registers,
 details of ram, rom, eeprom mmc, flash, etc etc and the coders write unique
code to package the boot-loader so as to be 'license free'. WE do not have
to go to that level, but surely encouraging and creating a plethora of
pathways to install gentoo is a good thing. 

Then folk can think of a variety of ways (catalyst, profile, world_file,
ansible etc etc) of how to put collections of packages and configa onto the
recent installs in an unattended fashion.   This will prepare us (gentoo) to
champion the future of VM, Containers and clusters is a very logical and
extensible way. NOBODY is bridging the divide between physical (actual HD)
semantics and those ethereal {vm, container, remote hosted etc) so that it
is one large, but logical endeavour. imho. This is where I believe gentoo
can dominate. Compiling from 100% sources, the gentoo way, is a killer
advantage and gentoo is very well positioned on that. You should read up on 
what D.Berk.... wrote some years ago about clusters and look at the who's
who list of research and commercial folks that used gentoo for clusters;
if you have not already done so. I have notice some of those docs
disappearing, but they are all at legacy archives......

Tuning clusters/Clouds is all about managing sources, keeping the
source-trees (gitignore) pristine and keeping the OS pristine. Likewise the
same thing need to happen to the underlying kernel. Like it or not the
Kernel_bloat is at an all time high and that is a separate but parallel
need. Gentoo supporting both OpenRC and systemd is allowing this distro to
morph into something unmatched in the linux world, imho. It's a very good
thing for Gentoo and I believe this will only benefit Gentoo, linux and the
open source communities. Like it or not, Gentoo is a power player. Folks
just try to keep it a secret, commercially. It's gonna explode everywhere,
once it is easy to install. Systemd has dis-lodged many linux users
and that is a wonderfui but time limited opportunity for Gentoo (a window in
time, if you like).

Combined efficiently (virtual and real), will allow the distro to prosper
beyond it's competition. Those non-rolling distros are at a huge
disadvantage on performance, security and maintainability imho. Look at
Suse's recent moves. All we lack is raw speed/simplicity in the installation
semantic(s). imho.



> Still, if I were actually deploying on a cluster I don't think any of
> this is the way I'd probably do it.  On a cluster I'd be more
> concerned with integration with a configuration management system.
> I'd be thinking more of things like openstack and coreos for the
> initial install, and then Gentoo is just something that goes on top
> (or in the case of coreos, underneath).  It is a bit like sticking
> your filesystem on top of lvm - it just makes things easier down the
> road with almost zero cost. 

Those distros that currently offer quickie installs of clusters,
are mostly pathetic at what's needed to run different, tuned or stripped
kernsl underneath. Kernel tuning supremacy  for those 'cluster distros'
is mostly not talked about much because that distracts from their 'distro'
goals. Gentoo is a wonderful platform to tune/test clusters optimized
kernels (heaptrack, kernelshark etc etc). We have all of that, and once
those become common topics on gentoo-users (gentoo-expert ?) it's going to
become yet another 'gentoo differentiator' technology.


You are right and you are wrong:: Openstack nor CoreOS are the best approach
for (BS) Big Science, imho. BS needs all resources solving and supporting a
single problem, with as low of latency as possible. Real-time (less than 200
ms) is the goal of all BS. It allows study and reformulation of problem
solution attempts much more efficiently.  It  (BS) allows you to study the
causal relationships of advance mathematics like never before. Go back a
week or so on gentoo user when those  rooooskies came to gentoo user to
implement their new, shiny math toy. Bircough () recognized the  needs
(wants and desires) he politely scolded them but put their codes into gentoo
ebuild form. I (unbenounced to the gentoo communities) bounce those codes
and Bircophs rapid response work into the faces of some folks I hassle all
the time. (no gentoo is not alone in having large point sources of angsts
toward me) and they got real quite for a few weeks. I have been pounding on
them to open up what they do for other 'mathematical pathways' to real time
solutions. I have few friend there, but they do listen and change course.
They are now opening up their work to different 'pathways' of mathematics to
real time problems. It's a small world in which we live. Gentoo is
fundamentally important in many realms, too, just do you know.


Compiling native sources on bare metal, with gcc5, RDMA and such is
leading the charge. Virtual, containers and partitioning only add latency
to the result of BS pathways. Why is BS so important to others? Simple.
First there are many, many unique needs, many are "well funded"; so if
gentoo_ness is uptaken so will gentoo centric jobs for coders and admins.
Second if you take the average VM, Container or virtual computer room and
add BS capabilities to it by refocusing say 1/3 to 1/2 of the existing 
systems, you will get a much more efficient Cloud/Cluster apparatus from a
given amount of hardware. These BS needs are driving the implementation of
better batch/multi-users/faster solutions for all
computation/sorting/searching needs of the digital world.


BS needs dictate a better usage of raw resources to solve those sets
of problems faster. BS system can be used to tremendously lower
the cost of batch processing, as speed kills. Cray and others have known
this for a long time. But nobody with that expertise wanted to make that
solution cost-effective. Now with clusters, BS innovations, gcc5 and rdma on
bare metal, there is another paradigm shift occurring .  BS  solutions will
extent to the rank and file capabilities of the average   computer rooms,
data centers, cloud and cluster thus generating lots of attention for
Gentoo. This installation situation of Gentoo is the largest impediment to
this reality, and it's just wrong and severely dated.

There is a 'vergence in the force' so get excited and get onboard!
Evolution has turned revolutionary:: keep your processors dry
and aim high......


hth,
James

[1] http://clonezilla.org/

[2] https://code.google.com/p/pentoo/wiki/PentooInstaller

Re: [gentoo-user] Re: Install PreQualifying Matrix

[Rich Freeman]

On Fri, Aug 21, 2015 at 3:28 PM, James <wireless@tampabay.rr.com> wrote:
> You are right and you are wrong:: Openstack nor CoreOS are the best approach
> for (BS) Big Science, imho. BS needs all resources solving and supporting a
> single problem, with as low of latency as possible.

What kind of latency are you expecting to get with Gentoo running on
CoreOS?  A process inside a container is no different from a process
outside a container as far as anything other than access/visibility
goes.  They're just processes as far as the kernel is concerned.
Sure, it isn't quite booting with init=myscieneapp but it is about as
close as you'll get to that.

-- 
Rich

[gentoo-user] Re: Install PreQualifying Matrix

[James]

Rich Freeman <rich0 <at> gentoo.org> writes:


> > for (BS) Big Science, imho. BS needs all resources solving and 
> > supporting  a single problem, with as low of latency as possible.

> What kind of latency are you expecting to get with Gentoo running on
> CoreOS?  A process inside a container is no different from a process
> outside a container as far as anything other than access/visibility
> goes.  They're just processes as far as the kernel is concerned.
> Sure, it isn't quite booting with init=myscieneapp but it is about as
> close as you'll get to that.


I'm not planning on running gentoo on CoreOS; so apologies if that is
confusing. I'm intending on running a stripped and optimized gentoo OS
and linux kernel as close to bare metal as I can. gcc5 is targeted at both
system, GPU and distributed resource compiling (RDMA).

Mesos + spark + tachyon + storm + RDMA + GCC-5.x is a killer platform
for clustering. It supports some traditional and well as radical frameworks.
Mesos is exploding with new Frameworks and is planning on support for many
languages. There is a bgo on apache-spark that needs a really talented Java
Hack to solve.  There is also an upcoming mesos conference in Ireland [1] that
any Euro_hack interested in Clustering should attend. Many companies are
hiring talent and paying a 50% premium, particularly if you can admin, code
and compile and know a bit of basic clustering.


[1] http://events.linuxfoundation.org/events/mesoscon-europe

Re: [gentoo-user] Re: Install PreQualifying Matrix

[Rich Freeman]

On Fri, Aug 21, 2015 at 11:29 PM, James <wireless@tampabay.rr.com> wrote:
> Rich Freeman <rich0 <at> gentoo.org> writes:
>> > for (BS) Big Science, imho. BS needs all resources solving and
>> > supporting  a single problem, with as low of latency as possible.
>
>> What kind of latency are you expecting to get with Gentoo running on
>> CoreOS?  A process inside a container is no different from a process
>> outside a container as far as anything other than access/visibility
>> goes.  They're just processes as far as the kernel is concerned.
>> Sure, it isn't quite booting with init=myscieneapp but it is about as
>> close as you'll get to that.
>
> I'm not planning on running gentoo on CoreOS; so apologies if that is
> confusing. I'm intending on running a stripped and optimized gentoo OS
> and linux kernel as close to bare metal as I can. gcc5 is targeted at both
> system, GPU and distributed resource compiling (RDMA).

Don't get me wrong - I appreciate the desire for bare-metal
performance in the high-performance computing world.  I've heard
stories/rumors of Gentoo getting attention elsewhere in this domain,
and we have a disproportionate number of physical scientists and such
in the community (including probably half of the Council - we joke
about it).  I've even heard of Gentoo used in high-throughput trading,
though a lot of that has moved on to ASICs and such and nobody talks
openly about what they're doing.

I was just trying to point out that containers are very different from
VMs, while generally trying to solve the same sorts of problems.  VMs
create continuous execution overhead and are memory-expensive.
Containers have zero execution overhead and are very memory-efficient.
Of course, if you throw 5x as many running processes on the same PC
you're still going to consume more RAM and CPU, but 5 containers
running on 1 PC tend to be pretty close to the CPU+RAM requirements of
linux hosts running on 5 PCs.  If you're just using containers for
configuration-management/etc and just run one container on a node,
then you're going to be very close to the same performance you'd get
running it on bare metal.

From the kernel's perspective every linux system uses containers.
They just tend to use a single container.  The kernel doesn't do
anything differently when a process spawns in a container.  When that
process looks out at the world the kernel shows it everything within
its namespaces.  That is true whether you have one set of namespaces
on the system or 50.  As far as I'm aware the system calls all take
just as long to run either way.  Containers really are just about
adding one more field to the keys in various kernel objects like
processes/tasks.

-- 
Rich

[gentoo-user] Re: Install PreQualifying Matrix

[James]

Rich Freeman <rich0 <at> gentoo.org> writes:


> Don't get me wrong - I appreciate the desire for bare-metal
> performance in the high-performance computing world.  I've heard
> stories/rumors of Gentoo getting attention elsewhere in this domain,
> and we have a disproportionate number of physical scientists and such
> in the community (including probably half of the Council - we joke
> about it).  I've even heard of Gentoo used in high-throughput trading,
> though a lot of that has moved on to ASICs and such and nobody talks
> openly about what they're doing.

Yep; lots of folks are putting their *nix expertise into FPGAs these
days as a way to protect "their Intellectual Property". Here's a
prime example in the drug discovery world [1].

Trading with Gentoo:: Yep. I was hustle via a NYC head_hunter
for several projects, some years back, but they would never disclose the
companies. One was some wealthy individual. They wanted to emcumber me
before they told me anything; not a good sign, besides I'm not too fond of
NYC. The more bad stuff I told the HH about myself, the more they liked me
as  candidate. The pay scale was way to high for my abilities anyway...
Yep:: nobody talks..... lots of real wise guys in NYC.


> I was just trying to point out that containers are very different from
> VMs, while generally trying to solve the same sorts of problems.  VMs
> create continuous execution overhead and are memory-expensive.
> Containers have zero execution overhead and are very memory-efficient.
> Of course, if you throw 5x as many running processes on the same PC
> you're still going to consume more RAM and CPU, but 5 containers
> running on 1 PC tend to be pretty close to the CPU+RAM requirements of
> linux hosts running on 5 PCs.  If you're just using containers for
> configuration-management/etc and just run one container on a node,
> then you're going to be very close to the same performance you'd get
> running it on bare metal.

VM are obsolete compared to containers, when you start looking closely
at timing and latencies which then effects throughput. That's pretty
much accepted mathematically by virtually all of the clustering devs
I interact with. It does not mean VMs are dead or not useful, but
they are not in the competition any more on performance driven needs.

Look Rich. Believe me, when you say things I listen. It's on my to do
list to evaluate CoreOS vs bare metal. Not to beat a dead horse but
I do need a fully unattended install semantic to do the regression testing
for routine cluster needs and my "half baked" ideas.... I do not believe
regression test results in vm or container setups. Maybe the first or
second digit of accuracy. I'm old school and I have to isolate
things on hardware. That's just how I roll:: I guess it's the EE in me.
Trust but verify......

So yes at some point I intend to vet the CoreOS thing, as it is 
very close to gentoo with ebuilds and such.... I think I'm the one
that pointed coreos out on the gentoo user list; some time ago,
as a derivative or rip-off of gentoo..... Folks said ChromeOS
was from Gentoo and CoreOS was from ChromeOS.... (ring any bells?).


> From the kernel's perspective every linux system uses containers.
> They just tend to use a single container.  The kernel doesn't do
> anything differently when a process spawns in a container.  When that
> process looks out at the world the kernel shows it everything within
> its namespaces.  That is true whether you have one set of namespaces
> on the system or 50.  As far as I'm aware the system calls all take
> just as long to run either way.  Containers really are just about
> adding one more field to the keys in various kernel objects like
> processes/tasks.


WE have kernel shark (via trace-cmd) now and heaptrack too. Those (2) tools
alone should let you gather actual data on what you have stated above and
publish it. If you want a bunch of links to kernel shark info and examples
just let me know. What would be keen (and is on my todo list) is to take
kernelshark and use it for some deep analysis work on gentoo. Then publish a
gentoo wiki page on KernelShark so the community can see a cool
example on Gentoo. Kernelshark bridges that kernel/OS barrier and can 
quantify the actual timing and latencies and problems in a full stack or
particular layer of the stack. It is addicting and can consume days of your
time, just so you know (in advance).


On another note:: What I'm missing (and it's definitely new learning
material to me) is a robust, flexible DAG tool(s). What do you know about
for DAG and such tools/codes?

James


[1] https://www.deshawresearch.com/