[gentoo-user] quick installs on older/embedded hardware

[gentoo-user] quick installs on older/embedded hardware

[James]

 Hello everybody,

I have a compliment of older x86 and amd64 boxes that I use for
testing things; new or specialized gentoo offerings and such. Mostly amd64.
Many of the older ide/ata drive systems have front loaded carriers
that have quick hot swap feature that make it easy to shut down
a machine. Swap the hardrive (and carrier cassette) and boot up
with a differnt OS for testing. Often I use one machine to work 
on several differnet problems. I try to avoid the VM approach,
because much of what I work on involves actual hardware issues
and hardware verifications. 

I could put a minimal distro, such as  system rescure, on to a usb stick,
CD/DVD for booting, mount the drive and dd over complete images
from various places, then a quick reboot to test a drive based distro.

I have many differnt such gyrations ongoing and I need to deploy
a singular semanctic for boot a myriad of offering to test/code on.

To just name a few: pentoo, lilblue, lxqt(4), lxqt(5), wrt,
embedded gentoo etc etc etc. It seems as though each is a "walk_once"
exercise and I'm loosing my mind. Throw on top of this, BTRFS, CEPTh
gluster, xfs, zfs I'm beginning to become too fragmented to stay focused on
the task(s) at hand.   

(Side rant) 
If you read some of my (broken english) postings, they are mostly due to 
hardware irritants that have the majoring of my limited brain cells
agitated to the point of illiteracy...... I use gmane  (to post to
gentoo-user) and it use to have spell checking, or I hacked it, but, for
what ever reason, spellcheck in gmane seems to be gone now........

Anyway, I'd like to hear all of the ideas, including various disciplined
(structured) approaches I can take to minimize uniqueness in my lab and what
I'm currently doing (mostly). I also have usb sticks, but I've
found booting various offernings on usb, particularly older hardware, to
be too biosed_burdened.

Bear in mind, I also have dozens of embedded boards, some x86, but
mostly arm based, that are also in the mix. As soon as some less expensive
arm64 (aarch64) boards become available, those too will become much more
prevalent in my lab. I need some new organizational (software and Image)
ideas. My hardware is very well organied on large, open racks with 
lots of UPS power and easy physical access to each box/board.

I'm also going to draw things up using (app-admin/rackview).
If/when  I can take my network security to the next level,
I'd like to open up 1/2 of the machines to the gentoo community
for testing and debug on actual hardware. A portal to actuall hardware
resources. I basically need the ability to move around complete
system images between differment boxes.


thoughts and comments are most welcome!

James

Re: [gentoo-user] quick installs on older/embedded hardware

[Alan McKinnon]

On 04/06/2014 21:16, James wrote:
>  Hello everybody,
> 
> I have a compliment of older x86 and amd64 boxes that I use for
> testing things; new or specialized gentoo offerings and such. Mostly amd64.
> Many of the older ide/ata drive systems have front loaded carriers
> that have quick hot swap feature that make it easy to shut down
> a machine. Swap the hardrive (and carrier cassette) and boot up
> with a differnt OS for testing. Often I use one machine to work 
> on several differnet problems. I try to avoid the VM approach,
> because much of what I work on involves actual hardware issues
> and hardware verifications. 
> 
> I could put a minimal distro, such as  system rescure, on to a usb stick,
> CD/DVD for booting, mount the drive and dd over complete images
> from various places, then a quick reboot to test a drive based distro.
> 
> I have many differnt such gyrations ongoing and I need to deploy
> a singular semanctic for boot a myriad of offering to test/code on.
> 
> To just name a few: pentoo, lilblue, lxqt(4), lxqt(5), wrt,
> embedded gentoo etc etc etc. It seems as though each is a "walk_once"
> exercise and I'm loosing my mind. Throw on top of this, BTRFS, CEPTh
> gluster, xfs, zfs I'm beginning to become too fragmented to stay focused on
> the task(s) at hand.   
> 
> (Side rant) 
> If you read some of my (broken english) postings, they are mostly due to 
> hardware irritants that have the majoring of my limited brain cells
> agitated to the point of illiteracy...... I use gmane  (to post to
> gentoo-user) and it use to have spell checking, or I hacked it, but, for
> what ever reason, spellcheck in gmane seems to be gone now........
> 
> Anyway, I'd like to hear all of the ideas, including various disciplined
> (structured) approaches I can take to minimize uniqueness in my lab and what
> I'm currently doing (mostly). I also have usb sticks, but I've
> found booting various offernings on usb, particularly older hardware, to
> be too biosed_burdened.
> 
> Bear in mind, I also have dozens of embedded boards, some x86, but
> mostly arm based, that are also in the mix. As soon as some less expensive
> arm64 (aarch64) boards become available, those too will become much more
> prevalent in my lab. I need some new organizational (software and Image)
> ideas. My hardware is very well organied on large, open racks with 
> lots of UPS power and easy physical access to each box/board.
> 
> I'm also going to draw things up using (app-admin/rackview).
> If/when  I can take my network security to the next level,
> I'd like to open up 1/2 of the machines to the gentoo community
> for testing and debug on actual hardware. A portal to actuall hardware
> resources. I basically need the ability to move around complete
> system images between differment boxes.
> 
> 
> thoughts and comments are most welcome!



I have no idea how to improve your situation. but I do have a thought:

You are not the only person to deal with this as there are many
manufacturers of embedded items and it must be quite common to test many
images with many hardware setups.

So how do other vendors do it? If we look at their workflow, perhaps a
useable method will filter up through the wetware :-)



-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] quick installs on older/embedded hardware

[Walter Dnes]

On Wed, Jun 04, 2014 at 07:16:33PM +0000, James wrote

> Anyway, I'd like to hear all of the ideas, including various
> disciplined (structured) approaches I can take to minimize uniqueness
> in my lab and what I'm currently doing (mostly). I also have usb
> sticks, but I've found booting various offernings on usb, particularly
> older hardware, to be too biosed_burdened.
> 
> Bear in mind, I also have dozens of embedded boards, some x86, but
> mostly arm based, that are also in the mix. As soon as some less expensive
> arm64 (aarch64) boards become available, those too will become much more
> prevalent in my lab. I need some new organizational (software and Image)
> ideas. My hardware is very well organied on large, open racks with 
> lots of UPS power and easy physical access to each box/board.

  You may be interested in buildroot http://buildroot.net/ and
http://buildroot.net/about.html  It's nominally aimed at cross-compiling
for embedded systems, but it looks like it handles just about
everything.  I'm not a developer, so I don't know if this is what you're
looking for, but it sounds interesting.

-- 
Walter Dnes <waltdnes@waltdnes.org>
I don't run "desktop environments"; I run useful applications

[gentoo-user] Re: quick installs on older/embedded hardware

[James]

Walter Dnes <waltdnes <at> waltdnes.org> writes:


>   You may be interested in buildroot http://buildroot.net/ and
> http://buildroot.net/about.html  It's nominally aimed at cross-compiling
> for embedded systems, but it looks like it handles just about
> everything.  I'm not a developer, so I don't know if this is what you're
> looking for, but it sounds interesting.


Buildroot is great for embedded system and cross compiling. It more
for after you get a basic system up, imho. What I need is a basic,
minimal image that I can dd over to a hard drive, and build out whatever
"gentoo" I need. In fact I may need several images:

(1) basic image to build up a amd64 workstation or server
(2) a basic image to build up a 586 (or greater) system from
(3) a basic image to build up x86 embedded systems from
(4) a basic image to build up 32 bit arm systems from
(5) a basic image to build up 64 bit arm images


The idea would be to dd these images to a blank HD and install the basic
gentoo system, then customize from there.

This would avoid hours and hours of custom (handbook) derived installation
hours for new system that are only slightly different and are all gentoo
systems, as the base system would have the file system inside the image,
the make.conf with minimal necessary files, profile etc etc. Just the
minimal necessary to get the kernel+system up. The basic system would 
change (be updated with new kernel) from time to time, but only every
few months. emerge --sync would bring the systems current.


Them, according to what I want to build, there would be a "guide sheet"
that details the flags necessary and the key packages (software-ebuild)
config file edits etc to realize the final test box(gentoo) system.

I'm thinking out loud here on  the list for folks to "chime in" and
refine the idea. Starting with (1) and (2) only above would get me started.
I'm quite certain this is going to be a work in progress.

maybe a custom cd/dvd via catalyst for each of the (5) categories ?
I do want as much as possible (practical and reasonable to support)
from the handbook into the boot image. I think most of what is therein
before entering "chroot" is a reasonable first draft idea?

And yes this would mean all the decisions like file system type, partitions,
fstab, mtab etc would be made ahead of time, generically
for one size works reasonable well with all setups. It has been pointed
out to me that "lilo" works best for the booting as there are no issues
wtih embedded builds or the various "C" libraries one can use:
glibc, uClibc, BSDs, bionic, musl.

I'll probable start with stable and one I have (1) and (2) working
move to a hardened version (at least the kernel and tool chain)....

It may not end up as clean as I like, but surely there is a way to
streamline various categories of systems, as a referenced starting point?
??????


James

[gentoo-user] Re: quick installs on older/embedded hardware

[James]

Alan McKinnon <alan.mckinnon <at> gmail.com> writes:


> So how do other vendors do it? If we look at their workflow, perhaps a
> useable method will filter up through the wetware 


I have no idea how to find and convince an appropriate company to share how
they setup for multi image testing on various hardware platforms. If you
know of such an opportunity, I'd be all ears on what they would be willing
to share.

Do tell.


James

Re: [gentoo-user] Re: quick installs on older/embedded hardware

[Alan McKinnon]

On 06/06/2014 19:20, James wrote:
> Alan McKinnon <alan.mckinnon <at> gmail.com> writes:
> 
> 
>> So how do other vendors do it? If we look at their workflow, perhaps a
>> useable method will filter up through the wetware 
> 
> 
> I have no idea how to find and convince an appropriate company to share how
> they setup for multi image testing on various hardware platforms. If you
> know of such an opportunity, I'd be all ears on what they would be willing
> to share.
> 
> Do tell.


Sorry, I have no idea myself. I was hoping that this stuff would be
reasonably common knowledge but apparently in the embedded space people
guard their stuff jealously.

In my space (ISPs) we tend to proudly show off our deployment magic at
every chance and every seminar, mostly to get bragging rights and
brownie points plus a good healthy dose of one-upmanship.

And everyone knows everyone else anyway, half of us have worked at every
other ISP at least once  - no such thing as keeping your internal
systems secret :-)



-- 
Alan McKinnon
alan.mckinnon@gmail.com