[gentoo-user] rpm or deb package installs

[gentoo-user] rpm or deb package installs

[James]

Hello,

So it's been some time for me, but there use to be easy ways 
to install .deb or rpm packages on gentoo; maybe in  /usr/local/portage. [1] 

I only find this guide on wiki.gentoo.org : [2].


So what I really want is a modern (safe) methodical way to quickly install
.deb or rpm packages (many should work) into /usr/local/portage
for quick testing and evaluation before I hack together an 
ebuild for it. So are there any newer (vetted) methods to
do this?  Anyone who does this quit a lot would surely have some
methods if not custom scripts addressing many little pitfalls?
Are rpm's better to install than .deb packages in general? What about 
cleanup and removal: semantics, syntax, or scripts?

I do see these in portage: app-arch/dpkg   and app-arch/rpm.


Any caveats or tricks anyone cares to share? 


James



[1] http://www.gentoo-wiki.info/TIP_install_programs_without_portage

[2] http://wiki.gentoo.org/wiki/RPM

Re: [gentoo-user] rpm or deb package installs

[Alan McKinnon]

On 13/02/2015 16:12, James wrote:
> Hello,
> 
> So it's been some time for me, but there use to be easy ways 
> to install .deb or rpm packages on gentoo; maybe in  /usr/local/portage. [1] 
> 
> I only find this guide on wiki.gentoo.org : [2].
> 
> 
> So what I really want is a modern (safe) methodical way to quickly install
> .deb or rpm packages (many should work) into /usr/local/portage
> for quick testing and evaluation before I hack together an 
> ebuild for it. So are there any newer (vetted) methods to
> do this?  Anyone who does this quit a lot would surely have some
> methods if not custom scripts addressing many little pitfalls?
> Are rpm's better to install than .deb packages in general? What about 
> cleanup and removal: semantics, syntax, or scripts?
> 
> I do see these in portage: app-arch/dpkg   and app-arch/rpm.
> 
> 
> Any caveats or tricks anyone cares to share? 



I doubt dpkg and rpm aren't going to be much use to you, unless you
really want to run two package managers. Besides, both are not
especially useful with the front ends apt* and yum.

Any special reason why you don't instead download the sources and build
them yourself with PREFIX=/usr/local ?




-- 
Alan McKinnon
alan.mckinnon@gmail.com

[gentoo-user] Re: rpm or deb package installs

[James]

Alan McKinnon <alan.mckinnon <at> gmail.com> writes:


> I doubt dpkg and rpm aren't going to be much use to you, unless you
> really want to run two package managers. Besides, both are not
> especially useful with the front ends apt* and yum.

I'd just use those to unpackage and maybe preprocess some of the codes.

Agreed. I do not want a full blown deb or rpm package manager just
a way to install and evaluate some of those codes before beginning a more
arduous  and comprehensive task. Maybe I should just put up a RH/centos box
and evaluate codes there. It seems *everything* I want to test and look at
in the cluster and hpc world, as a rpm or deb package; so I'm looking for a
time saver, to surf thru the myriad of codes I'm getting; many look very
cool  from the outside, but once I run them, they are pigs.......

Then a slick way to keep them secure and clean it out. Maybe I need chroot
jails too? I spend way to much time managing codes rather than I do actually
writing code. I feel confused often and cannot seem to master this
git_thingy.... I have not code seriously in a long time and now it is
becoming an obsession, but the old ways are draining my constitutional
powers.....


> Any special reason why you don't instead download the sources and build
> them yourself with PREFIX=/usr/local ?

Lots of errant codes flying everywhere so you have to pull a code audit
to see what's in the raw tarballs before building. That takes way too much
time. I'm working on setting up several more workstations for coding to
isolate them from my main system. This approach you suggest is: error prone,
takes too much time, and I'm lazy and sometimes even stupid.
I need a structure methodology to be a one man extreme_hack_prolific
system that prevents me from doing stupid things, whilst I'm distracted.


Maybe I should just put up a VM resources on the net, blast tons
of tests thru the vendors hardware and let them worry about the
security ramifications?  Some of it is these codes are based on 'functional
languages' and I just do not trust what I do not fully understand. Stuff
(files etc) goes everywhere and that makes me cautiously nervous. I have
/usr/local for manual work and /usr/local/portage for ovelays (layman) but
it's becoming a mess. There where to I put the work effort that is a  result
from repoman. Those codes seem to be parallel projects often
when the code I'm evaluating needs to be cleaned up or extend to properly
test. Furthermore I have a growing collection of file that result
from kernel profiling via  trace-cmd, valgrind, systemtap etc etc.
As soon as I delete something, I need to re-generated it for one
reason or another...... I just hope that this repo.conf effort
helps be get more structurally organized?  


Did you see/test 'travis-ci' yet? [1] I'm not sure it's the same
on github [2] but some of the devs are using it on github. 



James

[1] http://docs.travis-ci.com/

[2] https://github.com/travis-ci/travis-ci

Re: [gentoo-user] Re: rpm or deb package installs

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 737 bytes --]

On Fri, 13 Feb 2015 21:08:55 +0000 (UTC), James wrote:

> > I doubt dpkg and rpm aren't going to be much use to you, unless you
> > really want to run two package managers. Besides, both are not
> > especially useful with the front ends apt* and yum.  
> 
> I'd just use those to unpackage and maybe preprocess some of the codes.
> 
> Agreed. I do not want a full blown deb or rpm package manager just
> a way to install and evaluate some of those codes before beginning a
> more arduous  and comprehensive task.

In that case you ware deb2targz or rpm2targz to convert the package to a
tarball. then you can unpack it and inspect the contents.


-- 
Neil Bothwick

If it ain't broke, break it and charge for repair.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-user] Re: rpm or deb package installs

[Bill Kenworthy]

On 14/02/15 05:08, James wrote:
> Alan McKinnon <alan.mckinnon <at> gmail.com> writes:
> 
> 
...
>> Any special reason why you don't instead download the sources and build
>> them yourself with PREFIX=/usr/local ?
> 
> Lots of errant codes flying everywhere so you have to pull a code audit
> to see what's in the raw tarballs before building. That takes way too much
> time. I'm working on setting up several more workstations for coding to
> isolate them from my main system. This approach you suggest is: error prone,
> takes too much time, and I'm lazy and sometimes even stupid.
> I need a structure methodology to be a one man extreme_hack_prolific
> system that prevents me from doing stupid things, whilst I'm distracted.
> 

> 
> 

rpm is just a wrapper around a an archive with instructions on how to
build and or install it.  I have more experience with rpm's but I
believe debs are the same.  Just unwrap your .rpm/.deb file of choice
and install it manually (the binaries/code are usually in a zip inside
the rpm).  You should in most cases also be able to get a source rpm
(which I suspect you are talking about anyway, but binaries do work deps
permitting.

you can install rpm and then install your package via rpm - seem to
remember doing this with .debs somehow too.  deps are a problem but
usually workable.

and why set up a workstation? - this sort of thing is tailor made for
vm's.  Create a base for your experiments with essential packages,
settings etc, snapshot it (golden master) and then throwaway->restore
when finished with that iteration.

There are package managers besides gentoo/protage that can do a source
build/install and track the files on gentoo - though portage will not
know about it (rpm is one :)

and lastly, what do mean error prone? - to me a manual install is the
ONLY way you can build a proper ebuild that catches most of the
problems.  In the (admittedly few) ebuilds I have done an occasional
human error is nothing compared to system problems for a difficult package.

BillK

Re: [gentoo-user] Re: rpm or deb package installs

[Alan McKinnon]

On 13/02/2015 23:08, James wrote:
> Alan McKinnon <alan.mckinnon <at> gmail.com> writes:
> 
> 
>> I doubt dpkg and rpm aren't going to be much use to you, unless you
>> really want to run two package managers. Besides, both are not
>> especially useful with the front ends apt* and yum.
> 
> I'd just use those to unpackage and maybe preprocess some of the codes.
> 
> Agreed. I do not want a full blown deb or rpm package manager just
> a way to install and evaluate some of those codes before beginning a more
> arduous  and comprehensive task. Maybe I should just put up a RH/centos box
> and evaluate codes there. It seems *everything* I want to test and look at
> in the cluster and hpc world, as a rpm or deb package; so I'm looking for a
> time saver, to surf thru the myriad of codes I'm getting; many look very
> cool  from the outside, but once I run them, they are pigs.......
> 
> Then a slick way to keep them secure and clean it out. Maybe I need chroot
> jails too? I spend way to much time managing codes rather than I do actually
> writing code. I feel confused often and cannot seem to master this
> git_thingy.... I have not code seriously in a long time and now it is
> becoming an obsession, but the old ways are draining my constitutional
> powers.....


I see you are doing more than I thought you were doing :-)

rpms and debs are both cpio files so the easy way is to unpack them and
see what's going on:

rpm2cpio name.rpm | cpio -iv --make-directories
dpkg -x somepackage.deb ~/temp/

Considering the size of what you are doing, you are probably better off
running a Centos and Debian system to evaluate the code and discard the
rubbish. Once you've isolated the interesting ones, you can evaluate
them closer and maybe write ebuilds for them.




> 
> 
>> Any special reason why you don't instead download the sources and build
>> them yourself with PREFIX=/usr/local ?
> 
> Lots of errant codes flying everywhere so you have to pull a code audit
> to see what's in the raw tarballs before building. That takes way too much
> time. I'm working on setting up several more workstations for coding to
> isolate them from my main system. This approach you suggest is: error prone,
> takes too much time, and I'm lazy and sometimes even stupid.
> I need a structure methodology to be a one man extreme_hack_prolific
> system that prevents me from doing stupid things, whilst I'm distracted.
> 
> 
> Maybe I should just put up a VM resources on the net, blast tons
> of tests thru the vendors hardware and let them worry about the
> security ramifications?  Some of it is these codes are based on 'functional
> languages' and I just do not trust what I do not fully understand. Stuff
> (files etc) goes everywhere and that makes me cautiously nervous. I have
> /usr/local for manual work and /usr/local/portage for ovelays (layman) but
> it's becoming a mess. There where to I put the work effort that is a  result
> from repoman. Those codes seem to be parallel projects often
> when the code I'm evaluating needs to be cleaned up or extend to properly
> test. Furthermore I have a growing collection of file that result
> from kernel profiling via  trace-cmd, valgrind, systemtap etc etc.
> As soon as I delete something, I need to re-generated it for one
> reason or another...... I just hope that this repo.conf effort
> helps be get more structurally organized?  
> 
> 
> Did you see/test 'travis-ci' yet? [1] I'm not sure it's the same
> on github [2] but some of the devs are using it on github. 
> 
> 
> 
> James
> 
> [1] http://docs.travis-ci.com/
> 
> [2] https://github.com/travis-ci/travis-ci
> 
> 
> 
> 
> 
> 
> 
> 
> 


-- 
Alan McKinnon
alan.mckinnon@gmail.com

[gentoo-user] Re: rpm or deb package installs

[James]

Neil Bothwick <neil <at> digimed.co.uk> writes:


> > > I doubt dpkg and rpm aren't going to be much use to you, unless you
> > > really want to run two package managers. Besides, both are not
> > > especially useful with the front ends apt* and yum.  
> > 
> > I'd just use those to unpackage and maybe preprocess some of the codes.
> > 
> > Agreed. I do not want a full blown deb or rpm package manager just
> > a way to install and evaluate some of those codes before beginning a
> > more arduous  and comprehensive task.
> 
> In that case you ware deb2targz or rpm2targz to convert the package to a
> tarball. then you can unpack it and inspect the contents.


Agreed. Problem is my workflows it to test as is before looking that the
codes. If they do not do what they are suppose to (for clustering or HPC)
then why look under the hood.... Lots of pigs in clustering and HPC,
the stuff we use to run (and called it distributed or parallel) decades
ago was much faster.....  Putting bloat_ware on top of a cluster is
just plain stupid and that's what most are doing.... It negates the
entire point of distributed/hpc, imho.

thx,
James

[gentoo-user] Re: rpm or deb package installs

[James]

Alan McKinnon <alan.mckinnon <at> gmail.com> writes:


> I see you are doing more than I thought you were doing 
> 
> rpms and debs are both cpio files so the easy way is to unpack them and
> see what's going on:
> 
> rpm2cpio name.rpm | cpio -iv --make-directories
> dpkg -x somepackage.deb ~/temp/
> 
> Considering the size of what you are doing, you are probably better off
> running a Centos and Debian system to evaluate the code and discard the
> rubbish. Once you've isolated the interesting ones, you can evaluate
> them closer and maybe write ebuilds for them.

Workflow is the first test. Second unpack and look at the code
to maybe 5% of what I test. The tests I'm developing are low
level code profiling. Once I figure it all out what I need manually
looking at many (cluster/hpc) codes, then I hope to use CI
to automate the process of first performance profiling the codes.


THEN unpack and look at the codes iff it is impressive on performance. 

I am leaning towards a separate box for these evaluations, but a 
VM environment might be better in the long run...   dunno....


thx,
James 

[gentoo-user] Re: rpm or deb package installs

[James]

Bill Kenworthy <billk <at> iinet.net.au> writes:


> rpm is just a wrapper around a an archive with instructions on how to
> build and or install it.  I have more experience with rpm's but I
> believe debs are the same.  Just unwrap your .rpm/.deb file of choice
> and install it manually (the binaries/code are usually in a zip inside
> the rpm).  You should in most cases also be able to get a source rpm
> (which I suspect you are talking about anyway, but binaries do work deps
> permitting.

Yes the point is to be able to rapidly/easily be able to test and evaluate
many cluster/hpc codes, and other codes too, before I open them up
and look at them in detail, manually.


> you can install rpm and then install your package via rpm - seem to
> remember doing this with .debs somehow too.  deps are a problem but
> usually workable.

This is the argument for setting up an entire RH or debian workstation
to quickly evaluate the codes to find that less than %5 which are fast
and worthy of a deeper look. There are dozens of 'schedulers' for clusters
and then the concept of a 'framework' is even more loose or unconstrained
and the myriad of associated codes (modules) that one can use. It fact many
are pushing 'write your own framework'. It's hard to
categorize codes into a discernible 'stack' when it comes to HPC or cluster
offerings. Apache says one thing, a guy writing a book says another. It's
like the wild wild west with lots of folks (mostly large corps) shoot from
the hip and shooting off at the mouth. Many idiots and deceivers in this
space; and I intend to benchmark, profile and expose some of the myths.

So then I'll unpack the sources onto a gentoo system for deep examination
and low level (profile ) testing and evaluation and maybe creating an
ebuild if I like what I see. Distributing bloatware across a cluster
or on a HPC system, is a fools errand, imho, but that is exactly what
many are doing. It's very hard to figure out if they are 'flim_flam' artists
or sincerely belief their own musings.


> and why set up a workstation? - this sort of thing is tailor made for
> vm's.  Create a base for your experiments with essential packages,
> settings etc, snapshot it (golden master) and then throwaway->restore
> when finished with that iteration.

Ultimately, I think a VM or containers environment would be keen. I
liked docker, initially, but, like systemd, it is now bloating  so much that
it is becoming an entire operating system on it's own. Docker is too much
for me to worry about and learn. CoreOS is still attractive to me, but
it probably too much for me to divert into, at this time.  I'm really more
after HPC (high performance computing of scientific codes) than clustering
of a bizzilion processes (like log/web file analytics). That sort of stuff
is not hard to distribute, hadoop is sufficient for those routine, boring
codes and processes that are not time_critical. I'll do some of those
ordinary  tasks  but many cluster/hpc solutions already work good enough for
routine admin style processes, imho.


Big Science solutions using cluster or HPC are still very fragmented
if you look at some low level performance data. It takes a very specialized
focus to solve a given category of Big Science codes if you want to approach
any sort of reasonable robustness. It's like you need
a custom designed (HPCC) High Performance Computational Cluster with
a unique (at least highly customized and tweaked) implementation for each of
the many different possible configuration solutions. This part is evolving
on a daily basis, so apologies if it seems sketchy because it is sketchy at
best. I think where I'll end up is a myriad of HPCC, architecturally unique,
running along side of each other, mostly asleep until needed. It's even more
'fluid' when you consider the GPU resources, arm64 processors and the new
integrated CISC chips like the amd APU and the Intel+fpga chips.


> There are package managers besides gentoo/portage that can do a source
> build/install and track the files on gentoo - though portage will not
> know about it (rpm is one :)

Hmmmm. Where do I read more about this?


> and lastly, what do mean error prone? - to me a manual install is the
> ONLY way you can build a proper ebuild that catches most of the
> problems.  In the (admittedly few) ebuilds I have done an occasional
> human error is nothing compared to system problems for a difficult 
> package.

Error prone with manual steps, due to the large amount of codes I
am evaluating. Once I get down to less than 5% then the manual
processes are fine, if not superior, as I'm also reviewing the codes
of interest.  It's also me, if I do not impose self_structure, I get
sloppy with trying to manually speed things up. Also, if you go
back to the days of .configure and look at many different make files,
there is little coding consistency among different codes; boost and other
lib codes are the best you can hope for on consistency, imho. ymmv.

OK? Keep in mind that I'm an EE, hardware guy that started with discrete
transistors, native assemble and state machines; and C was a luxury, so my
perspectives do not often 'jive' with the entire OO world, but, I am trying
to get along....... [1]

> BillK

James

[1] http://www.phy.duke.edu/~rgb/Beowulf/c++_interview/c++_interview.html

Re: [gentoo-user] Re: rpm or deb package installs

[Tom H]

On Sat, Feb 14, 2015 at 1:30 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>
> rpms and debs are both cpio files so the easy way is to unpack them and
> see what's going on:
>
> rpm2cpio name.rpm | cpio -iv --make-directories
> dpkg -x somepackage.deb ~/temp/

For deb packages, you can use binutils' ar; there's no need for dpkg.
(IIRC, if you use rpm2tar, you don't need rpm installed unlike
rpm2cpio, but I'm not 100% sure.)

Re: [gentoo-user] Re: rpm or deb package installs

[R0b0t1]

> For deb packages, you can use binutils' ar; there's no need for dpkg.
> (IIRC, if you use rpm2tar, you don't need rpm installed unlike
> rpm2cpio, but I'm not 100% sure.)
>

You are right, rpm2targz doesn't require rpm to be installed. I found
I already had it installed yesterday (via libreoffice).