[gentoo-user] Using portage through NFS

[gentoo-user] Using portage through NFS

[Chris Lieb]

I have read the guide on gentoo-wiki about setting up portage to work
over NFS[0] and have it mostly working.  I have two issues that I would
like to work out:

1) I use sync-eix to update portage and my overlays (via layman).  I
want the client to still be able to run sync-eix, but have it only run
`emerge --metadata` (no `emerge --sync` or `layman --sync ALL`).  What
do I need to change in the eix-sync.conf?  (Man, that's a long man page :) )

Better yet, since my overlays are all in the exported NFS filesystem
(hence, the eix database would be the same across all clients), is it
possible to export my eix cache by hardlinking it into the NFS share?
If so, how do I make the client's eix use this database instead of the
one at /var/cache/eix?

2) I use the buildpkg feature on both the server and the client since
the client can usually use the packages for its own installations
(getbinpkg).  However, sometimes I require different use flags for the
client, but I still want to keep the package locally so I can restore it
later if I need to.  I have the NFS share mounted ro to keep the client
from overwriting what is on the server, so I am guessing that portage
will throw some kind of error when it tries to save the package to disk.

I was thinking of getting around this by using some kind of union mount.
 However, I don't understand how union mounts work or if they can be
used for my situation.  What I would like is to have some directory,
lets say /var/lib/portage/packages, that I union mount on top of the
exported NFS share, at /mnt/nfs_portage/packages.  I noticed in the
Portage w/ SquashFS/aufs howto[1], they used aufs to create a rw layer
on top of a ro SquashFS.  This sounds kind of what I want, except it
appears that aufs is memory-backed instead of disk-backed.  Is this so?
 The clients are all strapped for memory, so a memory-backed fs won't be
feasible.

Does anyone have any ideas or details on how I might implement this?

Thanks,
Chris Lieb

[0] http://en.gentoo-wiki.com/wiki/Sharing_Portage_over_NFS
[1] http://en.gentoo-wiki.com/wiki/Squashed_Portage_Tree

Re: [gentoo-user] Using portage through NFS

[Mike Kazantsev]

[-- Attachment #1: Type: text/plain, Size: 1596 bytes --]

On Thu, 05 Feb 2009 22:16:36 -0600
Chris Lieb <chris.lieb@gmail.com> wrote:

> 1) I use sync-eix to update portage and my overlays (via layman).  I
> want the client to still be able to run sync-eix, but have it only run
> `emerge --metadata` (no `emerge --sync` or `layman --sync ALL`).  What
> do I need to change in the eix-sync.conf?  (Man, that's a long man page :) )

You can use cluster-aware filesystems like GFS2 / OCFS2 to do anything
you like on any end, without having to worry about any inconsistency on
the other end.
That way you can do 'emerge --sync' or edit some overlay ebuild on any
end, seeing all the changes real-time on the other.


> 2) I use the buildpkg feature on both the server and the client since
> the client can usually use the packages for its own installations
> (getbinpkg).  However, sometimes I require different use flags for the
> client, but I still want to keep the package locally so I can restore it
> later if I need to.  I have the NFS share mounted ro to keep the client
> from overwriting what is on the server, so I am guessing that portage
> will throw some kind of error when it tries to save the package to disk.

Prehaps you should just 'mount /some/dev /usr/portage/packages' on each
machine, after you've mounted shared /usr/portage FS, so you'll have
unique packages on each server.
And if you'll need to copy some new packages from one machine to the
other without overwriting any of locally-compiled ones - you can just
use rsync, via cron, if it's kinda constant need.

-- 
Mike Kazantsev // fraggod.net

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

[gentoo-user] Re: Using portage through NFS

[Chris Lieb]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Chris Lieb wrote:
> I have read the guide on gentoo-wiki about setting up portage to work
> over NFS[0] and have it mostly working.  I have two issues that I would
> like to work out:
> 
> 1) I use sync-eix to update portage and my overlays (via layman).  I
> want the client to still be able to run sync-eix, but have it only run
> `emerge --metadata` (no `emerge --sync` or `layman --sync ALL`).  What
> do I need to change in the eix-sync.conf?  (Man, that's a long man page :) )
> 
> Better yet, since my overlays are all in the exported NFS filesystem
> (hence, the eix database would be the same across all clients), is it
> possible to export my eix cache by hardlinking it into the NFS share?
> If so, how do I make the client's eix use this database instead of the
> one at /var/cache/eix?

I got eix working by hardlinking the cache on the server into the NFS
share and setting EIX_CACHEFILE in /etc/eixrc on the client to point to
the mounted NFS share.  All tests show that eix on both the client and
server works fine after changing this.

> 2) I use the buildpkg feature on both the server and the client since
> the client can usually use the packages for its own installations
> (getbinpkg).  However, sometimes I require different use flags for the
> client, but I still want to keep the package locally so I can restore it
> later if I need to.  I have the NFS share mounted ro to keep the client
> from overwriting what is on the server, so I am guessing that portage
> will throw some kind of error when it tries to save the package to disk.
> 
> I was thinking of getting around this by using some kind of union mount.
>  However, I don't understand how union mounts work or if they can be
> used for my situation.  What I would like is to have some directory,
> lets say /var/lib/portage/packages, that I union mount on top of the
> exported NFS share, at /mnt/nfs_portage/packages.  I noticed in the
> Portage w/ SquashFS/aufs howto[1], they used aufs to create a rw layer
> on top of a ro SquashFS.  This sounds kind of what I want, except it
> appears that aufs is memory-backed instead of disk-backed.  Is this so?
>  The clients are all strapped for memory, so a memory-backed fs won't be
> feasible.
> 
> [0] http://en.gentoo-wiki.com/wiki/Sharing_Portage_over_NFS
> [1] http://en.gentoo-wiki.com/wiki/Squashed_Portage_Tree

After looking into aufs a little more and trying it, I discovered that,
in it's current state in the sunrise overlay, cannot fulfill my
requirements.  The most recent version of aufs does not have support for
mounting NFS filesystems (there might be a patch out there for it, but I
don't know how to configure this in an ebuild), and the last version of
aufs that supports NFS mounts doesn't work with newer kernels.

I don't want to do UnionFS since that requires me to patch the kernel,
which is more work than I have the time for.

Since I don't see any other options for stackable filesystems, I think
I'll just set the clients to not produce binpackages and wait for
UnionFS to get into the kernel or the Gentoo patchset.

Chris Lieb
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQEcBAEBAgAGBQJJjGwWAAoJEJWxx7fgsD+CxlcIAJwnlPvibmcdMGW/eNyaekYe
U1aGhsi09DoMU43aicKXNQ8p/rC2qwqXfocaxSXiBc4rrkrNpFhfTzPCDjwlhOSb
6fLtBpY7xo3m8CgP+g/Sv2tLFs4T/Gx4EqZ3TxfJh/xnvgUtFdlry+1BDqLPMsnl
XMJ5zetuJYZiOujoL4ZG8dS4Od0rVkomKegV0kP5pdiUbcdUN3tNMGMpa4pLvuIA
5DJtUfZk3Iyi5CyEqB/zNlUCmNS9z2hXHNw9y3pOVDQoeSCiufo+fGE8pUr6wXr9
sOy6EmmPvO5NQN/Z244smeWCgP8wNG6YSWEZik3lLU3Edw23VdeWA9KhOgNXAiQ=
=ypMH
-----END PGP SIGNATURE-----

Re: [gentoo-user] Re: Using portage through NFS

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 378 bytes --]

On Fri, 06 Feb 2009 10:57:58 -0600, Chris Lieb wrote:

> I don't want to do UnionFS since that requires me to patch the kernel,
> which is more work than I have the time for.

cd /usr/src/linux
patch -p1 </path/to/patchfile

You must be *really* short of time ;-)


-- 
Neil Bothwick

"Facts are stubborn things, but statistics are more pliable."
  - Mark Twain

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

[gentoo-user] Re: Using portage through NFS

[Chris Lieb]

Neil Bothwick wrote:
> On Fri, 06 Feb 2009 10:57:58 -0600, Chris Lieb wrote:
>
>> I don't want to do UnionFS since that requires me to patch the kernel,
>> which is more work than I have the time for.
>
> cd /usr/src/linux
> patch -p1 </path/to/patchfile
>
> You must be *really* short of time ;-)

My concern is more about making my own ebuild (based off of
gentoo-sources) that will patch the kernel source for me so that I can
easily distribute this patched kernel source to all of my computers.  I
need the process to be easy so that whom ever succeeds me doesn't have
to learn much more than just running a couple of scripts that I wrote.
Even how, I catch enough crap from my boss already for spending so much
time tinkering with servers instead of programming :)

A quick look at the gentoo-sources ebuild makes it look like the
patching must be happening in an eclass somewhere, but I'm no bash or
ebuild/eclass guru.  My skills end at fixing dependencies in packages
and doing simple ebuild copy version bumps.

Is there a guide out there for rolling my own kernel ebuild?

Chris

PS. Sorry if this is a double post.  The ml seems to silently reject my
posts that I do through gmane.

Re: [gentoo-user] Re: Using portage through NFS

[Roy Wright]

Chris Lieb wrote:
> Neil Bothwick wrote:
>> On Fri, 06 Feb 2009 10:57:58 -0600, Chris Lieb wrote:
>>
>>> I don't want to do UnionFS since that requires me to patch the kernel,
>>> which is more work than I have the time for.
>> cd /usr/src/linux
>> patch -p1 </path/to/patchfile
>>
>> You must be *really* short of time ;-)
> 
> My concern is more about making my own ebuild (based off of
> gentoo-sources) that will patch the kernel source for me so that I can
> easily distribute this patched kernel source to all of my computers.  I
> need the process to be easy so that whom ever succeeds me doesn't have
> to learn much more than just running a couple of scripts that I wrote.
> Even how, I catch enough crap from my boss already for spending so much
> time tinkering with servers instead of programming :)
> 
> Is there a guide out there for rolling my own kernel ebuild?
> 
> Chris

Take a peek at aufs on the sunrise overlay.  It has a section that 
patches the kernel, dependent on kernel version and USE flags.  It looks 
like it just informs the user that the kernel was patched.

HTH,
Roy