[gentoo-user] creating overlay to modify net-fs/samba-3.6.18

[gentoo-user] creating overlay to modify net-fs/samba-3.6.18

[Timur Aydin]

Today I had quite a bit of issues while creating a local overlay to
modify samba. Got through all of them, but there were the following
issues which I wanted to bring up, because I am not sure whether I have
done anything wrong:

- I had created the overlay under "/root/overlay", but this is causing
"permission denied" errors from ebuild when creating the Manifest file:

======================================================================
/usr/lib64/portage/bin/ebuild.sh: line 545:
/root/overlay/net-fs/samba/samba-3.6.18.ebuild: Permission denied
 * ERROR: net-fs/samba-3.6.18::myoverlay failed (depend phase):
 *   error sourcing ebuild
 *
 * Call stack:
 *   ebuild.sh, line 545:  Called die
 * The specific snippet of code:
 *                      source "$EBUILD" || die "error sourcing ebuild"
 *
 * If you need support, post the output of `emerge --info
'=net-fs/samba-3.6.18::myoverlay'`,
 * the complete build log and the output of `emerge -pqv
'=net-fs/samba-3.6.18::myoverlay'`.
 * Working directory: '/usr/lib64/portage/pym'
 * S: '/var/tmp/portage/net-fs/samba-3.6.18/work/samba-3.6.18'
======================================================================

After trying many different things without luck, I tried using
/usr/local/portage as the overlay folder and the problem went away! So
my first question is, does ebuild have a special security feature which
prevents it from processing overlays under /root ?

- The second issue is eclass related. After I moved net-fs/samba under
/usr/local/portage, I got the following error during Manifest creation:

======================================================================
bonsai samba # ebuild samba-3.6.18.ebuild manifest
 * ERROR: net-fs/samba-3.6.18::myoverlay failed (depend phase):
 *   pam.eclass could not be found by inherit()
 *
 * Call stack:
 *             ebuild.sh, line 545:  Called source
'/usr/local/portage/net-fs/samba/samba-3.6.18.ebuild'
 *   samba-3.6.18.ebuild, line   7:  Called inherit 'pam' 'versionator'
'multilib' 'eutils' 'systemd'
 *             ebuild.sh, line 257:  Called die
 * The specific snippet of code:
 *              [[ -z ${location} ]] && die "${1}.eclass could not be
found by inherit()"
 *
 * If you need support, post the output of `emerge --info
'=net-fs/samba-3.6.18::myoverlay'`,
 * the complete build log and the output of `emerge -pqv
'=net-fs/samba-3.6.18::myoverlay'`.
 * Working directory: '/usr/lib64/portage/pym'
 * S: '/var/tmp/portage/net-fs/samba-3.6.18/work/samba-3.6.18'
======================================================================

To resolve this, I just copied /usr/portage/eclass to
/usr/local/portage/eclass. Is this a proper solution?

--
Timur Aydin

Re: [gentoo-user] creating overlay to modify net-fs/samba-3.6.18

[Alan McKinnon]

On 10/09/2013 22:11, Timur Aydin wrote:
> Today I had quite a bit of issues while creating a local overlay to
> modify samba. Got through all of them, but there were the following
> issues which I wanted to bring up, because I am not sure whether I have
> done anything wrong:
> 
> - I had created the overlay under "/root/overlay", but this is causing
> "permission denied" errors from ebuild when creating the Manifest file:

Do you have userpriv set in FEATURES?

That makes emerge run as portage which cannot read /root.

Putting things like code, data and configs in /root is universally a bad
idea. Do not do this. Put such things in the usual places, protected by
orthodox permissions, just like every other package on your system. Your
local overlay is no different to the tree and does not require special
handling.

[snip]

> After trying many different things without luck, I tried using
> /usr/local/portage as the overlay folder and the problem went away! So
> my first question is, does ebuild have a special security feature which
> prevents it from processing overlays under /root ?

No, you just had the overlay in a highly unusual place.

> 
> - The second issue is eclass related. After I moved net-fs/samba under
> /usr/local/portage, I got the following error during Manifest creation:
> 
> ======================================================================
> bonsai samba # ebuild samba-3.6.18.ebuild manifest
>  * ERROR: net-fs/samba-3.6.18::myoverlay failed (depend phase):
>  *   pam.eclass could not be found by inherit()

[snip]


> To resolve this, I just copied /usr/portage/eclass to
> /usr/local/portage/eclass. Is this a proper solution?

No, that copies ALL the eclasses and is likely to really mess things up.
In fact, you should not have had to do anything with eclasses at all.

I suspect your make.conf is faulty.

What is the contents of make.conf?
what is ls -al $PORTDIR/eclass/pam.eclass


-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] creating overlay to modify net-fs/samba-3.6.18

[Timur Aydin]

On 09/11/13 00:01, Alan McKinnon wrote:
> Do you have userpriv set in FEATURES?
> 
> That makes emerge run as portage which cannot read /root.

Here is the value of FEATURES, based on the output of emerge --info:

FEATURES="assume-digests binpkg-logs config-protect-if-modified
distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch
preserve-libs protect-owned sandbox sfperms strict unknown-features-warn
unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"

So as you say, userpriv is set and it explains why /root/overlay doesn't
work.

> 
> I suspect your make.conf is faulty.
> 
> What is the contents of make.conf?
> what is ls -al $PORTDIR/eclass/pam.eclass
> 

Here are the contents of /etc/portage/make.conf:

======================================================================
# These settings were set by the catalyst build script that automatically
# built this stage.
# Please consult /usr/share/portage/config/make.conf.example for a more
# detailed example.
CFLAGS="-O2 -pipe"
CXXFLAGS="${CFLAGS}"
# WARNING: Changing your CHOST is not something that should be done lightly.
# Please consult http://www.gentoo.org/doc/en/change-chost.xml before
changing.
CHOST="x86_64-pc-linux-gnu"
# These are the USE flags that were used in addition to what is provided
by the
# profile used for building.
MAKEOPTS="-j8"
GENTOO_MIRRORS="ftp://ftp.linux.org.tr/pub/gentoo"
USE="nptl nptlonly unicode minizip tk udev python threads xvfb perl dlz
caps sqlite upcall systemd -gtk2 -gtk3 -acl"
ACCEPT_KEYWORDS="~amd64"
ACCEPT_LICENSE="*"
VIDEO_CARDS="intel"
INPUT_DEVICES="evdev keyboard mouse"
FEATURES="-test"
PORTDIR_OVERLAY="/usr/local/portage"
======================================================================

And here is the pam.eclass perms:

bonsai ~ # ls -al /usr/portage/eclass/pam.eclass
-rw-r--r-- 1 portage portage 6455 Aug  5  2012
/usr/portage/eclass/pam.eclass

Re: [gentoo-user] creating overlay to modify net-fs/samba-3.6.18

[Alan McKinnon]

On 11/09/2013 00:22, Timur Aydin wrote:
> On 09/11/13 00:01, Alan McKinnon wrote:

>> I suspect your make.conf is faulty.
>>
>> What is the contents of make.conf?
>> what is ls -al $PORTDIR/eclass/pam.eclass
>>
> 
> Here are the contents of /etc/portage/make.conf:
> 
> ======================================================================
> # These settings were set by the catalyst build script that automatically
> # built this stage.
> # Please consult /usr/share/portage/config/make.conf.example for a more
> # detailed example.
> CFLAGS="-O2 -pipe"
> CXXFLAGS="${CFLAGS}"
> # WARNING: Changing your CHOST is not something that should be done lightly.
> # Please consult http://www.gentoo.org/doc/en/change-chost.xml before
> changing.
> CHOST="x86_64-pc-linux-gnu"
> # These are the USE flags that were used in addition to what is provided
> by the
> # profile used for building.
> MAKEOPTS="-j8"
> GENTOO_MIRRORS="ftp://ftp.linux.org.tr/pub/gentoo"
> USE="nptl nptlonly unicode minizip tk udev python threads xvfb perl dlz
> caps sqlite upcall systemd -gtk2 -gtk3 -acl"
> ACCEPT_KEYWORDS="~amd64"
> ACCEPT_LICENSE="*"
> VIDEO_CARDS="intel"
> INPUT_DEVICES="evdev keyboard mouse"
> FEATURES="-test"
> PORTDIR_OVERLAY="/usr/local/portage"
> ======================================================================
> 
> And here is the pam.eclass perms:
> 
> bonsai ~ # ls -al /usr/portage/eclass/pam.eclass
> -rw-r--r-- 1 portage portage 6455 Aug  5  2012
> /usr/portage/eclass/pam.eclass



I can't reproduce that fault here, and google hits on the matter seem to
point towards stale metadata referencing eclasses that no longer exist.
I have a hunch you do not have valid metadata for your local overlay, so
I suggest:

1. delete the eclass directory from your overlay, run "emerge
--metadata" and emerge samba again. See what that does.

2. Set PORTDIR_CACHE_METHOD and/or OVERLAY_CACHE_METHOD explicitly in
make.conf, the best reference for these is in the eix man page


-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] creating overlay to modify net-fs/samba-3.6.18

[Timur Aydin]

On 09/11/13 23:13, Alan McKinnon wrote:
> I can't reproduce that fault here, and google hits on the matter seem to
> point towards stale metadata referencing eclasses that no longer exist.
> I have a hunch you do not have valid metadata for your local overlay, so
> I suggest:
> 
> 1. delete the eclass directory from your overlay, run "emerge
> --metadata" and emerge samba again. See what that does.

I have tried this, and the same problem happened, where emerge
complained that pam.eclass wasn't found.

> 2. Set PORTDIR_CACHE_METHOD and/or OVERLAY_CACHE_METHOD explicitly in
> make.conf, the best reference for these is in the eix man page

While reading up on the various methods in the eix man page, I started
to wonder whether a setting in /usr/local/portage/metadata/layout.conf
was causing this problem. When testing, layout.conf had only a single
entry, which was:

masters =

I remembered an error message from portage a while ago, which suggested
setting this to "gentoo" for backward compatibility. I tried that, and
the problem went away :) I was able to emerge samba from the overlay
successfully. Does this make sense?

--
Timur

Re: [gentoo-user] creating overlay to modify net-fs/samba-3.6.18

[Alan McKinnon]

On 11/09/2013 22:46, Timur Aydin wrote:
> On 09/11/13 23:13, Alan McKinnon wrote:
>> I can't reproduce that fault here, and google hits on the matter seem to
>> point towards stale metadata referencing eclasses that no longer exist.
>> I have a hunch you do not have valid metadata for your local overlay, so
>> I suggest:
>>
>> 1. delete the eclass directory from your overlay, run "emerge
>> --metadata" and emerge samba again. See what that does.
> 
> I have tried this, and the same problem happened, where emerge
> complained that pam.eclass wasn't found.
> 
>> 2. Set PORTDIR_CACHE_METHOD and/or OVERLAY_CACHE_METHOD explicitly in
>> make.conf, the best reference for these is in the eix man page
> 
> While reading up on the various methods in the eix man page, I started
> to wonder whether a setting in /usr/local/portage/metadata/layout.conf
> was causing this problem. When testing, layout.conf had only a single
> entry, which was:
> 
> masters =
> 
> I remembered an error message from portage a while ago, which suggested
> setting this to "gentoo" for backward compatibility. I tried that, and
> the problem went away :) I was able to emerge samba from the overlay
> successfully. Does this make sense?



/headbang :-)

Yes of course, it make a great deal of sense now. Basically, your local
overlay had no idea where the parent portage tree is or how to find it
so couldn't find the eclass directory.

As I understand it, this information used to be hard-coded magic and
overlays would "just know where to look". Since recently, you have to
configure it explicitly and not use hidden super-magic.

You would have been getting confusing messages in emerge output about a
faulty masters setting for overlays, pity we didn't spot that up front.
Double pity that there wasn't a clear message or news item about what
the error meant and the impact....


-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] creating overlay to modify net-fs/samba-3.6.18

[Timur Aydin]

On 09/12/13 01:00, Alan McKinnon wrote:
> Yes of course, it make a great deal of sense now. Basically, your local
> overlay had no idea where the parent portage tree is or how to find it
> so couldn't find the eclass directory.
> 
> As I understand it, this information used to be hard-coded magic and
> overlays would "just know where to look". Since recently, you have to
> configure it explicitly and not use hidden super-magic.
> 
> You would have been getting confusing messages in emerge output about a
> faulty masters setting for overlays, pity we didn't spot that up front.
> Double pity that there wasn't a clear message or news item about what
> the error meant and the impact....

Thanks, Alan, for the great help! SAMBA is now up and running with my
custom changes...

--
Timur

Re: [gentoo-user] creating overlay to modify net-fs/samba-3.6.18

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 1575 bytes --]

On Thursday 12 Sep 2013 01:28:00 Timur Aydin wrote:
> On 09/12/13 01:00, Alan McKinnon wrote:
> > Yes of course, it make a great deal of sense now. Basically, your local
> > overlay had no idea where the parent portage tree is or how to find it
> > so couldn't find the eclass directory.
> > 
> > As I understand it, this information used to be hard-coded magic and
> > overlays would "just know where to look". Since recently, you have to
> > configure it explicitly and not use hidden super-magic.
> > 
> > You would have been getting confusing messages in emerge output about a
> > faulty masters setting for overlays, pity we didn't spot that up front.
> > Double pity that there wasn't a clear message or news item about what
> > the error meant and the impact....


What happens when you run emerge with the new portage is that this error 
message pops up:

# emerge -uaDv world
!!! Repository 'x-portage' is missing masters attribute in 
'/usr/local/portage/metadata/layout.conf'
!!! Set 'masters = gentoo' in this file for future compatibility


However, as Alan says it is a puzzle why there wasn't a news item warning 
users of this new configuration requirement and how things may break if it is 
not complied with;  or why the ebuild does not create itself the 
'/usr/local/portage/metadata/' directory and populate layout.conf with default 
values - unless the user has already done so.

Either way it shouldn't let the user make WAGs as to what is now necessary for 
a properly functioning package manager.
-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-user] creating overlay to modify net-fs/samba-3.6.18

[Alan McKinnon]

On 14/09/2013 11:26, Mick wrote:
> On Thursday 12 Sep 2013 01:28:00 Timur Aydin wrote:
>> On 09/12/13 01:00, Alan McKinnon wrote:
>>> Yes of course, it make a great deal of sense now. Basically, your local
>>> overlay had no idea where the parent portage tree is or how to find it
>>> so couldn't find the eclass directory.
>>>
>>> As I understand it, this information used to be hard-coded magic and
>>> overlays would "just know where to look". Since recently, you have to
>>> configure it explicitly and not use hidden super-magic.
>>>
>>> You would have been getting confusing messages in emerge output about a
>>> faulty masters setting for overlays, pity we didn't spot that up front.
>>> Double pity that there wasn't a clear message or news item about what
>>> the error meant and the impact....
> 
> 
> What happens when you run emerge with the new portage is that this error 
> message pops up:
> 
> # emerge -uaDv world
> !!! Repository 'x-portage' is missing masters attribute in 
> '/usr/local/portage/metadata/layout.conf'
> !!! Set 'masters = gentoo' in this file for future compatibility
> 
> 
> However, as Alan says it is a puzzle why there wasn't a news item warning 
> users of this new configuration requirement and how things may break if it is 
> not complied with;  or why the ebuild does not create itself the 
> '/usr/local/portage/metadata/' directory and populate layout.conf with default 
> values - unless the user has already done so.
> 
> Either way it shouldn't let the user make WAGs as to what is now necessary for 
> a properly functioning package manager.


Is it just me, or has there really been an extraordinary number of
high-impact changes this year made WITHOUT news items?

I get a sense of a culture shift amongst the devs where news items are
considered less important than they should be. I don't have hard facts,
this is just my impression, but maybe the Council should do a PR
exercise to impress on devs that news items are really important


-- 
Alan McKinnon
alan.mckinnon@gmail.com