[gentoo-catalyst] [PATCH 1/2] Fix update_seed use by not using nor building binary packages during the seed update.

[gentoo-catalyst] [PATCH 1/2] Fix update_seed use by not using nor building binary packages during the seed update.

[Jorge Manuel B. S. Vicetto (jmbsvicetto)]

From: "Jorge Manuel B. S. Vicetto (jmbsvicetto)" <jmbsvicetto@gentoo.org>

---
 targets/stage1/stage1-chroot.sh     | 5 ++++-
 targets/support/chroot-functions.sh | 2 +-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/targets/stage1/stage1-chroot.sh b/targets/stage1/stage1-chroot.sh
index 3f628c2..ed83f38 100644
--- a/targets/stage1/stage1-chroot.sh
+++ b/targets/stage1/stage1-chroot.sh
@@ -30,12 +30,15 @@ if [ -n "${clst_update_seed}" ]; then
 		if [ -n "${clst_update_seed_command}" ]; then
 			clst_root_path=/ run_merge "--buildpkg=n ${clst_update_seed_command}"
 		else
-			clst_root_path=/ run_merge "--buildpkg=n --update --deep --newuse --onlydeps gcc"
+			clst_root_path=/ run_merge "--update --deep --newuse --complete-graph --rebuild-if-new-ver gcc"
 		fi
 	elif [ "${clst_update_seed}" != "no" ]; then
 		echo "Invalid setting for update_seed: ${clst_update_seed}"
 		exit 1
 	fi
+
+	# reset emerge options for the target
+	clst_update_seed=no setup_myemergeopts
 else
 	echo "Skipping seed stage update..."
 fi
diff --git a/targets/support/chroot-functions.sh b/targets/support/chroot-functions.sh
index 35ffbaa..0705b5b 100644
--- a/targets/support/chroot-functions.sh
+++ b/targets/support/chroot-functions.sh
@@ -133,7 +133,7 @@ setup_myemergeopts(){
 	then
 		export bootstrap_opts="${bootstrap_opts} -f"
 		export clst_myemergeopts="${clst_myemergeopts} -f"
-	elif [ -n "${clst_PKGCACHE}" ]
+	elif [ -n "${clst_PKGCACHE}" -a -z "${clst_update_seed}" ]
 	then
 		export clst_myemergeopts="${clst_myemergeopts} --usepkg --buildpkg --newuse"
 		export bootstrap_opts="${bootstrap_opts} -r"
-- 
1.8.1.5

[gentoo-catalyst] [PATCH 2/2] Fix bugs 407051 and 458536 by using FEATURES="preserve-libs" on the unmerge step of livecd-stage2 target.

[Jorge Manuel B. S. Vicetto (jmbsvicetto)]

From: "Jorge Manuel B. S. Vicetto (jmbsvicetto)" <jmbsvicetto@gentoo.org>

---
 targets/support/unmerge.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/targets/support/unmerge.sh b/targets/support/unmerge.sh
index 9f11273..907113e 100644
--- a/targets/support/unmerge.sh
+++ b/targets/support/unmerge.sh
@@ -2,6 +2,7 @@
 
 source /tmp/chroot-functions.sh
 
+FEATURES="${clst_myfeatures} preserve-libs"
 run_merge -C ${clst_packages}
 
 exit 0
-- 
1.8.1.5

Re: [gentoo-catalyst] [PATCH 2/2] Fix bugs 407051 and 458536 by using FEATURES="preserve-libs" on the unmerge step of livecd-stage2 target.

[Rick "Zero_Chaos" Farina]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/26/2013 11:09 PM, Jorge Manuel B. S. Vicetto (jmbsvicetto) wrote:
> From: "Jorge Manuel B. S. Vicetto (jmbsvicetto)" <jmbsvicetto@gentoo.org>
> 
I'm already having issues with preserve-libs even when I force disable
it in the env scripts.  I would really prefer this is to be made
optional before inclusion.  Even an esoteric/arcane option would make me
happy here.

- -ZC
> ---
>  targets/support/unmerge.sh | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/targets/support/unmerge.sh b/targets/support/unmerge.sh
> index 9f11273..907113e 100644
> --- a/targets/support/unmerge.sh
> +++ b/targets/support/unmerge.sh
> @@ -2,6 +2,7 @@
>  
>  source /tmp/chroot-functions.sh
>  
> +FEATURES="${clst_myfeatures} preserve-libs"
>  run_merge -C ${clst_packages}
>  
>  exit 0
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJRUxnVAAoJEKXdFCfdEflK970P/RyxNJrm0g2kVtQyxDB8dBUQ
APymvSpEEgvJpU42+++4A0+FoCxgWIG3dcnGCe+wqbtkIBGoEEGjAohQN+ydoqvd
LdUfknnE9i/CRU8tRRuGVrxt8wUfxHVkzMwQCcfzgbyOCvqVW7JULoyNrGHA5bhj
3moKOjX8+RBpX00wjNlSkUt4ti9AXAuR3QGVDCH3lC7uip39VX65v8UYzYF2laoN
MTnPQQA12sH61kc7wt01AUsvIYKlvs6ftE2ioii8Xxi8TrNndKX45VL/Xu4stvDJ
o2We0qrJX6VsQp+nYO5O/+YsjavLV+PT7a70WbvLb/w4wJhWKx5pRTv24iNuT9Dy
05oOdcZWVUgtn3rC5J+1+BYm5lTzkFkCABhYpXZzUWCpt5NE2Kn2QeWt1iuekEVD
Mt8j0VueQCLinO3NY8+OpoS+Z4sbzf1ng6lrJqSEPAmeaaVGSDmzD9UC/yf9hWiW
t9dAxsBSSf084YyZlA5YdrJykmmoiv/iu770TpXJDYKbjfM+LifCLgbjkUZjjOSG
tTO5jJyeAuH3rgxDqnICZleeB3AZhNY1YTWAQE7xsmtA83qNtduad/V/b8+1lR1h
Nr0Tr8mDA2HBxXvu2q4+vFQD4EwyqTXLycoLk85TlLYPxCrBXTCxR14Cv+8SMVxK
bBF3FWPONUQ6VWfaoLv8
=OkYN
-----END PGP SIGNATURE-----

Re: [gentoo-catalyst] [PATCH 1/2] Fix update_seed use by not using nor building binary packages during the seed update.

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 770 bytes --]

On Wed, Mar 27, 2013 at 03:09:56AM +0000, Jorge Manuel B. S. Vicetto (jmbsvicetto) wrote:
> -			clst_root_path=/ run_merge "--buildpkg=n --update --deep --newuse --onlydeps gcc"
> +			clst_root_path=/ run_merge "--update --deep --newuse --complete-graph --rebuild-if-new-ver gcc"

This looks like a reworking of Brian's "Fix broken seed stage
update..." [1].  It would probably be a good idea to explain the
differences between the two patches, since they seem to be solving the
same issues in slightly different ways.

Cheers,
Trevor

[1]: http://article.gmane.org/gmane.linux.gentoo.catalyst/2166/

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-catalyst] [PATCH 1/2] Fix update_seed use by not using nor building binary packages during the seed update.

[Rick "Zero_Chaos" Farina]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

It has come to light that this patch breaks binpkg use in stage1.
Personally I'd like to revert it entirely as the code is complex and
confusing to me for no reason.  Seperately I would fix the default
update_seed command because the "--rebuild-if-new-ver gcc" bit seems a
lot better to me.

Everyone approved this patch and it's been sitting in git for 2 months
without anyone realizing it's broken.  That alone should make it evident
that this code is too hard to read.

Can I get an ACK or NACK from the team members?

Thanks,
Zero

On 03/26/2013 11:09 PM, Jorge Manuel B. S. Vicetto (jmbsvicetto) wrote:
> From: "Jorge Manuel B. S. Vicetto (jmbsvicetto)" <jmbsvicetto@gentoo.org>
> 
> ---
>  targets/stage1/stage1-chroot.sh     | 5 ++++-
>  targets/support/chroot-functions.sh | 2 +-
>  2 files changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/targets/stage1/stage1-chroot.sh b/targets/stage1/stage1-chroot.sh
> index 3f628c2..ed83f38 100644
> --- a/targets/stage1/stage1-chroot.sh
> +++ b/targets/stage1/stage1-chroot.sh
> @@ -30,12 +30,15 @@ if [ -n "${clst_update_seed}" ]; then
>  		if [ -n "${clst_update_seed_command}" ]; then
>  			clst_root_path=/ run_merge "--buildpkg=n ${clst_update_seed_command}"
>  		else
> -			clst_root_path=/ run_merge "--buildpkg=n --update --deep --newuse --onlydeps gcc"
> +			clst_root_path=/ run_merge "--update --deep --newuse --complete-graph --rebuild-if-new-ver gcc"
>  		fi
>  	elif [ "${clst_update_seed}" != "no" ]; then
>  		echo "Invalid setting for update_seed: ${clst_update_seed}"
>  		exit 1
>  	fi
> +
> +	# reset emerge options for the target
> +	clst_update_seed=no setup_myemergeopts
>  else
>  	echo "Skipping seed stage update..."
>  fi
> diff --git a/targets/support/chroot-functions.sh b/targets/support/chroot-functions.sh
> index 35ffbaa..0705b5b 100644
> --- a/targets/support/chroot-functions.sh
> +++ b/targets/support/chroot-functions.sh
> @@ -133,7 +133,7 @@ setup_myemergeopts(){
>  	then
>  		export bootstrap_opts="${bootstrap_opts} -f"
>  		export clst_myemergeopts="${clst_myemergeopts} -f"
> -	elif [ -n "${clst_PKGCACHE}" ]
> +	elif [ -n "${clst_PKGCACHE}" -a -z "${clst_update_seed}" ]
>  	then
>  		export clst_myemergeopts="${clst_myemergeopts} --usepkg --buildpkg --newuse"
>  		export bootstrap_opts="${bootstrap_opts} -r"
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJRpDO+AAoJEKXdFCfdEflKr2QP/0Whgw5sSD6MmcKYcFRm5f/F
zJoKJ7bhGX5i2AOOFDj7C05ronjSPn5ADinEDNydBLLtrPkTSaMBvf+/XsJNRI8M
4IFAF39q3h4a7jBzRDI/SzjebXT1lJo2Tdx5HnF1lEJN28u8fyKoMW9SHRQXCqIt
mOj8tt6Ajw8SFXGgDpkmwii9nvp3MP5vIsO/yKoMnZoUm6cLXN5kT9JiE8RpPyYy
Gs7l4lVrBmwngGw40tpBNU5n1Koi3a7052qOQlUrdb4Bb29eVMEb53AbBGyvqZAb
vyO/b52qLJn2LmLDmhyZsDxOl76i98OxzHTgd/b5yNcF3J3kv4gI1iWI7fQUO6YI
gTpo5sZRzPvljAtqX3MfuLMtn5ZtsnxoKhSw1Oe30DBkcT0tzY36H/dNkdEA1WY0
Xh0i0UtwIgYKSU4WWMjsDErPtAbway2GuhfpSyjfGePaDiV84W3XvFQyBoBIQ6E8
g86t/y76mdINpqXKp7urYjb+o1CKt+zUQ+XoCOSkGDkA8yBhAheJu5RkLsztvvMd
C1hz2u/HxDGxqaWw40fWTWI3RKywJbL/JMC+pWULqI9Wz4Fv6DkaUoVM0BToqkGP
vgOVLbGc86t0mYYaSunUhZMQbSw99p0CdBUmxgnF8OyjPSJ9dd52ptiEFa3fllK/
vvGg8mlTjjWrmzliVBh4
=PNec
-----END PGP SIGNATURE-----

Re: [gentoo-catalyst] [PATCH 1/2] Fix update_seed use by not using nor building binary packages during the seed update.

[Brian Dolbec]

[-- Attachment #1: Type: text/plain, Size: 3781 bytes --]

On Tue, 2013-05-28 at 00:34 -0400, Rick "Zero_Chaos" Farina wrote:
> It has come to light that this patch breaks binpkg use in stage1.
...
> Thanks,
> Zero
> 
> On 03/26/2013 11:09 PM, Jorge Manuel B. S. Vicetto (jmbsvicetto) wrote:
> > From: "Jorge Manuel B. S. Vicetto (jmbsvicetto)" <jmbsvicetto@gentoo.org>
> > 
> > +	elif [ -n "${clst_PKGCACHE}" -a -z "${clst_update_seed}" ]

As it turns out, this one change was incorrect to evaluate all possible
scenarios.  My bash skills are poor, I originally had used nested if's ,
but that got optimized by others to the above line.

That line should be changed to:

elif [ -n "${clst_PKGCACHE}" ] && ( [ -z "${clst_update_seed}" ] || [ "${clst_update_seed}" == "no" ]

Which breaks down to (for those that can't follow it's logic):

# NOTE: clst_update_seed is currently only used for a stage1, so does not exist in the environment for other targets.
# spacing was added to the line so it lines up with the plain English text.

# if clst_PKGCACHE is not null   and  either clst_update_seed doesn't exist  or clst_update_seed is exactly equal to "no"
#if [ -n "${clst_PKGCACHE}" ]    &&  (        [ -z "${clst_update_seed}" ]   ||   [ "${clst_update_seed}" == "no" ] )


I have 3 reasons for wanting this change done this way

     1.  If a spec or config enables the PKGCACHE option.  It will add
        all the binpkg options to the emerge command.  Due to the
        possible problem with using/creating binpkgs during the
        update_seed process. These options should not be set.  Currently
        they are being set, but only --binpkg was being reset to =n in
        the stage1-chroot.sh update_seed command.  The other binpkg
        options were not being removed or reset.  This is poor coding.
        It is far better to just not add the unwanted options in the
        first place.
     2. While it is possible to work around not having the logic in
        setup_myemergeoptions(), it is hacky and complicates the
        stage1-chroot code more than neccessary.  It is also poor
        programming style.  In total it would mean the
        setup_myemergeoptions() would be called three times in total for
        the stage1 run.  Or temporarily saving the emerge options,
        unsetting clst_PKGCACHE, re-running setup_myemergeoptions() to
        get the correct options for update_seed, then later restoring
        the the two variables correctly. Both these coding options are
        poor programming style and more prone to bugs.  With the
        proposed change above it would be run twice.  Once keeping the
        PKGCACHE options off for the update_seed run, then reset
        according to the spec for the target emerge run.
     3. Placing the clst_update_seed logic to toggle the PKGCACHE
        options in the setup_myemergeoptions() simplifies the stage1
        code.  Plus it leaves the option to run an update_seed
        option/command for any future or current target without
        complicating/duplicating code.

I have attached a small test program I used to come up with the
correctly working patch change above.  It can be used to evaluate how
the line's logic is processed.  I have also attached the terminal output
it produced for all the possible conditions that could occur.  

Also I have slightly different PKGCACHE options in my rewrite branch.  I
have added --binpkg-respect-use to them.  It was brought to my attention
early in testing to put them in a config to eliminate the problem of
using binpkgs that were not compiled with them set correctly.

I will be pushing an updated rewrite branch with this and the other
changes made to master soon. So "git pull --force" to update your
checkout with the rewritten commits when I do.
-- 
Brian Dolbec <dolsen@gentoo.org>

[-- Attachment #2: tif --]
[-- Type: application/x-shellscript, Size: 764 bytes --]

[-- Attachment #3: tif.output --]
[-- Type: text/plain, Size: 1280 bytes --]

brian@big_daddy ~/Dev/git/catalyst $ ./tif "1"
YAY, PKGCACHE selected
brian@big_daddy ~/Dev/git/catalyst $ ./tif "1" no
clst_update_seed = no
YAY, PKGCACHE selected
brian@big_daddy ~/Dev/git/catalyst $ ./tif "" no
clst_update_seed = no
NOPE, PKGCACHE disabled
brian@big_daddy ~/Dev/git/catalyst $ ./tif ""
NOPE, PKGCACHE disabled
brian@big_daddy ~/Dev/git/catalyst $ clst_update_seed="" ./tif ""
NOPE, PKGCACHE disabled
brian@big_daddy ~/Dev/git/catalyst $ clst_update_seed="" ./tif "1"
YAY, PKGCACHE selected
brian@big_daddy ~/Dev/git/catalyst $ clst_update_seed="yes" ./tif "1"
NOPE, PKGCACHE disabled
brian@big_daddy ~/Dev/git/catalyst $ clst_update_seed="maybe" ./tif "1"
NOPE, PKGCACHE disabled
brian@big_daddy ~/Dev/git/catalyst $ clst_update_seed="maybe" ./tif "1"
NOPE, PKGCACHE disabled
brian@big_daddy ~/Dev/git/catalyst $ clst_update_seed="" ./tif ""
NOPE, PKGCACHE disabled
brian@big_daddy ~/Dev/git/catalyst $ ./tif "1"
YAY, PKGCACHE selected
brian@big_daddy ~/Dev/git/catalyst $ ./tif "1" no
setting clst_update_seed = no
YAY, PKGCACHE selected
brian@big_daddy ~/Dev/git/catalyst $ clst_update_seed="no" ./tif ""
NOPE, PKGCACHE disabled
brian@big_daddy ~/Dev/git/catalyst $ clst_update_seed="no" ./tif "1"
YAY, PKGCACHE selected
brian@big_daddy ~/Dev/git/catalyst $

Re: [gentoo-catalyst] [PATCH 1/2] Fix update_seed use by not using nor building binary packages during the seed update.

[Rick "Zero_Chaos" Farina]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/28/2013 12:42 PM, Brian Dolbec wrote:
> On Tue, 2013-05-28 at 00:34 -0400, Rick "Zero_Chaos" Farina wrote:
>> It has come to light that this patch breaks binpkg use in stage1.
> ...
>> Thanks,
>> Zero
>>
>> On 03/26/2013 11:09 PM, Jorge Manuel B. S. Vicetto (jmbsvicetto) wrote:
>>> From: "Jorge Manuel B. S. Vicetto (jmbsvicetto)" <jmbsvicetto@gentoo.org>
>>>
>>> +	elif [ -n "${clst_PKGCACHE}" -a -z "${clst_update_seed}" ]
> 
> As it turns out, this one change was incorrect to evaluate all possible
> scenarios.  My bash skills are poor, I originally had used nested if's ,
> but that got optimized by others to the above line.
> 
> That line should be changed to:
> 
> elif [ -n "${clst_PKGCACHE}" ] && ( [ -z "${clst_update_seed}" ] || [ "${clst_update_seed}" == "no" ]
> 
> Which breaks down to (for those that can't follow it's logic):
> 
> # NOTE: clst_update_seed is currently only used for a stage1, so does not exist in the environment for other targets.
> # spacing was added to the line so it lines up with the plain English text.
> 
> # if clst_PKGCACHE is not null   and  either clst_update_seed doesn't exist  or clst_update_seed is exactly equal to "no"
> #if [ -n "${clst_PKGCACHE}" ]    &&  (        [ -z "${clst_update_seed}" ]   ||   [ "${clst_update_seed}" == "no" ] )
> 
> 
> I have 3 reasons for wanting this change done this way
> 
>      1.  If a spec or config enables the PKGCACHE option.  It will add
>         all the binpkg options to the emerge command.  Due to the
>         possible problem with using/creating binpkgs during the
>         update_seed process. These options should not be set.  Currently
Can you please describe how using binpkgs during update_seed is an
issue?  I don't think all of us fully understand that, I know I don't.
>         they are being set, but only --binpkg was being reset to =n in
>         the stage1-chroot.sh update_seed command.  The other binpkg
>         options were not being removed or reset.  This is poor coding.
>         It is far better to just not add the unwanted options in the
>         first place.
>      2. While it is possible to work around not having the logic in
>         setup_myemergeoptions(), it is hacky and complicates the
>         stage1-chroot code more than neccessary.  It is also poor
>         programming style.  In total it would mean the
>         setup_myemergeoptions() would be called three times in total for
>         the stage1 run.  Or temporarily saving the emerge options,
>         unsetting clst_PKGCACHE, re-running setup_myemergeoptions() to
>         get the correct options for update_seed, then later restoring
>         the the two variables correctly. Both these coding options are
>         poor programming style and more prone to bugs.  With the
>         proposed change above it would be run twice.  Once keeping the
>         PKGCACHE options off for the update_seed run, then reset
>         according to the spec for the target emerge run.
>      3. Placing the clst_update_seed logic to toggle the PKGCACHE
>         options in the setup_myemergeoptions() simplifies the stage1
>         code.  Plus it leaves the option to run an update_seed
>         option/command for any future or current target without
>         complicating/duplicating code.
> 
> I have attached a small test program I used to come up with the
> correctly working patch change above.  It can be used to evaluate how
> the line's logic is processed.  I have also attached the terminal output
> it produced for all the possible conditions that could occur.  
> 
> Also I have slightly different PKGCACHE options in my rewrite branch.  I
> have added --binpkg-respect-use to them.  It was brought to my attention
> early in testing to put them in a config to eliminate the problem of
> using binpkgs that were not compiled with them set correctly.
We should probably make this default, if everyone agrees I can drum up a
quick patch and add it.  This is already default in my profile to
infinite success.

thanks,
Zero
> 
> I will be pushing an updated rewrite branch with this and the other
> changes made to master soon. So "git pull --force" to update your
> checkout with the rewritten commits when I do.
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJRpODDAAoJEKXdFCfdEflK51IP/0xZohUutpNMW+lpPKbkb7Kc
/peVQGs1wgnHZcvHbEnJb9F04oK4IIdal7eOpOi/cNagD/37RojNvvfHexe7zFXP
rhIoE/o++5Ma/iHjgHVY3EUk4mqdDKQMP9vht0g7TiAeNhjKa5bTAYeaq8vTBzCZ
/X9PFlLyPirs6MEszz/dWP0xkFVXSD9QCr4jF4g2brQIXjav0+P7jVFgITME/atS
H26F9tglDQeSZyM2mtTsbuzsPkHjau7Qkso7ITlyHNozore6M19hOLVLEXm9kfgq
8VDFq7rt8S3IsdkD85Xy8eIAw++wpFqMUiXubX1zCfa8qYPwf19yWWq4thQOt288
IRRtE/QGMaMi/V/rtoxnmQZEOHu8U5t5Uuw31DTNNmdVe1uRhs8WXdtbp/0ufFaG
BOGjv2yRkvgKx4d6TZcpn1T/LPJS/bFBEZYpPEyHV1G8leYOJi8CyasYZBSF7v7A
7U4GgfQul85+uogYTgNZ4I6d5aRfc5/FMH7/zycyIeKbNlso5iBow0s+XIKhhc4l
UYEyFirBNG55ZNIE18OxjQOCD2R9FjujwpJc1cuB20Hx93IRVifNIbcDZ6LaR5yq
fSLR/nzJJj2L3kv7Uq88akQClpVQMRruXQhNI7mTfhCjWmQtyHVbyeS8kOyJultV
yE4dIpKzkJ7OEuL4Dg0u
=ogZD
-----END PGP SIGNATURE-----

Re: [gentoo-catalyst] [PATCH 1/2] Fix update_seed use by not using nor building binary packages during the seed update.

[Brian Dolbec]

On Tue, 2013-05-28 at 12:52 -0400, Rick "Zero_Chaos" Farina wrote:
> On 05/28/2013 12:42 PM, Brian Dolbec wrote:
> > On Tue, 2013-05-28 at 00:34 -0400, Rick "Zero_Chaos" Farina wrote:

> > I have 3 reasons for wanting this change done this way
> > 
> >      1.  If a spec or config enables the PKGCACHE option.  It will add
> >         all the binpkg options to the emerge command.  Due to the
> >         possible problem with using/creating binpkgs during the
> >         update_seed process. These options should not be set.  Currently
> Can you please describe how using binpkgs during update_seed is an
> issue?  I don't think all of us fully understand that, I know I don't.

This was all discussed in irc and the catalyst list.  But somehow you
seemed to miss it all.

Early this year, mpc had an updated version.  
The existing update_seed command:

  clst_root_path=/ run_merge "--buildpkg=n --update --deep --newuse
--onlydeps gcc"

updated mpc, consequently, libmps.so,2 was deleted. libmpc.so.3 was
created.  This broke the existing gcc pkg due to the missing lib.
This means that gcc needs to be rebuilt in order to be linked to
libmpc.so.3. The existing command excluded gcc from being built (the
opposite to what is needed).  This failure in gcc did not show up until
stage2 I believe, and was a bit troublesome to pin down the cause. 

 Which is why I changed that line to:

clst_root_path=/ run_merge "--update --deep --newuse --complete-graph
--rebuild-if-new-ver gcc"

It may be slightly overkill with the --deep --complete-gragh but it
removes potential problems in revdep-rebuild, so...

NOW for how binpkgs play into this.

Existing gcc binpkgs have been linked to libmpc.so.2 and portage does
not check that all lib links exist before qualifying the binpkg to be
installed.  Therefore installing a gcc binpkg is a hit and miss
proposition.  Making it's use un-reliable.  Therefore until the
toolchain is migrated to eapi 5 with proper subslot use.  Using binpkgs
is unreliable for update_seed.

While this may slow down the stage1 process some.  It does make it more
reliable, so that the releng team won't be wasting time tracking down
errors that can be otherwise prevented.  Binpkgs can be created and
reused with some caution for the regular stage build.  But some errors
may similarly occur if the snapshot used is changed between runs without
clearing or checking existing binpkgs.  Again eapi 5 pkg migration will
solve this.  Hence the user beware warning.


> > Also I have slightly different PKGCACHE options in my rewrite branch.  I
> > have added --binpkg-respect-use to them.  It was brought to my attention
> > early in testing to put them in a config to eliminate the problem of
> > using binpkgs that were not compiled with them set correctly.
> We should probably make this default, if everyone agrees I can drum up a
> quick patch and add it.  This is already default in my profile to
> infinite success.
> 
> thanks,
> Zero
> > 
It was you that told me about that option, which is why i added it as
default when PKGCACHE is enabled.
That is something that Jorge did not pick up in my proposed patch.  I do
not know why.  Nothing was said as I recall.
-- 
Brian Dolbec <dolsen@gentoo.org>

Re: [gentoo-catalyst] [PATCH 1/2] Fix update_seed use by not using nor building binary packages during the seed update.

[Rick "Zero_Chaos" Farina]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/28/2013 01:59 PM, Brian Dolbec wrote:
> On Tue, 2013-05-28 at 12:52 -0400, Rick "Zero_Chaos" Farina wrote:
>> On 05/28/2013 12:42 PM, Brian Dolbec wrote:
>>> On Tue, 2013-05-28 at 00:34 -0400, Rick "Zero_Chaos" Farina wrote:
> 
>>> I have 3 reasons for wanting this change done this way
>>>
>>>      1.  If a spec or config enables the PKGCACHE option.  It will add
>>>         all the binpkg options to the emerge command.  Due to the
>>>         possible problem with using/creating binpkgs during the
>>>         update_seed process. These options should not be set.  Currently
>> Can you please describe how using binpkgs during update_seed is an
>> issue?  I don't think all of us fully understand that, I know I don't.
> 
> This was all discussed in irc and the catalyst list.  But somehow you
> seemed to miss it all.
> 
> Early this year, mpc had an updated version.  
> The existing update_seed command:
> 
>   clst_root_path=/ run_merge "--buildpkg=n --update --deep --newuse
> --onlydeps gcc"
> 
> updated mpc, consequently, libmps.so,2 was deleted. libmpc.so.3 was
> created.  This broke the existing gcc pkg due to the missing lib.
> This means that gcc needs to be rebuilt in order to be linked to
> libmpc.so.3. The existing command excluded gcc from being built (the
> opposite to what is needed).  This failure in gcc did not show up until
> stage2 I believe, and was a bit troublesome to pin down the cause. 
> 
>  Which is why I changed that line to:
> 
> clst_root_path=/ run_merge "--update --deep --newuse --complete-graph
> --rebuild-if-new-ver gcc"
> 
> It may be slightly overkill with the --deep --complete-gragh but it
> removes potential problems in revdep-rebuild, so...
> 
> NOW for how binpkgs play into this.
> 
> Existing gcc binpkgs have been linked to libmpc.so.2 and portage does
> not check that all lib links exist before qualifying the binpkg to be
> installed.  Therefore installing a gcc binpkg is a hit and miss
> proposition.  Making it's use un-reliable.  Therefore until the
> toolchain is migrated to eapi 5 with proper subslot use.  Using binpkgs
> is unreliable for update_seed.

In fact, the command is "--rebuild-if-new-ver" not
"--reinstall-if-new-ver".  As such, the original reporter of this bug
and I both seem to think that even with --usepkg, gcc should be rebuilt:

https://bugs.gentoo.org/show_bug.cgi?id=461422

That's obviously not how it is, but I feel we should focus our attention
on fixing this properly.
> 
> While this may slow down the stage1 process some.  It does make it more
> reliable, so that the releng team won't be wasting time tracking down
> errors that can be otherwise prevented.  Binpkgs can be created and
> reused with some caution for the regular stage build.  But some errors
> may similarly occur if the snapshot used is changed between runs without
> clearing or checking existing binpkgs.  Again eapi 5 pkg migration will
> solve this.  Hence the user beware warning.

As a rule, we don't want to hack around limitations in portage to make
catalyst work.  The toolchain team seems to have made it very clear they
aren't updating to eapi5 soon, but the portage team has been fixing
things left and right based on little more than my whims, I'd give them
a chance to fix this before throwing hacks into catalyst to work around
limitations in the package manager.

Thanks,
Zero
> 
> 
>>> Also I have slightly different PKGCACHE options in my rewrite branch.  I
>>> have added --binpkg-respect-use to them.  It was brought to my attention
>>> early in testing to put them in a config to eliminate the problem of
>>> using binpkgs that were not compiled with them set correctly.
>> We should probably make this default, if everyone agrees I can drum up a
>> quick patch and add it.  This is already default in my profile to
>> infinite success.
>>
>> thanks,
>> Zero
>>>
> It was you that told me about that option, which is why i added it as
> default when PKGCACHE is enabled.
> That is something that Jorge did not pick up in my proposed patch.  I do
> not know why.  Nothing was said as I recall.
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJRpPyQAAoJEKXdFCfdEflKM1oQALB5UV+N2Qeou3MVRuW4vx4w
88yHebh8Vm2wihhIk8iJeoUw05uHTpu6DlTcO0f7cjpqRpTc61ULYss+/YWFVbGv
PIB7uRquQv/aDFkfctSuUEYsVN94HmHB23T+tKHUPoGNWbmggvu4Y/z2nGAP6cNK
WbXx+6qvpMLK5LqN9+y9QB5Ml3of02i4brs1r42wUFxp6u/A4usuMq7s4hIoHUAh
RlxOIZPUCLxUV8BNNqsKsEOxFEZyawBnVAhUrp9C/GX4BvqsMeeT1R9KtgCjmcFI
lBtmyu4PVOUe/uBvfJd6CS3zpyx8ZY3/+BJzSEX7kB9ael2Ejg1JtGJFxQIC1RbL
/ctK2PBmiJ4BTGvzYN9HkgSqvsv6K+0IIsV2450oKd+CoyEeQ1M+Nj2NODVs+Io3
i3YyVDyZZzRqphdB8RBDLrrQ0ADuA9kRWNWTffaqla5l5pcdZs9EH87Kbvue2fZw
ISe50Bc1RPKePrhRthcP3OgaelU3YALDwOD2Apt+VGFAEmlByqwU2biqSXs6L/5v
zbPlvazqFjBPqDpJclIc8ETNUSqP5ODN2KesmZVd0pOSIsyDOjZoZL++HzRqtSrO
+vtRDXAq1kpFS25Frbd5yDbNQm+Qm9dVOWm6WJuG6kPwOW2AuI1m0lABFobo0nSr
NApls/UNICUhjIGZQsKZ
=5YP8
-----END PGP SIGNATURE-----

Re: [gentoo-catalyst] [PATCH 1/2] Fix update_seed use by not using nor building binary packages during the seed update.

[Brian Dolbec]

On Tue, 2013-05-28 at 14:50 -0400, Rick "Zero_Chaos" Farina wrote:
> On 05/28/2013 01:59 PM, Brian Dolbec wrote:

> > Existing gcc binpkgs have been linked to libmpc.so.2 and portage does
> > not check that all lib links exist before qualifying the binpkg to be
> > installed.  Therefore installing a gcc binpkg is a hit and miss
> > proposition.  Making it's use un-reliable.  Therefore until the
> > toolchain is migrated to eapi 5 with proper subslot use.  Using binpkgs
> > is unreliable for update_seed.
> 
> In fact, the command is "--rebuild-if-new-ver" not
> "--reinstall-if-new-ver".  As such, the original reporter of this bug
> and I both seem to think that even with --usepkg, gcc should be rebuilt:
> 
> https://bugs.gentoo.org/show_bug.cgi?id=461422
> 
> That's obviously not how it is, but I feel we should focus our attention
> on fixing this properly.
> > 

I think you did a typo there.  There are 2 portage options
--rebuild-if-new-ver and --rebuild-if-new-rev.  I opted for the *-ver
(version), since most revisions would not need to trigger a rebuild.
But the failure in that bug should be the same or similar fix to our
needs.

> As a rule, we don't want to hack around limitations in portage to make
> catalyst work.  The toolchain team seems to have made it very clear they
> aren't updating to eapi5 soon, but the portage team has been fixing
> things left and right based on little more than my whims, I'd give them
> a chance to fix this before throwing hacks into catalyst to work around
> limitations in the package manager.
> 
> Thanks,
> Zero
> > 

If someone wants to work on fixing it.  The solution will require
extracting the NEEDED.ELF.2 info from the binpkg and confirming that the
links exist.  If any of them don't, then trigger the build from source.
Rejecting the existing binpkg.

You may also want to talk to blueness, he created a utility for looking
for broken elf links.

Re: [gentoo-catalyst] [PATCH 1/2] Fix update_seed use by not using nor building binary packages during the seed update.

[Rick "Zero_Chaos" Farina]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/28/2013 03:33 PM, Brian Dolbec wrote:
> On Tue, 2013-05-28 at 14:50 -0400, Rick "Zero_Chaos" Farina wrote:
>> On 05/28/2013 01:59 PM, Brian Dolbec wrote:
> 
>>> Existing gcc binpkgs have been linked to libmpc.so.2 and portage does
>>> not check that all lib links exist before qualifying the binpkg to be
>>> installed.  Therefore installing a gcc binpkg is a hit and miss
>>> proposition.  Making it's use un-reliable.  Therefore until the
>>> toolchain is migrated to eapi 5 with proper subslot use.  Using binpkgs
>>> is unreliable for update_seed.
>>
>> In fact, the command is "--rebuild-if-new-ver" not
>> "--reinstall-if-new-ver".  As such, the original reporter of this bug
>> and I both seem to think that even with --usepkg, gcc should be rebuilt:
>>
>> https://bugs.gentoo.org/show_bug.cgi?id=461422
>>
>> That's obviously not how it is, but I feel we should focus our attention
>> on fixing this properly.
>>>
> 
> I think you did a typo there.  There are 2 portage options
> --rebuild-if-new-ver and --rebuild-if-new-rev.  I opted for the *-ver
> (version), since most revisions would not need to trigger a rebuild.
> But the failure in that bug should be the same or similar fix to our
> needs.

Not sure how *I* did a typo, looks like I typed the command properly
above, and I've not even commented on the bug yet... Yeah I do realize
that the bug is talking about -rev not -ver but the affected code
appears to be shared hence why it's brought up.
> 
>> As a rule, we don't want to hack around limitations in portage to make
>> catalyst work.  The toolchain team seems to have made it very clear they
>> aren't updating to eapi5 soon, but the portage team has been fixing
>> things left and right based on little more than my whims, I'd give them
>> a chance to fix this before throwing hacks into catalyst to work around
>> limitations in the package manager.
>>
>> Thanks,
>> Zero
>>>
> 
> If someone wants to work on fixing it.  The solution will require
> extracting the NEEDED.ELF.2 info from the binpkg and confirming that the
> links exist.  If any of them don't, then trigger the build from source.
> Rejecting the existing binpkg.
> 
> You may also want to talk to blueness, he created a utility for looking
> for broken elf links.

Here is where I admit to being lazy, I was planning on "fixing" it by
just triggering the src build no matter what when a rebuild occurs and
then hoping that someday people in gentoo actually use EAPI5 and make
this all unnecessary.

- -Zero
> 
> 
> 
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJRpQuJAAoJEKXdFCfdEflKtI8P/iNpLeHAKD6E8li0hm2Pw5RI
CM0HfvYmLioKdevAvlbvMjNiI/WgwgmOuF8Stj8biFZn80vYQpDYwuei6jEvnuSO
wTBNmnI+v6kC91rVb7LTdpIOXpfCg3V27kHRNOWqNnyVSd4WK6kKjp6c2YqHcnwl
2bxHFjvOXFK8s/n9PVrBBgn5VrMqQzlc4VqVPn5FHQJP2/aMBpVChwrAZBcS6nGI
8wMVYMMfjwypk8jdroSTQC32Etw/T1jwcWPvUhgqtU7BCmGRSC1zXu22RuwfiLmK
WPd2BmMl2jqBo6FOPSx83JfN+/OKumJy55aiz7LMVF+STPIfsSuzvvcipDJV2inv
s8mkVXybpSYs/fdbpUEcx2fRuAeseSTAf96iMVVpjcKeFEavMxWzTSOwwQqFOkIy
m8Gy8zON0YbpNZA/Zy5FnChYJ0R7ci8s0keEeVozMqE82XfgyAlkJFW771XQUs26
T+bK8sDgpANRxPx4c8AT18+Sudkplo7u06kDsRkQ+Ru2Plc9lsVYkpbI1FjjwG7X
uxCjiigCZA5Y+x4tnoDo6Rl8rAhllRTuHbzZpjbvZXxFlYHgrMsSQzPNgwW7UqgC
iIIYCd6rVrfSsc2qtgedJacdMYHTTGGy+Hq1SWNHWqsedhd063q4HJ3Wqk6Epivu
nAtvVC7SFo/f++WU36+z
=2knI
-----END PGP SIGNATURE-----