[gentoo-embedded] crossdev improvements

[gentoo-embedded] crossdev improvements

[Joakim Tjernlund]

Some improvements thoughts for crossdev:

- make crossdev respect PORTAGE_CONFIGROOT. We use our own portage
  tree for our pkgs, including the compiler, as we want to be able
  to recreate/rebuild the same sw versions in the future.

- a switch to control softfloat instead of encoding this in vendor
  name. The cross name is getting too long with softfloat in it.

- make crossdev build a binary cross compiler pkg which can be
  emerged by other machines.

 Jocke

Re: [gentoo-embedded] crossdev improvements

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 886 bytes --]

On Friday, September 24, 2010 12:01:21 Joakim Tjernlund wrote:
> - make crossdev respect PORTAGE_CONFIGROOT. We use our own portage
>   tree for our pkgs, including the compiler, as we want to be able
>   to recreate/rebuild the same sw versions in the future.

i dont know what this means.  crossdev doesnt touch PORTAGE_CONFIGROOT at all.

> - a switch to control softfloat instead of encoding this in vendor
>   name. The cross name is getting too long with softfloat in it.

except that softfloat checks go beyond the cross-compiler and packages in the 
tree need to be able to query support.  unless you have a clean way of 
handling this, there is no alternative.

> - make crossdev build a binary cross compiler pkg which can be
>   emerged by other machines.

crossdev is merely a frontend to emerge.  so do it yourself:
	crossdev ... --portage -B ...
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-embedded] crossdev improvements

[Joakim Tjernlund]

Mike Frysinger <vapier@gentoo.org> wrote on 2010/09/24 19:01:26:
>
> On Friday, September 24, 2010 12:01:21 Joakim Tjernlund wrote:
> > - make crossdev respect PORTAGE_CONFIGROOT. We use our own portage
> >   tree for our pkgs, including the compiler, as we want to be able
> >   to recreate/rebuild the same sw versions in the future.
>
> i dont know what this means.  crossdev doesnt touch PORTAGE_CONFIGROOT at all.

Exactly, it uses hardcoded paths to /etc et. all instead of PORTAGE_CONFIGROOT/etc

>
> > - a switch to control softfloat instead of encoding this in vendor
> >   name. The cross name is getting too long with softfloat in it.
>
> except that softfloat checks go beyond the cross-compiler and packages in the
> tree need to be able to query support.  unless you have a clean way of
> handling this, there is no alternative.

packages in the tree? I figured that only gcc and glibc(possibly binutils too)
needed that info?

I just tried to use EXTRA_ECONF=--without-fp instead of softfloat
and it seems to have worked, at least gcc and glibc builds without FPU insn's

hmm, wonder if COLLECT_GCC_OPTIONS could be used to see if softfloat is on?

>
> > - make crossdev build a binary cross compiler pkg which can be
> >   emerged by other machines.
>
> crossdev is merely a frontend to emerge.  so do it yourself:
>    crossdev ... --portage -B ...

That works? Guess I have to try it then :)
However, the sauna is waiting for me ATM so it will have to wait ..

 Jocke

Re: [gentoo-embedded] crossdev improvements

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 750 bytes --]

On Friday, September 24, 2010 13:42:04 Joakim Tjernlund wrote:
> Mike Frysinger <vapier@gentoo.org> wrote on 2010/09/24 19:01:26:
> > On Friday, September 24, 2010 12:01:21 Joakim Tjernlund wrote:
> > > - make crossdev respect PORTAGE_CONFIGROOT. We use our own portage
> > > 
> > >   tree for our pkgs, including the compiler, as we want to be able
> > >   to recreate/rebuild the same sw versions in the future.
> > 
> > i dont know what this means.  crossdev doesnt touch PORTAGE_CONFIGROOT at
> > all.
> 
> Exactly, it uses hardcoded paths to /etc et. all instead of
> PORTAGE_CONFIGROOT/etc

so you're talking about the /etc/portage/ file setup.  you should be explicit 
in your examples :P.

i'd file a bug about this.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-embedded] crossdev improvements

[Joakim Tjernlund]

Mike Frysinger <vapier@gentoo.org> wrote on 2010/09/25 07:37:49:
>
> On Friday, September 24, 2010 13:42:04 Joakim Tjernlund wrote:
> > Mike Frysinger <vapier@gentoo.org> wrote on 2010/09/24 19:01:26:
> > > On Friday, September 24, 2010 12:01:21 Joakim Tjernlund wrote:
> > > > - make crossdev respect PORTAGE_CONFIGROOT. We use our own portage
> > > >
> > > >   tree for our pkgs, including the compiler, as we want to be able
> > > >   to recreate/rebuild the same sw versions in the future.
> > >
> > > i dont know what this means.  crossdev doesnt touch PORTAGE_CONFIGROOT at
> > > all.
> >
> > Exactly, it uses hardcoded paths to /etc et. all instead of
> > PORTAGE_CONFIGROOT/etc
>
> so you're talking about the /etc/portage/ file setup.  you should be explicit
> in your examples :P.
>
> i'd file a bug about this.
> -mike

Until then, here is an initial hack(untested), against 20100814:

From 50337e535b7dff4fb473db097483ba5ffe7b355e Mon Sep 17 00:00:00 2001
From: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
Date: Sat, 25 Sep 2010 19:07:28 +0200
Subject: [PATCH] Initial support for PORTAGE_CONFIGROOT

---
 crossdev-20100814/crossdev |   32 +++++++++++++++++---------------
 1 files changed, 17 insertions(+), 15 deletions(-)

diff --git a/crossdev-20100814/crossdev b/crossdev-20100814/crossdev
index 54a6ed7..fc9b89e 100755
--- a/crossdev-20100814/crossdev
+++ b/crossdev-20100814/crossdev
@@ -12,7 +12,8 @@ if [[ ${ROOT:-/} != "/" ]] ; then
 	exit 2
 fi

-source /etc/init.d/functions.sh || exit 1
+setup_portage_vars
+source ${PORTAGE_CONFIGROOT}/etc/init.d/functions.sh || exit 1
 esyslog() { :; }
 die() {
 	echo
@@ -235,6 +236,7 @@ parse_target() {
 }

 setup_portage_vars() {
+	PORTAGE_CONFIGROOT=$(portageq envvar PORTAGE_CONFIGROOT)
 	PORTDIR_OVERLAYS=$(portageq envvar PORTDIR_OVERLAY)
 	PORTDIR_OVERLAY=${PORTDIR_OVERLAYS%% *}
 	PORTDIR=$(portageq envvar PORTDIR)
@@ -253,9 +255,9 @@ uninstall() {

 	[[ -d ${PORTDIR_OVERLAY}/cross-${CTARGET} ]] \
 		&& rm -r ${PORTDIR_OVERLAY}/cross-${CTARGET}
-	sed -i -e "/^cross-${CTARGET}$/d" /etc/portage/categories
+	sed -i -e "/^cross-${CTARGET}$/d" ${PORTAGE_CONFIGROOT}/etc/portage/categories
 	for f in package.{mask,keywords,use} ; do
-		f="/etc/portage/${f}"
+		f="${PORTAGE_CONFIGROOT}/etc/portage/${f}"
 		if [[ -d ${f} ]] ; then
 			rm -f "${f}"/cross-${CTARGET}
 			rmdir "${f}" 2>/dev/null
@@ -263,8 +265,8 @@ uninstall() {
 			sed -i -e "/cross-${CTARGET}\//d" "${f}"
 		fi
 	done
-	rm -rf /etc/portage/env/cross-${CTARGET}
-	rmdir /etc/portage/env 2>/dev/null
+	rm -rf ${PORTAGE_CONFIGROOT}/etc/portage/env/cross-${CTARGET}
+	rmdir ${PORTAGE_CONFIGROOT}/etc/portage/env 2>/dev/null

 	rmdir /var/db/pkg/cross-${CTARGET} 2>/dev/null
 	if [[ -d /var/db/pkg/cross-${CTARGET} ]] ; then
@@ -298,7 +300,7 @@ uninstall() {
 			[[ ${ans} == [Yy]* ]] && rm -rf "${d}"
 		fi
 	done
-	rm -f /etc/env.d/{binutils,gcc}/config-${CTARGET}
+	rm -f ${PORTAGE_CONFIGROOT}/etc/env.d/{binutils,gcc}/config-${CTARGET}
 	[[ -e /var/db/pkg/cross-${CTARGET} ]] && rmdir /var/db/pkg/cross-${CTARGET}

 	exit 0
@@ -445,14 +447,14 @@ fi

 # grab user settings
 for v in ABI UCLIBC_CPU USE BVER GVER KVER LVER STAGE CFLAGS LDFLAGS ASFLAGS ; do
-	if [[ -e /etc/portage/crossdev/${CTARGET}/${v} ]] ; then
+	if [[ -e ${PORTAGE_CONFIGROOT}/etc/portage/crossdev/${CTARGET}/${v} ]] ; then
 		# yes, quotes are needed in this instance (export $var="...")
-		export ${v}="$(</etc/portage/crossdev/${CTARGET}/${v})"
+		export ${v}="$(<${PORTAGE_CONFIGROOT}/etc/portage/crossdev/${CTARGET}/${v})"
 		einfo "Restoring user setting '${v}' to '${!v}'"
 	fi
-	if [[ -e /etc/portage/crossdev/${CTARGET}/env ]] ; then
+	if [[ -e ${PORTAGE_CONFIGROOT}/etc/portage/crossdev/${CTARGET}/env ]] ; then
 		einfo "Restoring generic user env settings"
-		source /etc/portage/crossdev/${CTARGET}/env
+		source ${PORTAGE_CONFIGROOT}/etc/portage/crossdev/${CTARGET}/env
 	fi
 done

@@ -600,12 +602,12 @@ set_portage() {
 	set_env ${pkg}
 }

-mkdir -p /etc/portage
-check_trailing_newline /etc/portage/categories
-grep -qs "^cross-${CTARGET}$" /etc/portage/categories \
-	|| echo cross-${CTARGET} >> /etc/portage/categories
+mkdir -p ${PORTAGE_CONFIGROOT}/etc/portage
+check_trailing_newline ${PORTAGE_CONFIGROOT}/etc/portage/categories
+grep -qs "^cross-${CTARGET}$" ${PORTAGE_CONFIGROOT}/etc/portage/categories \
+	|| echo cross-${CTARGET} >> ${PORTAGE_CONFIGROOT}/etc/portage/categories
 mkdir -p "${PORTDIR_OVERLAY}"/cross-${CTARGET}
-cd /etc/portage
+cd ${PORTAGE_CONFIGROOT}/etc/portage
 for f in package.{keywords,mask,use} ; do
 	[[ -f ${f} ]] && continue
 	mkdir -p ${f}
--
1.7.2.2

Re: [gentoo-embedded] crossdev improvements

[Joakim Tjernlund]

Mike Frysinger <vapier@gentoo.org> wrote on 2010/09/24 19:01:26:
>
> On Friday, September 24, 2010 12:01:21 Joakim Tjernlund wrote:
> > - make crossdev respect PORTAGE_CONFIGROOT. We use our own portage
> >   tree for our pkgs, including the compiler, as we want to be able
> >   to recreate/rebuild the same sw versions in the future.
>
> i dont know what this means.  crossdev doesnt touch PORTAGE_CONFIGROOT at all.
>
> > - a switch to control softfloat instead of encoding this in vendor
> >   name. The cross name is getting too long with softfloat in it.
>
> except that softfloat checks go beyond the cross-compiler and packages in the
> tree need to be able to query support.  unless you have a clean way of
> handling this, there is no alternative.
>
> > - make crossdev build a binary cross compiler pkg which can be
> >   emerged by other machines.
>
> crossdev is merely a frontend to emerge.  so do it yourself:
>    crossdev ... --portage -B ...

So I tried this(using --portage -b instead) and I don't quite get it to work.
My binpkg foo is pretty much non existing :)

So far I get a bin pkg in
/usr/portage/packages/cross/powerpc-4.4.4_softfloat-linux-gnu/cross-powerpc-4.4.4_softfloat-linux-gnu
and to get at it I must change PKGDIR to /usr/portage/packages/cross/powerpc-4.4.4_softfloat-linux-gnu/
get at the package. Does it have to be there or could it be moved to /usr/portage/packages/
so emerge finds it without messing with PKGDIR?

Next problem, how do I force other computers to use the bin pkg? Unless one
passes -G to emerge, emerge rebuilds the pkg and I don't want that
as I have passed EXTRA_ECONF args and added a small patch to glibc as well.

 Jocke

Re: [gentoo-embedded] crossdev improvements

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 1369 bytes --]

On Thursday, September 30, 2010 11:38:58 Joakim Tjernlund wrote:
> So I tried this(using --portage -b instead) and I don't quite get it to
> work. My binpkg foo is pretty much non existing :)
> 
> So far I get a bin pkg in
> /usr/portage/packages/cross/powerpc-4.4.4_softfloat-linux-gnu/cross-powerpc
> -4.4.4_softfloat-linux-gnu and to get at it I must change PKGDIR to
> /usr/portage/packages/cross/powerpc-4.4.4_softfloat-linux-gnu/ get at the
> package. Does it have to be there or could it be moved to
> /usr/portage/packages/ so emerge finds it without messing with PKGDIR?

crossdev itself forces a PKGDIR subdir.  this dates back 5.5 years at this 
point.  at the time, i was experimenting with same $CATEGORY value as the 
system and only differentiating by SLOT, but portage cant handle that: sys-
devel/gcc-4.4.4-r1:SLOT=i686-pc-linux-gnu will collide with sys-
devel/gcc-4.4.4-r1:SLOT=powerpc-unknown-linux-gnu in /var/db/pkg/.  since 
ultimately i didnt go that direction (we have the cross-xxx CATEGORY value 
now), i guess i can drop the PKGDIR munging completely from crossdev.

> Next problem, how do I force other computers to use the bin pkg? Unless one
> passes -G to emerge, emerge rebuilds the pkg and I don't want that
> as I have passed EXTRA_ECONF args and added a small patch to glibc as well.

man emerge -> -K
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-embedded] crossdev improvements

[Joakim Tjernlund]

Mike Frysinger <vapier@gentoo.org> wrote on 2010/10/01 02:36:44:
>
> On Thursday, September 30, 2010 11:38:58 Joakim Tjernlund wrote:
> > So I tried this(using --portage -b instead) and I don't quite get it to
> > work. My binpkg foo is pretty much non existing :)
> >
> > So far I get a bin pkg in
> > /usr/portage/packages/cross/powerpc-4.4.4_softfloat-linux-gnu/cross-powerpc
> > -4.4.4_softfloat-linux-gnu and to get at it I must change PKGDIR to
> > /usr/portage/packages/cross/powerpc-4.4.4_softfloat-linux-gnu/ get at the
> > package. Does it have to be there or could it be moved to
> > /usr/portage/packages/ so emerge finds it without messing with PKGDIR?
>
> crossdev itself forces a PKGDIR subdir.  this dates back 5.5 years at this
> point.  at the time, i was experimenting with same $CATEGORY value as the
> system and only differentiating by SLOT, but portage cant handle that: sys-
> devel/gcc-4.4.4-r1:SLOT=i686-pc-linux-gnu will collide with sys-
> devel/gcc-4.4.4-r1:SLOT=powerpc-unknown-linux-gnu in /var/db/pkg/.  since
> ultimately i didnt go that direction (we have the cross-xxx CATEGORY value
> now), i guess i can drop the PKGDIR munging completely from crossdev.

Great, the change looks very simple:

From 690c42cd1ba00ba9b979cf9665dba75aa76d8c98 Mon Sep 17 00:00:00 2001
From: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
Date: Fri, 1 Oct 2010 08:45:13 +0200
Subject: [PATCH] Remove PKGDIR munging

This isn't needed anymore.

Signed-off-by: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
---
 crossdev-20100814/crossdev |    1 -
 1 files changed, 0 insertions(+), 1 deletions(-)

diff --git a/crossdev-20100814/crossdev b/crossdev-20100814/crossdev
index fc9b89e..214ab7a 100755
--- a/crossdev-20100814/crossdev
+++ b/crossdev-20100814/crossdev
@@ -242,7 +242,6 @@ setup_portage_vars() {
 	PORTDIR=$(portageq envvar PORTDIR)
 	PORT_LOGDIR=$(portageq envvar PORT_LOGDIR)
 	PORT_LOGDIR=${PORT_LOGDIR:-/var/log/portage}
-	export PKGDIR=$(portageq envvar PKGDIR)/cross/${CTARGET}
 	export PORTAGE_TMPDIR=$(portageq envvar PORTAGE_TMPDIR)/cross/${CTARGET}
 	[[ ! -d ${PORT_LOGDIR} ]] && mkdir -p ${PORT_LOGDIR}
 	[[ ! -d ${PORTAGE_TMPDIR} ]] && mkdir -p ${PORTAGE_TMPDIR}
--
1.7.2.2

>
> > Next problem, how do I force other computers to use the bin pkg? Unless one
> > passes -G to emerge, emerge rebuilds the pkg and I don't want that
> > as I have passed EXTRA_ECONF args and added a small patch to glibc as well.
>
> man emerge -> -K

Thanks, but this is not quite what I want. This depends on the user actually passing
-K to emerge. I realize I want something like the different -bin ebuilds
in portage such as openoffice-bin or firefox-bin.
Any pointers on how to create a -bin ebuild from the binary pkgs I
get form crossdev ... -portage -b ?

       Jocke

Re: [gentoo-embedded] crossdev improvements

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 284 bytes --]

On Saturday, September 25, 2010 13:11:26 Joakim Tjernlund wrote:
>  setup_portage_vars() {
> +	PORTAGE_CONFIGROOT=$(portageq envvar PORTAGE_CONFIGROOT)

is this necessary ?  you cant really put PORTAGE_CONFIGROOT in /etc/make.conf 
as you hit a chicken & egg problem ...
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-embedded] crossdev improvements

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 865 bytes --]

On Friday, October 01, 2010 02:56:07 Joakim Tjernlund wrote:
> Mike Frysinger <vapier@gentoo.org> wrote on 2010/10/01 02:36:44:
> > On Thursday, September 30, 2010 11:38:58 Joakim Tjernlund wrote:
> > > Next problem, how do I force other computers to use the bin pkg? Unless
> > > one passes -G to emerge, emerge rebuilds the pkg and I don't want that
> > > as I have passed EXTRA_ECONF args and added a small patch to glibc as
> > > well.
> > 
> > man emerge -> -K
> 
> Thanks, but this is not quite what I want. This depends on the user
> actually passing -K to emerge. I realize I want something like the
> different -bin ebuilds in portage such as openoffice-bin or firefox-bin.
> Any pointers on how to create a -bin ebuild from the binary pkgs I
> get form crossdev ... -portage -b ?

portage has no support for changing PN on the fly
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-embedded] crossdev improvements

[Joakim Tjernlund]

Mike Frysinger <vapier@gentoo.org> wrote on 2010/10/08 04:54:07:
>
> On Saturday, September 25, 2010 13:11:26 Joakim Tjernlund wrote:
> >  setup_portage_vars() {
> > +   PORTAGE_CONFIGROOT=$(portageq envvar PORTAGE_CONFIGROOT)
>
> is this necessary ?  you cant really put PORTAGE_CONFIGROOT in /etc/make.conf
> as you hit a chicken & egg problem ...

Never got that far, tried it but run into some other problem related to
my private portage tree so I really don't know.

Removing PKGDIR worked OK though, I think my Remove PKGDIR patch is fine.

Re: [gentoo-embedded] crossdev improvements

[Joakim Tjernlund]

Mike Frysinger <vapier@gentoo.org> wrote on 2010/10/08 05:06:13:
>
> On Friday, October 01, 2010 02:56:07 Joakim Tjernlund wrote:
> > Mike Frysinger <vapier@gentoo.org> wrote on 2010/10/01 02:36:44:
> > > On Thursday, September 30, 2010 11:38:58 Joakim Tjernlund wrote:
> > > > Next problem, how do I force other computers to use the bin pkg? Unless
> > > > one passes -G to emerge, emerge rebuilds the pkg and I don't want that
> > > > as I have passed EXTRA_ECONF args and added a small patch to glibc as
> > > > well.
> > >
> > > man emerge -> -K
> >
> > Thanks, but this is not quite what I want. This depends on the user
> > actually passing -K to emerge. I realize I want something like the
> > different -bin ebuilds in portage such as openoffice-bin or firefox-bin.
> > Any pointers on how to create a -bin ebuild from the binary pkgs I
> > get form crossdev ... -portage -b ?
>
> portage has no support for changing PN on the fly

Yeah, I came to the same conclusion. I used qlist and tar instead:
TARGET="powerpc-e300c2-linux-gnu"
MY_ROOT="/usr/${TARGET}"
mkdir -p /tmp/${TARGET}
qlist --quiet --all ${TARGET} > /tmp/${TARGET}/cross-tools
ROOT=${MY_ROOT} qlist --quiet --all > /tmp/${TARGET}/pkgs
#prefix all paths with our target root prefix
sed -i s:/:${MY_ROOT}/: /tmp/${TARGET}/pkgs
tar czf /tmp/${TARGET}.tgz -T /tmp/${TARGET}/cross-tools \
    -T /tmp/${TARGET}/pkgs

Would be nice if one could get rid of the sed hack by adding
an option to qlist to list the true abs path.

       Jocke

Re: [gentoo-embedded] crossdev improvements

[Joakim Tjernlund]

Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote on 2010/10/08 08:37:51:
>
> Mike Frysinger <vapier@gentoo.org> wrote on 2010/10/08 05:06:13:
> >
> > On Friday, October 01, 2010 02:56:07 Joakim Tjernlund wrote:
> > > Mike Frysinger <vapier@gentoo.org> wrote on 2010/10/01 02:36:44:
> > > > On Thursday, September 30, 2010 11:38:58 Joakim Tjernlund wrote:
> > > > > Next problem, how do I force other computers to use the bin pkg? Unless
> > > > > one passes -G to emerge, emerge rebuilds the pkg and I don't want that
> > > > > as I have passed EXTRA_ECONF args and added a small patch to glibc as
> > > > > well.
> > > >
> > > > man emerge -> -K
> > >
> > > Thanks, but this is not quite what I want. This depends on the user
> > > actually passing -K to emerge. I realize I want something like the
> > > different -bin ebuilds in portage such as openoffice-bin or firefox-bin.
> > > Any pointers on how to create a -bin ebuild from the binary pkgs I
> > > get form crossdev ... -portage -b ?
> >
> > portage has no support for changing PN on the fly
>
> Yeah, I came to the same conclusion. I used qlist and tar instead:
> TARGET="powerpc-e300c2-linux-gnu"
> MY_ROOT="/usr/${TARGET}"
> mkdir -p /tmp/${TARGET}
> qlist --quiet --all ${TARGET} > /tmp/${TARGET}/cross-tools
> ROOT=${MY_ROOT} qlist --quiet --all > /tmp/${TARGET}/pkgs
> #prefix all paths with our target root prefix
> sed -i s:/:${MY_ROOT}/: /tmp/${TARGET}/pkgs
> tar czf /tmp/${TARGET}.tgz -T /tmp/${TARGET}/cross-tools \
>     -T /tmp/${TARGET}/pkgs
>
> Would be nice if one could get rid of the sed hack by adding
> an option to qlist to list the true abs path.

Perhaps a --root=${MY_ROOT} option?
Use like this:
  qlist --root=${MY_ROOT} --quiet --all
instead of
  ROOT=${MY_ROOT} qlist --quiet --all
The difference being --root lists the abs path, including ${MY_ROOT}

   Jocke

Re: [gentoo-embedded] crossdev improvements

[Joakim Tjernlund]

Mike Frysinger <vapier@gentoo.org> wrote on 2010/10/08 04:54:07:
>
> On Saturday, September 25, 2010 13:11:26 Joakim Tjernlund wrote:
> >  setup_portage_vars() {
> > +   PORTAGE_CONFIGROOT=$(portageq envvar PORTAGE_CONFIGROOT)
>
> is this necessary ?  you cant really put PORTAGE_CONFIGROOT in /etc/make.conf
> as you hit a chicken & egg problem ...

Perhaps this is better:

From 79d5bbe239f90383fa07fb17cd6fde1461bc0917 Mon Sep 17 00:00:00 2001
From: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
Date: Fri, 8 Oct 2010 10:01:30 +0200
Subject: [PATCH 2/2] Initial PORTAGE_CONFIGROOT support

This is untested

Signed-off-by: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
---
 crossdev |   30 ++++++++++++++++--------------
 1 files changed, 16 insertions(+), 14 deletions(-)

diff --git a/crossdev b/crossdev
index c35a2ed..854ae58 100755
--- a/crossdev
+++ b/crossdev
@@ -12,7 +12,9 @@ if [[ ${ROOT:-/} != "/" ]] ; then
 	exit 2
 fi

-source /etc/init.d/functions.sh || exit 1
+export PORTAGE_CONFIGROOT=${PORTAGE_CONFIGROOT:-/}
+
+source ${PORTAGE_CONFIGROOT}/etc/init.d/functions.sh || exit 1
 esyslog() { :; }
 die() {
 	echo
@@ -265,9 +267,9 @@ uninstall() {

 	[[ -d ${PORTDIR_OVERLAY}/cross-${CTARGET} ]] \
 		&& rm -r ${PORTDIR_OVERLAY}/cross-${CTARGET}
-	sed -i -e "/^cross-${CTARGET}$/d" /etc/portage/categories
+	sed -i -e "/^cross-${CTARGET}$/d" ${PORTAGE_CONFIGROOT}/etc/portage/categories
 	for f in package.{mask,keywords,use} ; do
-		f="/etc/portage/${f}"
+		f="${PORTAGE_CONFIGROOT}/etc/portage/${f}"
 		if [[ -d ${f} ]] ; then
 			rm -f "${f}"/cross-${CTARGET}
 			rmdir "${f}" 2>/dev/null
@@ -275,8 +277,8 @@ uninstall() {
 			sed -i -e "/cross-${CTARGET}\//d" "${f}"
 		fi
 	done
-	rm -rf /etc/portage/env/cross-${CTARGET}
-	rmdir /etc/portage/env 2>/dev/null
+	rm -rf ${PORTAGE_CONFIGROOT}/etc/portage/env/cross-${CTARGET}
+	rmdir ${PORTAGE_CONFIGROOT}/etc/portage/env 2>/dev/null

 	rmdir /var/db/pkg/cross-${CTARGET} 2>/dev/null
 	if [[ -d /var/db/pkg/cross-${CTARGET} ]] ; then
@@ -310,7 +312,7 @@ uninstall() {
 			[[ ${ans} == [Yy]* ]] && rm -rf "${d}"
 		fi
 	done
-	rm -f /etc/env.d/{binutils,gcc}/config-${CTARGET}
+	rm -f ${PORTAGE_CONFIGROOT}/etc/env.d/{binutils,gcc}/config-${CTARGET}
 	[[ -e /var/db/pkg/cross-${CTARGET} ]] && rmdir /var/db/pkg/cross-${CTARGET}

 	exit 0
@@ -461,14 +463,14 @@ fi

 # grab user settings
 for v in ABI UCLIBC_CPU USE BVER GVER KVER LVER STAGE CFLAGS LDFLAGS ASFLAGS ; do
-	if [[ -e /etc/portage/crossdev/${CTARGET}/${v} ]] ; then
+	if [[ -e ${PORTAGE_CONFIGROOT}/etc/portage/crossdev/${CTARGET}/${v} ]] ; then
 		# yes, quotes are needed in this instance (export $var="...")
 		export ${v}="$(</etc/portage/crossdev/${CTARGET}/${v})"
 		einfo "Restoring user setting '${v}' to '${!v}'"
 	fi
-	if [[ -e /etc/portage/crossdev/${CTARGET}/env ]] ; then
+	if [[ -e ${PORTAGE_CONFIGROOT}/etc/portage/crossdev/${CTARGET}/env ]] ; then
 		einfo "Restoring generic user env settings"
-		source /etc/portage/crossdev/${CTARGET}/env
+		source ${PORTAGE_CONFIGROOT}/etc/portage/crossdev/${CTARGET}/env
 	fi
 done

@@ -616,12 +618,12 @@ set_portage() {
 	set_env ${pkg} "${env}"
 }

-mkdir -p /etc/portage
-check_trailing_newline /etc/portage/categories
-grep -qs "^cross-${CTARGET}$" /etc/portage/categories \
-	|| echo cross-${CTARGET} >> /etc/portage/categories
+mkdir -p ${PORTAGE_CONFIGROOT}/etc/portage
+check_trailing_newline ${PORTAGE_CONFIGROOT}/etc/portage/categories
+grep -qs "^cross-${CTARGET}$" ${PORTAGE_CONFIGROOT}/etc/portage/categories \
+	|| echo cross-${CTARGET} >> ${PORTAGE_CONFIGROOT}/etc/portage/categories
 mkdir -p "${PORTDIR_OVERLAY}"/cross-${CTARGET}
-cd /etc/portage
+cd ${PORTAGE_CONFIGROOT}/etc/portage
 for f in package.{keywords,mask,use} ; do
 	[[ -f ${f} ]] && continue
 	mkdir -p ${f}
--
1.7.2.2

Re: [gentoo-embedded] crossdev improvements

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 1148 bytes --]

On Friday, October 08, 2010 03:32:55 Joakim Tjernlund wrote:
> Joakim Tjernlund wrote on 2010/10/08 08:37:51:
> > Yeah, I came to the same conclusion. I used qlist and tar instead:
> > TARGET="powerpc-e300c2-linux-gnu"
> > MY_ROOT="/usr/${TARGET}"
> > mkdir -p /tmp/${TARGET}
> > qlist --quiet --all ${TARGET} > /tmp/${TARGET}/cross-tools
> > ROOT=${MY_ROOT} qlist --quiet --all > /tmp/${TARGET}/pkgs
> > #prefix all paths with our target root prefix
> > sed -i s:/:${MY_ROOT}/: /tmp/${TARGET}/pkgs
> > tar czf /tmp/${TARGET}.tgz -T /tmp/${TARGET}/cross-tools \
> >     -T /tmp/${TARGET}/pkgs
> > 
> > Would be nice if one could get rid of the sed hack by adding
> > an option to qlist to list the true abs path.
> 
> Perhaps a --root=${MY_ROOT} option?
> Use like this:
>   qlist --root=${MY_ROOT} --quiet --all
> instead of
>   ROOT=${MY_ROOT} qlist --quiet --all
> The difference being --root lists the abs path, including ${MY_ROOT}

tar has a --transform option

i dont think the behavior you propose is intuitive.  better to have a 
dedicated option to control this behavior (if we choose to include it) ...
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-embedded] crossdev improvements

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 421 bytes --]

On Friday, October 08, 2010 04:09:17 Joakim Tjernlund wrote:
> -source /etc/init.d/functions.sh || exit 1
> +export PORTAGE_CONFIGROOT=${PORTAGE_CONFIGROOT:-/}
> +
> +source ${PORTAGE_CONFIGROOT}/etc/init.d/functions.sh || exit 1

why do you need to source this from the configroot ?  this seems to violate 
the intended purpose of the configroot variable ...

the rest looks straight forward enough ...
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-embedded] crossdev improvements

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 3211 bytes --]

On Friday, September 24, 2010 12:01:21 Joakim Tjernlund wrote:
> - make crossdev respect PORTAGE_CONFIGROOT. We use our own portage
>   tree for our pkgs, including the compiler, as we want to be able
>   to recreate/rebuild the same sw versions in the future.

try this:
diff --git a/crossdev b/crossdev
index c35a2ed..a782d19 100755
--- a/crossdev
+++ b/crossdev
@@ -12,6 +12,8 @@ if [[ ${ROOT:-/} != "/" ]] ; then
 	exit 2
 fi
 
+CONFIGROOT="${PORTAGE_CONFIGROOT}/etc/portage"
+
 source /etc/init.d/functions.sh || exit 1
 esyslog() { :; }
 die() {
@@ -265,9 +267,9 @@ uninstall() {
 
 	[[ -d ${PORTDIR_OVERLAY}/cross-${CTARGET} ]] \
 		&& rm -r ${PORTDIR_OVERLAY}/cross-${CTARGET}
-	sed -i -e "/^cross-${CTARGET}$/d" /etc/portage/categories
+	sed -i -e "/^cross-${CTARGET}$/d" "${CONFIGROOT}"/categories
 	for f in package.{mask,keywords,use} ; do
-		f="/etc/portage/${f}"
+		f="${CONFIGROOT}/${f}"
 		if [[ -d ${f} ]] ; then
 			rm -f "${f}"/cross-${CTARGET}
 			rmdir "${f}" 2>/dev/null
@@ -275,8 +277,8 @@ uninstall() {
 			sed -i -e "/cross-${CTARGET}\//d" "${f}"
 		fi
 	done
-	rm -rf /etc/portage/env/cross-${CTARGET}
-	rmdir /etc/portage/env 2>/dev/null
+	rm -rf "${CONFIGROOT}"/env/cross-${CTARGET}
+	rmdir "${CONFIGROOT}"/env 2>/dev/null
 
 	rmdir /var/db/pkg/cross-${CTARGET} 2>/dev/null
 	if [[ -d /var/db/pkg/cross-${CTARGET} ]] ; then
@@ -461,14 +463,15 @@ fi
 
 # grab user settings
 for v in ABI UCLIBC_CPU USE BVER GVER KVER LVER STAGE CFLAGS LDFLAGS ASFLAGS 
; do
-	if [[ -e /etc/portage/crossdev/${CTARGET}/${v} ]] ; then
+	d="${CONFIGROOT}/crossdev/${CTARGET}"
+	if [[ -e ${d}/${v} ]] ; then
 		# yes, quotes are needed in this instance (export $var="...")
-		export ${v}="$(</etc/portage/crossdev/${CTARGET}/${v})"
+		export ${v}="$(<"${d}"/${v})"
 		einfo "Restoring user setting '${v}' to '${!v}'"
 	fi
-	if [[ -e /etc/portage/crossdev/${CTARGET}/env ]] ; then
+	if [[ -e ${d}/env ]] ; then
 		einfo "Restoring generic user env settings"
-		source /etc/portage/crossdev/${CTARGET}/env
+		source "${d}"/env
 	fi
 done
 
@@ -509,6 +512,7 @@ einfo "Extra: insight:        DO IT"
 echo
 einfo "PORTDIR_OVERLAY:       ${PORTDIR_OVERLAY}"
 einfo "PORT_LOGDIR:           ${PORT_LOGDIR}"
+einfo "PORTAGE_CONFIGROOT:    ${PORTAGE_CONFIGROOT}"
 hr
 ) >& "${PORT_LOGDIR}"/cross-${CTARGET}-info.log || exit 1
 cat "${PORT_LOGDIR}"/cross-${CTARGET}-info.log
@@ -616,15 +620,15 @@ set_portage() {
 	set_env ${pkg} "${env}"
 }
 
-mkdir -p /etc/portage
-check_trailing_newline /etc/portage/categories
-grep -qs "^cross-${CTARGET}$" /etc/portage/categories \
-	|| echo cross-${CTARGET} >> /etc/portage/categories
-mkdir -p "${PORTDIR_OVERLAY}"/cross-${CTARGET}
-cd /etc/portage
+mkdir -p "${CONFIGROOT}"
+check_trailing_newline "${CONFIGROOT}"/categories
+grep -qs "^cross-${CTARGET}$" "${CONFIGROOT}"/categories \
+	|| echo cross-${CTARGET} >> "${CONFIGROOT}"/categories
+mkdir -p "${PORTDIR_OVERLAY}"/cross-${CTARGET} || exit 1
+cd "${CONFIGROOT}"
 for f in package.{keywords,mask,use} ; do
 	[[ -f ${f} ]] && continue
-	mkdir -p ${f}
+	mkdir -p ${f} || exit 1
 	rm -f ${f}/cross-${CTARGET}
 done
 
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-embedded] crossdev improvements

[Joakim Tjernlund]

Mike Frysinger <vapier@gentoo.org> wrote on 2010/10/08 10:23:24:
>
> On Friday, October 08, 2010 04:09:17 Joakim Tjernlund wrote:
> > -source /etc/init.d/functions.sh || exit 1
> > +export PORTAGE_CONFIGROOT=${PORTAGE_CONFIGROOT:-/}
> > +
> > +source ${PORTAGE_CONFIGROOT}/etc/init.d/functions.sh || exit 1
>
> why do you need to source this from the configroot ?  this seems to violate
> the intended purpose of the configroot variable ...

It felt right, but I don't really know. Remove it.

     Jocke

Re: [gentoo-embedded] crossdev improvements

[Joakim Tjernlund]

Mike Frysinger <vapier@gentoo.org> wrote on 2010/10/08 10:20:22:
>
> On Friday, October 08, 2010 03:32:55 Joakim Tjernlund wrote:
> > Joakim Tjernlund wrote on 2010/10/08 08:37:51:
> > > Yeah, I came to the same conclusion. I used qlist and tar instead:
> > > TARGET="powerpc-e300c2-linux-gnu"
> > > MY_ROOT="/usr/${TARGET}"
> > > mkdir -p /tmp/${TARGET}
> > > qlist --quiet --all ${TARGET} > /tmp/${TARGET}/cross-tools
> > > ROOT=${MY_ROOT} qlist --quiet --all > /tmp/${TARGET}/pkgs
> > > #prefix all paths with our target root prefix
> > > sed -i s:/:${MY_ROOT}/: /tmp/${TARGET}/pkgs
> > > tar czf /tmp/${TARGET}.tgz -T /tmp/${TARGET}/cross-tools \
> > >     -T /tmp/${TARGET}/pkgs
> > >
> > > Would be nice if one could get rid of the sed hack by adding
> > > an option to qlist to list the true abs path.
> >
> > Perhaps a --root=${MY_ROOT} option?
> > Use like this:
> >   qlist --root=${MY_ROOT} --quiet --all
> > instead of
> >   ROOT=${MY_ROOT} qlist --quiet --all
> > The difference being --root lists the abs path, including ${MY_ROOT}
>
> tar has a --transform option

Ah, tried it and it doesn't seem to work for me.
It appears tar applies the transform after reading the files
from the file system :(

>
> i dont think the behavior you propose is intuitive.  better to have a
> dedicated option to control this behavior (if we choose to include it) ...

OK, perhaps --abspath will do?

       jocke

Re: [gentoo-embedded] crossdev improvements

[Joakim Tjernlund]

Mike Frysinger <vapier@gentoo.org> wrote on 2010/10/08 11:01:57:
>
> On Friday, September 24, 2010 12:01:21 Joakim Tjernlund wrote:
> > - make crossdev respect PORTAGE_CONFIGROOT. We use our own portage
> >   tree for our pkgs, including the compiler, as we want to be able
> >   to recreate/rebuild the same sw versions in the future.
>
> try this:

My mail client trashed it, can you sent it as an attachment?

Re: [gentoo-embedded] crossdev improvements

[Joakim Tjernlund]

Mike Frysinger <vapier@gentoo.org> wrote on 2010/10/08 11:01:57:
>
> On Friday, September 24, 2010 12:01:21 Joakim Tjernlund wrote:
> > - make crossdev respect PORTAGE_CONFIGROOT. We use our own portage
> >   tree for our pkgs, including the compiler, as we want to be able
> >   to recreate/rebuild the same sw versions in the future.
>

Forgot to mention, I get this all the time now:

 * crossdev version:      @CDEVPV@
 * Host Portage ARCH:     x86
 * Target Portage ARCH:   ppc
 * Target System:         powerpc-native_softfloat-linux-gnu
 * Stage:                 3 (C compiler & libc)

 * binutils:              binutils-2.20.1-r1
 * gcc:                   gcc-4.4.4-r2
 * headers:               linux-headers-2.6.30-r1
 * libc:                  glibc-2.11.2

 * PORTDIR_OVERLAY:       /usr/local/portage/transmode
 * PORT_LOGDIR:           /var/log/portage
  _  -  ~  -  _  -  ~  -  _  -  ~  -  _  -  ~  -  _  -  ~  -  _  -  ~  -  _  -
/usr/portage/eclass/toolchain-funcs.eclass: line 283: KV_to_int: command not found
/usr/portage/eclass/toolchain-funcs.eclass: line 283: KV_to_int: command not found
/usr/portage/eclass/toolchain-funcs.eclass: line 285: KV_to_int: command not found
/usr/portage/eclass/toolchain-funcs.eclass: line 285: KV_to_int: command not found
 * Forcing the latest versions of {binutils,gcc}-config/gnuconfig ...

Something appears broken in latest crossdev.

   Jocke

Re: [gentoo-embedded] crossdev improvements

[Joakim Tjernlund]

Joakim Tjernlund/Transmode wrote on 2010/10/08 11:45:12:
>
> Mike Frysinger <vapier@gentoo.org> wrote on 2010/10/08 11:01:57:
> >
> > On Friday, September 24, 2010 12:01:21 Joakim Tjernlund wrote:
> > > - make crossdev respect PORTAGE_CONFIGROOT. We use our own portage
> > >   tree for our pkgs, including the compiler, as we want to be able
> > >   to recreate/rebuild the same sw versions in the future.
> >
> > try this:

> My mail client trashed it, can you sent it as an attachment?

I hand applied it and crossdev still works fine. I don't have a PORTAGE_CONFIGROOT
handy but at least it works as it used to.

      Jocke

Re: [gentoo-embedded] crossdev improvements

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 284 bytes --]

On Friday, October 08, 2010 06:14:48 Joakim Tjernlund wrote:
>  - /usr/portage/eclass/toolchain-funcs.eclass: line 283: KV_to_int:
> command not found
> 
> Something appears broken in latest crossdev.

as noted in your other thread, this has nothing to do with crossdev
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-embedded] crossdev improvements

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 767 bytes --]

On Friday, October 08, 2010 08:40:13 Joakim Tjernlund wrote:
> Joakim Tjernlund/Transmode wrote on 2010/10/08 11:45:12:
> > Mike Frysinger <vapier@gentoo.org> wrote on 2010/10/08 11:01:57:
> > > On Friday, September 24, 2010 12:01:21 Joakim Tjernlund wrote:
> > > > - make crossdev respect PORTAGE_CONFIGROOT. We use our own portage
> > > > 
> > > >   tree for our pkgs, including the compiler, as we want to be able
> > > >   to recreate/rebuild the same sw versions in the future.
> > > 
> > > try this:
> >
> > My mail client trashed it, can you sent it as an attachment?
> 
> I hand applied it and crossdev still works fine. I don't have a
> PORTAGE_CONFIGROOT handy but at least it works as it used to.

thanks.  ive applied & pushed it.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-embedded] crossdev improvements

[Joakim Tjernlund]

Mike Frysinger <vapier@gentoo.org> wrote on 2010/10/10 00:15:28:
>
> On Friday, October 08, 2010 06:14:48 Joakim Tjernlund wrote:
> >  - /usr/portage/eclass/toolchain-funcs.eclass: line 283: KV_to_int:
> > command not found
> >
> > Something appears broken in latest crossdev.
>
> as noted in your other thread, this has nothing to do with crossdev

OK, I didn't quite understand that. Then this is a pure portage problem
I guess

 Jocke

Re: [gentoo-embedded] crossdev improvements

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 811 bytes --]

On Sunday, October 10, 2010 03:30:52 Joakim Tjernlund wrote:
> Mike Frysinger <vapier@gentoo.org> wrote on 2010/10/10 00:15:28:
> > On Friday, October 08, 2010 06:14:48 Joakim Tjernlund wrote:
> > >  - /usr/portage/eclass/toolchain-funcs.eclass: line 283: KV_to_int:
> > > command not found
> > > 
> > > Something appears broken in latest crossdev.
> > 
> > as noted in your other thread, this has nothing to do with crossdev
> 
> OK, I didn't quite understand that. Then this is a pure portage problem
> I guess

sorry, i was wrong.  it's an issue between emerge-wrapper sourcing the 
eclasses to use tc-arch.  but only for powerpc targets as its code in 
toolchain-funcs.eclass is overly complicated.

i'll clean up the code in the eclass and it should fix these harmless 
warnings.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]