[gentoo-embedded] stages for x86 uclibc

[gentoo-embedded] stages for x86 uclibc

[Jude Pereira]

[-- Attachment #1: Type: text/plain, Size: 350 bytes --]

I've been able to create stages for x86 under the uclibc profile using
catalyst.
I'd like that they get upstream to the gentoo mirrors, so that its available
publicly.

I'm also willing to maintain them, I'd love to be a gentoo uclibc maintainer
for the x86 arch.

Let me know.

--Jude Pereira
(http://judepereira.com <http://judepereira.com/blog>/)

[-- Attachment #2: Type: text/html, Size: 483 bytes --]

Re: [gentoo-embedded] stages for x86 uclibc

[Ed W]

On 29/08/2011 15:23, Jude Pereira wrote:
> I've been able to create stages for x86 under the uclibc profile using
> catalyst.
> I'd like that they get upstream to the gentoo mirrors, so that its
> available publicly.
>
> I'm also willing to maintain them, I'd love to be a gentoo uclibc
> maintainer for the x86 arch.

Excellent idea. 

I am using latest uclibc (slightly newer git release than .32).  It is
so much better and easier to use than what is in portage that we
desperately need to dump the current portage uclibc for all archs that
support nptl + uclibc latest!  I don't think I have any local patches
now to build under this uclibc version...!

Additionally I have had some success enabling iconv on latest uclibc. 
This needs some tweaking to the uclibc src if you are building under a
uclibc toolchain (it only builds correctly under a glibc toolchain, but
easy to workaround). Patches are on the uclibc mailing list and I can
highlight them if needed. It adds around 50KB to my build size, but also
it allows git and a bunch of other ebuilds to build successfully without
patching - some people may like this trade off - I suspect that the
build size could be reduced with some work

I have asked if someone would proxy maintain this - I don't have a whole
bunch of time, but I would love to see portage get pulled up to date if
someone had the time to give it a push? My vote would be for .32 to
become stable (subject to normal arch signoffs) and a regular git
snapshot to be tracked and keyword masked

I think uclibc stages will mainly be useful to highlight that gentoo is
a good build environment for embedded and to provide a working toolchain
for cross compiling.  Love to see this become productionised

Cheers

Ed W

Re: [gentoo-embedded] stages for x86 uclibc

[Jude Pereira]

[-- Attachment #1: Type: text/plain, Size: 2146 bytes --]

I do have the time to give the uclibc profile in portage a good kick. I'm
very much into this as I'm working with uclibc toolchains under the x86
mainly.

I'll make an ebuild for the new one, and test it out.


--Jude Pereira
(http://judepereira.com/blog)


On Wed, Aug 31, 2011 at 4:27 PM, Ed W <lists@wildgooses.com> wrote:

> On 29/08/2011 15:23, Jude Pereira wrote:
> > I've been able to create stages for x86 under the uclibc profile using
> > catalyst.
> > I'd like that they get upstream to the gentoo mirrors, so that its
> > available publicly.
> >
> > I'm also willing to maintain them, I'd love to be a gentoo uclibc
> > maintainer for the x86 arch.
>
> Excellent idea.
>
> I am using latest uclibc (slightly newer git release than .32).  It is
> so much better and easier to use than what is in portage that we
> desperately need to dump the current portage uclibc for all archs that
> support nptl + uclibc latest!  I don't think I have any local patches
> now to build under this uclibc version...!
>
> Additionally I have had some success enabling iconv on latest uclibc.
> This needs some tweaking to the uclibc src if you are building under a
> uclibc toolchain (it only builds correctly under a glibc toolchain, but
> easy to workaround). Patches are on the uclibc mailing list and I can
> highlight them if needed. It adds around 50KB to my build size, but also
> it allows git and a bunch of other ebuilds to build successfully without
> patching - some people may like this trade off - I suspect that the
> build size could be reduced with some work
>
> I have asked if someone would proxy maintain this - I don't have a whole
> bunch of time, but I would love to see portage get pulled up to date if
> someone had the time to give it a push? My vote would be for .32 to
> become stable (subject to normal arch signoffs) and a regular git
> snapshot to be tracked and keyword masked
>
> I think uclibc stages will mainly be useful to highlight that gentoo is
> a good build environment for embedded and to provide a working toolchain
> for cross compiling.  Love to see this become productionised
>
> Cheers
>
> Ed W
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 2759 bytes --]

Re: [gentoo-embedded] stages for x86 uclibc

[Ed W]

[-- Attachment #1: Type: text/plain, Size: 2026 bytes --]

On 31/08/2011 13:16, Jude Pereira wrote:
> I do have the time to give the uclibc profile in portage a good kick.
> I'm very much into this as I'm working with uclibc toolchains under
> the x86 mainly.
>
> I'll make an ebuild for the new one, and test it out.

I didn't need to do much more than trim out all the old patches and bump
the ebuild.  Just to give a heads up, here are my main changes for
iconv/locales:


--- /usr/portage/sys-libs/uclibc/uclibc-0.9.30.1-r1.ebuild     
2011-04-20 19:31:26.000000000 +0100
+++ /usr/local/portage/sys-libs/uclibc/uclibc-0.9.33_pre20110617.ebuild
2011-06-23 19:43:24.000000000 +0100
@@ -170,6 +173,10 @@
                epatch "${WORKDIR}"/patch
        fi
 
+       epatch "${FILESDIR}"/${PN}-${PV}-clean.patch
+       epatch "${FILESDIR}"/gen_wc8bit.patch
+       epatch "${FILESDIR}"/gen_wctype.patch
+
        sed -i 's:getline:get_line:' extra/scripts/unifdef.c #277186
 
        ########## CPU SELECTION ##########
@@ -305,6 +312,14 @@
        cp .config myconfig
 
        emake -s clean > /dev/null || die "could not clean"
+
+       # TODO: These should depend on some useflag, eg iconv
+       # Run after make clean, otherwise files removed
+       find ./extra/locale/charmaps -name "*.pairs" >
extra/locale/codesets.txt
+       #cp ./extra/locale/LOCALES ./extra/locale/locales.txt
+       cp "${FILESDIR}"/locales.txt ./extra/locale/locales.txt
+       # TODO: Now edit locales as appropriate...
+       # FIXME: ...
 }


Also, not sure I remember why, but I have this change... 

-IUSE="build uclibc-compat debug hardened ssp ipv6 minimal wordexp
crosscompile_opts_headers-only"
+IUSE="build elibc_uclibc uclibc-compat debug hardened ssp ipv6 minimal
wordexp crosscompile_opts_headers-only"



The missing patch files are attached (note the _clean_ is not needed on
recent git).  Note the gen_wc patches should be credited/copyrighted to
the chap whos name I forget on the uclibc list who posted them about Feb
this year?

Bit of a mess, but hopefully helpful?

Ed W



[-- Attachment #2.1: Type: application/applefile, Size: 438 bytes --]

[-- Attachment #2.2: gen_wc8bit.patch --]
[-- Type: application/octet-stream, Size: 1192 bytes --]

--- a/extra/locale/gen_wc8bit.c
+++ b/extra/locale/gen_wc8bit.c
@@ -99,31 +99,29 @@
 	int total_size = 0;
 
 	if (!setlocale(LC_CTYPE, "en_US.UTF-8")) {
-		/* Silly foreigners disabling en_US locales */
-		FILE *fp = popen("locale -a", "r");
-		if (!fp)
-			goto locale_failure;
-
-		while (!feof(fp)) {
+		/* setlocale might be just a stub */
+		fp = popen("locale -a", "r");
+		if (!fp) {
+			printf("#error could not execute 'locale -a'\n");
+                        return EXIT_FAILURE;
+		}
+		
+	/*	while (!feof(fp)) {
 			char buf[256];
 			size_t len;
 
-			if (fgets(buf, sizeof(buf) - 10, fp) == NULL)
-				goto locale_failure;
+			if (fgets(buf, sizeof(buf) - 10, fp) == NULL) {
+				printf("#error could not read 'locale -a' output buffer\n");
+				return EXIT_FAILURE;
+			}
 
 			len = strlen(buf);
 			if (len > 0 && buf[len - 1] == '\n')
 				buf[--len] = '\0';
 			if (len < 5 || strcasecmp(&buf[len-5], ".UTF8") != 0)
 				strcat(buf, ".UTF8");
-			if (setlocale(LC_CTYPE, buf))
-				goto locale_success;
 		}
-
- locale_failure:
-		printf("could not find a UTF8 locale ... please enable en_US.UTF-8\n");
-		return EXIT_FAILURE;
- locale_success:
+	*/	
 		pclose(fp);
 	}
 

[-- Attachment #3.1: Type: application/applefile, Size: 459 bytes --]

[-- Attachment #3.2: uclibc-0.9.33_pre20110617-clean.patch --]
[-- Type: application/octet-stream, Size: 347 bytes --]

--- libubacktrace/Makefile.in.orig	2011-06-22 23:37:18.000000000 +0100
+++ libubacktrace/Makefile.in	2011-06-22 23:37:37.000000000 +0100
@@ -72,4 +72,4 @@
 
 CLEAN_libubacktrace:
 	$(do_rm) $(addprefix $(libubacktrace_OUT)/*., o os oS a) \
-	 $(addprefix $(libubacktrace_ARCH_OUT)/*., o os oS a)
+	 $(addprefix $(libubacktrace_OUT)/*., o os oS a)

[-- Attachment #4: locales.txt --]
[-- Type: text/plain, Size: 712 bytes --]

# First some @modifier mappings.  Internally, the modifier is signaled by
# replacing '_' in the locale name with a unique identifying character.
# For example, internally we map "ca_ES@euro" to "caeES".  This allows for
# smaller code and easier processing of locale names.

@euro e
@cyrillic c

#---------------------------------------------------------------------------
# Next, set to {y}es to enable and {n}o to disable the UTF-8 and the 8-bit
# codeset locales.  Of course, you must have built the c8tables.h and
# the wctables.h files appropriately.


UTF-8 yes
8-BIT yes

#---------------------------------------------------------------------------
# Now the locales

en_GB ISO-8859-1
en_GB.UTF-8 UTF-8


[-- Attachment #5.1: Type: application/applefile, Size: 438 bytes --]

[-- Attachment #5.2: gen_wctype.patch --]
[-- Type: application/octet-stream, Size: 6681 bytes --]

--- a/extra/locale/gen_wctype.c
+++ b/extra/locale/gen_wctype.c
@@ -227,11 +227,12 @@
 			++verbose;
 			continue;
 		}
-		if (!setlocale(LC_CTYPE, *argv)) {
+	/* setlocale might be just a stub */
+	/*	if (!setlocale(LC_CTYPE, *argv)) {
 			verbose_msg("setlocale(LC_CTYPE,%s) failed!  Skipping this locale...\n", *argv);
 			continue;
 		}
-
+	*/
 		if (!(totitle = wctrans("totitle"))) {
 			verbose_msg("no totitle transformation.\n");
 		}
@@ -306,43 +307,43 @@
 #endif
 #if 0
 				if (c < 256) {
-					unsigned int glibc;
+					unsigned int curr_stdclib;
 
-					glibc = 0;
-					if (isalnum(c)) ++glibc; glibc <<= 1;
-					if (isalpha(c)) ++glibc; glibc <<= 1;
-					if (isblank(c)) ++glibc; glibc <<= 1;
-					if (iscntrl(c)) ++glibc; glibc <<= 1;
-					if (isdigit(c)) ++glibc; glibc <<= 1;
-					if (isgraph(c)) ++glibc; glibc <<= 1;
-					if (islower(c)) ++glibc; glibc <<= 1;
-					if (isprint(c)) ++glibc; glibc <<= 1;
-					if (ispunct(c)) ++glibc; glibc <<= 1;
-					if (isspace(c)) ++glibc; glibc <<= 1;
-					if (isupper(c)) ++glibc; glibc <<= 1;
-					if (isxdigit(c)) ++glibc;
-					verbose_msg("%#8x : ctype %#4x\n", c, glibc);
+					curr_stdclib = 0;
+					if (isalnum(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (isalpha(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (isblank(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (iscntrl(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (isdigit(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (isgraph(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (islower(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (isprint(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (ispunct(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (isspace(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (isupper(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (isxdigit(c)) ++curr_stdclib;
+					verbose_msg("%#8x : ctype %#4x\n", c, curr_stdclib);
 				}
 #endif
 #if 1
 				/* Paranoid checking... */
 				{
-					unsigned int glibc;
+					unsigned int curr_stdclib;
 					unsigned int mine;
 
-					glibc = 0;
-					if (iswalnum(c)) ++glibc; glibc <<= 1;
-					if (iswalpha(c)) ++glibc; glibc <<= 1;
-					if (iswblank(c)) ++glibc; glibc <<= 1;
-					if (iswcntrl(c)) ++glibc; glibc <<= 1;
-					if (iswdigit(c)) ++glibc; glibc <<= 1;
-					if (iswgraph(c)) ++glibc; glibc <<= 1;
-					if (iswlower(c)) ++glibc; glibc <<= 1;
-					if (iswprint(c)) ++glibc; glibc <<= 1;
-					if (iswpunct(c)) ++glibc; glibc <<= 1;
-					if (iswspace(c)) ++glibc; glibc <<= 1;
-					if (iswupper(c)) ++glibc; glibc <<= 1;
-					if (iswxdigit(c)) ++glibc;
+					curr_stdclib = 0;
+					if (iswalnum(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (iswalpha(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (iswblank(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (iswcntrl(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (iswdigit(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (iswgraph(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (iswlower(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (iswprint(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (iswpunct(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (iswspace(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (iswupper(c)) ++curr_stdclib; curr_stdclib <<= 1;
+					if (iswxdigit(c)) ++curr_stdclib;
 
 					mine = 0;
 					if (mywalnum(d,c)) ++mine; mine <<= 1;
@@ -358,15 +359,15 @@
 					if (mywupper(d,c)) ++mine; mine <<= 1;
 					if (mywxdigit(d,c)) ++mine;
 
-					if (glibc != mine) {
-						verbose_msg("%#8x : glibc %#4x != %#4x mine  %u\n", c, glibc, mine, d);
+					if (curr_stdclib != mine) {
+						verbose_msg("%#8x : curr_stdclib %#4x != %#4x mine  %u\n", c, curr_stdclib, mine, d);
 						return EXIT_FAILURE;
 					}
 #if 0
 					if (iswctype(c,is_comb) || iswctype(c,is_comb3)) {
 /*						if (!iswpunct(c)) { */
 							verbose_msg("%#8x : %d %d %#4x\n",
-								   c, iswctype(c,is_comb),iswctype(c,is_comb3), glibc);
+								   c, iswctype(c,is_comb),iswctype(c,is_comb3), curr_stdclib);
 /*						} */
 					}
 #endif
@@ -553,7 +554,7 @@
 		for (c = 0; c <= 0x10ffffUL; c++)
 #endif
 		{
-			unsigned int glibc;
+			unsigned int curr_stdclib;
 			unsigned int mine;
 			unsigned int upper, lower;
 
@@ -568,19 +569,19 @@
 			}
 #endif
 #endif
-			glibc = 0;
-			if (iswalnum(c)) ++glibc; glibc <<= 1;
-			if (iswalpha(c)) ++glibc; glibc <<= 1;
-			if (iswblank(c)) ++glibc; glibc <<= 1;
-			if (iswcntrl(c)) ++glibc; glibc <<= 1;
-			if (iswdigit(c)) ++glibc; glibc <<= 1;
-			if (iswgraph(c)) ++glibc; glibc <<= 1;
-			if (iswlower(c)) ++glibc; glibc <<= 1;
-			if (iswprint(c)) ++glibc; glibc <<= 1;
-			if (iswpunct(c)) ++glibc; glibc <<= 1;
-			if (iswspace(c)) ++glibc; glibc <<= 1;
-			if (iswupper(c)) ++glibc; glibc <<= 1;
-			if (iswxdigit(c)) ++glibc;
+			curr_stdclib = 0;
+			if (iswalnum(c)) ++curr_stdclib; curr_stdclib <<= 1;
+			if (iswalpha(c)) ++curr_stdclib; curr_stdclib <<= 1;
+			if (iswblank(c)) ++curr_stdclib; curr_stdclib <<= 1;
+			if (iswcntrl(c)) ++curr_stdclib; curr_stdclib <<= 1;
+			if (iswdigit(c)) ++curr_stdclib; curr_stdclib <<= 1;
+			if (iswgraph(c)) ++curr_stdclib; curr_stdclib <<= 1;
+			if (iswlower(c)) ++curr_stdclib; curr_stdclib <<= 1;
+			if (iswprint(c)) ++curr_stdclib; curr_stdclib <<= 1;
+			if (iswpunct(c)) ++curr_stdclib; curr_stdclib <<= 1;
+			if (iswspace(c)) ++curr_stdclib; curr_stdclib <<= 1;
+			if (iswupper(c)) ++curr_stdclib; curr_stdclib <<= 1;
+			if (iswxdigit(c)) ++curr_stdclib;
 
 			{
 				unsigned int u;
@@ -630,8 +631,8 @@
 				if (mywupper(d,c)) ++mine; mine <<= 1;
 				if (mywxdigit(d,c)) ++mine;
 
-				if (glibc != mine) {
-					verbose_msg("%#8x : glibc %#4x != %#4x mine %d\n", c, glibc, mine, d);
+				if (curr_stdclib != mine) {
+					verbose_msg("%#8x : curr_stdclib %#4x != %#4x mine %d\n", c, curr_stdclib, mine, d);
 					if (c < 0x30000UL) {
 						verbose_msg("sc=%#x u=%#x n=%#x i0=%#x i1=%#x\n", sc, u, n, i0, i1);
 					}
@@ -655,17 +656,17 @@
 				}
 
 				if (towupper(c) != upper) {
-					verbose_msg("%#8x : towupper glibc %#4x != %#4x mine\n",
+					verbose_msg("%#8x : towupper curr_stdclib %#4x != %#4x mine\n",
 						   c, towupper(c), upper);
 				}
 
 				if (towlower(c) != lower) {
-					verbose_msg("%#8x : towlower glibc %#4x != %#4x mine   i0 = %d\n",
+					verbose_msg("%#8x : towlower curr_stdclib %#4x != %#4x mine   i0 = %d\n",
 						   c, towlower(c), lower, i0);
 				}
 
 				if (totitle && ((tt = towctrans(c, totitle)) != upper)) {
-					verbose_msg("%#8x : totitle glibc %#4lx != %#4x mine   i0 = %d\n",
+					verbose_msg("%#8x : totitle curr_stdclib %#4lx != %#4x mine   i0 = %d\n",
 						   c, tt, upper, i0);
 				}
 			}

Re: [gentoo-embedded] stages for x86 uclibc

[Jude Pereira]

[-- Attachment #1: Type: text/plain, Size: 2495 bytes --]

Hey Ed,
I bumped the ebuild to the snapshot of 20110831 last evening, I've
successfully compiled it, but then python goes crazy.

It says that it cant find libutil.so.0, but it is actually in the correct
lib dir. I'm going to look into this even more now.


--Jude Pereira
(http://judepereira.com/blog)


On Wed, Aug 31, 2011 at 8:49 PM, Ed W <lists@wildgooses.com> wrote:

> On 31/08/2011 13:16, Jude Pereira wrote:
> > I do have the time to give the uclibc profile in portage a good kick.
> > I'm very much into this as I'm working with uclibc toolchains under
> > the x86 mainly.
> >
> > I'll make an ebuild for the new one, and test it out.
>
> I didn't need to do much more than trim out all the old patches and bump
> the ebuild.  Just to give a heads up, here are my main changes for
> iconv/locales:
>
>
> --- /usr/portage/sys-libs/uclibc/uclibc-0.9.30.1-r1.ebuild
> 2011-04-20 19:31:26.000000000 +0100
> +++ /usr/local/portage/sys-libs/uclibc/uclibc-0.9.33_pre20110617.ebuild
> 2011-06-23 19:43:24.000000000 +0100
> @@ -170,6 +173,10 @@
>                epatch "${WORKDIR}"/patch
>        fi
>
> +       epatch "${FILESDIR}"/${PN}-${PV}-clean.patch
> +       epatch "${FILESDIR}"/gen_wc8bit.patch
> +       epatch "${FILESDIR}"/gen_wctype.patch
> +
>        sed -i 's:getline:get_line:' extra/scripts/unifdef.c #277186
>
>        ########## CPU SELECTION ##########
> @@ -305,6 +312,14 @@
>        cp .config myconfig
>
>        emake -s clean > /dev/null || die "could not clean"
> +
> +       # TODO: These should depend on some useflag, eg iconv
> +       # Run after make clean, otherwise files removed
> +       find ./extra/locale/charmaps -name "*.pairs" >
> extra/locale/codesets.txt
> +       #cp ./extra/locale/LOCALES ./extra/locale/locales.txt
> +       cp "${FILESDIR}"/locales.txt ./extra/locale/locales.txt
> +       # TODO: Now edit locales as appropriate...
> +       # FIXME: ...
>  }
>
>
> Also, not sure I remember why, but I have this change...
>
> -IUSE="build uclibc-compat debug hardened ssp ipv6 minimal wordexp
> crosscompile_opts_headers-only"
> +IUSE="build elibc_uclibc uclibc-compat debug hardened ssp ipv6 minimal
> wordexp crosscompile_opts_headers-only"
>
>
>
> The missing patch files are attached (note the _clean_ is not needed on
> recent git).  Note the gen_wc patches should be credited/copyrighted to
> the chap whos name I forget on the uclibc list who posted them about Feb
> this year?
>
> Bit of a mess, but hopefully helpful?
>
> Ed W
>
>
>

[-- Attachment #2: Type: text/html, Size: 3490 bytes --]

Re: [gentoo-embedded] stages for x86 uclibc

[Ed W]

On 01/09/2011 04:27, Jude Pereira wrote:
> Hey Ed,
> I bumped the ebuild to the snapshot of 20110831 last evening, I've
> successfully compiled it, but then python goes crazy.
>
> It says that it cant find libutil.so.0, but it is actually in the correct
> lib dir. I'm going to look into this even more now.
>

Hi, uclibc can break if you don't do a binary compatible upgrade.
Because it's so configurable, it's much easier to break it if you
accidently change something in the config which has big implications. 
For example I think it proved very difficult to switch uclibc from non
hardened to hardened because it breaks before you can re-install?

Additionally, I forget the details now, but I think this bug hit some
people before - not sure if it's properly resolved in current uclibc either?

    https://bugs.busybox.net/show_bug.cgi?id=1543

It's not something I recall running into though, so possibly a config
change has caused / will rectify things?

Ed W