public inbox for gentoo-dev@lists.gentoo.org
 help / color / mirror / Atom feed
From: Ryan Hill <dirtyepic@gentoo.org>
To: gentoo-dev@lists.gentoo.org
Subject: [gentoo-dev] Re: FEATURES=test, sys-devel/gcc ignored test failures
Date: Tue, 22 Mar 2011 17:25:40 -0600	[thread overview]
Message-ID: <20110322172540.1ccbd71f@gentoo.org> (raw)
In-Reply-To: 4D88918A.8050606@gentoo.org

[-- Attachment #1: Type: text/plain, Size: 2706 bytes --]

On Tue, 22 Mar 2011 13:09:46 +0100
"Paweł Hajdan, Jr." <phajdan.jr@gentoo.org> wrote:

> On 3/21/11 11:02 PM, Ryan Hill wrote:
> > It does to me, I use them all the time. ;)  The important part is that we
> > install the test results, which can then be used for regression testing when
> > rolling patchsets.
> 
> I see, it makes sense. I guess you're comparing the test results before
> and after rolling patchsets and look for regressions.
> 
> > I think that glibc and gcc tests and other testsuites that nearly always
> > fail shouldn't be run for the average user but should still be easily
> > accessible in a standard way.  I think we need a more finely grained test
> > setup, where we can say tests are "expensive" or "interesting only to
> > developers" or "known to fail", and let people opt-in to these on a
> > per-package basis. Right now you always have to opt-out using
> > package.use.mask which "works" but is unintuitive.
> 
> My main point is that the developer profile has FEATURES=test, and also
> arch testers and developers run with FEATURES=test. Being able to
> quickly rebuild gcc, glibc and others is a win.

Yes, I'm agreeing with you.  I'd like these off by default too.  We need a
standard way of enabling them however.  USE="test-dev" or something.  I
complain about this about once a year or so. ;)  Maybe I should just do it.

In the meantime:

echo -e 'sys-libs/glibc test\nsys-devel/gcc test' \
  >> /etc/portage/profile/package.use.mask
 
> I'm trying to understand the problem better - do you know what causes
> those test failures? I don't expect a "complete" answer because that'd
> probably be a half of actually fixing the failures.

The GCC testsuite isn't designed to pass.  It's designed to be a
regression test.  Check before and after you apply a patch, make sure you
don't cock things up.  From http://gcc.gnu.org/install/test.html: "It is
normal for some tests to report unexpected failures. At the current time the
testing harness does not allow fine grained control over whether or not a test
is expected to fail."  Look at http://gcc.gnu.org/buildstat.html and you'll
see this in practice.

One thing I know of that causes a bunch of failures is the fact that we
enable -Wformat, -Wformat-security, and -Wtrampolines by default.  Any
additional output during a test = fail.  I patched these recently for 4.5
though so they shouldn't be a problem going forward.


-- 
fonts, gcc-porting,                  it makes no sense how it makes no sense
toolchain, wxwidgets                           but i'll take it free anytime
@ gentoo.org                EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

      parent reply	other threads:[~2011-03-22 23:20 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-03-21 12:07 [gentoo-dev] FEATURES=test, sys-devel/gcc ignored test failures "Paweł Hajdan, Jr."
2011-03-21 12:29 ` Amadeusz Żołnowski
2011-03-21 22:02 ` [gentoo-dev] " Ryan Hill
2011-03-22 12:09   ` "Paweł Hajdan, Jr."
2011-03-22 15:01     ` Matt Turner
2011-03-22 23:25     ` Ryan Hill [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20110322172540.1ccbd71f@gentoo.org \
    --to=dirtyepic@gentoo.org \
    --cc=gentoo-dev@lists.gentoo.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox