[gentoo-dev] fricas[doc] now fails to emerge

[gentoo-dev] fricas[doc] now fails to emerge

[Andrey Grozin]

Hello *,

Until recently, fricas[doc] used to emerge fine. Today, when trying to do 
so, I get zillion

F: open_wr
S: deny
P: /dev/dri/card0
A: /dev/dri/card0
R: /dev/dri/card0
C: Xvfb -displayfd 1 -screen 0 1280x1024x24 +extension RANDR

and the line

virtx emake book

in the ebuild fails. I don't know what has changed. Any ideas?

Andrey

Re: [gentoo-dev] fricas[doc] now fails to emerge

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 857 bytes --]

On Sat, Aug 17, 2024 at 02:46:53PM +0000, Andrey Grozin wrote:
> Hello *,
> 
> Until recently, fricas[doc] used to emerge fine. Today, when trying to do 
> so, I get zillion
> 
> F: open_wr
> S: deny
> P: /dev/dri/card0
> A: /dev/dri/card0
> R: /dev/dri/card0
> C: Xvfb -displayfd 1 -screen 0 1280x1024x24 +extension RANDR
> 
> and the line
> 
> virtx emake book
> 
> in the ebuild fails. I don't know what has changed. Any ideas?
Maybe your Xvfb behavior?

/dev/dri/card* errors used to be a lot more common, so may be useful to
include "addpredict /dev/dri/" in virtualx.eclass's virtx function.

-- 
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation President & Treasurer
E-Mail   : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1113 bytes --]

Re: [gentoo-dev] fricas[doc] now fails to emerge

[negril.nx+gentoo]

Check the logfile at ${T}/Xvfb.log

Re: [gentoo-dev] fricas[doc] now fails to emerge

[Andrey Grozin]

On Sat, 17 Aug 2024, Paul Zander wrote:
> Check the logfile at ${T}/Xvfb.log
It says

_XSERVTransSocketUNIXCreateListener: ...SocketCreateListener() failed

and then zillion times

libEGL warning: failed to open /dev/dri/card0: Permission denied

Recently I've upgraded mesa to 24.2.0. Can this problem be related to this 
upgrade?

The day before yesterday, I've bumped clozurecl to 1.13. Naturally, I 
decided to check that it can compile maxima and fricas, and that they 
work. fricas emerged fine, with USE=doc. Then I started to emerge -auvND 
@world, in particular, it was going to rebuild maxima and fricas again, 
with sbcl (I normally use sbcl, not clozurecl). And energing fricas has 
failed. The same day, maybe, an hour later. In between a number of 
packaged were upgraded. The most suspicious one is mesa - it may be 
somehow related to the behavior of Xvfb.

Can anybody with the current mesa try to emerge fricas[doc] and tell us if 
it works (any lisp will do, probably, sbcl is the most reasonable one; by 
the way, clozurecl compiles fricas very quickly).

Andrey

Re: [gentoo-dev] fricas[doc] now fails to emerge

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1056 bytes --]

On Sun, Aug 18, 2024 at 04:20:05PM +0000, Andrey Grozin wrote:
> On Sat, 17 Aug 2024, Paul Zander wrote:
> > Check the logfile at ${T}/Xvfb.log
> Recently I've upgraded mesa to 24.2.0. Can this problem be related to this 
> upgrade?
...
> Can anybody with the current mesa try to emerge fricas[doc] and tell us if 
> it works (any lisp will do, probably, sbcl is the most reasonable one; by 
> the way, clozurecl compiles fricas very quickly).
Yes; I reproduced it.
- started w/ mesa-24.0.7 installed.
- emerge sbcl => success
- USE=doc emerge fricas => success
- emerge =mesa-24.2.0* => success
- USE=doc emerge fricas => fail w/ sandbox
- patch virtualx.eclass
- USE=doc emerge fricas => success

So Mesa's behavior changed, trying to accelerate Xvfb.

I pushed a fix to virtualx.eclass for you.

-- 
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation President & Treasurer
E-Mail   : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1113 bytes --]

Re: [gentoo-dev] fricas[doc] now fails to emerge

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 1139 bytes --]

>>>>> On Mon, 19 Aug 2024, Robin H Johnson wrote:

> On Sun, Aug 18, 2024 at 04:20:05PM +0000, Andrey Grozin wrote:
>> Can anybody with the current mesa try to emerge fricas[doc] and tell us if 
>> it works (any lisp will do, probably, sbcl is the most reasonable one; by 
>> the way, clozurecl compiles fricas very quickly).
> Yes; I reproduced it.
> - started w/ mesa-24.0.7 installed.
> - emerge sbcl => success
> - USE=doc emerge fricas => success
> - emerge =mesa-24.2.0* => success
> - USE=doc emerge fricas => fail w/ sandbox
> - patch virtualx.eclass
> - USE=doc emerge fricas => success

> So Mesa's behavior changed, trying to accelerate Xvfb.

> I pushed a fix to virtualx.eclass for you.

That addpredict looks like a workaround, not like a real fix of the
problem. Especially, the many warnings mentioned by grozin are still
there. With the patched virtualx.eclass, I still see more than thousand
messages in Xvfb.log:

   libEGL warning: failed to open /dev/dri/card0: Permission denied

Also, was this so urgent that you had to push the eclass change without
prior mailing list review?

Ulrich

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 507 bytes --]

Re: [gentoo-dev] fricas[doc] now fails to emerge

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 2101 bytes --]

On Mon, Aug 19, 2024 at 09:49:28PM +0200, Ulrich Mueller wrote:
> > I pushed a fix to virtualx.eclass for you.
> That addpredict looks like a workaround, not like a real fix of the
> problem.
It's a fix in that it correctly tells Sandbox that upstream mesas use a
predictive fopen(..., RDWR) and Gentoo expects it to *NOT* actually use the
device in the ebuild context.

> Especially, the many warnings mentioned by grozin are still
> there. With the patched virtualx.eclass, I still see more than thousand
> messages in Xvfb.log:
>    libEGL warning: failed to open /dev/dri/card0: Permission denied
The manual does correctly build despite that warning from mesa, because it
correctly falls back.

> Also, was this so urgent that you had to push the eclass change without
> prior mailing list review?
Low impact, fixes blockage.

Reminder to all, if you go looking for bugs, you'll find more than you expected.

Git bisect of mesa points to the relevant upstream commits that introduced the flaws.

1. The init behavior changed here: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30426/diffs
   This was added between mesa-24.2.0-rc3 and mesa-24.2.0-rc4
   It always probes those /dev/ files now.

2. I *also* found a similar failure upstream between 24.0.9 and 24.1.0; with USE=llvm specifically:
   https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27805/diffs#81e929dbdc766bd46257096f260549cbdeba18fc_1133_1161

sandbox snippet for USE=llvm:
F: open_wr
S: deny
P: /dev/udmabuf
A: /dev/udmabuf
R: /dev/udmabuf
C: Xvfb -displayfd 1 -screen 0 1280x1024x24 +extension RANDR 

"addpredict /dev/udmabuf" should also enable other cases to be stricter:
https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=48b4885e4f9a19ccc4c1489a387e38fb3b7d62b7

and re-enable tests here:
https://bugs.gentoo.org/933257

-- 
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation President & Treasurer
E-Mail   : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1113 bytes --]

Re: [gentoo-dev] fricas[doc] now fails to emerge

[Paul Zander]

[-- Attachment #1: Type: text/plain, Size: 3301 bytes --]

Am 20.08.24 um 16:52 schrieb Robin H. Johnson:
> On Mon, Aug 19, 2024 at 09:49:28PM +0200, Ulrich Mueller wrote:
>>> I pushed a fix to virtualx.eclass for you.
>> That addpredict looks like a workaround, not like a real fix of the
>> problem.
> It's a fix in that it correctly tells Sandbox that upstream mesas use a
> predictive fopen(..., RDWR) and Gentoo expects it to *NOT* actually use the
> device in the ebuild context.

It doesn't fix the underlying problem of Xvfb accessing /dev/dri/*. It 
works around it by breaking Xvfb via the sandbox.

>> Especially, the many warnings mentioned by grozin are still
>> there. With the patched virtualx.eclass, I still see more than thousand
>> messages in Xvfb.log:
>>     libEGL warning: failed to open /dev/dri/card0: Permission denied
> The manual does correctly build despite that warning from mesa, because it
> correctly falls back.
This is the hallmark of a workaround.
>> Also, was this so urgent that you had to push the eclass change without
>> prior mailing list review?
> Low impact, fixes blockage.

The impact of changing an eclass willy nilly without asking anyone for 
@X11 or anyone affected?

We've had mesa-24.2.0* in the tree for a while now. We had plenty of 
tinderbox runs testing. This is the only package that fails with those 
sandbox violations. No other @sci stuff, no @kde stuff.

The "Low impact, fixes blockage." fix would have been to add that 
addpredict line to the affected package, file a bug and ask the people 
that actually know about mesa.

> Reminder to all, if you go looking for bugs, you'll find more than you expected.
>
> Git bisect of mesa points to the relevant upstream commits that introduced the flaws.
>
> 1. The init behavior changed here:https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30426/diffs
>     This was added between mesa-24.2.0-rc3 and mesa-24.2.0-rc4
>     It always probes those /dev/ files now.
>
> 2. I *also* found a similar failure upstream between 24.0.9 and 24.1.0; with USE=llvm specifically:
>     https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27805/diffs#81e929dbdc766bd46257096f260549cbdeba18fc_1133_1161
>
> sandbox snippet for USE=llvm:
> F: open_wr
> S: deny
> P: /dev/udmabuf
> A: /dev/udmabuf
> R: /dev/udmabuf
> C: Xvfb -displayfd 1 -screen 0 1280x1024x24 +extension RANDR
>
> "addpredict /dev/udmabuf" should also enable other cases to be stricter:
> https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=48b4885e4f9a19ccc4c1489a387e38fb3b7d62b7
>
> and re-enable tests here:
> https://bugs.gentoo.org/933257

Oddly enough fricas also has an automagic dependency on xvfb-run...

FYI: this actually fixes the access for fricas:

diff --git a/eclass/virtualx.eclass b/eclass/virtualx.eclass
index f7318eafc59..21995b9dcc7 100644
--- a/eclass/virtualx.eclass
+++ b/eclass/virtualx.eclass
@@ -107,13 +107,12 @@ virtx() {
  
  	local i=0
  	local retval=0
-	local xvfbargs=( -screen 0 1280x1024x24 +extension RANDR )
+	local xvfbargs=( -screen 0 1280x1024x24 +extension RANDR -extension GLX )
  
  	debug-print "${FUNCNAME}: running Xvfb hack"
  	export XAUTHORITY=
  
  	einfo "Starting Xvfb ..."
-	addpredict /dev/dri/ # Needed for Xvfb w/ >=mesa-24.2.0
  
  	debug-print "${FUNCNAME}: Xvfb -displayfd 1 ${xvfbargs[*]}"
  	local logfile=${T}/Xvfb.log

[-- Attachment #2: Type: text/html, Size: 5678 bytes --]

Re: [gentoo-dev] fricas[doc] now fails to emerge

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 3754 bytes --]

On Tue, Aug 20, 2024 at 06:06:17PM +0200, Paul Zander wrote:
> >> Especially, the many warnings mentioned by grozin are still
> >> there. With the patched virtualx.eclass, I still see more than thousand
> >> messages in Xvfb.log:
> >>     libEGL warning: failed to open /dev/dri/card0: Permission denied
> > The manual does correctly build despite that warning from mesa, because it
> > correctly falls back.
> This is the hallmark of a workaround.
Hiding the errors also isn't a fix - it's just picking a different code path.

virtualx has detection automagic in what Mesa drivers are actually loading; and
that certainly contributes to the problems.

> >> Also, was this so urgent that you had to push the eclass change without
> >> prior mailing list review?
> > Low impact, fixes blockage.
> The impact of changing an eclass willy nilly without asking anyone for 
> @X11 or anyone affected?
Who was affected by the breakage? Grozin.
The tree has a sprawl of addpredict/addwrite /dev/dri/ entries; because this
isn't handled consistently at the Xvfb/mesa level.

> We've had mesa-24.2.0* in the tree for a while now. We had plenty of 
> tinderbox runs testing. This is the only package that fails with those 
> sandbox violations. No other @sci stuff, no @kde stuff.
See below about libeproxy - it's failing because of the same files.

> The "Low impact, fixes blockage." fix would have been to add that 
> addpredict line to the affected package, file a bug and ask the people 
> that actually know about mesa.
If I didn't know about mesa, why do you think I commented on this thread, and
provided bisect showing where upstream introduced the change in behavior?

> Oddly enough fricas also has an automagic dependency on xvfb-run...
@grozin: please do fix the automagic behavior in fricas; it's clearly not fatal
either, because my test environment does not have xvfb-run installed at all.

> FYI: this actually fixes the access for fricas:
Your patch would "fix" fricas, but I think it will break anything else
that explicitly relies on GLX inside Xvfb; and would need a tinderbox
run to verify what that is. There is an even worse potential outcome in your
patch: tests might skip some cases because they probe for the GLX extension and
skip those cases.

It would not surprise me if there are other packages already broken, and
not detected in the prior testing.

As a fast example, media-libs/libepoxy, fails already because it wants
GLX in src_test. It was broken before per bug #823786, and still broken
with my patch, and yours outright disables what it wants to test.  I think it's
mucking with sandbox in another way, because even with both config explicitly
permitting writes, it still shows Xvfb errors to them:

libepoxy-1.5.10-r3.ebuild:
==
multilib_src_test() {
    export MESA_LOADER_DRIVER_OVERRIDE=llvmpipe
    export LIBGL_KOPPER_DISABLE=true
    export LIBGL_KOPPER_DRI2=true
    export GALLIUM_DRIVER=llvmpipe
    export LIBGL_ALWAYS_SOFTWARE=true
    addwrite /dev/dri/card0
    addwrite /dev/udmabuf
    addwrite /dev/dri/renderD128
    virtx meson_src_test
}
==


$ head /var/tmp/portage-tmpfs/portage/media-libs/libepoxy-1.5.10-r3/temp/Xvfb.log 
_XSERVTransSocketUNIXCreateListener: ...SocketCreateListener() failed
_XSERVTransMakeAllCOTSServerListeners: server already running
libEGL warning: failed to open /dev/dri/card0: Permission denied
...

And yet running the same test outside of the ebuild works fine.

-- 
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation President & Treasurer
E-Mail   : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1113 bytes --]