[gentoo-dev] Should ${T} be defined in pkg_prepare ?

[gentoo-dev] Should ${T} be defined in pkg_prepare ?

[Maciej Grela]

Hi,

recently, I've tried to compile libreoffice using paludis and I've
noticed the following problem:

8< -------------------------------------------------
kraken ~ # cave resolve libreoffice
Done: 3905 steps              

These are the actions I will take, in order:

r   app-office/libreoffice:0::gentoo 3.5.2.2 to ::installed replacing
3.5.2.2
    (-aqua) -binfilter branding cups dbus eds gnome graphite gstreamer
gtk -java jemalloc -kde -mysql -nlpsolver nsplugin -odk opengl pdfimport
-postgres svg vba webdav xmlsec (-test) build_options: symbols=split
-optional_tests -trace -preserve_work
    Reasons: target, !=app-office/libreoffice-3.4.9999-r1 from
app-office/libreoffice-l10n, !=app-office/libreoffice-9999-r1 from
app-office/libreoffice-l10n, 2 more

Total: 1 reinstalls

Executing pretend actions: 1 of 1
libreoffice-3.5.2.2>  * Checking for at least 512 mebibytes RAM ...     
[ ok ]
libreoffice-3.5.2.2> DISK CHECK: path="", size="6G"
libreoffice-3.5.2.2>  * Checking for at least 6 gibibytes disk space at 
[ !! ]
libreoffice-3.5.2.2>
libreoffice-3.5.2.2>  * Couldn't determine disk space, skipping...
8< -------------------------------------------------

the check-reqs is not getting the "${T}" value properly when it's being
run from pkg_prepare. I've added an echo to the eclass to confirm this.
The ebuild detects free space properly when run under emerge:

8< -------------------------------------------------
kraken ~ # emerge --nodeps -p -v libreoffice

These are the packages that would be merged, in order:

[ebuild   R    ] app-office/libreoffice-3.5.2.2::gentoo
[3.5.2.2::__unknown__] USE="branding cups dbus graphite gtk nsplugin
opengl svg vba webdav xmlsec (-aqua) -binfilter -eds* -gnome*
-gstreamer* -java -jemalloc* -kde -mysql -nlpsolver -odk -pdfimport*
-postgres -test" 0 kB

Total: 1 package (1 reinstall), Size of downloads: 0 kB

 * IMPORTANT: 2 news items need reading for repository 'gentoo'.
 * Use eselect news to read news items.

kraken ~ # emerge --nodeps -v libreoffice

These are the packages that would be merged, in order:

[ebuild   R    ] app-office/libreoffice-3.5.2.2::gentoo
[3.5.2.2::__unknown__] USE="branding cups dbus graphite gtk nsplugin
opengl svg vba webdav xmlsec (-aqua) -binfilter -eds* -gnome*
-gstreamer* -java -jemalloc* -kde -mysql -nlpsolver -odk -pdfimport*
-postgres -test" 0 kB

Total: 1 package (1 reinstall), Size of downloads: 0 kB


>>> Verifying ebuild manifests
>>> Running pre-merge checks for app-office/libreoffice-3.5.2.2
 * Checking for at least 512 mebibytes RAM
...                                                                                                               
[ ok ]
DISK CHECK: path="/var/tmp/portage/app-office/libreoffice-3.5.2.2/temp",
size="6G"
 * Checking for at least 6 gibibytes disk space at
"/var/tmp/portage/app-office/libreoffice-3.5.2.2/temp"
...                                                 [ ok ]

>>> Emerging (1 of 1) app-office/libreoffice-3.5.2.2
^C

kraken ~ #
8< -------------------------------------------------


I've read the PMS and I haven't found information whether this variable
is supposed to be set during pkg_prepare or not. Therefore I ask, what
is the proper behaviour here ? Is there documentation on what special
env variables are supposed to be defined in each stage ?
Can this be considered as a bug in paludis ?

Thanks for your help.

Best regards,
Maciej Grela

Re: [gentoo-dev] Should ${T} be defined in pkg_prepare ?

[Ulrich Mueller]

>>>>> On Sat, 31 Mar 2012, Maciej Grela wrote:

> I've read the PMS and I haven't found information whether this variable
> is supposed to be set during pkg_prepare or not.

There is no such stage. You mean pkg_pretend, I suppose?

> Therefore I ask, what is the proper behaviour here ? Is there
> documentation on what special env variables are supposed to be
> defined in each stage ?

It's specified here:
<http://dev.gentoo.org/~ulm/pms/4/pms.html#x1-11900012.1>

| Variable   Legal in   Consistent?    Description
| ---------------------------------------------------------------------
| T          All        Partially⁴     The full path to a temporary
|                                      directory for use by the ebuild. 
|
| ⁴Consistent and preserved across a single connected sequence of
| install or uninstall phases, but not between install and uninstall.
| When reinstalling a package, this variable must have different
| values for the install and the replacement.

> Can this be considered as a bug in paludis ?

The spec seems to be clear that T is legal in all phases, including
pkg_pretend.

Ulrich

Re: [gentoo-dev] Should ${T} be defined in pkg_prepare ?

[Tiziano Müller]

[-- Attachment #1: Type: text/plain, Size: 1708 bytes --]

Am Samstag, den 31.03.2012, 14:44 +0200 schrieb Ulrich Mueller:
> >>>>> On Sat, 31 Mar 2012, Maciej Grela wrote:
> 
> > I've read the PMS and I haven't found information whether this variable
> > is supposed to be set during pkg_prepare or not.
> 
> There is no such stage. You mean pkg_pretend, I suppose?
> 
> > Therefore I ask, what is the proper behaviour here ? Is there
> > documentation on what special env variables are supposed to be
> > defined in each stage ?
> 
> It's specified here:
> <http://dev.gentoo.org/~ulm/pms/4/pms.html#x1-11900012.1>
> 
> | Variable   Legal in   Consistent?    Description
> | ---------------------------------------------------------------------
> | T          All        Partially⁴     The full path to a temporary
> |                                      directory for use by the ebuild. 
> |
> | ⁴Consistent and preserved across a single connected sequence of
> | install or uninstall phases, but not between install and uninstall.
> | When reinstalling a package, this variable must have different
> | values for the install and the replacement.
> 
> > Can this be considered as a bug in paludis ?
> 
> The spec seems to be clear that T is legal in all phases, including
> pkg_pretend.

Well, I'd say: there is no sane value you can assign to $T since you are
not allowed to write anything anyway:

"pkg_pretend must not write to the
filesystem." (http://dev.gentoo.org/~ulm/pms/4/pms.html#x1-9700010.1.2)

and since "pkg_pretend is run separately from the main phase function
sequence, and does not participate in any kind of environment saving" it
is not guaranteed to be set to the same $T later.

Cheers,
Tiziano

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 3551 bytes --]

Re: [gentoo-dev] Should ${T} be defined in pkg_prepare ?

[Ulrich Mueller]

>>>>> On Sat, 31 Mar 2012, Tiziano Müller wrote:

>> The spec seems to be clear that T is legal in all phases, including
>> pkg_pretend.

> Well, I'd say: there is no sane value you can assign to $T since you
> are not allowed to write anything anyway:

> "pkg_pretend must not write to the filesystem."
> (http://dev.gentoo.org/~ulm/pms/4/pms.html#x1-9700010.1.2)

That's not necessarily a contradiction. Writing being not allowed
doesn't imply that the directory must not exist.

> and since "pkg_pretend is run separately from the main phase
> function sequence, and does not participate in any kind of
> environment saving" it is not guaranteed to be set to the same $T
> later.

The problem is that apart from T (and maybe HOME), there seems to be
no other directory that check_reqs.eclass could use for its disk space
check in pkg_pretend. WORKDIR doesn't exist in pkg_* phases.

Ulrich

Re: [gentoo-dev] Should ${T} be defined in pkg_prepare ?

[Francesco Riosa]

2012/3/31 Tiziano Müller <dev-zero@gentoo.org>:
> Am Samstag, den 31.03.2012, 14:44 +0200 schrieb Ulrich Mueller:
>> >>>>> On Sat, 31 Mar 2012, Maciej Grela wrote:
>>
>> > I've read the PMS and I haven't found information whether this variable
>> > is supposed to be set during pkg_prepare or not.
>>
>> There is no such stage. You mean pkg_pretend, I suppose?
>>
>> > Therefore I ask, what is the proper behaviour here ? Is there
>> > documentation on what special env variables are supposed to be
>> > defined in each stage ?
>>
>> It's specified here:
>> <http://dev.gentoo.org/~ulm/pms/4/pms.html#x1-11900012.1>
>>
>> | Variable   Legal in   Consistent?    Description
>> | ---------------------------------------------------------------------
>> | T          All        Partially⁴     The full path to a temporary
>> |                                      directory for use by the ebuild.
>> |
>> | ⁴Consistent and preserved across a single connected sequence of
>> | install or uninstall phases, but not between install and uninstall.
>> | When reinstalling a package, this variable must have different
>> | values for the install and the replacement.
>>
>> > Can this be considered as a bug in paludis ?
>>
>> The spec seems to be clear that T is legal in all phases, including
>> pkg_pretend.
>
> Well, I'd say: there is no sane value you can assign to $T since you are
> not allowed to write anything anyway:

indeed it's not writing, check-reqs.eclass is reading space available
via 'df -Pm' there is something else it could use assuming
$PORTAGE_TMPDIR is worse for portability?

>
> "pkg_pretend must not write to the
> filesystem." (http://dev.gentoo.org/~ulm/pms/4/pms.html#x1-9700010.1.2)
>
> and since "pkg_pretend is run separately from the main phase function
> sequence, and does not participate in any kind of environment saving" it
> is not guaranteed to be set to the same $T later.
>
> Cheers,
> Tiziano

Re: [gentoo-dev] Should ${T} be defined in pkg_prepare ?

[Zac Medico]

On 03/31/2012 01:56 PM, Ulrich Mueller wrote:
>>>>>> On Sat, 31 Mar 2012, Tiziano Müller wrote:
>> and since "pkg_pretend is run separately from the main phase
>> function sequence, and does not participate in any kind of
>> environment saving" it is not guaranteed to be set to the same $T
>> later.
> 
> The problem is that apart from T (and maybe HOME), there seems to be
> no other directory that check_reqs.eclass could use for its disk space
> check in pkg_pretend. WORKDIR doesn't exist in pkg_* phases.

How about PWD?
-- 
Thanks,
Zac

Re: [gentoo-dev] Should ${T} be defined in pkg_prepare ?

[Ulrich Mueller]

>>>>> On Sat, 31 Mar 2012, Zac Medico wrote:

>> The problem is that apart from T (and maybe HOME), there seems to
>> be no other directory that check_reqs.eclass could use for its disk
>> space check in pkg_pretend. WORKDIR doesn't exist in pkg_* phases.

> How about PWD?

No sure. Do all package managers set it to a parent dir of WORKDIR
in pkg_pretend?

At least PMS doesn't say so:

| 10.1.1 Initial Working Directories
| [...] it may be set to anything and the ebuild must not rely upon
| a particular location for it.

Ulrich

Re: [gentoo-dev] Should ${T} be defined in pkg_prepare ?

[Zac Medico]

On 04/01/2012 01:29 AM, Ulrich Mueller wrote:
>>>>>> On Sat, 31 Mar 2012, Zac Medico wrote:
> 
>>> The problem is that apart from T (and maybe HOME), there seems to
>>> be no other directory that check_reqs.eclass could use for its disk
>>> space check in pkg_pretend. WORKDIR doesn't exist in pkg_* phases.
> 
>> How about PWD?
> 
> No sure. Do all package managers set it to a parent dir of WORKDIR
> in pkg_pretend?

Since bug 388593 [1], portage uses the normal PORTAGE_TMPDIR for
pkg_pretend. Before that is was using the default temp dir from python's
tempfile module (which defaults to /tmp).

> At least PMS doesn't say so:
> 
> | 10.1.1 Initial Working Directories
> | [...] it may be set to anything and the ebuild must not rely upon
> | a particular location for it.

Maybe something for EAPI 5?

[1] https://bugs.gentoo.org/show_bug.cgi?id=388593
-- 
Thanks,
Zac

Re: [gentoo-dev] Should ${T} be defined in pkg_prepare ?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 728 bytes --]

On Sat, 31 Mar 2012 14:10:50 +0200
Maciej Grela <maciej.grela@gmail.com> wrote:
> I've read the PMS and I haven't found information whether this
> variable is supposed to be set during pkg_prepare or not. Therefore I
> ask, what is the proper behaviour here ? Is there documentation on
> what special env variables are supposed to be defined in each stage ?
> Can this be considered as a bug in paludis ?

Mmm. The spec implies that it should be set to something sane... There's
no guarantee, of course, that the amount of space in ${T} in pkg_pretend
is anything like the amount of space that will be available for the
build, but check-reqs is deliberately designed to be wildly inaccurate.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Should ${T} be defined in pkg_prepare ?

[Maciej Grela]

On 01.04.2012 20:55, Ciaran McCreesh wrote:
> On Sat, 31 Mar 2012 14:10:50 +0200
> Maciej Grela <maciej.grela@gmail.com> wrote:
>> I've read the PMS and I haven't found information whether this
>> variable is supposed to be set during pkg_prepare or not. Therefore I
>> ask, what is the proper behaviour here ? Is there documentation on
>> what special env variables are supposed to be defined in each stage ?
>> Can this be considered as a bug in paludis ?
> Mmm. The spec implies that it should be set to something sane... There's
> no guarantee, of course, that the amount of space in ${T} in pkg_pretend
> is anything like the amount of space that will be available for the
> build, but check-reqs is deliberately designed to be wildly inaccurate.
>

Should I create a bug for this ?

Best regards,
-- 
Maciej Grela

Re: [gentoo-dev] Should ${T} be defined in pkg_prepare ?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1036 bytes --]

On Sun, 1 Apr 2012 19:55:44 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Sat, 31 Mar 2012 14:10:50 +0200
> Maciej Grela <maciej.grela@gmail.com> wrote:
> > I've read the PMS and I haven't found information whether this
> > variable is supposed to be set during pkg_prepare or not. Therefore
> > I ask, what is the proper behaviour here ? Is there documentation on
> > what special env variables are supposed to be defined in each
> > stage ? Can this be considered as a bug in paludis ?
> 
> Mmm. The spec implies that it should be set to something sane...
> There's no guarantee, of course, that the amount of space in ${T} in
> pkg_pretend is anything like the amount of space that will be
> available for the build, but check-reqs is deliberately designed to
> be wildly inaccurate.

Yes because being wildly accurate in this case is such a great
possibility here. Why don't you delight us with your vision of wildly
accurate space prediction, mister?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Should ${T} be defined in pkg_prepare ?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1081 bytes --]

On Sun, 1 Apr 2012 22:47:23 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> > There's no guarantee, of course, that the amount of space in ${T} in
> > pkg_pretend is anything like the amount of space that will be
> > available for the build, but check-reqs is deliberately designed to
> > be wildly inaccurate.
> 
> Yes because being wildly accurate in this case is such a great
> possibility here. Why don't you delight us with your vision of wildly
> accurate space prediction, mister?

You should probably look at who wrote check-reqs in the first place,
and why it was designed the way it was... You'll note in particular how
the code just has a best attempt at finding a solution, and how it
deliberately just gives a warning message, not a fatal error, if
either it can't figure out whether a requirement is met or if it
thinks that one isn't. You know, as if it were designed under the
assumption that accuracy wouldn't be possible...

Or you could just post more noise to the list in an attempt to stir up
trouble. Your call.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]