[gentoo-python] distutils-r1 -- support for out-of-source builds?

[gentoo-python] distutils-r1 -- support for out-of-source builds?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1259 bytes --]

Hello,

The original distutils.eclass uses out-of-source builds by default.
distutils-r1 (following the scheme of p-d-ng) just copies the sources
and does in-source builds unconditionally. I'm wondering about changing
that.

The change would mean that:

1) out-of-source (build-dir) builds are used by default,

2) in-source builds (and copying sources) is enabled through setting
DISTUTILS_IN_SOURCE_BUILD *or* whenever python_prepare() is declared
(since that function supposedly is used to modify sources),

3) a wrapper function for setup.py is introduced (esetuppy? edistsetup?)
which calls setup.py with correct build-dir,

4) ebuilds defining python_test() (and no python_prepare()) would need
to be updated to use the above-mentioned wrapper.

I'm wondering whether it's a good idea to change that. The main benefit
is that:

a) we avoid unnecessary copy in most cases,

b) we increase compatibility with distutils.eclass; e.g. the following
would start working:

  src_prepare() {
    distutils-r1_src_prepare
    sed ...
  }

which currently fails silently since sed is performed after copying
sources.

On the other hand, we would introduce a tiny bit of additional code.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-python] distutils-r1 -- support for out-of-source builds?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1678 bytes --]

On Sun, 28 Oct 2012 08:57:05 +0100
Michał Górny <mgorny@gentoo.org> wrote:

> On the other hand, we would introduce a tiny bit of additional code.

I did a bit research on the topic, and the code introduced is circa 100
lines, including docs and esetuppy() which would be probably added
anyway. There are other potential problems however.

The main issue is how distutils is designed. The only command accepting
build-dir is the 'build' command. Other commands which need to access
the build tree just use some weird method of getting parameters for
other commands. Effectively, if you need to provide a build-dir, you
always have to run 'build'.

So, the install command would grow from:

  ./setup.py install ...

to:

  ./setup.py build -b ... install ...

Of course, all other commands accessing the build-dir would need to
be prepended with the 'build -b ...' as well. For that reason,
the patches I'm going to attach add that thing to esetuppy() directly.

This has the following implications:

1) all calls to esetuppy() involve running build,

2) if build takes additional arguments, they may need to be passed
consistently to all esetuppy calls (like: 'build --foo' in args).
Otherwise, a rebuild with different options may happen (but that's
probably a case already anyway).

In any case, I wouldn't say it's very elegant of Python. It will work
flawlessly for the most of Python packages, and likely most
of the remaining ones would require running in-source builds anyway
(and that can be enabled with a single variable).

Well, best just look at the code and tell me what you think.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-python] distutils-r1 -- support for out-of-source builds?

[Mike Gilbert]

On Sun, Oct 28, 2012 at 11:55 AM, Michał Górny <mgorny@gentoo.org> wrote:
> On Sun, 28 Oct 2012 08:57:05 +0100
> Michał Górny <mgorny@gentoo.org> wrote:
>
>> On the other hand, we would introduce a tiny bit of additional code.
>
> I did a bit research on the topic, and the code introduced is circa 100
> lines, including docs and esetuppy() which would be probably added
> anyway. There are other potential problems however.
>
> The main issue is how distutils is designed. The only command accepting
> build-dir is the 'build' command. Other commands which need to access
> the build tree just use some weird method of getting parameters for
> other commands. Effectively, if you need to provide a build-dir, you
> always have to run 'build'.
>
> So, the install command would grow from:
>
>   ./setup.py install ...
>
> to:
>
>   ./setup.py build -b ... install ...
>
> Of course, all other commands accessing the build-dir would need to
> be prepended with the 'build -b ...' as well. For that reason,
> the patches I'm going to attach add that thing to esetuppy() directly.
>
> This has the following implications:
>
> 1) all calls to esetuppy() involve running build,
>
> 2) if build takes additional arguments, they may need to be passed
> consistently to all esetuppy calls (like: 'build --foo' in args).
> Otherwise, a rebuild with different options may happen (but that's
> probably a case already anyway).
>
> In any case, I wouldn't say it's very elegant of Python. It will work
> flawlessly for the most of Python packages, and likely most
> of the remaining ones would require running in-source builds anyway
> (and that can be enabled with a single variable).
>
> Well, best just look at the code and tell me what you think.
>

I don't really see any need for out-of-source builds, but if you want
to write the code for it I certainly don't mind having it there.

I will say that doing out-of-source builds occasionally makes writing
src_test a bit tricky; you have to play with the working directory and
PYTHONPATH.

Re: [gentoo-python] distutils-r1 -- support for out-of-source builds?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2774 bytes --]

On Sun, 28 Oct 2012 12:04:04 -0400
Mike Gilbert <floppym@gentoo.org> wrote:

> On Sun, Oct 28, 2012 at 11:55 AM, Michał Górny <mgorny@gentoo.org> wrote:
> > On Sun, 28 Oct 2012 08:57:05 +0100
> > Michał Górny <mgorny@gentoo.org> wrote:
> >
> >> On the other hand, we would introduce a tiny bit of additional code.
> >
> > I did a bit research on the topic, and the code introduced is circa 100
> > lines, including docs and esetuppy() which would be probably added
> > anyway. There are other potential problems however.
> >
> > The main issue is how distutils is designed. The only command accepting
> > build-dir is the 'build' command. Other commands which need to access
> > the build tree just use some weird method of getting parameters for
> > other commands. Effectively, if you need to provide a build-dir, you
> > always have to run 'build'.
> >
> > So, the install command would grow from:
> >
> >   ./setup.py install ...
> >
> > to:
> >
> >   ./setup.py build -b ... install ...
> >
> > Of course, all other commands accessing the build-dir would need to
> > be prepended with the 'build -b ...' as well. For that reason,
> > the patches I'm going to attach add that thing to esetuppy() directly.
> >
> > This has the following implications:
> >
> > 1) all calls to esetuppy() involve running build,
> >
> > 2) if build takes additional arguments, they may need to be passed
> > consistently to all esetuppy calls (like: 'build --foo' in args).
> > Otherwise, a rebuild with different options may happen (but that's
> > probably a case already anyway).
> >
> > In any case, I wouldn't say it's very elegant of Python. It will work
> > flawlessly for the most of Python packages, and likely most
> > of the remaining ones would require running in-source builds anyway
> > (and that can be enabled with a single variable).
> >
> > Well, best just look at the code and tell me what you think.
> >
> 
> I don't really see any need for out-of-source builds, but if you want
> to write the code for it I certainly don't mind having it there.
> 
> I will say that doing out-of-source builds occasionally makes writing
> src_test a bit tricky; you have to play with the working directory and
> PYTHONPATH.

Yes, I have fixed gentoopm to make it compile Python modules (in tests)
in build-dir rather than sourcedir.

https://bitbucket.org/mgorny/gentoopm/changeset/f46c4057d76
https://bitbucket.org/mgorny/gentoopm/changeset/975fd2dde55

The first commit should be enough for most packages out there.
The second one is necessary if any of the modules were loaded before
(e.g. in gentoopm to get PV). But probably getting the PV like that was
a bad idea anyway...

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]