From: "Michal Gorny (mgorny)" <mgorny@gentoo.org>
To: gentoo-commits@lists.gentoo.org
Subject: [gentoo-commits] gentoo commit in xml/htdocs/proj/en/Python/python-r1: dev-guide.xml
Date: Sat, 10 Nov 2012 08:26:20 +0000 (UTC) [thread overview]
Message-ID: <20121110082620.2800C20C63@flycatcher.gentoo.org> (raw)
mgorny 12/11/10 08:26:20
Added: dev-guide.xml
Log:
The initial version of python-r1 Developer's Guide.
Revision Changes Path
1.1 xml/htdocs/proj/en/Python/python-r1/dev-guide.xml
file : http://sources.gentoo.org/viewvc.cgi/gentoo/xml/htdocs/proj/en/Python/python-r1/dev-guide.xml?rev=1.1&view=markup
plain: http://sources.gentoo.org/viewvc.cgi/gentoo/xml/htdocs/proj/en/Python/python-r1/dev-guide.xml?rev=1.1&content-type=text/plain
Index: dev-guide.xml
===================================================================
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE guide SYSTEM "/dtd/guide.dtd">
<!-- $Header: /var/cvsroot/gentoo/xml/htdocs/proj/en/Python/python-r1/dev-guide.xml,v 1.1 2012/11/10 08:26:20 mgorny Exp $ -->
<guide lang="en">
<title>python-r1 Developer's Guide</title>
<author title="Author">
<mail link="mgorny@gentoo.org">Michał Górny</mail>
</author>
<author title="Editor">
<mail link="idella4@gentoo.org">Ian Delaney</mail>
</author>
<abstract>
This guide provides a basic insight to writing ebuilds using
the python-r1 and distutils-r1 eclasses.
</abstract>
<!-- The content of this document is licensed under the CC-BY-SA license -->
<!-- See http://creativecommons.org/licenses/by-sa/3.0/ -->
<license version="3.0"/>
<version>1</version>
<date>2012-11-04</date>
<chapter id="Introductory_info">
<title>Introductory information</title>
<section id="ii_When_to_use">
<title>When to use?</title>
<body>
<p>
The new Python eclasses, python-r1 and distutils-r1, are still
considered ‘testing grade’. Although they can be used
in the tree, they must not be used on stable packages
or packages which will need to be stabilized soon. For that
reason, it is usually a good idea to revbump the package when
converting.
</p>
<p>
It should be noted that the -r1 eclasses are not a drop-in
replacement for python.eclass. They differ in design and goals.
At the moment, they also lack some of the functionality.
The most important gap is lack of support for packages
not supporting installation for multiple Python implementations.
</p>
</body>
</section>
<section id="ii_Purpose_of_the_eclasses">
<title>Purpose of the eclasses</title>
<body>
<p>
The python-r1 suite consists of the python-r1 and distutils-r1
eclasses.
</p>
<p>
The python-r1 eclass is the fundamental eclass for all Python
packages. It covers the general aspects, helping ebuild
developers to handle building packages for multiple Python
implementations and write proper dependency strings. However,
it does not enforce any dependencies itself nor export phase
functions.
</p>
<p>
The distutils-r1 eclass extends python-r1 with a simple to use
framework to build and install packages using the standard
Python distutils build system. It follows the common practices
for build system eclasses, exporting phase functions which
handle applying patches, building and installing.
</p>
<p>
In the vast majority of cases, the distutils-r1 will the best
choice for your ebuild. The python-r1 should be mostly used
in packages which do not use the distutils build system or use
it in a non-standard way.
</p>
<note>
Please note that the distutils-r1 eclass inherits python-r1
implicitly. There is therefore no need to ever inherit both.
</note>
</body>
</section>
</chapter>
<chapter id="The_fundamentals_of_python_r1_eclass">
<title>The fundamentals of python-r1 eclass</title>
<section id="pr1_Listing_supported_Python_implementations">
<title>Listing supported Python implementations</title>
<body>
<p>
The first and most important task in writing ebuilds both
for python-r1 and distutils-r1 is to specify the list of Python
implementations supported by the ebuild.
</p>
<p>
The list of supported implementations is specified using
the <c>PYTHON_COMPAT</c> variable. It is an obligatory array
which has to be declared before the <c>inherit</c> command.
It should list <e>all</e> supported implementations using
the following naming scheme:
</p>
<ol>
<li><c>pythonX_Y</c> for CPython X.Y;</li>
<li><c>pypyX_Y</c> for PyPy X.Y;</li>
<li><c>jythonX_Y</c> for Jython X.Y.</li>
</ol>
<p>
<c>PYTHON_COMPAT</c> should be treated with respect similar
to the <c>KEYWORDS</c> variable. No implementation should
be added there without prior testing, and all package
dependencies must support that implementation.
</p>
<p>
The <c>PYTHON_COMPAT</c> variable can be considered
a replacement for python.eclass' <c>PYTHON_DEPEND</c>
and <c>RESTRICT_PYTHON_ABIS</c> variables.
</p>
<pre caption="Examples of PYTHON_COMPAT">
<comment># Any version of CPython 2.</comment>
<ident>PYTHON_COMPAT</ident>=( python2_5 python2_6 python2_7 )
<comment># Python 2 or 3, using brace expansion.</comment>
<ident>PYTHON_COMPAT</ident>=( python{2_5,2_6,2_7,3_1,3_2,3_3} )
<comment># Python 2.6+ and compliant implementations.</comment>
<ident>PYTHON_COMPAT</ident>=( python{2_6,2_7} pypy{1_8,1_9} )
<comment># inherit goes below PYTHON_COMPAT</comment>
<keyword>inherit</keyword> python-r1
</pre>
</body>
</section>
<section id="pr1_Depending_on_Python">
<title>Depending on Python</title>
<body>
<note>
Please note that this section applies to the sole use
of python-r1 eclass only. The distutils-r1 eclass
unconditionally adds Python dependency.
</note>
<p>
In order to efficiently handle various kinds of package
dependencies on Python, the python-r1 eclass does not set
the ebuild dependencies directly. Instead, it prepares
the proper dependency string and stores it in a variable named
<c>PYTHON_DEPS</c>.
</p>
<p>
The <c>PYTHON_DEPS</c> variable is unconditionally set
by the eclass to a list of proper USE-conditional dependencies
on enabled Python implementations. It also holds any additional
dependencies necessary — most notably, a dependency
on <c>dev-python/python-exec</c>.
</p>
<p>
The ebuild should reference the <c>PYTHON_DEPS</c> variable
in <c>RDEPEND</c> and/or <c>DEPEND</c> as necessary. It may need
to put those dependencies into an appropriate USE-conditional
block.
</p>
<pre caption="Example uses of PYTHON_DEPS">
<comment># Unconditional Python run-time + build-time dependency.</comment>
<ident>RDEPEND</ident>=${PYTHON_DEPS}
<ident>DEPEND</ident>=${RDEPEND}
<comment># Unconditional Python build-time dependency.</comment>
<ident>DEPEND</ident>=${PYTHON_DEPS}
<comment># USE-conditional Python run-time dependency.</comment>
<ident>RDEPEND</ident>="<const>python?</const> ( ${PYTHON_DEPS} )"
</pre>
</body>
</section>
<section id="pr1_Depending_on_other_Python_packages">
<title>Depending on other Python packages</title>
<body>
<p>
There are currently two kinds of Python packages in the tree;
one which explicitly specify enabled Python implementations via
<c>PYTHON_TARGETS</c>, and consists of ebuilds using python-r1
and python-distutils-ng eclasses, the other which lacks such
a support. These are packages using the ‘original’
python.eclass.
</p>
<p>
When a dependency against a package supporting
<c>PYTHON_TARGETS</c> is to be expressed, a USE dependency
should be specified to ensure that the dependencies
are installed for all required Python implementations.
The eclass provides the necessary dependency string fragment
in <c>PYTHON_USEDEP</c> variable.
</p>
<p>
The <c>PYTHON_USEDEP</c> variable is set unconditionally
by the eclass. It contains a bare compact USE dependency string,
enforcing a requirement on all selected Python implementations.
</p>
<p>
Depending on packages not supporting <c>PYTHON_TARGETS</c>
is discouraged. Whenever possible, please consider migrating
the dependant package instead. Such a dependency is unable
to enforce the fore-mentioned requirement, therefore making
the user vulnerable to unclear nuisance failures.
The dependencies on packages of that kind are expressed using
the regular (simple) dependency syntax.
</p>
<pre caption="Example use of PYTHON_USEDEP">
<comment># Simple dependency on a python-r1 package.</comment>
<ident>RDEPEND</ident>="<const>dev-python/setuptools</const>[${PYTHON_USEDEP}]"
<comment># Dependency with additional USE flags requested.</comment>
<ident>RDEPEND</ident>="<const>dev-python/lxml</const>[threads,${PYTHON_USEDEP}]"
<comment># Dependency on a package using python.eclass.</comment>
<ident>RDEPEND</ident>="<const>dev-python/wxpython:2.8</const>"
</pre>
</body>
</section>
<section id="pr1_Requesting_optional_Python_features">
<title>Requesting optional Python features (modules)</title>
<body>
<p>
Sometimes it is necessary to depend on a feature or module which
belongs to the Python implementation itself but is not always
available. The availability can depend both on Python version
and USE flags. These features can be divided into two types;
those having replacement modules and those lacking them.
</p>
<p>
If a particular module has a replacement package, the correct
way to require it is to depend on the respective virtual
package. For example, a script using the <c>argparse</c> module
should depend on <c>virtual/python-argparse</c>. The virtual
will either enforce a particular USE constraints or pull in
the replacement package as necessary.
</p>
<p>
If a particular module or feature is provided by the Python
implementation only, and is conditional upon the setting
of a USE flag, <c>PYTHON_REQ_USE</c> should be used instead.
</p>
<p>
The <c>PYTHON_REQ_USE</c> variable is an optional variable which
can be used to enforce a set of USE flag requirements
upon the Python implementation. It takes a bare USE dependency
string and appends it to the dependency on <e>every</e> Python
implementation supported. Sometimes it may be necessary to use
EAPI 4 USE defaults to handle cases when a particular flag
is enabled unconditionally in some of the supported
implementations.
</p>
<p>
The <c>PYTHON_REQ_USE</c> variable can be considered
a replacement for python.eclass' <c>PYTHON_USE_WITH</c>
variable.
</p>
<pre caption="Example uses of virtuals and PYTHON_REQ_USE">
<comment># Run-time dependency on argparse module (through virtuals).</comment>
<ident>RDEPEND</ident>="virtual/python-argparse[<var>${PYTHON_USEDEP}</var>]"
<comment># Simple dependency on ncurses support.</comment>
<ident>PYTHON_REQ_USE</ident>="ncurses"
<comment># Dependency on bzip2 support (conditional in PyPy, always enabled in CPython).</comment>
<ident>PYTHON_REQ_USE</ident>="bzip2(+)"
</pre>
</body>
</section>
<section id="pr1_Obtaining_Python_implementation_information">
<title>Obtaining Python implementation information</title>
<body>
<p>
Sometimes it is necessary to obtain information specific
to a particular Python implementations, in particular
interpreter-specific paths. The python-r1 eclass provides
the following means of obtaining that information:
</p>
<ol>
<li><c>python_export</c> function,</li>
<li><c>python_export_best</c> function,</li>
<li>getter functions.</li>
</ol>
<p>
The <c>python_export</c> and <c>python_export_best</c> functions
take an optional list of variable names and export
the requested variables. When no variable names are passed,
the default variable list is used.
</p>
<p>
The <c>python_export</c> can additionally take a Python
implementation as a first argument, either in the form of a USE
flag or an <c>EPYTHON</c> value. If such a form is used,
the values specific to the passed implementation are exported.
Otherwise, the currently used implementation (the <c>EPYTHON</c>
environment variable) is used.
</p>
<p>
The <c>python_export_best</c> exports variables for the ‘best’
of the currently enabled implementations; that is, the newest
interpreter version from the most preferred group. These groups
are, in order of preference:
</p>
<ol>
<li>CPython 2,</li>
<li>CPython 3,</li>
<li>PyPy,</li>
<li>Jython.</li>
</ol>
<p>
The getter functions print the value of a specific property
to standard output. The output can be captured using bash
command substitution. These functions take an optional parameter
specifying the requested Python implementation.
If not specified, the current one will be used.
</p>
<p>
The following table lists all the available variables, along
with the respective getter functions:
</p>
<table>
<tr>
<th>Variable name</th>
<th>Getter function</th>
<th>Default?</th>
<th>Description</th>
</tr>
<tr>
<ti><c>EPYTHON</c></ti>
<ti><c>python_get_EPYTHON</c></ti>
<ti>yes</ti>
<ti>
The Python implementation name (Gentoo-specific).
</ti>
</tr>
<tr>
<ti><c>PYTHON</c></ti>
<ti><c>python_get_PYTHON</c></ti>
<ti>yes</ti>
<ti>
Absolute path to the Python interpreter.
</ti>
</tr>
<tr>
<ti><c>PYTHON_SITEDIR</c></ti>
<ti><c>python_get_sitedir</c></ti>
<ti>no</ti>
<ti>
Path to the Python site-packages directory (where modules
should be installed).
</ti>
</tr>
</table>
</body>
</section>
</chapter>
<chapter id="The_distutils_r1_eclass">
<title>The distutils-r1 eclass</title>
<section id="dr1_Tasks_performed_by_distutils_r1">
<title>Tasks performed by distutils-r1</title>
<body>
<p>
The distutils-r1 exports a set of phase functions performing
common tasks related to building and installing the package.
In many cases, those tasks will suffice for a typical Python
package.
</p>
<p>
The tasks performed by distutils-r1 are (in execution order):
</p>
<ol>
<li>applying patches listed in the <c>PATCHES</c> array;</li>
<li>applying user patches (the <c>epatch_user</c> function);</li>
<li>building and installing the package using <c>setup.py</c>;</li>
<li>ensuring that Python scripts are installed in multiple
variants according to the enabled Python implementations;</li>
<li>installing additional documentation (using <c>DOCS</c>
and <c>HTML_DOCS</c> arrays).</li>
</ol>
<pre caption="Example on enabling patching and additional
documentation">
<keyword>PATCHES</keyword>=(
<comment># bug #nnnnnn, a random error with something.</comment>
"${FILESDIR}"/${P}-foo.patch
<comment># bug #nnnnnn, some other error.</comment>
"${FILESDIR}"/${P}-bar.patch
)
<keyword>DOCS</keyword>=( README.md )
<keyword>HTML_DOCS</keyword>=( doc/html/ )
</pre>
<note>
The listed variables can be either set in the global scope
or in the scope of the respective phase function.
The <c>PATCHES</c> variable can be set in <c>src_prepare()</c>
or <c>python_prepare_all()</c> instead, while <c>DOCS</c>
and <c>HTML_DOCS</c> can be set in <c>src_install()</c>
or <c>python_install_all()</c>.
</note>
</body>
</section>
<section id="dr1_The_partial_phase_functions">
<title>The ‘partial’ phase functions</title>
<body>
<p>
The distutils-r1 eclass utilises a mechanism inspired by phase
functions to make writing ebuilds relatively easy. For each
phase function handled by distutils-r1, namely all <c>src_*</c>
phases, two ‘partial’ phases are used; the per-implementation
sub-phase and the common sub-phase.
</p>
<p>
The sub-phase functions are named for the corresponding
<c>src_*</c> phase but with the <c>src</c> prefix replaced
by <c>python</c>. For example, the compilation sub-phase
function has to be named <c>python_compile</c>. These functions
are called once for each enabled Python implementation
and perform tasks specific to it. They are called
with the standard implementation information variables exported
(<c>EPYTHON</c>, <c>PYTHON</c>…).
</p>
<p>
An additional <c>_all</c> suffix is appended to the common
sub-phase functions. For example, the installation sub-phase
function is subsequently named <c>python_install_all</c>. These
functions are called only once in the main source directory
and they perform tasks common to all enabled implementations.
</p>
<p>
The distutils-r1 eclass provides default functions for all
per-implementation sub-phases and for <c>python_prepare_all</c>
and <c>python_install_all</c>. If you are replacing any
of those phase functions, you ought call the respective
<c>distutils-r1_*</c> phase function in it.
</p>
<pre caption="Example of defining sub-phase functions">
<keyword>python_compile_all()</keyword> {
if use doc; then
python_export_best
"${PYTHON}" setup.py doc || die
fi
}
<keyword>python_test()</keyword> {
"${PYTHON}" setup.py test || die
}
<keyword>python_install_all()</keyword> {
use doc && local HTML_DOCS=( doc/* )
<keyword>distutils-r1_python_install_all</keyword>
}</pre>
</body>
</section>
<section id="dr1_Out_of_source_and_in_source_builds">
<title>Out-of-source and in-source builds</title>
<body>
<p>
There are two modes of building packages with distutils-r1;
‘out-of-source builds’ (the default) and ‘in-source builds’.
An out-of-source build uses a common source directory, reads
files from that directory but locates the built files
in a separate directory. An in-source build locates both
the source and output files in the same directory.
</p>
<p>
When an out-of-source build is used, the phase functions are run
in the <c>${S}</c> directory. The build directory used
for the particular implementation is stored
in the <c>BUILD_DIR</c> variable, all calls to <c>setup.py</c>
pass additional <c>build --build-base…</c> parameters pointing
to it and the <path>lib</path> subdirectory of the build
directory is added to <c>PYTHONPATH</c>. As a side effect,
the package is built on the first <c>setup.py</c> invocation.
</p>
<p>
The in-source builds are implemented for more problematic
packages where out-of-source builds may not work due to bugs
in the build system. They are enabled either explicitly
when the <c>DISTUTILS_IN_SOURCE_BUILD</c> variable is set
(to any value) or implicitly when the <c>python_prepare</c>
(per-implementation one) sub-phase is declared.
</p>
<p>
When in-source builds are enabled, the default
<c>python_prepare_all</c> first creates a copy of all
the sources in the <c>BUILD_DIR</c> for each implementation.
Common sub-phase functions are still executed in the source
directory but per-implementation sub-phases are executed
in the build directory for each implementation. There is
no explicit linking between the source copies, therefore
the build system is allowed to modify them freely.
</p>
</body>
</section>
</chapter>
<!-- vim:se tw=72 ts=2 sts=2 sw=2 :-->
</guide>
next reply other threads:[~2012-11-10 8:26 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-11-10 8:26 Michal Gorny (mgorny) [this message]
-- strict thread matches above, loose matches on Subject: below --
2012-11-24 21:34 [gentoo-commits] gentoo commit in xml/htdocs/proj/en/Python/python-r1: dev-guide.xml Michal Gorny (mgorny)
2012-11-26 12:14 Michal Gorny (mgorny)
2012-11-27 17:47 Michal Gorny (mgorny)
2012-11-27 20:41 Michal Gorny (mgorny)
2012-11-28 19:49 Michal Gorny (mgorny)
2012-11-30 11:04 Michal Gorny (mgorny)
2012-12-01 9:27 Michal Gorny (mgorny)
2012-12-06 16:15 Michal Gorny (mgorny)
2012-12-07 18:00 Michal Gorny (mgorny)
2012-12-09 14:39 Michal Gorny (mgorny)
2013-01-04 22:34 Michal Gorny (mgorny)
2013-01-11 21:16 Michal Gorny (mgorny)
2013-01-12 23:34 Michal Gorny (mgorny)
2013-01-30 11:30 Michal Gorny (mgorny)
2013-02-13 11:40 Michal Gorny (mgorny)
2013-02-15 14:35 Michal Gorny (mgorny)
2013-06-27 2:09 Robin H. Johnson (robbat2)
2013-09-20 21:15 Michal Gorny (mgorny)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20121110082620.2800C20C63@flycatcher.gentoo.org \
--to=mgorny@gentoo.org \
--cc=gentoo-commits@lists.gentoo.org \
--cc=gentoo-dev@lists.gentoo.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox