* [gentoo-portage-dev] Pending changes to Portage ebuild & behavior
@ 2014-07-28 15:03 Michał Górny
2014-07-28 17:53 ` Sebastian Luther
0 siblings, 1 reply; 2+ messages in thread
From: Michał Górny @ 2014-07-28 15:03 UTC (permalink / raw
To: gentoo-portage-dev
[-- Attachment #1.1: Type: text/plain, Size: 1743 bytes --]
Hello, everyone.
Following the last meeting approval, I've started working on cleaning up
sys-apps/portage, and making it use EAPI=5 and python-r1. While at it,
I found a few difficulties, pretty closely related to how we install
portage now.
I believe we need to change that to make the ebuild cleaner. If someone
has some spare time, I'd appreciate fixing the portage code.
Current install
---------------
1. .py files are installed in /usr/lib/portage/pym and byte-compiled
with active python interp,
2. emerge and other tools prepend /usr/lib/portage/pym to use those
files [bytecode conflict possible],
3. site-packages contains only symlinks to .py files,
4. some of python modules in portage use paths relative to .py file to
access /usr/lib/portage/bin.
New install
-----------
1. no /usr/lib/portage/pym (it's not really necessary with
python_targets),
2. all python modules & bytecode in site-packages,
3. emerge and other tools load portage from site-packages [proper
bytecode used],
4. but python modules need to be able to locate /usr/lib/portage/bin
somehow.
What needs to be done
---------------------
1. either:
a) we make portage capable of finding /usr/lib/portage/bin when .py
files are in site-packages (but don't break running from checkout!),
b) we install all of bin/* to site-packages as well (some python
packages simply do that).
2. extra kudos for replacing that ugly Makefile with setup.py.
Any thoughts? Anyone willing to do the work? Otherwise, I'll do it
myself if nobody beats me up to it.
Attaching work-in-progress portage-9999-r1 ebuild. Don't install it or
your system will be broken :).
--
Best regards,
Michał Górny
[-- Attachment #1.2: portage-9999-r1.ebuild --]
[-- Type: text/plain, Size: 7714 bytes --]
# Copyright 1999-2014 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: /var/cvsroot/gentoo-x86/sys-apps/portage/portage-9999.ebuild,v 1.96 2014/07/26 09:48:43 mgorny Exp $
EAPI=5
PYTHON_COMPAT=( pypy python{3_2,3_3,3_4,2_7} )
PYTHON_REQ_USE='bzip2(+),ssl(+)'
inherit git-r3 eutils multilib python-r1
DESCRIPTION="Portage is the package management and distribution system for Gentoo"
HOMEPAGE="http://www.gentoo.org/proj/en/portage/index.xml"
LICENSE="GPL-2"
KEYWORDS=""
SLOT="0"
IUSE="build doc epydoc +ipc linguas_ru selinux xattr"
# make-3.82 is for bug #455858
DEPEND="${python_dep}
>=sys-devel/make-3.82
>=sys-apps/sed-4.0.5 sys-devel/patch
doc? ( app-text/xmlto ~app-text/docbook-xml-dtd-4.4 )
epydoc? ( >=dev-python/epydoc-2.0 )"
# Require sandbox-2.2 for bug #288863.
# For xattr, we can spawn getfattr and setfattr from sys-apps/attr, but that's
# quite slow, so it's not considered in the dependencies as an alternative to
# to python-3.3 / pyxattr. Also, xattr support is only tested with Linux, so
# for now, don't pull in xattr deps for other kernels.
# For whirlpool hash, require python[ssl] or python-mhash [XXX] (bug #425046).
# For compgen, require bash[readline] (bug #445576).
RDEPEND="${python_dep}
!build? ( >=sys-apps/sed-4.0.5
|| ( >=app-shells/bash-4.2_p37[readline] ( <app-shells/bash-4.2_p37 >=app-shells/bash-3.2_p17 ) )
>=app-admin/eselect-1.2
${PYTHON_DEPS}
)
elibc_FreeBSD? ( sys-freebsd/freebsd-bin )
elibc_glibc? ( >=sys-apps/sandbox-2.2 )
elibc_uclibc? ( >=sys-apps/sandbox-2.2 )
>=app-misc/pax-utils-0.1.17
selinux? ( || ( >=sys-libs/libselinux-2.0.94[python] <sys-libs/libselinux-2.0.94 ) )
xattr? ( kernel_linux? (
$(python_gen_cond_dep 'dev-python/pyxattr[${PYTHON_USEDEP}]' python{2_7,3_2} pypy)
) )
!<app-shells/bash-3.2_p17
!<app-admin/logrotate-3.8.0"
PDEPEND="
!build? (
>=net-misc/rsync-2.6.4
userland_GNU? ( >=sys-apps/coreutils-6.4 )
)"
# coreutils-6.4 rdep is for date format in emerge-webrsync #164532
# NOTE: FEATURES=installsources requires debugedit and rsync
SRC_ARCHIVES="http://dev.gentoo.org/~dolsen/releases/portage"
prefix_src_archives() {
local x y
for x in ${@}; do
for y in ${SRC_ARCHIVES}; do
echo ${y}/${x}
done
done
}
EGIT_REPO_URI="git://git.overlays.gentoo.org/proj/portage.git
https://github.com/gentoo/portage.git"
EGIT_MIN_CLONE_TYPE=single
src_prepare() {
epatch_user
einfo "Producing ChangeLog from Git history..."
git log ebcf8975b37a8aae9735eb491a9b4cb63549bd5d^.. \
> "${S}"/ChangeLog || die
local _version=$(git describe --tags | sed -e 's|-\([0-9]\+\)-.\+$|_p\1|')
_version=${_version:1}
einfo "Setting portage.VERSION to ${_version} ..."
sed -e "s/^VERSION =.*/VERSION = '${_version}'/" -i pym/portage/__init__.py || \
die "Failed to patch portage.VERSION"
sed -e "1s/VERSION/${_version}/" -i doc/fragment/version || \
die "Failed to patch VERSION in doc/fragment/version"
sed -e "1s/VERSION/${_version}/" -i $(find man -type f) || \
die "Failed to patch VERSION in man page headers"
if ! use ipc ; then
einfo "Disabling ipc..."
sed -e "s:_enable_ipc_daemon = True:_enable_ipc_daemon = False:" \
-i pym/_emerge/AbstractEbuildProcess.py || \
die "failed to patch AbstractEbuildProcess.py"
fi
if use xattr && use kernel_linux ; then
einfo "Adding FEATURES=xattr to make.globals ..."
echo -e '\nFEATURES="${FEATURES} xattr"' >> cnf/make.globals \
|| die "failed to append to make.globals"
fi
if [[ -n ${EPREFIX} ]] ; then
einfo "Setting portage.const.EPREFIX ..."
sed -e "s|^\(SANDBOX_BINARY[[:space:]]*=[[:space:]]*\"\)\(/usr/bin/sandbox\"\)|\\1${EPREFIX}\\2|" \
-e "s|^\(FAKEROOT_BINARY[[:space:]]*=[[:space:]]*\"\)\(/usr/bin/fakeroot\"\)|\\1${EPREFIX}\\2|" \
-e "s|^\(BASH_BINARY[[:space:]]*=[[:space:]]*\"\)\(/bin/bash\"\)|\\1${EPREFIX}\\2|" \
-e "s|^\(MOVE_BINARY[[:space:]]*=[[:space:]]*\"\)\(/bin/mv\"\)|\\1${EPREFIX}\\2|" \
-e "s|^\(PRELINK_BINARY[[:space:]]*=[[:space:]]*\"\)\(/usr/sbin/prelink\"\)|\\1${EPREFIX}\\2|" \
-e "s|^\(EPREFIX[[:space:]]*=[[:space:]]*\"\).*|\\1${EPREFIX}\"|" \
-i pym/portage/const.py || \
die "Failed to patch portage.const.EPREFIX"
einfo "Prefixing shebangs ..."
while read -r -d $'\0' ; do
local shebang=$(head -n1 "$REPLY")
if [[ ${shebang} == "#!"* && ! ${shebang} == "#!${EPREFIX}/"* ]] ; then
sed -i -e "1s:.*:#!${EPREFIX}${shebang:2}:" "$REPLY" || \
die "sed failed"
fi
done < <(find . -type f -print0)
einfo "Adjusting make.globals ..."
sed -e "s|\(/usr/portage\)|${EPREFIX}\\1|" \
-e "s|^\(PORTAGE_TMPDIR=\"\)\(/var/tmp\"\)|\\1${EPREFIX}\\2|" \
-i cnf/make.globals || die "sed failed"
einfo "Adjusting repos.conf ..."
sed -e "s|^\(main-repo = \).*|\\1gentoo_prefix|" \
-e "s|^\\[gentoo\\]|[gentoo_prefix]|" \
-e "s|^\(location = \)\(/usr/portage\)|\\1${EPREFIX}\\2|" \
-e "s|^\(sync-uri = \).*|\\1rsync://prefix.gentooexperimental.org/gentoo-portage-prefix|" \
-i cnf/repos.conf || die "sed failed"
einfo "Adding FEATURES=force-prefix to make.globals ..."
echo -e '\nFEATURES="${FEATURES} force-prefix"' >> cnf/make.globals \
|| die "failed to append to make.globals"
fi
cd "${S}/cnf" || die
if [ -f "make.conf.example.${ARCH}".diff ]; then
patch make.conf.example "make.conf.example.${ARCH}".diff || \
die "Failed to patch make.conf.example"
else
eerror ""
eerror "Portage does not have an arch-specific configuration for this arch."
eerror "Please notify the arch maintainer about this issue. Using generic."
eerror ""
fi
}
src_compile() {
python_setup
if use doc; then
emake docbook || die
fi
if use epydoc; then
einfo "Generating api docs"
emake epydoc || die
fi
}
src_test() {
./runtests.sh || die "tests failed"
}
src_install() {
emake DESTDIR="${D}" \
sysconfdir="${EPREFIX}/etc" \
prefix="${EPREFIX}/usr" \
install
python_install() {
# The Makefile intends on using /usr/lib/portage/pym, so move files.
python_domodule "${ED%/}"/usr/lib/portage/pym/*
python_optimize
}
python_parallel_foreach_impl python_install
# Kill the legacy location to avoid creating caches there.
rm -r "${ED%/}"/usr/lib/portage/pym || die
# Now, wrap the scripts.
local f shebang
for f in "${ED%/}"/usr/bin/*; do
read shebang < "${f}"
[[ ${shebang} == *python* ]] && python_replicate_script "${f}"
done
}
current_python_has_xattr() {
[[ $("${PYTHON}" -c 'import sys ; sys.stdout.write(sys.hexversion >= 0x3030000 and "yes" or "no")') = yes ]] || \
"${PYTHON}" -c 'import xattr' 2>/dev/null
}
pkg_preinst() {
if [[ $ROOT == / ]] ; then
# Run some minimal tests as a sanity check.
local test_runner=$(find "${ED}" -name runTests)
if [[ -n $test_runner && -x $test_runner ]] ; then
einfo "Running preinst sanity tests..."
"$test_runner" || die "preinst sanity tests failed"
fi
fi
if use xattr && ! current_python_has_xattr ; then
ewarn "For optimal performance in xattr handling, install"
ewarn "dev-python/pyxattr, or use Python 3.3 or newer."
fi
# elog dir must exist to avoid logrotate error for bug #415911.
# This code runs in preinst in order to bypass the mapping of
# portage:portage to root:root which happens after src_install.
keepdir /var/log/portage/elog
# This is allowed to fail if the user/group are invalid for prefix users.
if chown portage:portage "${ED}"var/log/portage{,/elog} 2>/dev/null ; then
chmod g+s,ug+rwx "${ED}"var/log/portage{,/elog}
fi
}
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 949 bytes --]
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: [gentoo-portage-dev] Pending changes to Portage ebuild & behavior
2014-07-28 15:03 [gentoo-portage-dev] Pending changes to Portage ebuild & behavior Michał Górny
@ 2014-07-28 17:53 ` Sebastian Luther
0 siblings, 0 replies; 2+ messages in thread
From: Sebastian Luther @ 2014-07-28 17:53 UTC (permalink / raw
To: gentoo-portage-dev
Am 28.07.2014 17:03, schrieb Michał Górny:
> Hello, everyone.
>
> New install -----------
>
> 1. no /usr/lib/portage/pym (it's not really necessary with
> python_targets),
We need to make that emerge -C prevents the removal of the last python
version for which portage is installed then. That may already work.
>
> 2. all python modules & bytecode in site-packages,
>
> 3. emerge and other tools load portage from site-packages [proper
> bytecode used],
Other tools already do that. Emerge is a different thing. It's not a
tool based on portage. emerge and portage are two large,
interconnected things. (run "git grep _emerge" in pym/portage)
>
> 4. but python modules need to be able to locate
> /usr/lib/portage/bin somehow.
>
>
What about hardcoding this path at install time? Looking at pkgcore
might give you some hints.
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2014-07-28 17:53 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-07-28 15:03 [gentoo-portage-dev] Pending changes to Portage ebuild & behavior Michał Górny
2014-07-28 17:53 ` Sebastian Luther
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox