[gentoo-dev] [PATCH 0/2] lua-utils.eclass: support LuaJIT and unslotted Lua

[gentoo-dev] [PATCH 0/2] lua-utils.eclass: support LuaJIT and unslotted Lua

[Marek Szuba]

I have decided to add support for dev-lang/lua:0 because having thought
quite a lot about the transition path from Lua ebuilds as they are to
the new eclasses, it seems to be the most sane approach. Still, I hope
this will not stay around for too long.

Nothing particularly noteworthy about LuaJIT support, possibly apart
from the fact that we take advantage of it being allegedly being a
drop-in replacement for lua-5.1.

[gentoo-dev] [PATCH 1/2] lua-utils.eclass: Support luajit

[Marek Szuba]

According to discussions on IRC, luajit should work as a drop-in
replacement for lua5.1 - and indeed, at least for x11-wm/awesome
it has worked.

Note that for the time being dev-lang/luajit uses the same module
directories as dev-lang/lua:5.1, which may lead to weird behaviour in
multi-impl ebuilds supporting both lua5-1 and luajit. Hopefully we will
get luajit to use its own directories so that it is fully independent,
same as we install pypy3 modules in their own directory hierarchy in
spite of compatibility with cpython-3.6.

Signed-off-by: Marek Szuba <marecki@gentoo.org>
---
 eclass/lua-utils.eclass              | 21 +++++++++++++++++----
 profiles/desc/lua_single_target.desc |  1 +
 profiles/desc/lua_targets.desc       |  1 +
 3 files changed, 19 insertions(+), 4 deletions(-)

diff --git a/eclass/lua-utils.eclass b/eclass/lua-utils.eclass
index 490d19a0019..24ef67635d5 100644
--- a/eclass/lua-utils.eclass
+++ b/eclass/lua-utils.eclass
@@ -14,8 +14,6 @@
 # A utility eclass providing functions to query Lua implementations,
 # install Lua modules and scripts.
 #
-# Please note that for the time being this eclass does NOT support luajit.
-#
 # This eclass neither sets any metadata variables nor exports any phase
 # functions. It can be inherited safely.
 
@@ -39,6 +37,7 @@ inherit toolchain-funcs
 # @DESCRIPTION:
 # All supported Lua implementations, most preferred last
 _LUA_ALL_IMPLS=(
+	luajit
 	lua5-1
 	lua5-2
 	lua5-3
@@ -141,9 +140,16 @@ _lua_wrapper_setup() {
 		local ELUA LUA
 		_lua_export "${impl}" ELUA LUA
 
-		# Lua interpreter and compiler
+		# Lua interpreter
 		ln -s "${EPREFIX}"/usr/bin/${ELUA} "${workdir}"/bin/lua || die
-		ln -s "${EPREFIX}"/usr/bin/${ELUA/a/ac} "${workdir}"/bin/luac || die
+
+		# Lua compiler, or a stub for it in case of luajit
+		if [[ ${ELUA} == luajit ]]; then
+			# Just in case
+			ln -s "${EPREFIX}"/bin/true "${workdir}"/bin/luac || die
+		else
+			ln -s "${EPREFIX}"/usr/bin/${ELUA/a/ac} "${workdir}"/bin/luac || die
+		fi
 
 		# pkg-config
 		ln -s "${EPREFIX}"/usr/$(get_libdir)/pkgconfig/${ELUA}.pc \
@@ -201,6 +207,10 @@ _lua_export() {
 	local impl var
 
 	case "${1}" in
+		luajit)
+			impl=${1}
+			shift
+			;;
 		lua*)
 			impl=${1/-/.}
 			shift
@@ -259,6 +269,9 @@ _lua_export() {
 			LUA_PKG_DEP)
 				local d
 				case ${impl} in
+					luajit)
+						LUA_PKG_DEP="dev-lang/luajit:="
+						;;
 					lua*)
 						LUA_PKG_DEP="dev-lang/lua:${impl#lua}"
 						;;
diff --git a/profiles/desc/lua_single_target.desc b/profiles/desc/lua_single_target.desc
index 1bee02b6978..c3d422e434d 100644
--- a/profiles/desc/lua_single_target.desc
+++ b/profiles/desc/lua_single_target.desc
@@ -7,3 +7,4 @@ lua5-1 - Build for Lua 5.1 only
 lua5-2 - Build for Lua 5.2 only
 lua5-3 - Build for Lua 5.3 only
 lua5-4 - Build for Lua 5.4 only
+luajit - Build for LuaJIT only
diff --git a/profiles/desc/lua_targets.desc b/profiles/desc/lua_targets.desc
index 2575de0bcfd..75b9e0f86af 100644
--- a/profiles/desc/lua_targets.desc
+++ b/profiles/desc/lua_targets.desc
@@ -7,3 +7,4 @@ lua5-1 - Build with Lua 5.1
 lua5-2 - Build with Lua 5.2
 lua5-3 - Build with Lua 5.3
 lua5-4 - Build with Lua 5.4
+luajit - Build with LuaJIT
-- 
2.26.2

[gentoo-dev] [PATCH 2/2] lua-utils.eclass: support dev-lang/lua:0

[Marek Szuba]

This is to make it possible for the maintainers of Lua-dependent ebuilds
to transition to lua{,-single}.eclass without unmasking slotted
dev-lang/lua.

Signed-off-by: Marek Szuba <marecki@gentoo.org>
---
 eclass/lua-utils.eclass              | 8 ++++++++
 profiles/desc/lua_single_target.desc | 1 +
 profiles/desc/lua_targets.desc       | 1 +
 3 files changed, 10 insertions(+)

diff --git a/eclass/lua-utils.eclass b/eclass/lua-utils.eclass
index 24ef67635d5..b84fb6e9a68 100644
--- a/eclass/lua-utils.eclass
+++ b/eclass/lua-utils.eclass
@@ -38,6 +38,7 @@ inherit toolchain-funcs
 # All supported Lua implementations, most preferred last
 _LUA_ALL_IMPLS=(
 	luajit
+	lua0
 	lua5-1
 	lua5-2
 	lua5-3
@@ -211,6 +212,10 @@ _lua_export() {
 			impl=${1}
 			shift
 			;;
+		lua0)
+			impl="lua"
+			shift
+			;;
 		lua*)
 			impl=${1/-/.}
 			shift
@@ -272,6 +277,9 @@ _lua_export() {
 					luajit)
 						LUA_PKG_DEP="dev-lang/luajit:="
 						;;
+					lua)
+						LUA_PKG_DEP="dev-lang/lua:0"
+						;;
 					lua*)
 						LUA_PKG_DEP="dev-lang/lua:${impl#lua}"
 						;;
diff --git a/profiles/desc/lua_single_target.desc b/profiles/desc/lua_single_target.desc
index c3d422e434d..04f71b1fe58 100644
--- a/profiles/desc/lua_single_target.desc
+++ b/profiles/desc/lua_single_target.desc
@@ -3,6 +3,7 @@
 
 # This file contains descriptions of LUA_SINGLE_TARGET USE_EXPAND flags.
 
+lua0 - Build for unslotted Lua only
 lua5-1 - Build for Lua 5.1 only
 lua5-2 - Build for Lua 5.2 only
 lua5-3 - Build for Lua 5.3 only
diff --git a/profiles/desc/lua_targets.desc b/profiles/desc/lua_targets.desc
index 75b9e0f86af..9f296fe2499 100644
--- a/profiles/desc/lua_targets.desc
+++ b/profiles/desc/lua_targets.desc
@@ -3,6 +3,7 @@
 
 # This file contains descriptions of LUA_TARGETS USE_EXPAND flags.
 
+lua0 - Build with unslotted Lua
 lua5-1 - Build with Lua 5.1
 lua5-2 - Build with Lua 5.2
 lua5-3 - Build with Lua 5.3
-- 
2.26.2

Re: [gentoo-dev] [PATCH 0/2] lua-utils.eclass: support LuaJIT and unslotted Lua

[Azamat Hackimov]

Hi.

пн, 5 окт. 2020 г. в 21:01, Marek Szuba <marecki@gentoo.org>:
>
> I have decided to add support for dev-lang/lua:0 because having thought
> quite a lot about the transition path from Lua ebuilds as they are to
> the new eclasses, it seems to be the most sane approach. Still, I hope
> this will not stay around for too long.

Is there a slotmove that can be applied? Can be lua:0 and lua:5.1
coexist? Will there be more problems with that approach?
I think a better idea is to migrate the base set of lua packages to
slotted Lua (maybe under package.mask) and slowly migrate the rest of
the portage.

-- 
From Siberia with Love!

Re: [gentoo-dev] [PATCH 0/2] lua-utils.eclass: support LuaJIT and unslotted Lua

[Marek Szuba]

[-- Attachment #1.1: Type: text/plain, Size: 1354 bytes --]

On 2020-10-05 20:30, Azamat Hackimov wrote:

> Is there a slotmove that can be applied?

I am afraid I do not understand the question. What do you want to move,
and why?

> Can be lua:0 and lua:5.1 coexist?

In theory they can (I say "in theory" because app-eselect/eselect-lua
ebuild currently block dev-lang/lua:0), except having :0 installed would
conflict with using eselect-lua - which I guess might be acceptable
during the transition period. Anyway, there are no file collisions
between :0 and :5.1, and while the two use the same CMOD and LMOD
directories (for the record, so does LuaJIT for now) this should be
harmless - files in LMOD would be very much identical, and from what I
recall from discussions on IRC compiled modules in CMOD must not link
against Lua libraries themselves so they should be
implementation-independent.

> I think a better idea is to migrate the base set of lua packages

Define "base set of lua packages".

> (maybe under package.mask)

...meaning that at some point in the future we would have to unmask
slotted Lua AND all the ebuilds depending on it, potentially unleashing
a torrent of errors. Although we cannot really avoid this step
altogether, I would rather let maintainers adapt their Lua ebuilds to
the eclasses first and handle slotted Lua as a separate step.

-- 
MS


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] [PATCH 0/2] lua-utils.eclass: support LuaJIT and unslotted Lua

[Azamat Hackimov]

пн, 5 окт. 2020 г. в 22:20, Marek Szuba <marecki@gentoo.org>:

> > Is there a slotmove that can be applied?
>
> I am afraid I do not understand the question. What do you want to move,
> and why?

Currently portage is mostly lua:5.1 aware and the first thing is move
dev-lang/lua:0 to 5.1 slot by slotmove in profiles/updates:

slotmove dev-lang/lua 0 5.1

and update all ebuilds with dev-lang/lua:0 dependency (there are about
200 ebuilds with that dependency, maybe there need to be an issue news
item, since there may need to rebuild all dependencies).
After that we need to unmask and stabilize "true" slotted lua:5.1
version, so this would be step 0 on slotted lua migration. After that
migration of rest packages to new lua.eclass will be a lot easier.

> > I think a better idea is to migrate the base set of lua packages
>
> Define "base set of lua packages".

Well, core build ecosystem of lua packages:

dev-lua/ldoc
dev-lua/lpeg
dev-lua/luacov
dev-lua/luacrypto
dev-lua/luadoc
dev-lua/luarocks
dev-lua/luasystem
dev-lua/lua-utf8
dev-lua/lua-zlib

maybe more.

-- 
From Siberia with Love!

Re: [gentoo-dev] [PATCH 0/2] lua-utils.eclass: support LuaJIT and unslotted Lua

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 393 bytes --]

>>>>> On Mon, 05 Oct 2020, Azamat Hackimov wrote:

> Currently portage is mostly lua:5.1 aware and the first thing is move
> dev-lang/lua:0 to 5.1 slot by slotmove in profiles/updates:

> slotmove dev-lang/lua 0 5.1

This would mean that slot 0 can never again be used for the package.
My advice would be to limit the version matching the slotmove, like
=dev-lang/lua-5.1* or similar.

Ulrich

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 507 bytes --]

Re: [gentoo-dev] [PATCH 0/2] lua-utils.eclass: support LuaJIT and unslotted Lua

[Marek Szuba]

[-- Attachment #1.1: Type: text/plain, Size: 1263 bytes --]

On 2020-10-05 23:17, Azamat Hackimov wrote:

>>> Is there a slotmove that can be applied?
>>
>> I am afraid I do not understand the question. What do you want to move,
>> and why?
> 
> Currently portage is mostly lua:5.1 aware and the first thing is move
> dev-lang/lua:0 to 5.1 slot by slotmove in profiles/updates:
> 
> slotmove dev-lang/lua 0 5.1

The file structure of dev-lang/lua:0 differs from dev-lang/lua:5.x,
meaning that introducing such a slot move would require Lua eclasses to
treat lua5-1 differently from other targets. In other words, we would
*still* need lua0 support but without a clearly distinct name.

>> Define "base set of lua packages".
> 
> Well, core build ecosystem of lua packages:
> 
> dev-lua/ldoc
> dev-lua/lpeg
> dev-lua/luacov
> dev-lua/luacrypto
> dev-lua/luadoc
> dev-lua/luarocks
> dev-lua/luasystem
> dev-lua/lua-utf8
> dev-lua/lua-zlib
> 
> maybe more.

And the advantage of having those migrated would be? It would make sense
if these really were core to Lua support in other packages, like e.g.
setuptools and the like in the Python world - but this varies wildly
from package to package, with a large number of them depending on
nothing but the Lua interpreter itself.

-- 
MS


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] [PATCH 0/2] lua-utils.eclass: support LuaJIT and unslotted Lua

[Azamat Hackimov]

вт, 6 окт. 2020 г. в 15:04, Marek Szuba <marecki@gentoo.org>:
>
> On 2020-10-05 23:17, Azamat Hackimov wrote:
>
> >>> Is there a slotmove that can be applied?
> >>
> >> I am afraid I do not understand the question. What do you want to move,
> >> and why?
> >
> > Currently portage is mostly lua:5.1 aware and the first thing is move
> > dev-lang/lua:0 to 5.1 slot by slotmove in profiles/updates:
> >
> > slotmove dev-lang/lua 0 5.1
>
> The file structure of dev-lang/lua:0 differs from dev-lang/lua:5.x,
> meaning that introducing such a slot move would require Lua eclasses to
> treat lua5-1 differently from other targets. In other words, we would
> *still* need lua0 support but without a clearly distinct name.

So there will be a new stabilized lua:5.1 version as I proposed.
Again, here is a quick todo list:

1. slotmove dev-lang/lua
2. Change all dependencies to change from lua:0 to lua:5.1
3. Stabilize & unmask slotted version 5.1.5-r103
4. Release news & rebuild with emerge -a --oneshot $(equery depends
dev-lang/lua | awk '{print " ="$1}')
5. Now we have lua:5.1 portage ready to lua.eclass migration
6. PROFIT

1-4 can be done with one git push

-- 
From Siberia with Love!

Re: [gentoo-dev] [PATCH 0/2] lua-utils.eclass: support LuaJIT and unslotted Lua

[Brian Evans]

[-- Attachment #1.1.1: Type: text/plain, Size: 1501 bytes --]

On 10/6/2020 8:45 AM, Azamat Hackimov wrote:
> вт, 6 окт. 2020 г. в 15:04, Marek Szuba <marecki@gentoo.org>:
>>
>> On 2020-10-05 23:17, Azamat Hackimov wrote:
>>
>>>>> Is there a slotmove that can be applied?
>>>>
>>>> I am afraid I do not understand the question. What do you want to move,
>>>> and why?
>>>
>>> Currently portage is mostly lua:5.1 aware and the first thing is move
>>> dev-lang/lua:0 to 5.1 slot by slotmove in profiles/updates:
>>>
>>> slotmove dev-lang/lua 0 5.1
>>
>> The file structure of dev-lang/lua:0 differs from dev-lang/lua:5.x,
>> meaning that introducing such a slot move would require Lua eclasses to
>> treat lua5-1 differently from other targets. In other words, we would
>> *still* need lua0 support but without a clearly distinct name.
> 
> So there will be a new stabilized lua:5.1 version as I proposed.
> Again, here is a quick todo list:
> 
> 1. slotmove dev-lang/lua
> 2. Change all dependencies to change from lua:0 to lua:5.1
> 3. Stabilize & unmask slotted version 5.1.5-r103
> 4. Release news & rebuild with emerge -a --oneshot $(equery depends
> dev-lang/lua | awk '{print " ="$1}')

Please don't use 'equery depends' for such things.  It shows what *can* 
depend and not what *actually* depends on a given system because of 
metadata shortcuts. (It won't consider USE for example.)

If a library name is moving, the best tool is revdep-rebuild with the 
--library option with the old soname to look for.

Brian

[-- Attachment #1.1.2: OpenPGP_0x4E15E2F167C78E1D.asc --]
[-- Type: application/pgp-keys, Size: 17905 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [gentoo-dev] [PATCH 0/2] lua-utils.eclass: support LuaJIT and unslotted Lua

[Azamat Hackimov]

вт, 6 окт. 2020 г. в 15:54, Brian Evans <grknight@gentoo.org>:
>
> On 10/6/2020 8:45 AM, Azamat Hackimov wrote:
> > вт, 6 окт. 2020 г. в 15:04, Marek Szuba <marecki@gentoo.org>:
> >>
> >> On 2020-10-05 23:17, Azamat Hackimov wrote:
> >>
> >>>>> Is there a slotmove that can be applied?
> >>>>
> >>>> I am afraid I do not understand the question. What do you want to move,
> >>>> and why?
> >>>
> >>> Currently portage is mostly lua:5.1 aware and the first thing is move
> >>> dev-lang/lua:0 to 5.1 slot by slotmove in profiles/updates:
> >>>
> >>> slotmove dev-lang/lua 0 5.1
> >>
> >> The file structure of dev-lang/lua:0 differs from dev-lang/lua:5.x,
> >> meaning that introducing such a slot move would require Lua eclasses to
> >> treat lua5-1 differently from other targets. In other words, we would
> >> *still* need lua0 support but without a clearly distinct name.
> >
> > So there will be a new stabilized lua:5.1 version as I proposed.
> > Again, here is a quick todo list:
> >
> > 1. slotmove dev-lang/lua
> > 2. Change all dependencies to change from lua:0 to lua:5.1
> > 3. Stabilize & unmask slotted version 5.1.5-r103
> > 4. Release news & rebuild with emerge -a --oneshot $(equery depends
> > dev-lang/lua | awk '{print " ="$1}')
>
> Please don't use 'equery depends' for such things.  It shows what *can*
> depend and not what *actually* depends on a given system because of
> metadata shortcuts. (It won't consider USE for example.)
>
> If a library name is moving, the best tool is revdep-rebuild with the
> --library option with the old soname to look for.
>
> Brian

Depend packages can not use a library for linking, so it may be hard
to retrieve the actual list with revdep-rebuild.


--
From Siberia with Love!

Re: [gentoo-dev] [PATCH 0/2] lua-utils.eclass: support LuaJIT and unslotted Lua

[Marek Szuba]

[-- Attachment #1.1.1: Type: text/plain, Size: 971 bytes --]

On 2020-10-05 20:01, Marek Szuba wrote:

> I have decided to add support for dev-lang/lua:0 because having thought
> quite a lot about the transition path from Lua ebuilds as they are to
> the new eclasses, it seems to be the most sane approach. Still, I hope
> this will not stay around for too long.

NOT pushed because having poked around various Lua revdeps in the tree I 
realised that with the way Lua detection works in e.g. CMake, the only 
way of making sure an ebuild really honours the values of LUA_TARGETS / 
LUA_SINGLE_TARGET, or indeed LUA_COMPAT, is to have multiple Lua 
versions installed side by side. In other words, migrating ebuilds to 
lua{,-single}.eclass prior to unmasking dev-lang/lua:5.x could in fact 
result in quite a mess.

> Nothing particularly noteworthy about LuaJIT support, possibly apart
> from the fact that we take advantage of it being allegedly being a
> drop-in replacement for lua-5.1.

Pushed.

-- 
MS

[-- Attachment #1.1.2: OpenPGP_0xD4945FFDD7FE7E37_and_old_rev.asc --]
[-- Type: application/pgp-keys, Size: 122125 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]