[gentoo-portage-dev] PATCH: initial EAPI awareness

[gentoo-portage-dev] PATCH: initial EAPI awareness

[Brian Harring]

[-- Attachment #1.1: Type: text/plain, Size: 1518 bytes --]

Hola.

Attached is a patch that 
A) adds EAPI awareness to portage; mainly, if >0, complain and be 
   unwilling to merge the package
B) tweaks to portage_db_flat, addition of portage_db_metadata, and 
   portage_db_flat_hash

Flat_hash is the replacement cache format for metadata/cache; assuming 
nobody does anything stupid with EAPI, it will allow us to deploy new 
metadata in the tree without forcing a breakage every time we do so.

portage_db_metadata.py is a compability implementation; basically 
detects if it's flat_list (portage_db_flat), or flat_hash and calls 
accordingly.

Aside from that, tried to keep changes as minimal as possible.  The 
patch isn't complete due to the fact we need to add an EAPI cache 
compatibility check for --metadata transfer, but can't do that till I 
get some feedback from -dev ml regarding a bit of tree cleanup that 
will go along with it.

Please test this out; if you want to test the EAPI checking, tag 
EAPI=1 into an ebuild, and try making emerge bail.

If you're less adventurous, please test the compatibility cache 
testing; 
in /etc/portage/modules
portdbapi.metadbmodule=portage_db_metadata.database

Will enable the autodetection.  Additionally, 
portdbapi.auxdbmodule=portage_db_flat_hash.database

being flipped on would be appreciated, although you will have to wipe 
your cache and run emerge --metadata; with the settings above, you're 
testing the auto-detect functionality, and the new format.

Thanks,
~harring

[-- Attachment #1.2: 2.0.51-eapi-awareness.patch --]
[-- Type: text/plain, Size: 11596 bytes --]

diff -urN portage-old/bin/emerge portage/bin/emerge
--- portage-old/bin/emerge	2005-08-27 05:28:09.000000000 -0500
+++ portage/bin/emerge	2005-08-27 05:27:17.000000000 -0500
@@ -612,7 +612,7 @@
 	help()
 	sys.exit(0)
 elif portage.secpass!=2:
-	if myaction in ["search", "info", "regen"]:
+	if myaction in ["search", "info", "regen", "metadata"]:
 		pass
 	elif (not myaction) and (not myfiles):
 		pass
@@ -1497,7 +1497,7 @@
 		myfetchlist=[]
 		for x in mylist:
 			fetch=" "
-
+			stupid_if_logic_hack = True
 			if x[0]=="blocks":
 				addl=""+red("B")+"  "+fetch+"  "
 				resolved=portage.db[x[1]]["vartree"].resolve_key(x[2])
@@ -1512,7 +1512,22 @@
 						print red("(is blocking "+x[3]+")")
 					else:
 						print
+				stupid_if_logic_hack = False
+
 			else:
+				if x[0] == "ebuild":	source = "porttree"
+				elif x[0] == "binary":	source = "bintree"
+				elif x[0] == ("nomerge", "blocker"):	source = None
+				else:
+					print "EAPI check, unknown source "+str(x[0])+" for "+x[2]+", this shouldn't occur",mylist
+					sys.exit(1)
+				if source:
+					needed = portage.db[x[1]][source].dbapi.aux_get(x[2], ["EAPI"])[0]
+					if needed > portage.EAPI:
+						print '[%s] %s %s, EAPI:%i installed portage EAPI:%i' % (x[0].ljust(13), x[2], red("UNMERGABLE"),needed, portage.EAPI)
+						stupid_if_logic_hack = True
+
+			if stupid_if_logic_hack:
 				if (x[0]!="binary") and ("fetch" in string.split(portage.portdb.aux_get(x[2],["RESTRICT"])[0])):
 					fetch = red("F")
 					if portage.portdb.fetch_check(x[2], portage.settings):
@@ -1830,6 +1845,34 @@
 		self.pkgsettings["FEATURES"]=string.join(myfeat)
 
 		mergecount=0
+		
+		unmergable = []
+		for source, root, cpv, action in mymergelist:
+			if source == "ebuild":	source = "porttree"
+			elif source == "binary": source = "bintree"
+			elif source in ("nomerge", "blocks"):
+				continue
+			else:
+				print "EAPI check, unknown source "+str(source)+" for "+cpv+", this shouldn't occur",mymergelist
+				sys.exit(1)
+			needed = portage.db[root][source].dbapi.aux_get(cpv, ["EAPI"])[0]
+			if portage.EAPI < needed:
+				unmergable.append((needed, cpv))
+
+		if len(unmergable):
+			print
+			print red("!!!")
+			print red("!!!")+" %s" % red("Ebuild API incompatibility")
+			print red("!!!")+" installed portage EAPI is %i, the following packages require a higher version" % portage.EAPI
+			print red("!!!")+" This version of portage is incapable of installing the higher version"
+			print red("!!!")+" A portage upgrade should solve this"
+			print red("!!!")
+			for needed, cpv in unmergable:
+				print ">>>    %s requires EAPI %s" % (cpv, red(str(needed)))
+			print
+			if not ("--fetchonly" in myopts or "--fetch-all-uri" in myopts or "--pretend" in myopts):
+				sys.exit(1)
+
 		for x in mymergelist:
 			mergecount+=1
 			myroot=x[1]
diff -urN portage-old/pym/portage.py portage/pym/portage.py
--- portage-old/pym/portage.py	2005-08-27 05:28:02.000000000 -0500
+++ portage/pym/portage.py	2005-08-27 05:27:26.000000000 -0500
@@ -81,7 +81,7 @@
 	  MOVE_BINARY, PRELINK_BINARY, WORLD_FILE, MAKE_CONF_FILE, MAKE_DEFAULTS_FILE, \
 	  DEPRECATED_PROFILE_FILE, USER_VIRTUALS_FILE, EBUILD_SH_ENV_FILE, \
 	  INVALID_ENV_FILE, CUSTOM_MIRRORS_FILE, SANDBOX_PIDS_FILE, CONFIG_MEMORY_FILE,\
-	  INCREMENTALS, STICKIES
+	  INCREMENTALS, STICKIES, EAPI
 
 	from portage_data import ostype, lchown, userland, secpass, uid, wheelgid, \
 	                         portage_uid, portage_gid
@@ -5077,9 +5071,9 @@
   'DEPEND',    'RDEPEND',   'SLOT',      'SRC_URI',
 	'RESTRICT',  'HOMEPAGE',  'LICENSE',   'DESCRIPTION',
 	'KEYWORDS',  'INHERITED', 'IUSE',      'CDEPEND',
-	'PDEPEND',   'PROVIDE',
+	'PDEPEND',   'PROVIDE', 'EAPI',
 	'UNUSED_01', 'UNUSED_02', 'UNUSED_03', 'UNUSED_04',
-	'UNUSED_05', 'UNUSED_06', 'UNUSED_07', 'UNUSED_08',
+	'UNUSED_05', 'UNUSED_06', 'UNUSED_07',
 	]
 auxdbkeylen=len(auxdbkeys)
 
@@ -5373,10 +5367,15 @@
 		mydata   = self.auxdb[mylocation][cat][pkg]
 		returnme = []
 		for x in mylist:
-			if mydata.has_key(x):
-				returnme.append(mydata[x])
-			else:
-				returnme.append("")
+			returnme.append(mydata.get(x,""))
+
+		if "EAPI" in mylist:
+			idx = mylist.index("EAPI")
+			try:
+				returnme[idx] = int(returnme[idx])
+			except ValueError:
+				# string
+				returnme[idx] = 0
 
 		return returnme
 
diff -urN portage-old/pym/portage_db_flat.py portage/pym/portage_db_flat.py
--- portage-old/pym/portage_db_flat.py	2005-08-27 05:27:59.000000000 -0500
+++ portage/pym/portage_db_flat.py	2005-08-27 05:27:43.000000000 -0500
@@ -9,6 +9,10 @@
 
 import portage_db_template
 
+# since this format is massively deprecated, 
+# we're hardcoding the previously weird line count
+magic_line_count = 22
+
 class database(portage_db_template.database):
 	def module_init(self):
 		self.lastkey  = None # Cache
@@ -42,39 +46,39 @@
 				mykeys += [x]
 		return mykeys
 
-	def get_values(self,key):
+	def get_values(self,key, data=None):
+		""" do not use data unless you know what it does."""
+
 		if not key:
 			raise KeyError, "key is not set to a valid value"
 
-		try:
-			# give buffering a hint of the pretty much maximal cache size we deal with
-			myf = open(self.fullpath+key, "r", 8192)
-		except OSError, oe:
-			# either the file didn't exist, or it was removed under our feet.
-			return None 
-		
+		mydict = {}
+		if data == None:
+			try:
+				# give buffering a hint of the pretty much maximal cache size we deal with
+				myf = open(self.fullpath+key, "r", 8192)
+			except OSError:
+				# either the file didn't exist, or it was removed under our feet.
+				raise KeyError("failed reading key")
+
+			# nuke the newlines right off the batt.
+			data = myf.read().splitlines()
+			mydict["_mtime_"] = os.fstat(myf.fileno()).st_mtime
+			myf.close()
+		else:
+			mydict["_mtime_"] = data.pop(-1)
 
-		# nuke the newlines right off the batt.
-		data = myf.read().splitlines()
-		mdict = {}
-		
 		# rely on exceptions to note differing line counts.
 		try:
-			for x in range(0, len(self.dbkeys)):
-				mdict[self.dbkeys[x]] = data[x]
-
-			# do this now, rather then earlier- possible that earlier it might have been wasted
-			# if key count mismatched
-			mdict["_mtime_"] = os.fstat(myf.fileno()).st_mtime
+			for x in range(magic_line_count):
+				mydict[self.dbkeys[x]] = data[x]
 
 		except IndexError:
-			myf.close()
 			raise ValueError, "Key count mistmatch"
 
-		myf.close()
-		return mdict
+		return mydict
 	
-	def set_values(self,key,val):
+	def set_values(self,key, val, raw=False):
 		if not key:
 			raise KeyError, "No key provided. key:%s val:%s" % (key,val)
 		if not val:
@@ -86,12 +90,19 @@
 
 		update_fp = self.fullpath + ".update." + str(os.getpid()) + "." + key
 		myf = open(update_fp,"w")
-		myf.writelines( [ val[x] +"\n" for x in self.dbkeys] )
+		if not raw:
+			myf.writelines( [ val[x] +"\n" for x in self.dbkeys] )
+			if len(self.dbkeys) != magic_line_count:
+				myf.writelines(["\n"] * len(self.dbkeys) - magic_line_count)
+			mtime = val["_mtime_"]
+		else:
+			mtime = val.pop(-1)
+			myf.writelines(val)
 		myf.close()
 		
 		os.chown(update_fp, self.uid, self.gid)
 		os.chmod(update_fp, 0664)
-		os.utime(update_fp, (-1,long(val["_mtime_"])))
+		os.utime(update_fp, (-1,long(mtime)))
 		os.rename(update_fp, self.fullpath+key)
 
 	def del_key(self,key):
diff -urN portage-old/pym/portage_db_flat_hash.py portage/pym/portage_db_flat_hash.py
--- portage-old/pym/portage_db_flat_hash.py	1969-12-31 18:00:00.000000000 -0600
+++ portage/pym/portage_db_flat_hash.py	2005-08-27 05:27:43.000000000 -0500
@@ -0,0 +1,38 @@
+# Copyright 2004 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+# $Header: /var/cvsroot/gentoo-src/portage/pym/Attic/portage_db_flat.py,v 1.13.2.6 2005/04/19 07:14:17 ferringb Exp $
+cvs_id_string="$Id: portage_db_flat.py,v 1.13.2.6 2005/04/19 07:14:17 ferringb Exp $"[5:-2]
+
+import portage_db_flat, os
+
+class database(portage_db_flat.database):
+	
+	def get_values(self, key, data=None):
+		""" do not specify data unless you know what it does"""
+		if not key:
+			raise KeyError("key is not valid")
+		
+		if data == None:
+			try:
+				myf = open(self.fullpath + key, "r")
+			except OSError:
+				raise KeyError("failed pulling key")
+
+			data = dict(map(lambda x: x.split("=",1), myf.read().splitlines()))
+			data["_mtime_"] = os.fstat(myf.fileno()).st_mtime
+			myf.close()
+
+		mydict = {}
+		for x in self.dbkeys:
+			mydict[x] = data.get(x, "")
+		mydict["_mtime_"] = long(data["_mtime_"])
+		return mydict
+		
+	def set_values(self, key, values):
+		l = []
+		for x in values.keys():
+			if values[x] and x != "_mtime_":
+				l.append("%s=%s\n" % (x, values[x]))
+		l.append(values["_mtime_"])
+		portage_db_flat.database.set_values(self, key, l, raw=True)
+		
diff -urN portage-old/pym/portage_db_metadata.py portage/pym/portage_db_metadata.py
--- portage-old/pym/portage_db_metadata.py	1969-12-31 18:00:00.000000000 -0600
+++ portage/pym/portage_db_metadata.py	2005-08-27 05:27:43.000000000 -0500
@@ -0,0 +1,49 @@
+# Copyright 2004 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+# $Header: /var/cvsroot/gentoo-src/portage/pym/Attic/portage_db_flat.py,v 1.13.2.6 2005/04/19 07:14:17 ferringb Exp $
+cvs_id_string="$Id: portage_db_flat.py,v 1.13.2.6 2005/04/19 07:14:17 ferringb Exp $"[5:-2]
+
+import os, portage_db_flat_hash, portage_db_flat
+
+class database(portage_db_flat_hash.database):
+	
+	def get_values(self, key):
+		if not key:
+			raise KeyError("key is not valid")
+		
+		try:
+			myf = open(self.fullpath + key, "r")
+		except OSError:
+			raise KeyError("key is not valid")
+		mtime = os.fstat(myf.fileno()).st_mtime
+		data = myf.read().splitlines()
+		
+		# easy attempt first.
+		if len(data) != portage_db_flat.magic_line_count:
+			d = dict(map(lambda x: x.split("=",1), data))
+			d["_mtime_"] = mtime
+			return portage_db_flat_hash.database.get_values(self, key, d)
+		# this one's interesting.
+		d = {}
+
+		for line in data:
+			# yes, meant to iterate over a string.
+			hashed = False
+			for idx, c in enumerate(line):
+				if not c.isalpha():
+					if c == "=" and idx > 0:
+						hashed = True
+						d[line[:idx]] = line[idx + 1:]
+					elif c == "_" or c.isdigit():
+						continue
+					break
+				elif not c.isupper():
+					break
+
+			if not hashed:
+				# non hashed.
+				data.append(mtime)
+				return portage_db_flat.database.get_values(self, key, data=data)
+
+		d["_mtime_"] = mtime
+		return portage_db_flat_hash.database.get_values(self, key, data=d)
--- portage-old/pym/portage_const.py	2005-04-28 23:56:35.000000000 -0500
+++ portage/pym/portage_const.py	2005-08-27 03:59:09.000000000 -0500
@@ -43,6 +43,9 @@
 INCREMENTALS=["USE","FEATURES","ACCEPT_KEYWORDS","ACCEPT_LICENSE","CONFIG_PROTECT_MASK","CONFIG_PROTECT","PRELINK_PATH","PRELINK_PATH_MASK"]
 STICKIES=["KEYWORDS_ACCEPT","USE","CFLAGS","CXXFLAGS","MAKEOPTS","EXTRA_ECONF","EXTRA_EINSTALL","EXTRA_EMAKE"]
 
+# supported EBUILD api versions.
+EAPI = 0
+
 # ===========================================================================
 # END OF CONSTANTS -- END OF CONSTANTS -- END OF CONSTANTS -- END OF CONSTANT
 # ===========================================================================

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Jason Stubbs]

[-- Attachment #1: Type: text/plain, Size: 439 bytes --]

On Saturday 27 August 2005 19:53, Brian Harring wrote:
> Please test this out; if you want to test the EAPI checking, tag
> EAPI=1 into an ebuild, and try making emerge bail.

+ print "EAPI check, unknown source "+str(x[0])+" for "+x[2]+", this 
shouldn't occur",mylist
+ sys.exit(1)

These two are in the patch. Can that be made an exception please? That way 
--debug could still be useful if the bug hits.

-- 
Jason Stubbs

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 742 bytes --]

On Sun, Aug 28, 2005 at 02:46:28PM +0900, Jason Stubbs wrote:
> On Saturday 27 August 2005 19:53, Brian Harring wrote:
> > Please test this out; if you want to test the EAPI checking, tag
> > EAPI=1 into an ebuild, and try making emerge bail.
> 
> + print "EAPI check, unknown source "+str(x[0])+" for "+x[2]+", this 
> shouldn't occur",mylist
> + sys.exit(1)
> 
> These two are in the patch. Can that be made an exception please? That way 
> --debug could still be useful if the bug hits.
That's debugging; not knowing (due to design flaws) all potentials for 
x[0], stuck that in debugging for anyone running the patch; people 
hitting it reporting back (hopefully), for debugging

Final version won't include it.
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Zac Medico]

[-- Attachment #1: Type: text/plain, Size: 234 bytes --]

Brian Harring wrote:
> Please test this out; if you want to test the EAPI checking, tag 
> EAPI=1 into an ebuild, and try making emerge bail.

I needed to patch ebuild.sh so that EAPI would be parsed.  It bails out properly now.

Zac

[-- Attachment #2: 2.0.51-eapi-awareness-ebuild.sh.patch --]
[-- Type: text/x-patch, Size: 741 bytes --]

Index: portage/bin/ebuild.sh
===================================================================
--- portage.orig/bin/ebuild.sh
+++ portage/bin/ebuild.sh
@@ -1850,6 +1850,7 @@ for myarg in $*; do
 		echo `echo "$CDEPEND"`     >> $dbkey
 		echo `echo "$PDEPEND"`     >> $dbkey
 		echo `echo "$PROVIDE"`     >> $dbkey
+		echo `echo "$EAPI"`        >> $dbkey
 		echo `echo "$UNUSED_01"`   >> $dbkey
 		echo `echo "$UNUSED_02"`   >> $dbkey
 		echo `echo "$UNUSED_03"`   >> $dbkey
@@ -1857,7 +1858,6 @@ for myarg in $*; do
 		echo `echo "$UNUSED_05"`   >> $dbkey
 		echo `echo "$UNUSED_06"`   >> $dbkey
 		echo `echo "$UNUSED_07"`   >> $dbkey
-		echo `echo "$UNUSED_08"`   >> $dbkey
 		set +f
 		#make sure it is writable by our group:
 		exit 0

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 395 bytes --]

On Sun, Aug 28, 2005 at 02:31:24AM -0700, Zac Medico wrote:
> Brian Harring wrote:
> >Please test this out; if you want to test the EAPI checking, tag 
> >EAPI=1 into an ebuild, and try making emerge bail.
> 
> I needed to patch ebuild.sh so that EAPI would be parsed.  It bails out 
> properly now.
Crud, didn't include that file.
Should be echo `echo "${EAPI:-0}"` actually
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[warnera6]

> On Sun, Aug 28, 2005 at 02:46:28PM +0900, Jason Stubbs wrote:
>> On Saturday 27 August 2005 19:53, Brian Harring wrote:
>> > Please test this out; if you want to test the EAPI checking, tag
>> > EAPI=1 into an ebuild, and try making emerge bail.
>>
>> + print "EAPI check, unknown source "+str(x[0])+" for "+x[2]+", this
>> shouldn't occur",mylist
>> + sys.exit(1)
>>
>> These two are in the patch. Can that be made an exception please? That
>> way
>> --debug could still be useful if the bug hits.
> That's debugging; not knowing (due to design flaws) all potentials for
> x[0], stuck that in debugging for anyone running the patch; people
> hitting it reporting back (hopefully), for debugging
>
> Final version won't include it.
> ~harring
>

pppfft, you were just complaining about all the sys.exit()'s in the code
and then you go and add one yourself.  raise something instead, even if
it's just an Exception, that way it saves anyone from grepping through the
code for the error message.

-Alec

PS:  Squirrelmail sucks for actual work :P


-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 927 bytes --]

On Saturday 27 August 2005 12:53, Brian Harring wrote:
> Hola.
>
> Attached is a patch that
> A) adds EAPI awareness to portage; mainly, if >0, complain and be
>    unwilling to merge the package
> B) tweaks to portage_db_flat, addition of portage_db_metadata, and
>    portage_db_flat_hash

I just realised that there are possible issues with incompatble eclasses 
and ebuilds. What (should) happen(s) when an eclass supports EAPI=1 with 
src_configure and src_make, while the ebuild supports/expects EAPI=0 with 
src_compile.

This means that in some way eclasses should state which EAPI versions they 
support. Possibly with making EAPI a space separated list of api's 
accepted. This would mean some checking in the inherit code AND that EAPI 
in the ebuild should be defined before the inherit.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 1517 bytes --]

On Fri, Sep 02, 2005 at 08:31:26AM +0200, Marius Mauch wrote:
> On 08/27/05  Brian Harring wrote:
> 
> > Hola.
> > 
> > Attached is a patch that 
> > A) adds EAPI awareness to portage; mainly, if >0, complain and be 
> >    unwilling to merge the package
> 
> Actually I just wrote also a patch for it (for 2.1), however instead of
> complaining I just masked them (without unmask ability), just add a
> check to gvisible() and getmaskingstatus() (actually just calling
> dbapi.eapicheck()). That way it should be more transparent to users IMO.
> Won't stop people from using ebuild(1) though.
Masking makes a bit more sense, 'cept that information needs to be 
dumped out when resolution cannot occur.

Further, it's a mild gotcha for users who wonder wth portage is 
masking a node.

Better approach imo though, and would like to see that implemented 
rather then what I did with the stable patch.

Incremental'ing EAPI during inherit is a no go; EAPI1 and 
EAPI0 are incompatible for execution, due to the src_compile 
configure/make breakup.

Somebody care to split a masking patch for stable rather then the 
emerge modifications I did btw?  I'm poking at ensuring an eapi=0 
portage's generated eapi=1 cache entries are not used by an eapi=1 
portage without a forced regeneration atm.

That and the fact the 2.1 state should be decided, if we're going to 
have (effectively) two branches of development going at once, vs 
developmental line and maintenance branch.
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Zac Medico]

[-- Attachment #1: Type: text/plain, Size: 826 bytes --]

Brian Harring wrote:
> 
> Please test this out; if you want to test the EAPI checking, tag 
> EAPI=1 into an ebuild, and try making emerge bail.
> 

Well, it bails too often. :)

It seems that an explicit integer conversion is needed for > and < comparisons with mixed types (2.0.51-eapi-awareness-emerge-int-conversion.patch).  

> If you're less adventurous, please test the compatibility cache 
> testing; 
> in /etc/portage/modules
> portdbapi.metadbmodule=portage_db_metadata.database
> 

We need to make sure EAPI defaults to 0 when using old metadata.  It seems like db_template.__getitem__() is a nice central place to do that (2.0.51-eapi-awareness-db_template.patch).  I made another patch for bindbapi.aux_get() which does the same for old metadata from binpkgs (2.0.51-eapi-awareness-bindbapi.aux_get.patch).

Zac

[-- Attachment #2: 2.0.51-eapi-awareness-emerge-int-conversion.patch --]
[-- Type: text/x-patch, Size: 907 bytes --]

Index: portage-2.0.51.22/bin/emerge
===================================================================
--- portage-2.0.51.22.orig/bin/emerge
+++ portage-2.0.51.22/bin/emerge
@@ -1523,7 +1523,7 @@ class depgraph:
 					sys.exit(1)
 				if source:
 					needed = portage.db[x[1]][source].dbapi.aux_get(x[2], ["EAPI"])[0]
-					if needed > portage.EAPI:
+					if int(needed) > portage.EAPI:
 						print '[%s] %s %s, EAPI:%i installed portage EAPI:%i' % (x[0].ljust(13), x[2], red("UNMERGABLE"),needed, portage.EAPI)
 						stupid_if_logic_hack = True
 
@@ -1856,7 +1856,7 @@ class depgraph:
 				print "EAPI check, unknown source "+str(source)+" for "+cpv+", this shouldn't occur",mymergelist
 				sys.exit(1)
 			needed = portage.db[root][source].dbapi.aux_get(cpv, ["EAPI"])[0]
-			if portage.EAPI < needed:
+			if portage.EAPI < int(needed):
 				unmergable.append((needed, cpv))
 
 		if len(unmergable):

[-- Attachment #3: 2.0.51-eapi-awareness-db_template.patch --]
[-- Type: text/x-patch, Size: 567 bytes --]

Index: portage-2.0.51.22/pym/portage_db_template.py
===================================================================
--- portage-2.0.51.22.orig/pym/portage_db_template.py
+++ portage-2.0.51.22/pym/portage_db_template.py
@@ -71,6 +71,10 @@ class database:
 		if self.has_key(key):
 			try:
 				values = self.get_values(key)
+				values.setdefault("EAPI","0")
+				if values["EAPI"]=="":
+					# for backward compatibility with old metadata from various sources
+					values["EAPI"]="0"
 				self.__addCache(key,values)
 				return values
 			except SystemExit, e:

[-- Attachment #4: 2.0.51-eapi-awareness-bindbapi.aux_get.patch --]
[-- Type: text/x-patch, Size: 718 bytes --]

Index: portage-2.0.51.22/pym/portage.py
===================================================================
--- portage-2.0.51.22.orig/pym/portage.py
+++ portage-2.0.51.22/pym/portage.py
@@ -4517,13 +4517,18 @@ class bindbapi(fakedbapi):
 				if self.bintree.remotepkgs[tbz2name].has_key(x):
 					mylist.append(self.bintree.remotepkgs[tbz2name][x][:]) # [:] Copy String
 				else:
-					mylist.append("")
+					if x=="EAPI":
+						mylist.append("0")
+					else:
+						mylist.append("")
 			else:
 				myval = tbz2.getfile(x)
 				if myval == None:
 					myval = ""
 				else:
 					myval = string.join(myval.split(),' ')
+				if x=="EAPI" and myval=="":
+					myval="0"
 				mylist.append(myval)
 
 		return mylist

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Brian Harring]

[-- Attachment #1.1: Type: text/plain, Size: 2257 bytes --]

On Mon, Aug 29, 2005 at 01:52:50PM -0700, Zac Medico wrote:
> Brian Harring wrote:
> >
> >Please test this out; if you want to test the EAPI checking, tag 
> >EAPI=1 into an ebuild, and try making emerge bail.
> >
> 
> Well, it bails too often. :)
Mainly cause I generated something of a crap patch leaving a bit of 
stuff out from the looks of it :)

> It seems that an explicit integer conversion is needed for > and < 
> comparisons with mixed types 
> (2.0.51-eapi-awareness-emerge-int-conversion.patch).  
> >If you're less adventurous, please test the compatibility cache 
> >testing; 
> >in /etc/portage/modules
> >portdbapi.metadbmodule=portage_db_metadata.database
> >
> 
> We need to make sure EAPI defaults to 0 when using old metadata.  It seems 
> like db_template.__getitem__() is a nice central place to do that 
> (2.0.51-eapi-awareness-db_template.patch).  I made another patch for 
> bindbapi.aux_get() which does the same for old metadata from binpkgs 
> (2.0.51-eapi-awareness-bindbapi.aux_get.patch).
The default'ing should occur outside of the cache layer imo, due to 
the fact it has to occur in every dbapi.
Attached is a patch doing so, using gvisible to mask ebuild's EAPI > 
portage's EAPI, and a check in doebuild to block people from trying to 
do things with an ebuild that has an EAPI greater then portage's.

The doebuild mod is strictly for non depend phases; the data from a 
depend's phase gets nuked and EAPI set to -EAPI if it's a later 
version that what portage knows about.

Discussed on irc a bit, but repeating the reasons for those on the ml 
who've missed the logic; effectively, an EAPI=0 portage generating an 
EAPI=1 cache entry cannot be guranteed to generate it correctly, 
contain all needed metadata.

So you flag it specially (negated EAPI), and don't store anything.  
Add a few checks in so when an EAPI=1 portage finds the EAPI=1 cache 
entry, which has been negated to mean effectively "someone who is 
EAPI=1, please regenerate me", it gets regenerated, and the entry can 
be trusted to hold all required EAPI1 metadata.

Attached is a patch culminating the feedback, with a few tweaks.  
Should work a helluva lot better this time around :)
~harring

[-- Attachment #1.2: 2.0.51-eapi-awareness-2.patch --]
[-- Type: text/plain, Size: 13792 bytes --]

diff -purN portage-old/bin/ebuild.sh portage/bin/ebuild.sh
--- portage-old/bin/ebuild.sh	2005-08-29 02:16:41.000000000 -0500
+++ portage/bin/ebuild.sh	2005-08-29 17:02:00.000000000 -0500
@@ -1831,6 +1831,7 @@ for myarg in $*; do
 		echo `echo "$CDEPEND"`     >> $dbkey
 		echo `echo "$PDEPEND"`     >> $dbkey
 		echo `echo "$PROVIDE"`     >> $dbkey
+		echo `echo "${EAPI:-0}"`   >> $dbkey
 		echo `echo "$UNUSED_01"`   >> $dbkey
 		echo `echo "$UNUSED_02"`   >> $dbkey
 		echo `echo "$UNUSED_03"`   >> $dbkey
@@ -1838,7 +1839,6 @@ for myarg in $*; do
 		echo `echo "$UNUSED_05"`   >> $dbkey
 		echo `echo "$UNUSED_06"`   >> $dbkey
 		echo `echo "$UNUSED_07"`   >> $dbkey
-		echo `echo "$UNUSED_08"`   >> $dbkey
 		set +f
 		#make sure it is writable by our group:
 		exit 0
diff -purN portage-old/pym/portage.py portage/pym/portage.py
--- portage-old/pym/portage.py	2005-08-27 05:28:02.000000000 -0500
+++ portage/pym/portage.py	2005-08-29 17:11:30.000000000 -0500
@@ -81,7 +81,7 @@ try:
 	  MOVE_BINARY, PRELINK_BINARY, WORLD_FILE, MAKE_CONF_FILE, MAKE_DEFAULTS_FILE, \
 	  DEPRECATED_PROFILE_FILE, USER_VIRTUALS_FILE, EBUILD_SH_ENV_FILE, \
 	  INVALID_ENV_FILE, CUSTOM_MIRRORS_FILE, SANDBOX_PIDS_FILE, CONFIG_MEMORY_FILE,\
-	  INCREMENTALS, STICKIES
+	  INCREMENTALS, STICKIES, EAPI
 
 	from portage_data import ostype, lchown, userland, secpass, uid, wheelgid, \
 	                         portage_uid, portage_gid
@@ -2402,6 +2396,15 @@ def doebuild(myebuild,mydo,myroot,mysett
 			raise
 		except:
 			pass
+		try:
+			eapi = abs(db[root][tree].dbapi.aux_get(mycpv, ["EAPI"])[0])
+			if portage_const.EAPI != eapi:
+				# can't do anything with this.
+				raise Exception("Unable to do any operations on '%s', due to the fact it's EAPI is higher then this portage versions.  Please upgrade to a portage version that supports EAPI %i" % (mycpv, eapi))
+		except SystemExit, e:
+			raise
+		except Exception, e:
+			raise Exception("Unable to pull EAPI from cpv %s, tree %s; can't confirm that it's supported by this portage, thus unable to merge it: Exception was '%s'" % (mycpv, tree, e))
 
 	if mysplit[2] == "r0":
 		mysettings["PVR"]=mysplit[1]
@@ -3911,7 +3914,10 @@ def getmaskingstatus(mycpv):
 					rValue.append("package.mask")
 
 	# keywords checking
-	mygroups = portdb.aux_get(mycpv, ["KEYWORDS"])[0].split()
+	mygroups, eapi = portdb.aux_get(mycpv, ["KEYWORDS", "EAPI"])
+	if abs(eapi) > portage_const.EAPI:
+		return ["required EAPI %i, supported EAPI %i" % (eapi, portage_const.EAPI)]
+	mygroups = mygroups.split()
 	pgroups=groups[:]
 	myarch = settings["ARCH"]
 	pkgdict = settings.pkeywordsdict
@@ -4531,7 +4537,12 @@ class bindbapi(fakedbapi):
 				else:
 					myval = string.join(myval.split(),' ')
 				mylist.append(myval)
-
+		if "EAPI" in wants:
+			idx = wants.index("EAPI")
+			try:
+				mylist[idx] = abs(int(mylist[idx]))
+			except ValueError:
+				mylist[idx] = 0
 		return mylist
 
 
@@ -4811,6 +4822,12 @@ class vardbapi(dbapi):
 			else:
 				myd = ""
 			results.append(myd)
+		if "EAPI" in wants:
+			idx = wants.index("EAPI")
+			try:
+				results[idx] = abs(int(wants[idx]))
+			except ValueError:
+				results[idx] = 0 
 		return results
 
 
@@ -5077,9 +5094,9 @@ auxdbkeys=[
   'DEPEND',    'RDEPEND',   'SLOT',      'SRC_URI',
 	'RESTRICT',  'HOMEPAGE',  'LICENSE',   'DESCRIPTION',
 	'KEYWORDS',  'INHERITED', 'IUSE',      'CDEPEND',
-	'PDEPEND',   'PROVIDE',
+	'PDEPEND',   'PROVIDE', 'EAPI',
 	'UNUSED_01', 'UNUSED_02', 'UNUSED_03', 'UNUSED_04',
-	'UNUSED_05', 'UNUSED_06', 'UNUSED_07', 'UNUSED_08',
+	'UNUSED_05', 'UNUSED_06', 'UNUSED_07',
 	]
 auxdbkeylen=len(auxdbkeys)
 
@@ -5293,11 +5310,18 @@ class portdbapi(dbapi):
 		# we use cache files, which is usually on /usr/portage/metadata/cache/.
 		if doregen and mylocation==self.mysettings["PORTDIR"] and metacachedir and self.metadb[cat].has_key(pkg):
 			metadata=self.metadb[cat][pkg]
+
+			if metadata.setdefault("EAPI", 0) != portage_const.EAPI:
+				# intentionally wipe keys.
+				metadata = {"EAPI":-metadata["EAPI"]}
+			
+			# fun...
+			# we know that metadata cache entries *always* have all keys
 			self.eclassdb.update_package(mylocation,cat,pkg,metadata["INHERITED"].split())
 			self.auxdb[mylocation][cat][pkg] = metadata
 			self.auxdb[mylocation][cat].sync()
-		elif doregen:
 
+		elif doregen:
 			writemsg("doregen: %s %s\n" % (doregen,mycpv), 2)
 			writemsg("Generating cache entry(0) for: "+str(myebuild)+"\n",1)
 
@@ -5362,6 +5386,14 @@ class portdbapi(dbapi):
 				if mylines[x][-1] == '\n':
 					mylines[x] = mylines[x][:-1]
 				mydata[auxdbkeys[x]] = mylines[x]
+
+			eapi = int(mydata["EAPI"])
+			if eapi > portage_const.EAPI:
+				# if newer version, wipe everything and negate eapi
+				mydata = {}
+				map(lambda x:mydata.setdefault(x, ""), auxdbkeys)
+				mydata["EAPI"] = -eapi
+
 			mydata["_mtime_"] = emtime
 
 			self.auxdb[mylocation][cat][pkg] = mydata
@@ -5373,10 +5405,15 @@ class portdbapi(dbapi):
 		mydata   = self.auxdb[mylocation][cat][pkg]
 		returnme = []
 		for x in mylist:
-			if mydata.has_key(x):
-				returnme.append(mydata[x])
-			else:
-				returnme.append("")
+			returnme.append(mydata.get(x,""))
+
+		if "EAPI" in mylist:
+			idx = mylist.index("EAPI")
+			try:
+				returnme[idx] = abs(int(returnme[idx]))
+			except ValueError:
+				# string
+				returnme[idx] = 0
 
 		return returnme
 
@@ -5625,14 +5662,14 @@ class portdbapi(dbapi):
 			#we need to update this next line when we have fully integrated the new db api
 			auxerr=0
 			try:
-				myaux=db["/"]["porttree"].dbapi.aux_get(mycpv, ["KEYWORDS"])
+				keys, eapi = db["/"]["porttree"].dbapi.aux_get(mycpv, ["KEYWORDS", "EAPI"])
 			except (KeyError,IOError,TypeError):
 				continue
-			if not myaux[0]:
+			if not keys:
 				# KEYWORDS=""
 				#print "!!! No KEYWORDS for "+str(mycpv)+" -- Untested Status"
 				continue
-			mygroups=myaux[0].split()
+			mygroups=keys.split()
 			pgroups=groups[:]
 			match=0
 			cp = dep_getkey(mycpv)
@@ -5660,7 +5697,8 @@ class portdbapi(dbapi):
 			if not match and ((hastesting and "~*" in pgroups) or (hasstable and "*" in pgroups)):
 				match=1
 			if match:
-				newlist.append(mycpv)
+				if eapi == portage_const.EAPI:
+					newlist.append(mycpv)
 		return newlist
 
 class binarytree(packagetree):
diff -purN portage-old/pym/portage_const.py portage/pym/portage_const.py
--- portage-old/pym/portage_const.py	2005-08-29 17:05:10.000000000 -0500
+++ portage/pym/portage_const.py	2005-08-29 17:06:02.000000000 -0500
@@ -43,6 +43,8 @@ CONFIG_MEMORY_FILE      = PRIVATE_PATH +
 INCREMENTALS=["USE","FEATURES","ACCEPT_KEYWORDS","ACCEPT_LICENSE","CONFIG_PROTECT_MASK","CONFIG_PROTECT","PRELINK_PATH","PRELINK_PATH_MASK"]
 STICKIES=["KEYWORDS_ACCEPT","USE","CFLAGS","CXXFLAGS","MAKEOPTS","EXTRA_ECONF","EXTRA_EINSTALL","EXTRA_EMAKE"]
 
+EAPI = 0
+
 # ===========================================================================
 # END OF CONSTANTS -- END OF CONSTANTS -- END OF CONSTANTS -- END OF CONSTANT
 # ===========================================================================
diff -purN portage-old/pym/portage_db_flat.py portage/pym/portage_db_flat.py
--- portage-old/pym/portage_db_flat.py	2005-08-27 05:27:59.000000000 -0500
+++ portage/pym/portage_db_flat.py	2005-08-27 05:27:43.000000000 -0500
@@ -9,6 +9,10 @@ import stat
 
 import portage_db_template
 
+# since this format is massively deprecated, 
+# we're hardcoding the previously weird line count
+magic_line_count = 22
+
 class database(portage_db_template.database):
 	def module_init(self):
 		self.lastkey  = None # Cache
@@ -42,39 +46,39 @@ class database(portage_db_template.datab
 				mykeys += [x]
 		return mykeys
 
-	def get_values(self,key):
+	def get_values(self,key, data=None):
+		""" do not use data unless you know what it does."""
+
 		if not key:
 			raise KeyError, "key is not set to a valid value"
 
-		try:
-			# give buffering a hint of the pretty much maximal cache size we deal with
-			myf = open(self.fullpath+key, "r", 8192)
-		except OSError, oe:
-			# either the file didn't exist, or it was removed under our feet.
-			return None 
-		
+		mydict = {}
+		if data == None:
+			try:
+				# give buffering a hint of the pretty much maximal cache size we deal with
+				myf = open(self.fullpath+key, "r", 8192)
+			except OSError:
+				# either the file didn't exist, or it was removed under our feet.
+				raise KeyError("failed reading key")
+
+			# nuke the newlines right off the batt.
+			data = myf.read().splitlines()
+			mydict["_mtime_"] = os.fstat(myf.fileno()).st_mtime
+			myf.close()
+		else:
+			mydict["_mtime_"] = data.pop(-1)
 
-		# nuke the newlines right off the batt.
-		data = myf.read().splitlines()
-		mdict = {}
-		
 		# rely on exceptions to note differing line counts.
 		try:
-			for x in range(0, len(self.dbkeys)):
-				mdict[self.dbkeys[x]] = data[x]
-
-			# do this now, rather then earlier- possible that earlier it might have been wasted
-			# if key count mismatched
-			mdict["_mtime_"] = os.fstat(myf.fileno()).st_mtime
+			for x in range(magic_line_count):
+				mydict[self.dbkeys[x]] = data[x]
 
 		except IndexError:
-			myf.close()
 			raise ValueError, "Key count mistmatch"
 
-		myf.close()
-		return mdict
+		return mydict
 	
-	def set_values(self,key,val):
+	def set_values(self,key, val, raw=False):
 		if not key:
 			raise KeyError, "No key provided. key:%s val:%s" % (key,val)
 		if not val:
@@ -86,12 +90,19 @@ class database(portage_db_template.datab
 
 		update_fp = self.fullpath + ".update." + str(os.getpid()) + "." + key
 		myf = open(update_fp,"w")
-		myf.writelines( [ val[x] +"\n" for x in self.dbkeys] )
+		if not raw:
+			myf.writelines( [ val[x] +"\n" for x in self.dbkeys] )
+			if len(self.dbkeys) != magic_line_count:
+				myf.writelines(["\n"] * len(self.dbkeys) - magic_line_count)
+			mtime = val["_mtime_"]
+		else:
+			mtime = val.pop(-1)
+			myf.writelines(val)
 		myf.close()
 		
 		os.chown(update_fp, self.uid, self.gid)
 		os.chmod(update_fp, 0664)
-		os.utime(update_fp, (-1,long(val["_mtime_"])))
+		os.utime(update_fp, (-1,long(mtime)))
 		os.rename(update_fp, self.fullpath+key)
 
 	def del_key(self,key):
diff -purN portage-old/pym/portage_db_flat_hash.py portage/pym/portage_db_flat_hash.py
--- portage-old/pym/portage_db_flat_hash.py	1969-12-31 18:00:00.000000000 -0600
+++ portage/pym/portage_db_flat_hash.py	2005-08-27 05:27:43.000000000 -0500
@@ -0,0 +1,38 @@
+# Copyright 2004 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+# $Header: /var/cvsroot/gentoo-src/portage/pym/Attic/portage_db_flat.py,v 1.13.2.6 2005/04/19 07:14:17 ferringb Exp $
+cvs_id_string="$Id: portage_db_flat.py,v 1.13.2.6 2005/04/19 07:14:17 ferringb Exp $"[5:-2]
+
+import portage_db_flat, os
+
+class database(portage_db_flat.database):
+	
+	def get_values(self, key, data=None):
+		""" do not specify data unless you know what it does"""
+		if not key:
+			raise KeyError("key is not valid")
+		
+		if data == None:
+			try:
+				myf = open(self.fullpath + key, "r")
+			except OSError:
+				raise KeyError("failed pulling key")
+
+			data = dict(map(lambda x: x.split("=",1), myf.read().splitlines()))
+			data["_mtime_"] = os.fstat(myf.fileno()).st_mtime
+			myf.close()
+
+		mydict = {}
+		for x in self.dbkeys:
+			mydict[x] = data.get(x, "")
+		mydict["_mtime_"] = long(data["_mtime_"])
+		return mydict
+		
+	def set_values(self, key, values):
+		l = []
+		for x in values.keys():
+			if values[x] and x != "_mtime_":
+				l.append("%s=%s\n" % (x, values[x]))
+		l.append(values["_mtime_"])
+		portage_db_flat.database.set_values(self, key, l, raw=True)
+		
diff -purN portage-old/pym/portage_db_metadata.py portage/pym/portage_db_metadata.py
--- portage-old/pym/portage_db_metadata.py	1969-12-31 18:00:00.000000000 -0600
+++ portage/pym/portage_db_metadata.py	2005-08-27 05:27:43.000000000 -0500
@@ -0,0 +1,49 @@
+# Copyright 2004 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+# $Header: /var/cvsroot/gentoo-src/portage/pym/Attic/portage_db_flat.py,v 1.13.2.6 2005/04/19 07:14:17 ferringb Exp $
+cvs_id_string="$Id: portage_db_flat.py,v 1.13.2.6 2005/04/19 07:14:17 ferringb Exp $"[5:-2]
+
+import os, portage_db_flat_hash, portage_db_flat
+
+class database(portage_db_flat_hash.database):
+	
+	def get_values(self, key):
+		if not key:
+			raise KeyError("key is not valid")
+		
+		try:
+			myf = open(self.fullpath + key, "r")
+		except OSError:
+			raise KeyError("key is not valid")
+		mtime = os.fstat(myf.fileno()).st_mtime
+		data = myf.read().splitlines()
+		
+		# easy attempt first.
+		if len(data) != portage_db_flat.magic_line_count:
+			d = dict(map(lambda x: x.split("=",1), data))
+			d["_mtime_"] = mtime
+			return portage_db_flat_hash.database.get_values(self, key, d)
+		# this one's interesting.
+		d = {}
+
+		for line in data:
+			# yes, meant to iterate over a string.
+			hashed = False
+			for idx, c in enumerate(line):
+				if not c.isalpha():
+					if c == "=" and idx > 0:
+						hashed = True
+						d[line[:idx]] = line[idx + 1:]
+					elif c == "_" or c.isdigit():
+						continue
+					break
+				elif not c.isupper():
+					break
+
+			if not hashed:
+				# non hashed.
+				data.append(mtime)
+				return portage_db_flat.database.get_values(self, key, data=data)
+
+		d["_mtime_"] = mtime
+		return portage_db_flat_hash.database.get_values(self, key, data=d)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Brian Harring]

[-- Attachment #1.1: Type: text/plain, Size: 391 bytes --]

Round 3, fixed all uglyness.
You *will* see uglyness for the changeover from flat_list to flat_hash 
if you're setting portdbapi.auxdbmodule to flat_hash, but that's a one 
time hit, and is the reason we blow away the cache on portage 
upgrades.

Either way, full patch, just correction of a few instances where it 
user visible warnings were popping up, but not required.
~harring

[-- Attachment #1.2: 2.0.51-eapi-awareness-3.patch --]
[-- Type: text/plain, Size: 13954 bytes --]

diff -purN portage-old/bin/ebuild.sh portage/bin/ebuild.sh
--- portage-old/bin/ebuild.sh	2005-08-29 02:16:41.000000000 -0500
+++ portage/bin/ebuild.sh	2005-08-29 17:02:00.000000000 -0500
@@ -1831,6 +1831,7 @@ for myarg in $*; do
 		echo `echo "$CDEPEND"`     >> $dbkey
 		echo `echo "$PDEPEND"`     >> $dbkey
 		echo `echo "$PROVIDE"`     >> $dbkey
+		echo `echo "${EAPI:-0}"`   >> $dbkey
 		echo `echo "$UNUSED_01"`   >> $dbkey
 		echo `echo "$UNUSED_02"`   >> $dbkey
 		echo `echo "$UNUSED_03"`   >> $dbkey
@@ -1838,7 +1839,6 @@ for myarg in $*; do
 		echo `echo "$UNUSED_05"`   >> $dbkey
 		echo `echo "$UNUSED_06"`   >> $dbkey
 		echo `echo "$UNUSED_07"`   >> $dbkey
-		echo `echo "$UNUSED_08"`   >> $dbkey
 		set +f
 		#make sure it is writable by our group:
 		exit 0
diff -purN portage-old/pym/portage.py portage/pym/portage.py
--- portage-old/pym/portage.py	2005-08-27 05:28:02.000000000 -0500
+++ portage/pym/portage.py	2005-08-29 19:28:34.000000000 -0500
@@ -81,7 +81,7 @@ try:
 	  MOVE_BINARY, PRELINK_BINARY, WORLD_FILE, MAKE_CONF_FILE, MAKE_DEFAULTS_FILE, \
 	  DEPRECATED_PROFILE_FILE, USER_VIRTUALS_FILE, EBUILD_SH_ENV_FILE, \
 	  INVALID_ENV_FILE, CUSTOM_MIRRORS_FILE, SANDBOX_PIDS_FILE, CONFIG_MEMORY_FILE,\
-	  INCREMENTALS, STICKIES
+	  INCREMENTALS, STICKIES, EAPI
 
 	from portage_data import ostype, lchown, userland, secpass, uid, wheelgid, \
 	                         portage_uid, portage_gid
@@ -2402,6 +2396,15 @@ def doebuild(myebuild,mydo,myroot,mysett
 			raise
 		except:
 			pass
+		try:
+			eapi = abs(db[root][tree].dbapi.aux_get(mycpv, ["EAPI"])[0])
+			if portage_const.EAPI != eapi:
+				# can't do anything with this.
+				raise Exception("Unable to do any operations on '%s', due to the fact it's EAPI is higher then this portage versions.  Please upgrade to a portage version that supports EAPI %i" % (mycpv, eapi))
+		except SystemExit, e:
+			raise
+		except Exception, e:
+			raise Exception("Unable to pull EAPI from cpv %s, tree %s; can't confirm that it's supported by this portage, thus unable to merge it: Exception was '%s'" % (mycpv, tree, e))
 
 	if mysplit[2] == "r0":
 		mysettings["PVR"]=mysplit[1]
@@ -3911,7 +3914,10 @@ def getmaskingstatus(mycpv):
 					rValue.append("package.mask")
 
 	# keywords checking
-	mygroups = portdb.aux_get(mycpv, ["KEYWORDS"])[0].split()
+	mygroups, eapi = portdb.aux_get(mycpv, ["KEYWORDS", "EAPI"])
+	if abs(eapi) > portage_const.EAPI:
+		return ["required EAPI %i, supported EAPI %i" % (eapi, portage_const.EAPI)]
+	mygroups = mygroups.split()
 	pgroups=groups[:]
 	myarch = settings["ARCH"]
 	pkgdict = settings.pkeywordsdict
@@ -4531,7 +4537,12 @@ class bindbapi(fakedbapi):
 				else:
 					myval = string.join(myval.split(),' ')
 				mylist.append(myval)
-
+		if "EAPI" in wants:
+			idx = wants.index("EAPI")
+			try:
+				mylist[idx] = abs(int(mylist[idx]))
+			except ValueError:
+				mylist[idx] = 0
 		return mylist
 
 
@@ -4811,6 +4822,12 @@ class vardbapi(dbapi):
 			else:
 				myd = ""
 			results.append(myd)
+		if "EAPI" in wants:
+			idx = wants.index("EAPI")
+			try:
+				results[idx] = abs(int(wants[idx]))
+			except ValueError:
+				results[idx] = 0 
 		return results
 
 
@@ -5077,9 +5094,9 @@ auxdbkeys=[
   'DEPEND',    'RDEPEND',   'SLOT',      'SRC_URI',
 	'RESTRICT',  'HOMEPAGE',  'LICENSE',   'DESCRIPTION',
 	'KEYWORDS',  'INHERITED', 'IUSE',      'CDEPEND',
-	'PDEPEND',   'PROVIDE',
+	'PDEPEND',   'PROVIDE', 'EAPI',
 	'UNUSED_01', 'UNUSED_02', 'UNUSED_03', 'UNUSED_04',
-	'UNUSED_05', 'UNUSED_06', 'UNUSED_07', 'UNUSED_08',
+	'UNUSED_05', 'UNUSED_06', 'UNUSED_07',
 	]
 auxdbkeylen=len(auxdbkeys)
 
@@ -5293,11 +5310,22 @@ class portdbapi(dbapi):
 		# we use cache files, which is usually on /usr/portage/metadata/cache/.
 		if doregen and mylocation==self.mysettings["PORTDIR"] and metacachedir and self.metadb[cat].has_key(pkg):
 			metadata=self.metadb[cat][pkg]
-			self.eclassdb.update_package(mylocation,cat,pkg,metadata["INHERITED"].split())
+			try:
+				eapi = int(metadata["EAPI"])
+			except (KeyError, ValueError):
+				eapi = 0
+				metadata["EAPI"] = eapi
+			if eapi != portage_const.EAPI:
+				# intentionally wipe keys.
+				metadata = {"EAPI":-eapi}
+			else:
+				# eclass updates only if we haven't nuked the entry.
+				self.eclassdb.update_package(mylocation,cat,pkg,metadata["INHERITED"].split())
+
 			self.auxdb[mylocation][cat][pkg] = metadata
 			self.auxdb[mylocation][cat].sync()
-		elif doregen:
 
+		elif doregen:
 			writemsg("doregen: %s %s\n" % (doregen,mycpv), 2)
 			writemsg("Generating cache entry(0) for: "+str(myebuild)+"\n",1)
 
@@ -5362,6 +5390,14 @@ class portdbapi(dbapi):
 				if mylines[x][-1] == '\n':
 					mylines[x] = mylines[x][:-1]
 				mydata[auxdbkeys[x]] = mylines[x]
+
+			eapi = int(mydata["EAPI"])
+			if eapi > portage_const.EAPI:
+				# if newer version, wipe everything and negate eapi
+				mydata = {}
+				map(lambda x:mydata.setdefault(x, ""), auxdbkeys)
+				mydata["EAPI"] = -eapi
+
 			mydata["_mtime_"] = emtime
 
 			self.auxdb[mylocation][cat][pkg] = mydata
@@ -5373,10 +5409,15 @@ class portdbapi(dbapi):
 		mydata   = self.auxdb[mylocation][cat][pkg]
 		returnme = []
 		for x in mylist:
-			if mydata.has_key(x):
-				returnme.append(mydata[x])
-			else:
-				returnme.append("")
+			returnme.append(mydata.get(x,""))
+
+		if "EAPI" in mylist:
+			idx = mylist.index("EAPI")
+			try:
+				returnme[idx] = abs(int(returnme[idx]))
+			except ValueError:
+				# string
+				returnme[idx] = 0
 
 		return returnme
 
@@ -5625,14 +5666,14 @@ class portdbapi(dbapi):
 			#we need to update this next line when we have fully integrated the new db api
 			auxerr=0
 			try:
-				myaux=db["/"]["porttree"].dbapi.aux_get(mycpv, ["KEYWORDS"])
+				keys, eapi = db["/"]["porttree"].dbapi.aux_get(mycpv, ["KEYWORDS", "EAPI"])
 			except (KeyError,IOError,TypeError):
 				continue
-			if not myaux[0]:
+			if not keys:
 				# KEYWORDS=""
 				#print "!!! No KEYWORDS for "+str(mycpv)+" -- Untested Status"
 				continue
-			mygroups=myaux[0].split()
+			mygroups=keys.split()
 			pgroups=groups[:]
 			match=0
 			cp = dep_getkey(mycpv)
@@ -5660,7 +5701,8 @@ class portdbapi(dbapi):
 			if not match and ((hastesting and "~*" in pgroups) or (hasstable and "*" in pgroups)):
 				match=1
 			if match:
-				newlist.append(mycpv)
+				if eapi == portage_const.EAPI:
+					newlist.append(mycpv)
 		return newlist
 
 class binarytree(packagetree):
diff -purN portage-old/pym/portage_const.py portage/pym/portage_const.py
--- portage-old/pym/portage_const.py	2005-08-29 17:05:10.000000000 -0500
+++ portage/pym/portage_const.py	2005-08-29 17:06:02.000000000 -0500
@@ -43,6 +43,8 @@ CONFIG_MEMORY_FILE      = PRIVATE_PATH +
 INCREMENTALS=["USE","FEATURES","ACCEPT_KEYWORDS","ACCEPT_LICENSE","CONFIG_PROTECT_MASK","CONFIG_PROTECT","PRELINK_PATH","PRELINK_PATH_MASK"]
 STICKIES=["KEYWORDS_ACCEPT","USE","CFLAGS","CXXFLAGS","MAKEOPTS","EXTRA_ECONF","EXTRA_EINSTALL","EXTRA_EMAKE"]
 
+EAPI = 0
+
 # ===========================================================================
 # END OF CONSTANTS -- END OF CONSTANTS -- END OF CONSTANTS -- END OF CONSTANT
 # ===========================================================================
diff -purN portage-old/pym/portage_db_flat.py portage/pym/portage_db_flat.py
--- portage-old/pym/portage_db_flat.py	2005-08-27 05:27:59.000000000 -0500
+++ portage/pym/portage_db_flat.py	2005-08-29 19:34:32.000000000 -0500
@@ -9,6 +9,10 @@ import stat
 
 import portage_db_template
 
+# since this format is massively deprecated, 
+# we're hardcoding the previously weird line count
+magic_line_count = 22
+
 class database(portage_db_template.database):
 	def module_init(self):
 		self.lastkey  = None # Cache
@@ -42,39 +46,39 @@ class database(portage_db_template.datab
 				mykeys += [x]
 		return mykeys
 
-	def get_values(self,key):
+	def get_values(self,key, data=None):
+		""" do not use data unless you know what it does."""
+
 		if not key:
 			raise KeyError, "key is not set to a valid value"
 
-		try:
-			# give buffering a hint of the pretty much maximal cache size we deal with
-			myf = open(self.fullpath+key, "r", 8192)
-		except OSError, oe:
-			# either the file didn't exist, or it was removed under our feet.
-			return None 
-		
+		mydict = {}
+		if data == None:
+			try:
+				# give buffering a hint of the pretty much maximal cache size we deal with
+				myf = open(self.fullpath+key, "r", 8192)
+			except OSError:
+				# either the file didn't exist, or it was removed under our feet.
+				raise KeyError("failed reading key")
+
+			# nuke the newlines right off the batt.
+			data = myf.read().splitlines()
+			mydict["_mtime_"] = os.fstat(myf.fileno()).st_mtime
+			myf.close()
+		else:
+			mydict["_mtime_"] = data.pop(-1)
 
-		# nuke the newlines right off the batt.
-		data = myf.read().splitlines()
-		mdict = {}
-		
 		# rely on exceptions to note differing line counts.
 		try:
-			for x in range(0, len(self.dbkeys)):
-				mdict[self.dbkeys[x]] = data[x]
-
-			# do this now, rather then earlier- possible that earlier it might have been wasted
-			# if key count mismatched
-			mdict["_mtime_"] = os.fstat(myf.fileno()).st_mtime
+			for x in range(magic_line_count):
+				mydict[self.dbkeys[x]] = data[x]
 
 		except IndexError:
-			myf.close()
 			raise ValueError, "Key count mistmatch"
 
-		myf.close()
-		return mdict
+		return mydict
 	
-	def set_values(self,key,val):
+	def set_values(self,key, val, raw=False):
 		if not key:
 			raise KeyError, "No key provided. key:%s val:%s" % (key,val)
 		if not val:
@@ -86,12 +90,19 @@ class database(portage_db_template.datab
 
 		update_fp = self.fullpath + ".update." + str(os.getpid()) + "." + key
 		myf = open(update_fp,"w")
-		myf.writelines( [ val[x] +"\n" for x in self.dbkeys] )
+		if not raw:
+			myf.writelines( [ str(val[x]) +"\n" for x in self.dbkeys] )
+			if len(self.dbkeys) != magic_line_count:
+				myf.writelines(["\n"] * len(self.dbkeys) - magic_line_count)
+			mtime = val["_mtime_"]
+		else:
+			mtime = val.pop(-1)
+			myf.writelines(val)
 		myf.close()
 		
 		os.chown(update_fp, self.uid, self.gid)
 		os.chmod(update_fp, 0664)
-		os.utime(update_fp, (-1,long(val["_mtime_"])))
+		os.utime(update_fp, (-1,long(mtime)))
 		os.rename(update_fp, self.fullpath+key)
 
 	def del_key(self,key):
diff -purN portage-old/pym/portage_db_flat_hash.py portage/pym/portage_db_flat_hash.py
--- portage-old/pym/portage_db_flat_hash.py	1969-12-31 18:00:00.000000000 -0600
+++ portage/pym/portage_db_flat_hash.py	2005-08-27 05:27:43.000000000 -0500
@@ -0,0 +1,38 @@
+# Copyright 2004 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+# $Header: /var/cvsroot/gentoo-src/portage/pym/Attic/portage_db_flat.py,v 1.13.2.6 2005/04/19 07:14:17 ferringb Exp $
+cvs_id_string="$Id: portage_db_flat.py,v 1.13.2.6 2005/04/19 07:14:17 ferringb Exp $"[5:-2]
+
+import portage_db_flat, os
+
+class database(portage_db_flat.database):
+	
+	def get_values(self, key, data=None):
+		""" do not specify data unless you know what it does"""
+		if not key:
+			raise KeyError("key is not valid")
+		
+		if data == None:
+			try:
+				myf = open(self.fullpath + key, "r")
+			except OSError:
+				raise KeyError("failed pulling key")
+
+			data = dict(map(lambda x: x.split("=",1), myf.read().splitlines()))
+			data["_mtime_"] = os.fstat(myf.fileno()).st_mtime
+			myf.close()
+
+		mydict = {}
+		for x in self.dbkeys:
+			mydict[x] = data.get(x, "")
+		mydict["_mtime_"] = long(data["_mtime_"])
+		return mydict
+		
+	def set_values(self, key, values):
+		l = []
+		for x in values.keys():
+			if values[x] and x != "_mtime_":
+				l.append("%s=%s\n" % (x, values[x]))
+		l.append(values["_mtime_"])
+		portage_db_flat.database.set_values(self, key, l, raw=True)
+		
diff -purN portage-old/pym/portage_db_metadata.py portage/pym/portage_db_metadata.py
--- portage-old/pym/portage_db_metadata.py	1969-12-31 18:00:00.000000000 -0600
+++ portage/pym/portage_db_metadata.py	2005-08-27 05:27:43.000000000 -0500
@@ -0,0 +1,49 @@
+# Copyright 2004 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+# $Header: /var/cvsroot/gentoo-src/portage/pym/Attic/portage_db_flat.py,v 1.13.2.6 2005/04/19 07:14:17 ferringb Exp $
+cvs_id_string="$Id: portage_db_flat.py,v 1.13.2.6 2005/04/19 07:14:17 ferringb Exp $"[5:-2]
+
+import os, portage_db_flat_hash, portage_db_flat
+
+class database(portage_db_flat_hash.database):
+	
+	def get_values(self, key):
+		if not key:
+			raise KeyError("key is not valid")
+		
+		try:
+			myf = open(self.fullpath + key, "r")
+		except OSError:
+			raise KeyError("key is not valid")
+		mtime = os.fstat(myf.fileno()).st_mtime
+		data = myf.read().splitlines()
+		
+		# easy attempt first.
+		if len(data) != portage_db_flat.magic_line_count:
+			d = dict(map(lambda x: x.split("=",1), data))
+			d["_mtime_"] = mtime
+			return portage_db_flat_hash.database.get_values(self, key, d)
+		# this one's interesting.
+		d = {}
+
+		for line in data:
+			# yes, meant to iterate over a string.
+			hashed = False
+			for idx, c in enumerate(line):
+				if not c.isalpha():
+					if c == "=" and idx > 0:
+						hashed = True
+						d[line[:idx]] = line[idx + 1:]
+					elif c == "_" or c.isdigit():
+						continue
+					break
+				elif not c.isupper():
+					break
+
+			if not hashed:
+				# non hashed.
+				data.append(mtime)
+				return portage_db_flat.database.get_values(self, key, data=data)
+
+		d["_mtime_"] = mtime
+		return portage_db_flat_hash.database.get_values(self, key, data=d)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 841 bytes --]

On Monday 29 August 2005 22:52, Zac Medico wrote:
> Brian Harring wrote:
> > Please test this out; if you want to test the EAPI checking, tag
> > EAPI=1 into an ebuild, and try making emerge bail.
>
> Well, it bails too often. :)
>
> It seems that an explicit integer conversion is needed for > and <
> comparisons with mixed types
> (2.0.51-eapi-awareness-emerge-int-conversion.patch).

I don't think it is a wise path to interpret EAPI's as integers at all. 
There should not be guarantees of forward or backward compatibility 
between versions. Interpreting them as integers seems to imply that. 
Basically I think that EAPI could be anything wanted, similar to SLOTS. 
Of course using digits is easy for humans.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Marius Mauch]

On 08/30/05  Paul de Vrieze wrote:

> On Monday 29 August 2005 22:52, Zac Medico wrote:
> > Brian Harring wrote:
> > > Please test this out; if you want to test the EAPI checking, tag
> > > EAPI=1 into an ebuild, and try making emerge bail.
> >
> > Well, it bails too often. :)
> >
> > It seems that an explicit integer conversion is needed for > and <
> > comparisons with mixed types
> > (2.0.51-eapi-awareness-emerge-int-conversion.patch).
> 
> I don't think it is a wise path to interpret EAPI's as integers at
> all.  There should not be guarantees of forward or backward
> compatibility  between versions. Interpreting them as integers seems
> to imply that.  Basically I think that EAPI could be anything wanted,
> similar to SLOTS.  Of course using digits is easy for humans.

I've to agree with Paul here, portage should hold a list of all EAPI
values it understands, not a maximum value.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 1281 bytes --]

On Tue, Aug 30, 2005 at 12:38:16PM +0200, Marius Mauch wrote:
> On 08/30/05  Paul de Vrieze wrote:
> 
> > On Monday 29 August 2005 22:52, Zac Medico wrote:
> > > Brian Harring wrote:
> > > > Please test this out; if you want to test the EAPI checking, tag
> > > > EAPI=1 into an ebuild, and try making emerge bail.
> > >
> > > Well, it bails too often. :)
> > >
> > > It seems that an explicit integer conversion is needed for > and <
> > > comparisons with mixed types
> > > (2.0.51-eapi-awareness-emerge-int-conversion.patch).
> > 
> > I don't think it is a wise path to interpret EAPI's as integers at
> > all.  There should not be guarantees of forward or backward
> > compatibility  between versions. Interpreting them as integers seems
> > to imply that.  Basically I think that EAPI could be anything wanted,
> > similar to SLOTS.  Of course using digits is easy for humans.
> 
> I've to agree with Paul here, portage should hold a list of all EAPI
> values it understands, not a maximum value.
Stable knows of one version- so containment check versus > is moot; 
negated values are disallowed for eapi anyways.
Note that the rewrite will do containment tests instead, but rewrite 
will also support eapi1 and eapi0 rather then just one.
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Marius Mauch]

On 08/30/05  Brian Harring wrote:

> On Tue, Aug 30, 2005 at 12:38:16PM +0200, Marius Mauch wrote:
> > On 08/30/05  Paul de Vrieze wrote:
> > 
> > > On Monday 29 August 2005 22:52, Zac Medico wrote:
> > > > Brian Harring wrote:
> > > > > Please test this out; if you want to test the EAPI checking,
> > > > > tag EAPI=1 into an ebuild, and try making emerge bail.
> > > >
> > > > Well, it bails too often. :)
> > > >
> > > > It seems that an explicit integer conversion is needed for > and
> > > > < comparisons with mixed types
> > > > (2.0.51-eapi-awareness-emerge-int-conversion.patch).
> > > 
> > > I don't think it is a wise path to interpret EAPI's as integers at
> > > all.  There should not be guarantees of forward or backward
> > > compatibility  between versions. Interpreting them as integers
> > > seems to imply that.  Basically I think that EAPI could be
> > > anything wanted, similar to SLOTS.  Of course using digits is easy
> > > for humans.
> > 
> > I've to agree with Paul here, portage should hold a list of all EAPI
> > values it understands, not a maximum value.
> Stable knows of one version- so containment check versus > is moot; 
> negated values are disallowed for eapi anyways.
> Note that the rewrite will do containment tests instead, but rewrite 
> will also support eapi1 and eapi0 rather then just one.

Problem is that you then rely on python always evaluating "somestring" >
0 as True which I don't think is a good idea (it holds true even for "0"
> 0), if you treat 0 as a string you get problems (as some strings are
> "smaller" than 0) and you can't convert all strings to ints.

What's the point of using > anyway?

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 1418 bytes --]

On Tue, Aug 30, 2005 at 03:15:15PM +0200, Marius Mauch wrote:
> On 08/30/05  Brian Harring wrote:
> Problem is that you then rely on python always evaluating "somestring" >
> 0 as True which I don't think is a good idea (it holds true even for "0"
> > 0), if you treat 0 as a string you get problems (as some strings are
> > "smaller" than 0) and you can't convert all strings to ints.

In the initial proposal of EAPI, it was intended as version ebuild 
format #; think format specifications.

Float is probably better, but strings being slipped in I don't see any 
good reason to allow, nor has it been requrested (it was 
specifically ixnayed when the idea was hashed out actually).

Aside from that, again, stable is capable of a single eapi version; if 
it's a string, the code catches the value error and knocks it down to 
eapi0 due to the reasons described above.

Further reason why string is a no go indicated in the code; if eapi0 
portage tries regening an eapi1 cache entry, it stores negated eapi 
version with no other metadata.  Allowing strings nukes that approach, 
unless you disallow - as the first character (which would be 
demonstration of strings not being incredibly well suited for eapi 
settings imo).

> What's the point of using > anyway?
Simplicity in the code right now, since stable will *never* support 
anything but eapi0.  It's an easy check.

~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 3707 bytes --]

On Tuesday 30 August 2005 15:28, Brian Harring wrote:
> On Tue, Aug 30, 2005 at 03:15:15PM +0200, Marius Mauch wrote:
> > On 08/30/05  Brian Harring wrote:
> > Problem is that you then rely on python always evaluating
> > "somestring" > 0 as True which I don't think is a good idea (it holds
> > true even for "0"
> >
> > > 0), if you treat 0 as a string you get problems (as some strings
> > > are "smaller" than 0) and you can't convert all strings to ints.
>
> In the initial proposal of EAPI, it was intended as version ebuild
> format #; think format specifications.
>
> Float is probably better, but strings being slipped in I don't see any
> good reason to allow, nor has it been requrested (it was
> specifically ixnayed when the idea was hashed out actually).

What the format of the values of EAPI is is irrelevant. The matter is that 
there is no way to know that version 5 is in any way compatible with 
version 4, which in turn is also incompatible with version 3. Seeing 
versions as numbers would imply an ordering in the versions. Such an 
ordering would seem to imply compatibility or something the like, while 
versions could differ significantly.

As any portage version should still check which EAPI the ebuild has to be 
able to support it correctly any range checks would be moot anyway.

About the ordering, what if there was some new format that would enable to 
very easilly write packages for perl modules but sucked at anything else? 
This would be a different format alltogether, not a different version.

If EAPI's were more than just strings, at some point it might even be 
possible to associate packages with EAPI's so that portage would 
automatically signal the EAPI was unsupported, find the relevant package 
and merge the plugin. Restart itself and merge the ebuild that needed the 
new EAPI. The plugin ebuild could ensure that the portage version is 
compatible.

> Aside from that, again, stable is capable of a single eapi version; if
> it's a string, the code catches the value error and knocks it down to
> eapi0 due to the reasons described above.

Then the code is broken. It should check whether the value is "0" or the 
empty string "" or unset. If it is the latter it should change it to "0", 
if it is anything else it should just treat it as an unsupported format 
and ignore that particular ebuild. (and try to use other versions of the 
package that are still in the old format).

>
> Further reason why string is a no go indicated in the code; if eapi0
> portage tries regening an eapi1 cache entry, it stores negated eapi
> version with no other metadata.  Allowing strings nukes that approach,
> unless you disallow - as the first character (which would be
> demonstration of strings not being incredibly well suited for eapi
> settings imo).

As long as it does not generate "-0" this broken behaviour should not 
break anything as long as portage 2.1 does a forced regen or cache 
evaluation on installation (when detecting a 2.0 version present) or 
first run.

>
> > What's the point of using > anyway?
>
> Simplicity in the code right now, since stable will *never* support
> anything but eapi0.  It's an easy check.

Is it really harder to check for either "" or unset or "0". That should be 
very little extra code, and would be correct regardless of future 
decisions regarding EAPI.

EAPI should be left as free as possible, portage just needs to notice that 
it doesn't recognize the value. If this would be in a not-a-number 
exception that would be ok too.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Marius Mauch]

On 08/30/05  Brian Harring wrote:

> > What's the point of using > anyway?
> Simplicity in the code right now, since stable will *never* support 
> anything but eapi0.  It's an easy check.

You really want to tell me that you consider
   if myeapi > 0:
as simpler than
   EAPI_COMPATIBLE="0"
   if myeapi in ['']+EAPI_COMPATIBLE.split():
?

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Marius Mauch]

On 08/29/05  Brian Harring wrote:

> Somebody care to split a masking patch for stable rather then the 
> emerge modifications I did btw?  I'm poking at ensuring an eapi=0 
> portage's generated eapi=1 cache entries are not used by an eapi=1 
> portage without a forced regeneration atm.

Well, the masking part should apply to stable as well as AFAIK those
code regions haven't changed.

> That and the fact the 2.1 state should be decided, if we're going to 
> have (effectively) two branches of development going at once, vs 
> developmental line and maintenance branch.

Well, basically I wanted to deploy elog and Jason mentioned some other
changes as well, and while talking about it we couldn't find much in
head that we didn't want out. Unfortunately you weren't around at that
time. Also I think we could use 2.1 to add all the hacks we need for
transitioning (like the EAPI and Manifest stuff).
However recently I met at least one nasty bug in the 2.1 dep resolver
which (if I understood right) Jason classified as more or less
unfixable, so I'm not that sure how viable 2.1 is anymore.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Jason Stubbs]

[-- Attachment #1: Type: text/plain, Size: 474 bytes --]

On Wednesday 31 August 2005 02:46, Marius Mauch wrote:
> However recently I met at least one nasty bug in the 2.1 dep resolver
> which (if I understood right) Jason classified as more or less
> unfixable, so I'm not that sure how viable 2.1 is anymore.

This same bug is in stable as well. The difference is that 2.1 keeps adding 
deps whereas 2.0 skips out the first chance it gets. That, for all intense 
purposes, completely masks the issue.

-- 
Jason Stubbs

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 5387 bytes --]

On Tue, Aug 30, 2005 at 04:47:51PM +0200, Paul de Vrieze wrote:
> On Tuesday 30 August 2005 15:28, Brian Harring wrote:
> > On Tue, Aug 30, 2005 at 03:15:15PM +0200, Marius Mauch wrote:
> > > On 08/30/05  Brian Harring wrote:
> > > Problem is that you then rely on python always evaluating
> > > "somestring" > 0 as True which I don't think is a good idea (it holds
> > > true even for "0"
> > >
> > > > 0), if you treat 0 as a string you get problems (as some strings
> > > > are "smaller" than 0) and you can't convert all strings to ints.
> >
> > In the initial proposal of EAPI, it was intended as version ebuild
> > format #; think format specifications.
> >
> > Float is probably better, but strings being slipped in I don't see any
> > good reason to allow, nor has it been requrested (it was
> > specifically ixnayed when the idea was hashed out actually).
> 
> What the format of the values of EAPI is is irrelevant. The matter is that 
> there is no way to know that version 5 is in any way compatible with 
> version 4, which in turn is also incompatible with version 3. Seeing 
> versions as numbers would imply an ordering in the versions. Such an 
> ordering would seem to imply compatibility or something the like, while 
> versions could differ significantly.

There actually is an ordering to the versions though- they're going to 
map out to later portage releases (eapi3 release's portage version >= 
eapi4 release's portage version >= eapi5 releases' portage version).

Why am I being so damned stubborn about numbers for this?  Cause I 
don't want users having to dink around with knowing that eapi="really 
cool version" somehow maps out to 3.1.  Further, eapi version *likely* 
will wind up as the slotting for any split out portage-ebuild package, 
and slots traditionally have always been ints.


> About the ordering, what if there was some new format that would enable to 
> very easilly write packages for perl modules but sucked at anything else? 
> This would be a different format alltogether, not a different version.
> 
> If EAPI's were more than just strings, at some point it might even be 
> possible to associate packages with EAPI's so that portage would 
> automatically signal the EAPI was unsupported, find the relevant package 
> and merge the plugin. Restart itself and merge the ebuild that needed the 
> new EAPI. The plugin ebuild could ensure that the portage version is 
> compatible.

The slotting comment above is in regard to it.  People will hit 
annoyances along the way, but slot==EAPI for portage-ebuild ought to 
allow a sane resolver to dig it's way back.

Won't work for stable, due to the fact stable's resolver isn't smart 
enough, but rewrite's should support the "lets hop from portage 
version to portage version in upgrading" bit.

Aside from that, EAPI is specific to *ebuilds*.  New format is a 
seperate beast that shouldn't be shoe horned into ebuilds imo.


> > Aside from that, again, stable is capable of a single eapi version; if
> > it's a string, the code catches the value error and knocks it down to
> > eapi0 due to the reasons described above.
> 
> Then the code is broken. It should check whether the value is "0" or the 
> empty string "" or unset. If it is the latter it should change it to "0", 
> if it is anything else it should just treat it as an unsupported format 
> and ignore that particular ebuild. (and try to use other versions of the 
> package that are still in the old format).

Yielding on this, since I agree on it.  Will update the next round of 
patches so it's stricter.


> > Further reason why string is a no go indicated in the code; if eapi0
> > portage tries regening an eapi1 cache entry, it stores negated eapi
> > version with no other metadata.  Allowing strings nukes that approach,
> > unless you disallow - as the first character (which would be
> > demonstration of strings not being incredibly well suited for eapi
> > settings imo).
> 
> As long as it does not generate "-0" this broken behaviour should not 
> break anything as long as portage 2.1 does a forced regen or cache 
> evaluation on installation (when detecting a 2.0 version present) or 
> first run.

Wouldn't call it broken behaviour; the reasoning behind doing this is 
that people, regardless of any warning we give them, *will* have 
caches that are used by multiple portage versions.  Bad setups will 
results in older portage installations updating a central cache; the 
negating allows for older portage installations and newer to play nice 
in the cache, not pissing each other off as occurs with stable and 
>=2.1 (or any cache change for that matter).
Granted that's a combination of flat_hash and eapi, but eapi's 
negation still is a major factor in the compatibility.


> Is it really harder to check for either "" or unset or "0". That should be 
> very little extra code, and would be correct regardless of future 
> decisions regarding EAPI.
> 
> EAPI should be left as free as possible, portage just needs to notice that 
> it doesn't recognize the value. If this would be in a not-a-number 
> exception that would be ok too.

Flattening the EAPI version down to 0 is wrong; re: "" existing, it 
gets corrected by the layers that wrap the cache so it always winds up 
as a number.


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 1623 bytes --]

On Tue, Aug 30, 2005 at 07:46:24PM +0200, Marius Mauch wrote:
> On 08/29/05  Brian Harring wrote:
> > That and the fact the 2.1 state should be decided, if we're going to 
> > have (effectively) two branches of development going at once, vs 
> > developmental line and maintenance branch.
> 
> Well, basically I wanted to deploy elog and Jason mentioned some other
> changes as well, and while talking about it we couldn't find much in
> head that we didn't want out. Unfortunately you weren't around at that
> time. Also I think we could use 2.1 to add all the hacks we need for
> transitioning (like the EAPI and Manifest stuff).

I'd rather tag the hacks into stable release, as what the EAPI 
patch is intended to do.  Reasoning is pretty straightforward; I trust 
stable code to hold less user visible bugs now, then what 2.1 would 
hold- stable has been shoved through the ringer; 2.1 hasn't been 
shoved through a comparable ringer.  Further, if we're tagging 
compatibility hacks for 2.0.51 -> 3.0, the thing that matters is the 
compatibility additions, not extra (potentially buggy) features.

Don't get me wrong- I'm still watching 2.1 bugs, but mainly for 
correction of stuff w/in rewrite.

2.1 *could* be made into a full release line, I just am convinced the 
time to do so has come and gone already.  Rewrite isn't complete, 
but the base of it is saner then 2.x's, and people (beyond me) are 
actively working on it.

Further, people are sniffing around re: capabilities the rewrite has 
natively, N portdir's for example for the -osx crew.

My 2 cents, at least.
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 6648 bytes --]

On Wednesday 31 August 2005 01:46, Brian Harring wrote:
> > What the format of the values of EAPI is is irrelevant. The matter is
> > that there is no way to know that version 5 is in any way compatible
> > with version 4, which in turn is also incompatible with version 3.
> > Seeing versions as numbers would imply an ordering in the versions.
> > Such an ordering would seem to imply compatibility or something the
> > like, while versions could differ significantly.
>
> There actually is an ordering to the versions though- they're going to
> map out to later portage releases (eapi3 release's portage version >=
> eapi4 release's portage version >= eapi5 releases' portage version).

Of course there is some kind of ordering. My point is that this ordering 
should be for users only, but portage should totally disregard this.

> Why am I being so damned stubborn about numbers for this?  Cause I
> don't want users having to dink around with knowing that eapi="really
> cool version" somehow maps out to 3.1.  Further, eapi version *likely*
> will wind up as the slotting for any split out portage-ebuild package,
> and slots traditionally have always been ints.

I don't mind you using numbers or something like "3.1-strict" meaning that 
it was introduced with portage 3.1. Just do it in such a way that when in 
the future it is decided that text can be in EAPI specifiers, portage 
will not choke on newer ebuilds that do that.

> > If EAPI's were more than just strings, at some point it might even be
> > possible to associate packages with EAPI's so that portage would
> > automatically signal the EAPI was unsupported, find the relevant
> > package and merge the plugin. Restart itself and merge the ebuild
> > that needed the new EAPI. The plugin ebuild could ensure that the
> > portage version is compatible.
>
> The slotting comment above is in regard to it.  People will hit
> annoyances along the way, but slot==EAPI for portage-ebuild ought to
> allow a sane resolver to dig it's way back.

My main focus is not on automatic finding of compatible versions of 
portage for an EAPI value. It is on ensuring forward compatibility. This 
is easiest when portage makes very few assumptions on the contents of 
EAPI.

> Won't work for stable, due to the fact stable's resolver isn't smart
> enough, but rewrite's should support the "lets hop from portage
> version to portage version in upgrading" bit.
>
> Aside from that, EAPI is specific to *ebuilds*.  New format is a
> seperate beast that shouldn't be shoe horned into ebuilds imo.

EAPI defines which format the ebuild/eclass has. Whether that is a small 
change or a big change is not that relevant. Currently the formats all 
have bash-compatible syntax, and I don't argue to remove that. I just 
want to keep it possible as long as EAPI is recognizable, and bash 
doesn't bail out on it too early. Hence the reason that I argue that EAPI 
should be the first item after the whitelines and comments in an ebuild. 
This allows *fast* and failsafe detection of the required EAPI. Even when 
what follows is totally incompatible with the default parser. (bash 
is/contains also a parser)

> > > Further reason why string is a no go indicated in the code; if
> > > eapi0 portage tries regening an eapi1 cache entry, it stores
> > > negated eapi version with no other metadata.  Allowing strings
> > > nukes that approach, unless you disallow - as the first character
> > > (which would be demonstration of strings not being incredibly well
> > > suited for eapi settings imo).
> >
> > As long as it does not generate "-0" this broken behaviour should not
> > break anything as long as portage 2.1 does a forced regen or cache
> > evaluation on installation (when detecting a 2.0 version present) or
> > first run.
>
> Wouldn't call it broken behaviour; the reasoning behind doing this is
> that people, regardless of any warning we give them, *will* have
> caches that are used by multiple portage versions.  Bad setups will
> results in older portage installations updating a central cache; the
> negating allows for older portage installations and newer to play nice
> in the cache, not pissing each other off as occurs with stable and
>
> >=2.1 (or any cache change for that matter).
>
> Granted that's a combination of flat_hash and eapi, but eapi's
> negation still is a major factor in the compatibility.

This would mean that "-" as the first character in an EAPI string should 
be disallowed until we can be "certain" that the portage versions in use 
do not need this feature anymore.

> > Is it really harder to check for either "" or unset or "0". That
> > should be very little extra code, and would be correct regardless of
> > future decisions regarding EAPI.
> >
> > EAPI should be left as free as possible, portage just needs to notice
> > that it doesn't recognize the value. If this would be in a
> > not-a-number exception that would be ok too.
>
> Flattening the EAPI version down to 0 is wrong; re: "" existing, it
> gets corrected by the layers that wrap the cache so it always winds up
> as a number.

Why does the cache do that. It doesn't with IUSE, DEPENDS, and most other 
variables. It should be easy to have it have the same functioning with 
EAPI. Btw. if it only maps unset or "" to 0 I have no problem with that, 
but if it maps "funky-no_number%string" to 0 or any number that would be 
broken.

Paul

ps. My point is that one never knows what the future holds. The EAPI 
feature is one that must be as forward compatible as possible. Designing 
forward compatibility is hard. As such the contents of EAPI should be as 
free as possible, and retrieving EAPI should be as easy as possible (by 
putting it in the start of the file).

pps. For being able to state that one eclass/ebuild is compatible with two 
(overlapping) formats I also argue to make EAPI a space separated list of 
supported API's. As an example for why this would make sense take for 
example the depend.apache eclass. This eclass is compatible with both the 
proposed EAPI1 as the current EAPI0 as it does not use/provide 
src_compile. Writing separate eclasses with just different EAPI versions 
is rather pointless, so allowing a list of supported ones makes sense. I 
do realise that this makes things a bit harder as first the EAPI of an 
ebuild(including it's eclasses) must be decided, and only then might 
proper parsing be possible to happen.

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Marius Mauch]

On 08/30/05  Brian Harring wrote:

> Why am I being so damned stubborn about numbers for this?  Cause I 
> don't want users having to dink around with knowing that eapi="really 
> cool version" somehow maps out to 3.1.  Further, eapi version *likely*
> will wind up as the slotting for any split out portage-ebuild package,
> and slots traditionally have always been ints.

Thanks for that last comment, it actually prooves our point ;)
Yes, SLOTs are *generally* numeric strings, as they are generally tied
to major versions. But as soon as more than the major part of the
version is important ints don't work anymore (and never ever mention
float again, version numbers aren't floats), and in some cases you find
non-version data in SLOT (like $CTARGET).

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Zac Medico]

Marius Mauch wrote:
> On 08/30/05  Paul de Vrieze wrote:
>>I don't think it is a wise path to interpret EAPI's as integers at
>>all.  There should not be guarantees of forward or backward
>>compatibility  between versions. Interpreting them as integers seems
>>to imply that.  Basically I think that EAPI could be anything wanted,
>>similar to SLOTS.  Of course using digits is easy for humans.
> 
> 
> I've to agree with Paul here, portage should hold a list of all EAPI
> values it understands, not a maximum value.
> 

I also agree.  It seems like a list of string identifiers will give us a high level of forward flexibility.

For the implementation we can have a chain of EAPI handlers that are capable of determining whether or not they support a given EAPI.  The first handler in the chain to accept the package will be allowed to handle it.  Internally, these handlers may be able to share libraries and/or inherit from eachother.

Zac
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 7884 bytes --]

Hokay, this is pretty much my response to the string vs int stuff, 
slammed into one email.

First clarifying the assumption I'm after a max test for eapi testing; 
not the case- stated so already in this thread.

Restating it again, for stable, a check of if it's > 0 (with int 
conversion failures being knocked up to 1 fex) is effectively the same 
thing as if str(portage_const.EAPI) in ("", "0").  Stated also that 
for rewrite, this approach won't work, and you're stuck specifying 
either a range of supported EAPI (max *and* min), or specifying every 
single individual eapi identifier, which I'll clarify before is going 
to burn any maintainer of such code.

On Wed, Aug 31, 2005 at 12:52:53PM +0200, Marius Mauch wrote:
> On 08/30/05  Brian Harring wrote:
> 
> > Why am I being so damned stubborn about numbers for this?  Cause I 
> > don't want users having to dink around with knowing that eapi="really 
> > cool version" somehow maps out to 3.1.  Further, eapi version *likely*
> > will wind up as the slotting for any split out portage-ebuild package,
> > and slots traditionally have always been ints.
> 
> Thanks for that last comment, it actually prooves our point ;)
> Yes, SLOTs are *generally* numeric strings, as they are generally tied
> to major versions. But as soon as more than the major part of the
> version is important ints don't work anymore (and never ever mention
> float again, version numbers aren't floats), and in some cases you find
> non-version data in SLOT (like $CTARGET).

Summation of string based failings.

First off, a modifier of an EAPI, say EAPI=3-strict isn't valid; why?  
strict is most likely a restriction, a specific negation of some 
aspect of eapi3.
  
Restrict is the place for it, or a similar metadata key.  Yes this 
doesn't hold across all cases (someone could've just decided 3-strict 
was a cool name fex), but throwing it out there to point out that eapi 
is ebuild templates, not restrictions.

Further, any deviation where loss of backwards compatibility would 
occur (limiting or extending bash syntax) is implicitly another 
format, thus beyond eapi's keen.  EAPI is ebuild (the format) spec/api 
*only*, see comments below re: format support for further explanation. 

Re: tagging EAPI at the top of a file, infra would probably shoot me 
for doing such- till a live, fully compatible and *roughly* equivalent 
parser is available, portage would have to do a bit of grepping, 
jacking up the regen times.

One thing I'm going to clarify also, is that the rewrite *does not* 
make it hard to tag new formats in.  There's no reason to jam 
everything into ebuilds, especially considering we are bound by 
existing definitions, and a body of 20,000 live ebuilds users/devs 
would get rather pissed off about if we forced a change across it.

If it's not an adjustment, a tweak, an extension/subtraction of ebuild
functionality, eapi isn't applicable imo, and a seperate format is the 
route to go (one that could easily live alongside in the tree).

EAPI is just for ebuild format.  alt formats (whether sh based or 
otherwise) would be wise to version their version also.


Either way, getting to the point of why strings suck badly in the 
point where it matters- as format version identifiers for the code 
that does the actual work.

Example code for numeric, say we have eapi 0-5 at this point.

def configure(self):
	if self.pkg.eapi > 0:
		...call configure phase...
	return


example code for strings

def configure(self):
	if self.pkg.eapi in ("0", "1", "2", "3", "4", "5"):
		...call configure phase...
	return

This sucks.

the response is "well, stick it into a list somewhere".  Can do, but 
you have to mangle each list everytime you add a new eapi.  That's a 
good way to lead to dumb ass bugs when tinkering with a new eapi 
level.

or... capabilities, as suggested.

capabilities = 	{"0":["configure"],
		"1":["configure"],
		"2":["configure"],
		"3":["configure"],
		"4":["configure"],
		"5":["configure"],}
		
def has_capability(eapi, capability):
	try:			return capability in capabilities[eapi]:
	except KeyError:	return False


def configure(self):
	if has_capability(self.pkg.eapi, "configure"):
		...call configure phase...
	return


or (slightly nastier example)

# eapi5 being eapi4, just with bash side changes.  implicit is that 
# their is bash side changes between each eapi, just stating it 
# explicitly to avoid the 'wtf' 

capabilities = 	{"0":["configure"],
		"1":["configure", "test_src_uri"],
		"2":["configure", "test_src_uri"],
		"3":["configure", "test_src_uri", "full_depriving"],
		"4":["configure", "test_src_uri", "full_depriving", "user_management"],
		"5":["configure", "test_src_uri", "full_depriving", "user_management"],}

def configure(self):
	if "configure" in capabilities.get(eapi, []):
		...call configure phase...
	return

Suggestions regarding using enumerations still dodge that point that 
via strings, you're building a finite set of matching strings to 
execute a block of code under, rather then establishing a min/max 
during which an int eapi should execute a block of code.  Latter's a 
helluva lot simpler then the former.

The arguement here is that it's somehow cleaner; it's not really.  You 
still have magic strings ("configure"), and a massive table of 
capabilities.  I used a single capability; if you're doing 
"has_capability" for every logic check, you're going to get into some 
very ugly terrain of magic const's all over the place.

Further, if attempting to map out what has changed between each eapi 
version, it's the wrong place to do so; that's implementation right 
there, specifically python side knowledge of how to support that 
version.

Proper design would actually use a magic constant at the top of the 
file, CONFIGURE_REQUIRED = 1 or something of the sort.

If I want to work on the next extension of eapi, I create an ebuild 
that has the new incremented #, and I can *already* have that eapi up 
and running, usurping the previous eapi definition.  No forced 
manual redefinition of stuff that is an easy way to induce bugs 
(increased manual work == bugs).  I can start extending the previous 
definition, tweaking it to I want without having to gut half of the 
controlling code.

Yes, I'm talking about it from the code perspective, but remember that 
this is the area that ultimately *matters*; control of the ebuild env 
is via the python side of things.  Keeping that as bug free, and clean 
as possible is what matters from where I sit; we can extend the ebuild 
env/template without issue, but swiveling from template to template is 
totally controlled by python side portage, which means that bit of 
code best be clean and robust.

That said, if you dig through my earlier patch, and comments about 
needing to handle int conversion better rather then flattening it to 
0, the code *is* forwards compatible if someone decides to stick strings 
into EAPI.  In other words, it's a moot debate due to the fact that 
the internal representation of eapi (is it a string, or is it an int?) 
is specific to that portage version; whatever version supports an eapi 
with strings tagged into has the fun of addressing the issues I've 
brought up above.

So basically the debate over string vs int can be left to when someone 
tries to add a non int eapi; forewarning: I'll bring up these points 
again when it's raised.  It's a Bad Idea (tm), and there is a damn 
good reason you don't see it elsewhere when dealing with versions of 
something :)

Final comment on this; there's a reason pkgs are release with 
major/minor as numeric; detection of capabilities, handling 
capabilities is *much* cleaner via such an approach.

~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 2727 bytes --]

On Wed, Aug 31, 2005 at 11:55:40AM +0200, Paul de Vrieze wrote:
> pps. For being able to state that one eclass/ebuild is compatible with two 
> (overlapping) formats I also argue to make EAPI a space separated list of 
> supported API's. As an example for why this would make sense take for 
> example the depend.apache eclass. This eclass is compatible with both the 
> proposed EAPI1 as the current EAPI0 as it does not use/provide 
> src_compile. Writing separate eclasses with just different EAPI versions 
> is rather pointless, so allowing a list of supported ones makes sense. I 
> do realise that this makes things a bit harder as first the EAPI of an 
> ebuild(including it's eclasses) must be decided, and only then might 
> proper parsing be possible to happen.

Preferred solution (imo) for such a case is eapi defined prior to the 
inherit iff the eclass is capable of multiple eapi levels.

It's effectively the same thing, but allows for the ebuild/eclass 
authors to have total control over it, and doesn't require a second 
duff eclass.

Regarding having the eclass specify compatible eapi versions, and 
portage somehow tracking ebuild eapi and comparing the two (and 
bailing if the containment test fails), not much for.  Reasoning is 
thus- new eapi3 is available, ebuild dev bumps (fex) apache so that 
it uses eapi3; if the eclass limits to <3, the ebuild *cannot* use the 
eclass.  Need a duplicate eclass, or the ebuild gets the eclasses code 
jammed into it, or (I really don't like this case) eclass gets eapi3 
tagged in, despite the fact the eclass may not be eclass compatible 
without the ebuild doing a little dance (for eapi0/1, defining it's 
own src_configure and src_compile fex).

Or the ebuild is kept at eapi2, till eapi3 is available in the eclass, 
slightly saner approach, but also effectively adoption of new eapis 
due to the fact that you cannot localize the build up of eapi3 
compatibility crap in the ebuild, then port it into the eclass (then 
marking the eclass as explicitly eapi3 compatible).

Eclass templates I prefer to keep as flexible as possible; granted 
people can shoot themselves in the foot, but that's always the case 
(add a safety to the gun, they'll find the safety and maybe gutshot 
themselves instead while playing with the safety :).  Point I'm 
getting at is that artificial limiters people find a way around, with 
the workaround being usually a helluva worse then what the limitation 
is trying to block.

Imo, at least.
Either way, it's something that once applied, cannot be reversed; 
would rather leave it loose, then tighten it down the line if people 
seem chronically apt to screw it up.
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Zac Medico]

Marius Mauch wrote:

> On 08/30/05  Paul de Vrieze wrote:
>
>> I don't think it is a wise path to interpret EAPI's as integers at
>> all.  There should not be guarantees of forward or backward
>> compatibility  between versions. Interpreting them as integers seems
>> to imply that.  Basically I think that EAPI could be anything wanted,
>> similar to SLOTS.  Of course using digits is easy for humans.
>
>
>
> I've to agree with Paul here, portage should hold a list of all EAPI
> values it understands, not a maximum value.
>

I also agree.  It seems like a list of string identifiers will give us a high level of forward flexibility.

For the implementation we can have a chain of EAPI handlers that are capable of determining whether or not they support a given EAPI.  The first handler in the chain to accept the package will be allowed to handle it.  Internally, these handlers may be able to share libraries and/or inherit from eachother.

Brian Harring wrote:
> 
> Either way, getting to the point of why strings suck badly in the 
> point where it matters- as format version identifiers for the code 
> that does the actual work.
> 
> Example code for numeric, say we have eapi 0-5 at this point.
> 
> def configure(self):
> 	if self.pkg.eapi > 0:
> 		...call configure phase...
> 	return
> 
> 
> example code for strings
> 
> def configure(self):
> 	if self.pkg.eapi in ("0", "1", "2", "3", "4", "5"):
> 		...call configure phase...
> 	return
> 
> This sucks.
> 

Yes, the above seems bad.  That is why I have suggested that a chain of handlers be used.  Select the first handler in the chain that recognizes the ebuild's eapi.  The selected handler would not need the above conditionals because it would be designed specifically for the eapi that it handles.

[snip]
> 
> capabilities = 	{"0":["configure"],
> 		"1":["configure", "test_src_uri"],
> 		"2":["configure", "test_src_uri"],
> 		"3":["configure", "test_src_uri", "full_depriving"],
> 		"4":["configure", "test_src_uri", "full_depriving", "user_management"],
> 		"5":["configure", "test_src_uri", "full_depriving", "user_management"],}
> 
> def configure(self):
> 	if "configure" in capabilities.get(eapi, []):
> 		...call configure phase...
> 	return
> 

Where appropriate, handlers could inherit from eachother or use shared libraries in order to provide the above capabilities.

[snip]
> 
> That said, if you dig through my earlier patch, and comments about 
> needing to handle int conversion better rather then flattening it to 
> 0, the code *is* forwards compatible if someone decides to stick strings 
> into EAPI.  In other words, it's a moot debate due to the fact that 
> the internal representation of eapi (is it a string, or is it an int?) 
> is specific to that portage version; whatever version supports an eapi 
> with strings tagged into has the fun of addressing the issues I've 
> brought up above.
> 

Okay, if it works and it's forward compatible with string lists, then I'm happy with it. :)

Zac
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 5475 bytes --]

On Wednesday 31 August 2005 14:58, Brian Harring wrote:
> On Wed, Aug 31, 2005 at 11:55:40AM +0200, Paul de Vrieze wrote:
> > pps. For being able to state that one eclass/ebuild is compatible
> > with two (overlapping) formats I also argue to make EAPI a space
> > separated list of supported API's. As an example for why this would
> > make sense take for example the depend.apache eclass. This eclass is
> > compatible with both the proposed EAPI1 as the current EAPI0 as it
> > does not use/provide src_compile. Writing separate eclasses with just
> > different EAPI versions is rather pointless, so allowing a list of
> > supported ones makes sense. I do realise that this makes things a bit
> > harder as first the EAPI of an ebuild(including it's eclasses) must
> > be decided, and only then might proper parsing be possible to happen.
>
> Preferred solution (imo) for such a case is eapi defined prior to the
> inherit iff the eclass is capable of multiple eapi levels.
>
> It's effectively the same thing, but allows for the ebuild/eclass
> authors to have total control over it, and doesn't require a second
> duff eclass.
>
> Regarding having the eclass specify compatible eapi versions, and
> portage somehow tracking ebuild eapi and comparing the two (and
> bailing if the containment test fails), not much for.  Reasoning is
> thus- new eapi3 is available, ebuild dev bumps (fex) apache so that
> it uses eapi3; if the eclass limits to <3, the ebuild *cannot* use the
> eclass.  Need a duplicate eclass, or the ebuild gets the eclasses code
> jammed into it, or (I really don't like this case) eclass gets eapi3
> tagged in, despite the fact the eclass may not be eclass compatible
> without the ebuild doing a little dance (for eapi0/1, defining it's
> own src_configure and src_compile fex).

If the eclass get's ported to eapi3 then all hell breaks loose (probably) 
as there are many ebuilds (some in the user's overlays and out of our 
control) that expect it to be eapi2. If the port was the case of just 
specifying eapi3 by the merit of being compatible to both (the change 
being outside the region of the eclass) this is unneeded.

Further there is a simple scheme for using EAPI as a list:

- portage specifies EAPI=0 before loading the ebuild:
- ebuild specifies EAPI="eapi1 eapi2 0"
- at inherit the current EAPI gets stored and EAPI is set to EAPI=0
- the eclass is sourced.
- The eclass specifies EAPI="eapi1 0"
- inherit finished the sourcing of the ebuild and compares EAPI_STORED
  with EAPI and keeps the common elements. Now EAPI="eapi1 0". Other
  eclasses might now be imported as if 'import a b c d' means 'import a'
  'import b' etc..
- After the ebuild is finally sourced completely EAPI has a value. This
  EAPI value is then masked with the supported EAPI levels. If the masking
  results in an empty list while the original list had items portage must
  be updated. If the original list was allready empty the ebuild is
  invalid. Finally if the masked list results in more than 1 atom, portage
  uses some internal list to select the best EAPI. This preference system
  could very well be internal to portage.

A feature that could be added would be that inherit would bail out as soon 
as the EAPI list is empty, and the ebuild can allready be seen as 
invalid.

> Or the ebuild is kept at eapi2, till eapi3 is available in the eclass,
> slightly saner approach, but also effectively adoption of new eapis
> due to the fact that you cannot localize the build up of eapi3
> compatibility crap in the ebuild, then port it into the eclass (then
> marking the eclass as explicitly eapi3 compatible).

Many eclasses can be compatible with multiple versions, but some cannot. 
That's why I argue that eclasses should list all api's that they are 
compatible with. Suppose for arguments sake that for example api3 was an 
api that was soon found to be not such usefull, and too far a deviation 
from api4. Then api4 is introduced that takes most usefull features from 
api3, but gets a higher compatibility with api2. An eclass could then be 
compatible with api2 and api4 without any compatibility layer.

>
> Eclass templates I prefer to keep as flexible as possible; granted
> people can shoot themselves in the foot, but that's always the case
> (add a safety to the gun, they'll find the safety and maybe gutshot
> themselves instead while playing with the safety :).  Point I'm
> getting at is that artificial limiters people find a way around, with
> the workaround being usually a helluva worse then what the limitation
> is trying to block.

Alternatively one could put all responsibility in ebuild authors. This 
would however be a disaster waiting to happen as an eclass gets updated 
to suddenly becomming incompatible with an ebuild in the user's overlay 
or our own tree.

> Imo, at least.
> Either way, it's something that once applied, cannot be reversed;
> would rather leave it loose, then tighten it down the line if people
> seem chronically apt to screw it up.

I just think that eclasses should specify what api they work with. If 
someone finds it does support their api of choice, but doesn't state so, 
well, that someone can just add the api of choice to the list. Way easier 
than even the fastest kludge.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Zac Medico]

Brian Harring wrote:
> Round 3, fixed all uglyness.
> You *will* see uglyness for the changeover from flat_list to flat_hash 
> if you're setting portdbapi.auxdbmodule to flat_hash, but that's a one 
> time hit, and is the reason we blow away the cache on portage 
> upgrades.
> 
> Either way, full patch, just correction of a few instances where it 
> user visible warnings were popping up, but not required.
> ~harring
> 

This patch worked well for me with your suggested settings in /etc/portage/modules:

portdbapi.metadbmodule=portage_db_metadata.database
portdbapi.auxdbmodule=portage_db_flat_hash.database

With "emerge metadata" I was able to convert the existing metadata from the rsync mirrors into flat_hash format.  There were no problems emerging ebuilds and I was also able to emerge binpkgs with old metadata.

Zac
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 9150 bytes --]

On Wednesday 31 August 2005 14:57, Brian Harring wrote:
>
> First off, a modifier of an EAPI, say EAPI=3-strict isn't valid; why?
> strict is most likely a restriction, a specific negation of some
> aspect of eapi3.

No 3-strict would be a variant of version 3 in the way that 
html-4.0-strict and html-4.0-transitional are related. They are very 
similar versions, but for a computer just different as computers don't 
handle and shouldn't handle "a little bit different". As such a portage 
implementation would either support the api called "3-strict" or it 
wouldn't. This would be independent of it's support of some other api 
that was called "3". The same numbers would be for user aid only.

> Restrict is the place for it, or a similar metadata key.  Yes this
> doesn't hold across all cases (someone could've just decided 3-strict
> was a cool name fex), but throwing it out there to point out that eapi
> is ebuild templates, not restrictions.

EAPI is not templates. It is API. It specifies what format the 
ebuild/eclass has. Currently the planned versions are only implemented by 
small differences, but to portage the different api's should be 
completely separate. It just happens that a lot of code can and will be 
shared between the api implementations.

> Further, any deviation where loss of backwards compatibility would
> occur (limiting or extending bash syntax) is implicitly another
> format, thus beyond eapi's keen.  EAPI is ebuild (the format) spec/api
> *only*, see comments below re: format support for further explanation.

Why should this be beyond EAPI. As long as any portage in run can read the 
ebuild well enough to extract the EAPI value and decide that it doesn't 
recognize it, things should be transparent. Or do you propose that for 
future/experimental ebuild formats we introduce yet another variable 
"EFORMAT" to specify what format the ebuild has. For me it's kind of 
double to do that.

> Re: tagging EAPI at the top of a file, infra would probably shoot me
> for doing such- till a live, fully compatible and *roughly* equivalent
> parser is available, portage would have to do a bit of grepping,
> jacking up the regen times.

If in cache EAPI can be gotten from the cache. If not, I don't think it 
matters where in the file EAPI occurs from the standpoint of getting it's 
value. The only thing would be that in the future a fast EAPI parser 
could be made that would just look at EAPI and get its version. I could 
easilly write you such a parser.

> One thing I'm going to clarify also, is that the rewrite *does not*
> make it hard to tag new formats in.  There's no reason to jam
> everything into ebuilds, especially considering we are bound by
> existing definitions, and a body of 20,000 live ebuilds users/devs
> would get rather pissed off about if we forced a change across it.

What I want is a way to be able to in the future change the ebuild format 
to aleviate some of it's deficiencies. And this in such a way that all 
current ebuilds can still be used, and that is compatible with any 
reasonable portage version. (So format variations should wait for some 
time for older portages to get out of view).

> If it's not an adjustment, a tweak, an extension/subtraction of ebuild
> functionality, eapi isn't applicable imo, and a seperate format is the
> route to go (one that could easily live alongside in the tree).

You mean a different file extension. That could be possible, but for a 
proper forward compatible format this is not necessary. We now have the 
possibility to make the current ebuild format forward compatible. Why not 
do this immediately. The nice thing is that this forward compatibility is 
even compatible with current (and old) portage versions as long as no too 
big changes are made before the non-eapi-aware portage versions are 
phased out.

> EAPI is just for ebuild format.  alt formats (whether sh based or
> otherwise) would be wise to version their version also.

Why not combine them. I really don't see why we should have dual version 
systems.

> Either way, getting to the point of why strings suck badly in the
> point where it matters- as format version identifiers for the code
> that does the actual work.
>
> Example code for numeric, say we have eapi 0-5 at this point.
>
> def configure(self):
> 	if self.pkg.eapi > 0:
> 		...call configure phase...
> 	return
>
>
> example code for strings

This is broken as portage should not allow formats it doesn't know about.

>
> def configure(self):
> 	if self.pkg.eapi in ("0", "1", "2", "3", "4", "5"):
> 		...call configure phase...
> 	return
>
> This sucks.

What about this pseudocode:
API=self.pkg.eapi
if not isnumber(API):
    error "unsupported api or malformatted api"
else
    if asnumber(API) >= 0 and asnumber(API) <= 5:
        ...call configure phase...
    return


And as you should remember from programming 101 you should check for 
faulty inputs anyway.

>
> the response is "well, stick it into a list somewhere".  Can do, but
> you have to mangle each list everytime you add a new eapi.  That's a
> good way to lead to dumb ass bugs when tinkering with a new eapi
> level.

Support for the api should be added anyway. Along with the code to handle 
this new api it shouldn't be hard to have this code register itself in 
the list with handled api's.

> Suggestions regarding using enumerations still dodge that point that
> via strings, you're building a finite set of matching strings to
> execute a block of code under, rather then establishing a min/max
> during which an int eapi should execute a block of code.  Latter's a
> helluva lot simpler then the former.

there will anyway only be a finite fixed set of api's that any portage 
version should handle. If it does know about 4 and 5 but not about 4.5 it 
shouldn't handle 4.5.

> The arguement here is that it's somehow cleaner; it's not really.  You
> still have magic strings ("configure"), and a massive table of
> capabilities.  I used a single capability; if you're doing
> "has_capability" for every logic check, you're going to get into some
> very ugly terrain of magic const's all over the place.

The idea is that there is a class for every api version. Some classes 
share lot's of code with other classes and are siblings. Some are hardly 
alike. There then only needs to be one check and that calls the 
approprate class.

> If I want to work on the next extension of eapi, I create an ebuild
> that has the new incremented #, and I can *already* have that eapi up
> and running, usurping the previous eapi definition.  No forced
> manual redefinition of stuff that is an easy way to induce bugs
> (increased manual work == bugs).  I can start extending the previous
> definition, tweaking it to I want without having to gut half of the
> controlling code.

You mean subclassing the new eapi in the portage code, and registering it 
under the new name. That's just about or maybe even more easy than 
anything else.

> Yes, I'm talking about it from the code perspective, but remember that
> this is the area that ultimately *matters*; control of the ebuild env
> is via the python side of things.  Keeping that as bug free, and clean
> as possible is what matters from where I sit; we can extend the ebuild
> env/template without issue, but swiveling from template to template is
> totally controlled by python side portage, which means that bit of
> code best be clean and robust.

From my perspective the sh part of portage is even messier than the python 
part. From my perspective doing things at the python side is a lot 
better.

> That said, if you dig through my earlier patch, and comments about
> needing to handle int conversion better rather then flattening it to
> 0, the code *is* forwards compatible if someone decides to stick
> strings into EAPI.  In other words, it's a moot debate due to the fact
> that the internal representation of eapi (is it a string, or is it an
> int?) is specific to that portage version; whatever version supports an
> eapi with strings tagged into has the fun of addressing the issues I've
> brought up above.

If that is the case I don't care. I don't believe that strings are an 
issue, but I'm happy as long as when I stick EAPI="pauls_test_format" 
into my ebuild your portage bails out.

> So basically the debate over string vs int can be left to when someone
> tries to add a non int eapi; forewarning: I'll bring up these points
> again when it's raised.  It's a Bad Idea (tm), and there is a damn
> good reason you don't see it elsewhere when dealing with versions of
> something :)

I still argue that treating api versions as numbers is the bad idea (tm), 
and using ranges to check them even worse. Each version should be treated 
differently anyway so at that point the check needs to be done anyway. 
More than one check is ugly code.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 988 bytes --]

On Wed, Aug 31, 2005 at 08:10:35AM -0700, Zac Medico wrote:
> Brian Harring wrote:
> >Round 3, fixed all uglyness.
> >You *will* see uglyness for the changeover from flat_list to flat_hash 
> >if you're setting portdbapi.auxdbmodule to flat_hash, but that's a one 
> >time hit, and is the reason we blow away the cache on portage 
> >upgrades.
> >
> >Either way, full patch, just correction of a few instances where it 
> >user visible warnings were popping up, but not required.
> >~harring
> >
> 
> This patch worked well for me with your suggested settings in 
> /etc/portage/modules:
> 
> portdbapi.metadbmodule=portage_db_metadata.database
> portdbapi.auxdbmodule=portage_db_flat_hash.database
> 
> With "emerge metadata" I was able to convert the existing metadata from the 
> rsync mirrors into flat_hash format.  There were no problems emerging 
> ebuilds and I was also able to emerge binpkgs with old metadata.
cool, thank you for testing that.
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Zac Medico]

[-- Attachment #1: Type: text/plain, Size: 628 bytes --]

Brian Harring wrote:
> Round 3, fixed all uglyness.
> You *will* see uglyness for the changeover from flat_list to flat_hash 
> if you're setting portdbapi.auxdbmodule to flat_hash, but that's a one 
> time hit, and is the reason we blow away the cache on portage 
> upgrades.
> 
> Either way, full patch, just correction of a few instances where it 
> user visible warnings were popping up, but not required.
> ~harring
> 

I have created a patch for forward compatibility with arbitrary EAPI strings.  The only change is to substitute an integer constant EAPI_UNRECOGNIZED whenever an unrecognized string is encountered.

Zac

[-- Attachment #2: 2.0.51-eapi-awareness-3-arbitrary-string-forward-compatibility.patch --]
[-- Type: text/x-patch, Size: 2687 bytes --]

Index: portage/pym/portage.py
===================================================================
--- portage.orig/pym/portage.py
+++ portage/pym/portage.py
@@ -81,7 +81,7 @@ try:
 	  MOVE_BINARY, PRELINK_BINARY, WORLD_FILE, MAKE_CONF_FILE, MAKE_DEFAULTS_FILE, \
 	  DEPRECATED_PROFILE_FILE, USER_VIRTUALS_FILE, EBUILD_SH_ENV_FILE, \
 	  INVALID_ENV_FILE, CUSTOM_MIRRORS_FILE, SANDBOX_PIDS_FILE, CONFIG_MEMORY_FILE,\
-	  INCREMENTALS, STICKIES, EAPI
+	  INCREMENTALS, STICKIES, EAPI, EAPI_UNRECOGNIZED
 
 	from portage_data import ostype, lchown, userland, secpass, uid, wheelgid, \
 	                         portage_uid, portage_gid
@@ -4542,7 +4542,10 @@ class bindbapi(fakedbapi):
 			try:
 				mylist[idx] = abs(int(mylist[idx]))
 			except ValueError:
-				mylist[idx] = 0
+				if mylist[idx]=="":
+					mylist[idx] = 0
+				else:
+					mylist[idx] = EAPI_UNRECOGNIZED
 		return mylist
 
 
@@ -4827,7 +4830,10 @@ class vardbapi(dbapi):
 			try:
 				results[idx] = abs(int(wants[idx]))
 			except ValueError:
-				results[idx] = 0 
+				if wants[idx]=="":
+					results[idx] = 0 
+				else:
+					results[idx] = EAPI_UNRECOGNIZED
 		return results
 
 
@@ -5313,7 +5319,10 @@ class portdbapi(dbapi):
 			try:
 				eapi = int(metadata["EAPI"])
 			except (KeyError, ValueError):
-				eapi = 0
+				if metadata.has_key("EAPI") and metadata["EAPI"]!="":
+					eapi = EAPI_UNRECOGNIZED
+				else:
+					eapi = 0
 				metadata["EAPI"] = eapi
 			if eapi != portage_const.EAPI:
 				# intentionally wipe keys.
@@ -5391,7 +5400,10 @@ class portdbapi(dbapi):
 					mylines[x] = mylines[x][:-1]
 				mydata[auxdbkeys[x]] = mylines[x]
 
-			eapi = int(mydata["EAPI"])
+			try:
+				eapi = int(mydata["EAPI"])
+			except ValueError:
+				eapi = EAPI_UNRECOGNIZED
 			if eapi > portage_const.EAPI:
 				# if newer version, wipe everything and negate eapi
 				mydata = {}
@@ -5417,7 +5429,10 @@ class portdbapi(dbapi):
 				returnme[idx] = abs(int(returnme[idx]))
 			except ValueError:
 				# string
-				returnme[idx] = 0
+				if returnme[idx]=="":
+					returnme[idx] = 0
+				else:
+					returnme[idx] = EAPI_UNRECOGNIZED
 
 		return returnme
 
Index: portage/pym/portage_const.py
===================================================================
--- portage.orig/pym/portage_const.py
+++ portage/pym/portage_const.py
@@ -44,6 +44,7 @@ INCREMENTALS=["USE","FEATURES","ACCEPT_K
 STICKIES=["KEYWORDS_ACCEPT","USE","CFLAGS","CXXFLAGS","MAKEOPTS","EXTRA_ECONF","EXTRA_EINSTALL","EXTRA_EMAKE"]
 
 EAPI = 0
+EAPI_UNRECOGNIZED = 65536
 
 # ===========================================================================
 # END OF CONSTANTS -- END OF CONSTANTS -- END OF CONSTANTS -- END OF CONSTANT

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Zac Medico]

Paul de Vrieze wrote:
> On Wednesday 31 August 2005 14:57, Brian Harring wrote:
> 
>> Re: tagging EAPI at the top of a file, infra would probably shoot me for doing such- till a live, fully compatible and *roughly* equivalent parser is available, portage would have to do a bit of grepping, jacking up the regen times.
> 
> 
> If in cache EAPI can be gotten from the cache. If not, I don't think it matters where in the file EAPI occurs from the standpoint of getting it's value. The only thing would be that in the future a fast EAPI parser could be made that would just look at EAPI and get its version. I could easilly write you such a parser.

It is impossible write a parser for an unconstrained and unknown format that may exist in the future.  If we put a constraint on the format, in order to parse the EAPI, then we contradict our original goal (to unconstrain the format).

A better approach IMO would be to store the EAPI in a separate file such as metadata.xml.  This would allow *absolute* flexibility in the "ebuild" format.  Portage would be able to select an appropriate parser with no need to examine the "ebuild" itself.

Zac
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 2376 bytes --]

On Thu, Sep 01, 2005 at 10:27:44PM -0700, Zac Medico wrote:
> Paul de Vrieze wrote:
> >On Wednesday 31 August 2005 14:57, Brian Harring wrote:
> >
> >>Re: tagging EAPI at the top of a file, infra would probably shoot me for 
> >>doing such- till a live, fully compatible and *roughly* equivalent parser 
> >>is available, portage would have to do a bit of grepping, jacking up the 
> >>regen times.
> >
> >
> >If in cache EAPI can be gotten from the cache. If not, I don't think it 
> >matters where in the file EAPI occurs from the standpoint of getting it's 
> >value. The only thing would be that in the future a fast EAPI parser could 
> >be made that would just look at EAPI and get its version. I could easilly 
> >write you such a parser.
> 
> It is impossible write a parser for an unconstrained and unknown format 
> that may exist in the future.  If we put a constraint on the format, in 
> order to parse the EAPI, then we contradict our original goal (to 
> unconstrain the format).
> 
> A better approach IMO would be to store the EAPI in a separate file such as 
> metadata.xml.  This would allow *absolute* flexibility in the "ebuild" 
> format.  Portage would be able to select an appropriate parser with no need 
> to examine the "ebuild" itself.
Disallows eclasses from ever setting eapi.

Like I've said, EAPI is ebuild specific.  Ebuild is a format; eapi 
defines revisions of it, in my mind a minor revision of the ebuild 1 
format.  Any form of loss of backwards compatability *should* be a 
different format, .ebuild2 for all I care.

Trying to use EAPI to allow for N different formats into one format is 
wrong from where I sit; you would need a container format for it, 
which ebuild wasn't designed for (nor is it easily extensible to be 
made so I posit).

EAPI's original specification was for handling addition of new funcs, 
different hooks in the ebuild; I prefer it remain as this.  The core 
rewrite is format agnostic, if a new format is defined (whether a 
massively managled version of ebuild or flat out new), it's a seperate 
format and should be handled via the core, not via ebuild specific 
package handling.

There's no reason a repository can't hold multiple formats internally; 
the capability is there, use that rather then trying to jam too much 
into EAPI, imo at least.
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Marius Mauch]

[-- Attachment #1: Type: text/plain, Size: 774 bytes --]

On 08/27/05  Brian Harring wrote:

> Hola.
> 
> Attached is a patch that 
> A) adds EAPI awareness to portage; mainly, if >0, complain and be 
>    unwilling to merge the package

Actually I just wrote also a patch for it (for 2.1), however instead of
complaining I just masked them (without unmask ability), just add a
check to gvisible() and getmaskingstatus() (actually just calling
dbapi.eapicheck()). That way it should be more transparent to users IMO.
Won't stop people from using ebuild(1) though.

Marius

PS: I'm aware that the cache changes probably aren't 100% correct.

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.

[-- Attachment #2: eapi.patch --]
[-- Type: text/x-patch, Size: 5345 bytes --]

--- pym/portage.py.org	2005-08-23 03:23:59.000000000 +0200
+++ pym/portage.py	2005-09-02 05:49:50.000000000 +0200
@@ -91,7 +90,7 @@
 	  MOVE_BINARY, PRELINK_BINARY, WORLD_FILE, MAKE_CONF_FILE, MAKE_DEFAULTS_FILE, \
 	  DEPRECATED_PROFILE_FILE, USER_VIRTUALS_FILE, EBUILD_SH_ENV_FILE, \
 	  INVALID_ENV_FILE, CUSTOM_MIRRORS_FILE, SANDBOX_PIDS_FILE, CONFIG_MEMORY_FILE,\
-	  INCREMENTALS, STICKIES
+	  INCREMENTALS, STICKIES, EAPI_COMPATIBLE
 
 	from portage_data import ostype, lchown, userland, secpass, uid, wheelgid, \
 	                         portage_uid, portage_gid
@@ -2057,6 +2056,13 @@
 		'return: ["0",">=sys-libs/bar-1.0","http://www.foo.com"] or [] if mycpv not found'
 		raise NotImplementedError
 
+	def eapicheck(self, mycpv, verbose=0):
+		myeapi = self.aux_get(mycpv, ["EAPI"])[0]
+		rval = (myeapi in ['']+EAPI_COMPATIBLE.split())
+		if not rval and verbose:
+			writemsg("Warning: EAPI check failed for %s (has EAPI '%s')" % (mycpv, myeapi))
+		return rval
+
 	def match(self,origdep,use_cache=1):
 		mydep=dep_expand(origdep,mydb=self)
 		mykey=portage_dep.dep_getkey(mydep)
@@ -2667,9 +2673,9 @@
   'DEPEND',    'RDEPEND',   'SLOT',      'SRC_URI',
 	'RESTRICT',  'HOMEPAGE',  'LICENSE',   'DESCRIPTION',
 	'KEYWORDS',  'INHERITED', 'IUSE',      'CDEPEND',
-	'PDEPEND',   'PROVIDE',
+	'PDEPEND',   'PROVIDE', 'EAPI'
 	'UNUSED_01', 'UNUSED_02', 'UNUSED_03', 'UNUSED_04',
-	'UNUSED_05', 'UNUSED_06', 'UNUSED_07', 'UNUSED_08',
+	'UNUSED_05', 'UNUSED_06', 'UNUSED_07', 
 	]
 auxdbkeylen=len(auxdbkeys)
 
@@ -2737,7 +2743,7 @@
 			raise KeyError("CPV %s does not exist" % mycpv)
 		mycp=mysplit[0]+"/"+mysplit[1]
 
-		if settings.pmaskdict.has_key(mycp):
+		if "package.mask" in getmaskingstatus(mycpv) and settings.pmaskdict.has_key(mycp):
 			for x in settings.pmaskdict[mycp]:
 				if mycpv in self.xmatch("match-all", x):
 					pmaskfile = open(settings["PORTDIR"]+"/profiles/package.mask")
@@ -2768,6 +2774,11 @@
 	
 		rValue = []
 
+		# EAPI checking
+		
+		if not db["/"]["porttree"].dbapi.eapicheck(mycpv):
+			rValue.append("EAPI")
+
 		# profile checking
 		revmaskdict=settings.prevmaskdict
 		if revmaskdict.has_key(mycp):
@@ -3286,7 +3297,7 @@
 		return newlist
 
 	def gvisible(self,mylist):
-		"strip out group-masked (not in current group) entries"
+		"strip out group-masked (not in current group) entries and entries with wrong EAPIs"
 		global groups
 		if mylist==None:
 			return []
@@ -3297,7 +3308,7 @@
 			#we need to update this next line when we have fully integrated the new db api
 			auxerr=0
 			try:
-				myaux=db["/"]["porttree"].dbapi.aux_get(mycpv, ["KEYWORDS"])
+				myaux=db["/"]["porttree"].dbapi.aux_get(mycpv, ["KEYWORDS", "EAPI"])
 			except (KeyError,IOError,TypeError):
 				continue
 			if not myaux[0]:
@@ -3305,6 +3316,7 @@
 				#print "!!! No KEYWORDS for "+str(mycpv)+" -- Untested Status"
 				continue
 			mygroups=myaux[0].split()
+			myeapi=myaux[1]
 			pgroups=groups[:]
 			cp = portage_dep.dep_getkey(mycpv)
 			if cp in pkgdict:
@@ -3334,7 +3346,7 @@
 						if gp[0] != "-":
 							match=1
 							break
-			if match:
+			if match and db["/"]["porttree"].dbapi.eapicheck(mycpv):
 				newlist.append(mycpv)
 		return newlist
 
--- bin/ebuild.sh.org	2005-08-23 03:23:46.000000000 +0200
+++ bin/ebuild.sh	2005-09-02 04:29:37.000000000 +0200
@@ -640,6 +640,7 @@
 			[ "$CDEPEND:-unset}" != "unset" ] && 		speak "CDEPEND=$(echo $CDEPEND)"
 			[ "$PDEPEND:-unset}" != "unset" ] && 		speak "PDEPEND=$(echo $PDEPEND)"
 			[ "$PROVIDE:-unset}" != "unset" ] && 		speak "PROVIDE=$(echo $PROVIDE)"
+			[ "$EAPI:-unset}" != "unset" ] && 		speak "EAPI=$(echo $EAPI)"
 			speak 'end_keys'
 			set +f
 			;;
--- bin/ebuild-default-functions.sh.org	2005-08-23 03:24:26.000000000 +0200
+++ bin/ebuild-default-functions.sh	2005-09-02 04:30:05.000000000 +0200
@@ -344,6 +340,7 @@
 	echo "$RDEPEND"		> RDEPEND
 	echo "$RESTRICT"	> RESTRICT
 	echo "$SLOT"		> SLOT
+	echo "$EAPI"		> EAPI
 	echo "$USE"		> USE
 	export_environ "${PORTAGE_BUILDDIR}/build-info/environment.bz2" 'bzip2 -c9'
 	cp "${EBUILD}" "${PF}.ebuild"
--- pym/portage_const.py.org	2005-08-23 01:18:53.000000000 +0200
+++ pym/portage_const.py	2005-09-02 05:44:43.000000000 +0200
@@ -63,3 +63,4 @@
 CVS_BIN                 = "/usr/bin/cvs"
 
 EBUILD_PHASES			= "setup unpack compile test install preinst postinst prerm postrm"
+EAPI_COMPATIBLE			= "0"
--- pym/cache/flat_list.py.org	2005-08-23 01:17:51.000000000 +0200
+++ pym/cache/flat_list.py	2005-09-02 05:14:27.000000000 +0200
@@ -11,7 +11,7 @@
 	auxdbkey_order=('DEPEND', 'RDEPEND', 'SLOT', 'SRC_URI',
 		'RESTRICT',  'HOMEPAGE',  'LICENSE', 'DESCRIPTION',
 		'KEYWORDS',  'IUSE', 'CDEPEND',
-		'PDEPEND',   'PROVIDE','_eclasses_')
+		'PDEPEND',   'PROVIDE', 'EAPI', '_eclasses_')
 
 	def __init__(self, label, auxdbkeys, **config):
 		super(database,self).__init__(label, auxdbkeys, **config)
--- pym/cache/metadata.py.org	2005-08-23 01:17:51.000000000 +0200
+++ pym/cache/metadata.py	2005-09-02 05:53:41.000000000 +0200
@@ -8,7 +8,7 @@
 	auxdbkey_order=('DEPEND', 'RDEPEND', 'SLOT', 'SRC_URI',
 		'RESTRICT',  'HOMEPAGE',  'LICENSE', 'DESCRIPTION',
 		'KEYWORDS',  'INHERITED', 'IUSE', 'CDEPEND',
-		'PDEPEND',   'PROVIDE')
+		'PDEPEND',   'PROVIDE', 'EAPI')
 
 	def __init__(self, label, auxdbkeys, **config):
 		super(database,self).__init__(label, auxdbkeys, **config)

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Zac Medico]

Brian Harring wrote:
> On Thu, Sep 01, 2005 at 10:27:44PM -0700, Zac Medico wrote:
>>
>>A better approach IMO would be to store the EAPI in a separate file such as 
>>metadata.xml.  This would allow *absolute* flexibility in the "ebuild" 
>>format.  Portage would be able to select an appropriate parser with no need 
>>to examine the "ebuild" itself.
> 
> Disallows eclasses from ever setting eapi.
> 

In order to adapt my above suggestion, to allow for eclasses to define the eapi, we would specify a parser identifier inside metadata.xml rather than an eapi identifier.  Indeed, the parser identifier (or file extension) would be a separate thing from eapi.

Zac
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 2274 bytes --]

On Friday 02 September 2005 07:27, Zac Medico wrote:
> Paul de Vrieze wrote:
> > On Wednesday 31 August 2005 14:57, Brian Harring wrote:
> >> Re: tagging EAPI at the top of a file, infra would probably shoot me
> >> for doing such- till a live, fully compatible and *roughly*
> >> equivalent parser is available, portage would have to do a bit of
> >> grepping, jacking up the regen times.
> >
> > If in cache EAPI can be gotten from the cache. If not, I don't think
> > it matters where in the file EAPI occurs from the standpoint of
> > getting it's value. The only thing would be that in the future a fast
> > EAPI parser could be made that would just look at EAPI and get its
> > version. I could easilly write you such a parser.
>
> It is impossible write a parser for an unconstrained and unknown format
> that may exist in the future.  If we put a constraint on the format, in
> order to parse the EAPI, then we contradict our original goal (to
> unconstrain the format).

It would be possible to make these restrictions small. A workable 
restriction could be to put the EAPI statement in the front of the file 
only preceded by whitespace and comments. After the line with the EAPI 
statement anything could happen. Even a completely binary format could be 
used. But indeed no constraints is impossible, so putting the format 
statement in the start of the file is the best solution. Besides the fact 
that it makes detecting the format that much faster, and allows bailing 
out before parsing the whole file if an unsupported api is detected.

> A better approach IMO would be to store the EAPI in a separate file
> such as metadata.xml.  This would allow *absolute* flexibility in the
> "ebuild" format.  Portage would be able to select an appropriate parser
> with no need to examine the "ebuild" itself.

Please don't. Parsing xml is a lot harder than recognizing lines. A simple 
EAPI parser would probably use canned (library) line reading functions 
and then look if the line starts with "EAPI=". If so it reads the EAPI, 
else it checks the next line. No need to understand anything else of the 
file format.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 2605 bytes --]

On Friday 02 September 2005 08:04, Brian Harring wrote:
>
> Like I've said, EAPI is ebuild specific.  Ebuild is a format; eapi
> defines revisions of it, in my mind a minor revision of the ebuild 1
> format.  Any form of loss of backwards compatability *should* be a
> different format, .ebuild2 for all I care.

The new proposed format loses backwards compatibility. If there is 
backwards compatibility no new format or api version is needed. EAPI 
should work on the python level, not only on the ebuild.sh level.

> Trying to use EAPI to allow for N different formats into one format is
> wrong from where I sit; you would need a container format for it,
> which ebuild wasn't designed for (nor is it easily extensible to be
> made so I posit).

No it would state that the eclass is 100% compatible with both by the 
formats overlapping and the ebuild not threading outside the overlapped 
area. Take for an example many simple kde applications. Those use the kde 
eclass to do all the work and only contain a skeleton of variables 
themselves. These ebuilds are compatible with both the current as the 
proposed new API. When marked so, they could be used as EAPI=1 as soon as 
the kde eclass is ported to EAPI=1. Similarly many eclasses do not 
provide src_compile, and as such are compatible with both EAPI versions.

> EAPI's original specification was for handling addition of new funcs,
> different hooks in the ebuild; I prefer it remain as this.  The core
> rewrite is format agnostic, if a new format is defined (whether a
> massively managled version of ebuild or flat out new), it's a seperate
> format and should be handled via the core, not via ebuild specific
> package handling.

EAPI now is going to be used for the above. It can however with very 
little effort be made such that future ebuild format revisions are 
possible. Also don't be mistaken that splitting out configure and make 
stages do need support from the python part.

>
> There's no reason a repository can't hold multiple formats internally;
> the capability is there, use that rather then trying to jam too much
> into EAPI, imo at least.

How would you suggest to do this then. The ebuilds/eclasses are completely 
the same except for their EAPI definition. They also have the same 
(file)name. And that is not counting the fact that two files containing 
the same is bad design as there will allways be an issue of one file 
being updated and the other not.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] PATCH: initial EAPI awareness

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 10965 bytes --]

On Fri, Sep 02, 2005 at 10:53:05AM +0200, Paul de Vrieze wrote:
> On Friday 02 September 2005 08:04, Brian Harring wrote:
> >
> > Like I've said, EAPI is ebuild specific.  Ebuild is a format; eapi
> > defines revisions of it, in my mind a minor revision of the ebuild 1
> > format.  Any form of loss of backwards compatability *should* be a
> > different format, .ebuild2 for all I care.
> 
> The new proposed format loses backwards compatibility. If there is 
> backwards compatibility no new format or api version is needed. EAPI 
> should work on the python level, not only on the ebuild.sh level.

The two are utterly intertwined.

Doesn't matter if portage thinks it's eapi 1 or 2, what matters is if the 
ebuild*sh env *creates* an eapi 1 or 2 env for execution.  Python side 
just behaves a bit differently, ebuild*sh is where all of the actual 
execution occurs (and varies dependant on eapi).

> > Trying to use EAPI to allow for N different formats into one format is
> > wrong from where I sit; you would need a container format for it,
> > which ebuild wasn't designed for (nor is it easily extensible to be
> > made so I posit).
> 
> No it would state that the eclass is 100% compatible with both by the 
> formats overlapping and the ebuild not threading outside the overlapped 
> area.

And how is this going to work when it's more then just a split of a 
func?  

I'm not saying it can't be done, I'm saying it *shouldn't* be done.  
It's begging for screwups; relying on portage to pick and choose 
between eapi levels for an ebuild's execution env dependant on an 
eclass *requires* the ebuild to be fully compatible to those 
eapis, same for the eclass author.  That's not so easy to do, and will 
lead to a buttload of case tests of EAPI, which isn't going to improve 
the code (imo).

People aren't going to do it, is my opinion.  It's unnecessarily 
complex with minimal gain; the gain being decreased due to the 
inherent complexity of 
A) pulling this off in portage
B) comprehending *exactly* what portage will do in each case
C) the ebuild/eclass author not only managing to get usage of *one* 
   eapi correct, getting potentially N eapi's correct.
D) eclasses that inherit eclasses requiring the same eapi 
   compatability checks
E) within two bodies of code, devs  keeping seperated in their mind the 
   differences between each eapi, and *never* blurring them.

Note I'm not pulling the "everyone else is ignorant" crap that popped 
up on -dev ml; I'm stating that writing to an api is manual work, and 
people will screw up on it, just the same as in writing the
implementation of that api, *we* will screw up on it.

Bugs per line, it's unavoidable.  Not advocating that we dumb 
everything down, advocating we don't add something that (assuming we 
get it right in implementation) is one hell of a pandora's box from 
the standpoint of complex ebuilds combined with complex eclasses.

That's also not even getting into the fact that what you're proposing, 
effectively greping EAPI from the file is 

A) helluva lot slower on regen
B) trying to turn ebuilds into a container format.  They're not a 
   container format, nor should they be (kitchen sink shouldn't be 
   included).
C) core's more then capable of supporting seperated formats; the 
   'container' should be the repository, not jammed within the package 
   data.
D) It's a helluva lot easier, and less chance for screwup to just 
   define a new format, use a new extension, and have the repository 
   implementation use a different pkg format for that file.
E) New, non eapi=1 style change with a different extension wouldn't 
   even be *seen* by the noncompliant repositories.  Helluva lot 
   simpler.

Not advocating we define a new format every day; I'm advocating that 
we define a format (v1), we do tweaks to it, and define a new 
*seperate* format when we need stuff that is beyond the keen of the 
existing format.  Shifting away from declarative syntax, moving away 
from bash syntax, etc.  If it isn't akin to our existing 
definition of an ebuild, it's not an ebuild and should *not* be jammed 
into the ebuild format.

It's a new beast, and should be seperated.

That's also ignoring the additional interaction of elibs thrown into 
the mix.  If what you're suggesting is implemented, eclasses *and* 
ebuilds would be litered with 

if [ "$EAPI" == 2 ]; then
	eapi2 specific stuff
elif [ "$EAPI" == 1]; then
	eapi1 specific stuff
else
	general, all eapi stuff
fi

Note that the code above also has a nice hidden bug in it that's not 
obvious till you think about it; there is no way to predict at the 
time of the ebuilds writing that EAPI3 will work for the else block, 
yet logic structures of that sort *will* exist if you open up the N 
EAPI compatibility for an ebuild/eclass.  Any bumping of the potential 
eapi's for an ebuild/eclass will expose it.

That and I'd hope if it were ever implemented, devs would be using 
functions within the logic block; even with, any non-trivial bit of 
code is going to get nasty as it's littered with a hundred and one 
checks of eapi tweaking the flow ever so slightly.

It's needlessly complex, and a recipe for disaster.


> Take for an example many simple kde applications. Those use the kde 
> eclass to do all the work and only contain a skeleton of variables 
> themselves. These ebuilds are compatible with both the current as the 
> proposed new API. When marked so, they could be used as EAPI=1 as soon as 
> the kde eclass is ported to EAPI=1. Similarly many eclasses do not 
> provide src_compile, and as such are compatible with both EAPI versions.

Lacking src_compile, they're compatible.  They're also quite likely an 
elib, which is a different beast designed to address the fact that 
library code of eclasses (true library), should exist outside of the 
eclass.  Eclasses are purely templates for ebuilds, a derived 
template/class from the base template, that an ebuild uses to avoid 
defining common chunks of code.

Common code blocks, say what eutils/toolchain does are elibs (yes I 
know about the patch dep on eutils, ignore it for the sake of 
arguement).

> > EAPI's original specification was for handling addition of new funcs,
> > different hooks in the ebuild; I prefer it remain as this.  The core
> > rewrite is format agnostic, if a new format is defined (whether a
> > massively managled version of ebuild or flat out new), it's a seperate
> > format and should be handled via the core, not via ebuild specific
> > package handling.
> 
> EAPI now is going to be used for the above. It can however with very 
> little effort be made such that future ebuild format revisions are 
> possible. Also don't be mistaken that splitting out configure and make 
> stages do need support from the python part.

Well aware of the complexities involved in keeping the python and bash 
side in sync; it's part of the reason I oppose what you're proposing.

Change in terminology here btw; eapi (imo) has never been defined as a 
format revision, it's a change in the portage exported bash api, and 
how it potentially calls into the ebuild.sh api (addition of 
src_configure).

People split off a new format (seperate extension fex) when they no 
longer can make changes to the api, where grand changes are required.  
EAPI isn't designed to address that; it's designed to address 
ebuilds/eclasses requiring new ebuild*sh functionality, addition of a 
new do* or change to # of phases called.

eapi0 -> eapi1 is probably going to be the nastiest change over from 
where I sit; it's the transition from a totally unversioned ebuild*sh 
env to one that has saner handling.

Total change over of ebuild format, as you're hinting at is not eapi's 
intention; that's covered by the core being format agnostic, and 
repository abstractions.


> > There's no reason a repository can't hold multiple formats internally;
> > the capability is there, use that rather then trying to jam too much
> > into EAPI, imo at least.
> 
> How would you suggest to do this then. The ebuilds/eclasses are completely 
> the same except for their EAPI definition. They also have the same 
> (file)name. And that is not counting the fact that two files containing 
> the same is bad design as there will allways be an issue of one file 
> being updated and the other not.

I'm saying we stick with what already occurs in the tree; eclass 
functionality is either prototyped outside of the tree, and 
introduced, or an ebuild has the eclass functionality prototyped 
within it, and that code migrated to the eclass.

For example- I've got kde eclass that's eapi0, and want to migrate to 
eapi1.  I break the kde eclass up so it's eapi1 compatible (at the 
func level), but with the exported src_compile eapi0 so I don't break 
the existance ebuilds.

I then add a shim eclass that *is* eapi1 (calling the chunked kde 
internals), and migrate pkgs gradually over to it.  Once it's fully in 
use, convert the underlying kde eclass to eapi1 (move the shim code 
in), leave the shim as indirection, and remove the shim from the 
testing ebuilds.

Or.

I want to migrate my kde eclass to eapi1.  Like any eclass change, I 
find what would be affected.  I break the kde eclass into eapi1 
compatible chunks (seperated configure and compile), with a eapi0 
src_compile that calls both.

I then go and clean up the ebuilds that override src_compile, and 
convert them over to the new funcs in the eclass (bumping the ebuild's 
eapi in the process).  Once they're addressed, all that remains is the 
ebuilds that use the eclasses exported src_compile; remove the eapi0 
src_compile shim in the eclass, and set EAPI=1 within the eclass.  All 
ebuilds that rely on the eclass to define eapi and the majority of 
their code (those that didn't required modification), are now eapi1, 
as is the eclass.

Heading off the "make the eclass eapi 0 and 1 compliant with portage 
choosing between them", what I'm stating is that the eclass and ebuilds 
authors do it, they be explicit and manage the conversion themselves 
(if they see the need to bump to eapiN).
They can manage the code and keep it sane, rather then trying to ifdef 
N different eapis and hoping portage doesn't screw up (it will screw 
up invariably, as will they in both my solution and yours).  

Mine just is simpler, from where I sit, less chance of screwups beyond 
usual bonehead mistakes people always make.

Either solution expects ebuild/eclass devs to get it right; having 
portage somehow swivel the eclass so that it's exported api/env is of 
a certain eapi expects the devs to have that code working properly, 
and for a *perfect* implementation that can do this without screwup.

It's not realistic, frankly.
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]