From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 3348 invoked by uid 1002); 12 Nov 2003 18:48:03 -0000 Mailing-List: contact gentoo-dev-help@gentoo.org; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@gentoo.org Received: (qmail 27680 invoked from network); 12 Nov 2003 18:47:56 -0000 From: Alastair Tse To: gentoo-dev@gentoo.org Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-0d6+aXSdhxvkmFOB7yC4" Message-Id: <1068662803.18867.134.camel@huggins.eng.cam.ac.uk> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.4 Date: Wed, 12 Nov 2003 18:46:44 +0000 X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ X-Cam-AntiVirus: No virus found X-Cam-SpamDetails: scanned, SpamAssassin (score=-4.9, PGP_SIGNATURE_2 -2.45, USER_AGENT_XIMIAN -2.35) Subject: [gentoo-dev] python-2.3.2 testing required X-Archives-Salt: e1c84005-db87-4a89-9a97-a6ec811aed95 X-Archives-Hash: 72392e1e45da34b5344742b9c35d0c5c --=-0d6+aXSdhxvkmFOB7yC4 Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Hi All, I need some more testing for python-2.3.2 before it is released into ~x86. This is a call to Gentoo developers and people interested in helping to test.=20 If you test, make sure you know what you are doing. Portage requires python. It is best to have a binary package of your current portage and python, just in case something majorly bad happens. I would greatly appreciate any feedback on bugs and niggles with the upgrade process. Report bugs on bugs.gentoo.org please. I'll list some of the changes, points of interest and errata. How to Upgrade: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 1. Unmask _both_ =3Ddev-lang/python-2.3* and >=3Dsys-apps/portage-2.0.49-r1= 6 in package.mask. 2. run: emerge -u portage python=20 (note that -r16 should work with both 2.2 and 2.3) 3. run: /usr/portage/dev-lang/python/python-updater Major Changes: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Unicode support has a choice of UCS2 or UCS4.=20 --------------------------------------------- UCS4 uses significantly more memory than UCS2. For those who are not familiar with unicode, UCS2 means representing unicode characters with 16 bit words and UCS4 means representing unicode characters in 32 bit words. Not that both Redhat (>9) and Debian (unstable) have python-2.3 compiled with UCS4 by default. Currently, you get UCS4 if you have "cjk" in your USE flags because the only language to use the extra pane in UCS4 are CJK langauges. I've been using UCS4 for a while now and it works without any problems.=20 The reason why I'm not making this default is because UCS4 python uses more memory. An example is supybot (Python IRC bot) that uses 8M for UCS2 and 13M for UCS4. But note that this example is not scientific because the machines were different in kernel version, compiler and compiler optimisations. I'm willing to listen to any ideas about whether to support UCS4 by default or not. The only problem that may occur is if you change between UCS2 and UCS4, you will need to recompile any external python codec packages like japanesecodec, koreancodec, cjkcodecs, iconvcodec. For those who want to know more about UCS and Unicode, you maybe interested in some articles like: http://uche.ogbuji.net/tech/akara/nodes/2003-01-01/characters http://www.python.org/cgi-bin/faqw.py?req=3Dshow&file=3Dfaq04.107.htp http://www.debian.org/doc/manuals/intro-i18n/ch-codes.en.html#s-unicodes-cc= s Handling *.pyc *.pyo --------------------- The only other major change in the handling of python is with byte-compiled modules. With previous versions of python, portage had a problem tracking *.pyc and *.pyo files. The reason is that these files were registered with portage at install time, but may be modified at runtime if either the timestamp on the python executable has changed or the version has been upgraded. This traditionally created two problems. Firstly, *.pyc and *.pyo were left behind because their timestamps and md5sum didn't match when they were installed[1]. The second problem is that sometimes python gets run inside sandbox and suddenly requests certain .pyc and .pyo to be re-compiled, creating sandbox violations. You will notice special workarounds in libsandbox to avoid this problem. The solution I am now employing is the not let portage track the .pyc and .pyo, and generating them at pkg_postinst(). For package removals, a pkg_postrm(), orphaned *.pyc *.pyo (orphaned means the .pyc that don't have a corresponding .py) will be removed. The other problem with python recompiling modules at runtime is now solved by setting an environment variable (PY_DONTCOMPILE) during merging that prevents python from attempting to recompile modules. This is now implemented in python.eclass and distutils.eclass. If you maintain anything that creates python modules, you should either read the eclass to find out what needs to change for python-2.3 or email me the packages that need to be looked at. Most packages that use distutils should work automatically with this new scheme. [1] .. http://bugs.gentoo.org/show_bug.cgi?id=3D8804 Python Updater -------------- A neccessary evil for the upgrade is to recompile all the modules that depend on Python. I use a very simple rule to determine that. The rule is if a package has a file in /usr/lib/python2.2, then it will need to be re-emerged. There is some special magic that does that in /usr/portage/dev-lang/python/files/python-updater which is able to get all the packages, sort the dependencies to avoid merging problems and output a log file to /tmp/python-updater.log. I'd appreciate as much feedback on this as possible. Especially on whether I should run it in pkg_postinst() to make sure people who don't read the trailing einfo's actually upgrade their python modules. It might be technically unsound for me to run it in pkg_postinst() (nested emerge's sound like a bad idea.) Other Python Improvements -------------------------- * New version, supposedly faster by 30%. * bsddb support for 4.x * Supports generators (very spiffy for implementing weightless threads) * A new idle (python gui console) Known Problems: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D * I haven't tried this on a install from scratch. Maybe someone can help me to create a stage with only 2.3 and see how things work. * Even though python-2.3.2 requires portage-2.0.49-r16, I can't put a dep in otherwise it may create a circular dependency. Probably the best solution would be to have a pkg_setup check before it goes ahead. * On some machines, portageq doesn't work after python upgrade. Make sure you are using the latest python version. (http://bugs.gentoo.org/show_bug.cgi?id=3D32374) Plan for Python's Future =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D * I'm aiming to push Python-2.3.2 to ~x86 in the next week or so. There are most likely packages that don't work under 2.3 that need to be upgraded. If you find one, please report it. * Maybe we should think about using a non version specific directory for Python modules to ease upgrade. That's all folks! Thanks, --=20 Alastair 'liquidx' Tse >> Gentoo Developer >> http://www.liquidx.net/ | http://dev.gentoo.org/~liquidx/ --=-0d6+aXSdhxvkmFOB7yC4 Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (GNU/Linux) iD8DBQA/soATOM4cezkHFPYRApk2AJwL5fdE4sS3gcL8ac2mtlnyY1jhmwCeNQnn vnv1dDrl+GQTF2i3VfBodCQ= =/l0e -----END PGP SIGNATURE----- --=-0d6+aXSdhxvkmFOB7yC4--