[gentoo-dev] preserving mtimes

[gentoo-dev] preserving mtimes

[Marijn Schouten (hkBst)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi list,

files installed by portage to ${ROOT} do not have the same time stamps as the same files
in ${D}. This creates problems for Common Lisp implementations and one Scheme
implementation in our overlay, explained in [1]. Current work-around is tarring up and
untarring to preserve mtimes. Fixes mentioned in [2] could reduce that hack to a touch of
some generated files to make them older than their sources, at least in our case.

Unfortunately not all package managers implement the same behaviour and I don't think PMS
says anything about it.

With reference counting implemented there doesn't seem to be any reason not to preserve
mtimes by default anymore and I think that would be the correct thing to do, but either
way I'd like PMS to specify what should happen wrt to mtimes, so that I can rely on that.

Marijn

[1]:http://bugs.gentoo.org/show_bug.cgi?id=16162#c32
[2]:http://bugs.gentoo.org/show_bug.cgi?id=16162#c52

- --
Marijn Schouten (hkBst), Gentoo Lisp project, Gentoo ML
<http://www.gentoo.org/proj/en/lisp/>, #gentoo-{lisp,ml} on FreeNode
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkgg30wACgkQp/VmCx0OL2xnQwCfayTo5PATYpCPRgcROP+9p0ES
jroAn3H2XJ103UC3V7XglDGSWZLHPDRH
=4pVG
-----END PGP SIGNATURE-----
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] preserving mtimes

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 750 bytes --]

On Wed, 07 May 2008 00:44:28 +0200
"Marijn Schouten (hkBst)" <hkBst@gentoo.org> wrote:
> and I think that would be the correct thing to do, but either way I'd
> like PMS to specify what should happen wrt to mtimes, so that I can
> rely on that.

PMS makes no guarantee as to what happens with mtimes, which means you
can't rely upon things happening one way or the other. This is
deliberate -- preserving mtimes leads to all kinds of weirdness on
packages that are generated from a raw tar file rather than from a
build system.

> Current work-around is tarring up and untarring to preserve mtimes.

That's not really any good either. The proper solution would be to fix
whatever it is that's mtime-sensitive.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] preserving mtimes

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 2033 bytes --]

On Wed, May 07, 2008 at 01:52:19AM +0100, Ciaran McCreesh wrote:
> On Wed, 07 May 2008 00:44:28 +0200
> "Marijn Schouten (hkBst)" <hkBst@gentoo.org> wrote:
> > and I think that would be the correct thing to do, but either way I'd
> > like PMS to specify what should happen wrt to mtimes, so that I can
> > rely on that.
> 
> PMS makes no guarantee as to what happens with mtimes, which means you
> can't rely upon things happening one way or the other. This is
> deliberate -- preserving mtimes leads to all kinds of weirdness on
> packages that are generated from a raw tar file rather than from a
> build system.

I'd be curious what weirdness you're referring to, since pkgcore has 
preserved mtime from the get go.

Yet to see a single issue from it, although *I* could think of a few 
instances where it would be problematic- that said, those pkgs already 
do postinst resetting of mtime to work around portage/paludis 
resetting mtime, so those issues doesn't appear.


> > Current work-around is tarring up and untarring to preserve mtimes.
> 
> That's not really any good either. The proper solution would be to fix
> whatever it is that's mtime-sensitive.

That's not exactly much of a solution; simplest example, that results 
in python.eclass:python_mod_compile, invoked during postinst to reset 
the cached bytecode mtimes (essentially).  Aside from this being 
uncontrolled/untracked access to the live fs, this slows down merges 
due to redundant work.

Finally, it also trashes the chksums that the manager records upon 
merging to the fs- so an mtime/chksum based unmerger can/will orphan 
those files.

Frankly, the mtime issue keeps rearing its head and needs killing- 
it's been an issue for near 4 years even, back in the OSX days we had 
to rewrite .a TOC's since the linker was mtime aware.

See no reason to preserve this misfeature.  Can't comment on paludis, 
but it shouldn't be an issue for portage to make the leap from what 
I've seen source wise.

~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] preserving mtimes

[Zac Medico]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Marijn Schouten (hkBst) wrote:
> Hi list,
> 
> files installed by portage to ${ROOT} do not have the same time stamps
> as the same files
> in ${D}.

Any timestamp difference here is not due to portage itself (since
portage-2.1.3). The timestamp change is usually due to the default
values of these variables which are defined in ebuild.sh:

export INSOPTIONS="-m0644"
export EXEOPTIONS="-m0755"
export LIBOPTIONS="-m0644"
export DIROPTIONS="-m0755"

It's currently possible for ebuilds to call the insopts, diropts,
exeopts, and libopts functions to modify these variables. If they
add the -p option, then timestamps will be preserved. I suppose we
can add -p to the default options if that's what everybody wants.

Zac
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAkghFakACgkQ/ejvha5XGaOLSQCeNOXhp5BY7pIeB/dfQ0lQGkEM
7doAoL9y/VH24DAQ9xDnmV4BlwB2Q5rt
=fW6M
-----END PGP SIGNATURE-----
-- 
gentoo-dev@lists.gentoo.org mailing list

[gentoo-dev] Re: preserving mtimes

[Steve Long]

Zac Medico wrote:
> It's currently possible for ebuilds to call the insopts, diropts,
> exeopts, and libopts functions to modify these variables. If they
> add the -p option, then timestamps will be preserved. I suppose we
> can add -p to the default options if that's what everybody wants.
> 
Gets my vote (or new-fangled backport from pkgcore if it's more efficient.)

-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] Re: preserving mtimes

[Zac Medico]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Steve Long wrote:
> Zac Medico wrote:
>> It's currently possible for ebuilds to call the insopts, diropts,
>> exeopts, and libopts functions to modify these variables. If they
>> add the -p option, then timestamps will be preserved. I suppose we
>> can add -p to the default options if that's what everybody wants.
>>
> Gets my vote (or new-fangled backport from pkgcore if it's more efficient.)
> 

It's a pretty simple thing to change. We just have to do a couple of
things:

1) Add -p to the default INSOPTIONS, EXEOPTIONS, LIBOPTIONS, and
DIROPTIONS variables.

2) Modify the corresponding insopts, diropts, exeopts, and libopts
functions to force -p into the options if the caller hasn't included
that option.

Zac
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAkgl97cACgkQ/ejvha5XGaOa0wCgnZsLdyaCi/pa8dP8EAWOl7EP
WzwAoMd+RCzpUpK+UCZ7zwizscIMg0jM
=lz2T
-----END PGP SIGNATURE-----
-- 
gentoo-dev@lists.gentoo.org mailing list