[gentoo-portage-dev] idea: strict run-time dependencies to remove perl-cleaner, python-updater...

[gentoo-portage-dev] idea: strict run-time dependencies to remove perl-cleaner, python-updater...

[pclouds]

This is an idea.
Everytime you update python, perl to a major version, you have to run
perl-cleaner/python-updater to re-emerge all packages that depend on
previous version. The situation is probably similar to kernel
packages. When you re-compile and install new kernel, you need to
re-emerge kernel packages to make sure it work properly. I'd rather a
consistent approach to solve these problems instead seperate updater
for separate programs such as python-updater, perl-cleaner.

The idea is to use a number to identify compatibility. Those versions
which have the same number are expected to work well with other
installed programs. If the number is different, then all other depend
programs should be re-emerged.

 When portage merge a package (A) to system, it allow the package to
specify a number (or string, whatever) to specify a compatibility
number. When other packages (e.g B) that depend on package A are
merged, it will keep A's special number in /var/db/pkg. If package A
is re-emerged and break compatibility, this number should change.
Otherwise, this number remains the same.
Those program like perl-cleaner, python-updater may benefit from these
numbers. It may check for numbers stored in /var/db/pkg's B and number
in A's. If these numbers differ, B should be re-emerged.
As for python, all python 2.4's number should be "2.4" and python
2.2.x  should be "2.2". Perl is similar. As for kernel, the number
should be timestamp. Suppose i can feed a kernel ebuild with a config
file :), then everytime i change config, re-emege kernel,
compatibility number should change. That's the signature for, say,
kernel-updater to update all kernel packages.

To implement this feature, ebuild should provide a function, say,
pkg_version() that return compatibility number. Portage should store
this number in /var/db/pkg. Also, when emerge an ebuild, portage
should store current compatibility numbers of all direct dependencies.
The rest of work is left to perl-cleaner, python-updater,
kernel-updater ..., checking compatibility numbers and update packages
appropriately.

How about this?
--
Bi Cờ Lao

-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] idea: strict run-time dependencies to remove perl-cleaner, python-updater...

[Finn Thain]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3952 bytes --]


On Wed, 21 Sep 2005, pclouds wrote:

> This is an idea.
> Everytime you update python, perl to a major version, you have to run
> perl-cleaner/python-updater to re-emerge all packages that depend on
> previous version. The situation is probably similar to kernel
> packages. When you re-compile and install new kernel, you need to
> re-emerge kernel packages to make sure it work properly. I'd rather a
> consistent approach to solve these problems instead seperate updater
> for separate programs such as python-updater, perl-cleaner.
> 
> The idea is to use a number to identify compatibility. Those versions
> which have the same number are expected to work well with other
> installed programs. If the number is different, then all other depend
> programs should be re-emerged.

I assume there is no such thing as a kernel-updater (at least there isn't 
one on my system). Your suggestion would make it easier to write one. But, 
since kernel symbol versioning is precise enough to detect ABI 
compatibility between versions, isn't the problem largely solved in the 
kernel? I'm not sure that such a script is needed other than to save you 
from having to figure out what pkg contains the module that just broke?

There is also the problem that some portion of gentoo users roll their own 
kernels, without telling portage.

So, if you wanted to write a kernel-updater, wouldn't it be better to 
exhaustively check the module symbols against the Module.symvers file? 
(There is also the vermagic field in modinfo, not sure where that comes 
from. I'm assuming CONFIG_MODVERSIONS, but those not using that hopefully 
know what they are doing anyway :-).

>  When portage merge a package (A) to system, it allow the package to
> specify a number (or string, whatever) to specify a compatibility
> number. When other packages (e.g B) that depend on package A are
> merged, it will keep A's special number in /var/db/pkg. If package A
> is re-emerged and break compatibility, this number should change.
> Otherwise, this number remains the same.
> Those program like perl-cleaner, python-updater may benefit from these
> numbers. It may check for numbers stored in /var/db/pkg's B and number
> in A's. If these numbers differ, B should be re-emerged.
> As for python, all python 2.4's number should be "2.4" and python
> 2.2.x  should be "2.2". Perl is similar. As for kernel, the number
> should be timestamp.

The rebuilder/module ebuilds don't need to know that version x is 
compatible with version x+n if the rebuilder is only ever run upon ebuild 
advise after an upgrade.

Consequently, you could just have portage record the actual versions of 
the deps that were in use at the time a pkg was emerged (if it doesn't 
already?). Then the rebuilder just has to look for any kernel modules that 
were emerged while depending on a kernel older than the present one. (I 
don't know if this would help with package.provided kernels and 
!CONFIG_MODVERSIONS kernels, it might.)

IMHO an exhaustive search is still the best approach.

(But don't ask me what you do if the module is not slotted and the 
original kernel is still installed as a fall back...)

-f

> Suppose i can feed a kernel ebuild with a config file :), then everytime 
> i change config, re-emege kernel, compatibility number should change. 
> That's the signature for, say, kernel-updater to update all kernel 
> packages.
> 
> To implement this feature, ebuild should provide a function, say,
> pkg_version() that return compatibility number. Portage should store
> this number in /var/db/pkg. Also, when emerge an ebuild, portage
> should store current compatibility numbers of all direct dependencies.
> The rest of work is left to perl-cleaner, python-updater,
> kernel-updater ..., checking compatibility numbers and update packages
> appropriately.
> 
> How about this?
> --
> Bi Cờ Lao
> 
> 

Re: [gentoo-portage-dev] idea: strict run-time dependencies to remove perl-cleaner, python-updater...

[pclouds]

Well, my aim is mainly perl, python, nautilus, gtk+ modules... As for
kernel modules, i'd rather all gentoo-provided kernel modules are
updated properly without exaustive search (too slow). People who
manually install kernel modules should take care of themselves :)

On 9/22/05, Finn Thain <fthain@telegraphics.com.au> wrote:
>
> On Wed, 21 Sep 2005, pclouds wrote:
>
> > This is an idea.
> > Everytime you update python, perl to a major version, you have to run
> > perl-cleaner/python-updater to re-emerge all packages that depend on
> > previous version. The situation is probably similar to kernel
> > packages. When you re-compile and install new kernel, you need to
> > re-emerge kernel packages to make sure it work properly. I'd rather a
> > consistent approach to solve these problems instead seperate updater
> > for separate programs such as python-updater, perl-cleaner.
> >
> > The idea is to use a number to identify compatibility. Those versions
> > which have the same number are expected to work well with other
> > installed programs. If the number is different, then all other depend
> > programs should be re-emerged.
>
> I assume there is no such thing as a kernel-updater (at least there isn't
> one on my system). Your suggestion would make it easier to write one. But,
> since kernel symbol versioning is precise enough to detect ABI
> compatibility between versions, isn't the problem largely solved in the
> kernel? I'm not sure that such a script is needed other than to save you
> from having to figure out what pkg contains the module that just broke?
>
> There is also the problem that some portion of gentoo users roll their own
> kernels, without telling portage.
>
> So, if you wanted to write a kernel-updater, wouldn't it be better to
> exhaustively check the module symbols against the Module.symvers file?
> (There is also the vermagic field in modinfo, not sure where that comes
> from. I'm assuming CONFIG_MODVERSIONS, but those not using that hopefully
> know what they are doing anyway :-).
>
> >  When portage merge a package (A) to system, it allow the package to
> > specify a number (or string, whatever) to specify a compatibility
> > number. When other packages (e.g B) that depend on package A are
> > merged, it will keep A's special number in /var/db/pkg. If package A
> > is re-emerged and break compatibility, this number should change.
> > Otherwise, this number remains the same.
> > Those program like perl-cleaner, python-updater may benefit from these
> > numbers. It may check for numbers stored in /var/db/pkg's B and number
> > in A's. If these numbers differ, B should be re-emerged.
> > As for python, all python 2.4's number should be "2.4" and python
> > 2.2.x  should be "2.2". Perl is similar. As for kernel, the number
> > should be timestamp.
>
> The rebuilder/module ebuilds don't need to know that version x is
> compatible with version x+n if the rebuilder is only ever run upon ebuild
> advise after an upgrade.
>
> Consequently, you could just have portage record the actual versions of
> the deps that were in use at the time a pkg was emerged (if it doesn't
> already?). Then the rebuilder just has to look for any kernel modules that
> were emerged while depending on a kernel older than the present one. (I
> don't know if this would help with package.provided kernels and
> !CONFIG_MODVERSIONS kernels, it might.)
>
> IMHO an exhaustive search is still the best approach.
>
> (But don't ask me what you do if the module is not slotted and the
> original kernel is still installed as a fall back...)
>
> -f
>
> > Suppose i can feed a kernel ebuild with a config file :), then everytime
> > i change config, re-emege kernel, compatibility number should change.
> > That's the signature for, say, kernel-updater to update all kernel
> > packages.
> >
> > To implement this feature, ebuild should provide a function, say,
> > pkg_version() that return compatibility number. Portage should store
> > this number in /var/db/pkg. Also, when emerge an ebuild, portage
> > should store current compatibility numbers of all direct dependencies.
> > The rest of work is left to perl-cleaner, python-updater,
> > kernel-updater ..., checking compatibility numbers and update packages
> > appropriately.
> >
> > How about this?
> > --
> > Bi Cờ Lao
> >
> >
>


--
Bi Cờ Lao

-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] idea: strict run-time dependencies to remove perl-cleaner, python-updater...

[Finn Thain]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 5620 bytes --]



On Thu, 22 Sep 2005, pclouds wrote:

> Well, my aim is mainly perl, python, nautilus, gtk+ modules... As for 
> kernel modules, i'd rather all gentoo-provided kernel modules are 
> updated properly without exaustive search (too slow).

I don't think it would be slow. I say that because it only takes the 
kernel an instant to fail insmod on a incorrectly versioned module, on a 
slow machine (25 MHz). However, it would be slow to rebuild more modules 
than necessary.

Preventing that requires a heuristic that says, "module foo compiled 
against baz-x will not work with new baz-y, where y > x+n". This n is 
often an unknown when a module is emerged, so these rule can't be stated 
in the module ebuild. They have to be stated in the rebuilder that 
ships with baz-y. But preparing those rules would make it more difficult 
to prepare each new kernel release.

I agree that portage could make rebuilders rather easy to write for those 
things where you have a simple rule like "modules built against dep 
version 2.x work with any dep version 2.y". But Linux kernel versioning 
doesn't produce simple rules, and exhaustive searches don't need them.

-f

> People who manually install kernel modules should take care of 
> themselves :)
>
> On 9/22/05, Finn Thain <fthain@telegraphics.com.au> wrote:
> >
> > On Wed, 21 Sep 2005, pclouds wrote:
> >
> > > This is an idea.
> > > Everytime you update python, perl to a major version, you have to run
> > > perl-cleaner/python-updater to re-emerge all packages that depend on
> > > previous version. The situation is probably similar to kernel
> > > packages. When you re-compile and install new kernel, you need to
> > > re-emerge kernel packages to make sure it work properly. I'd rather a
> > > consistent approach to solve these problems instead seperate updater
> > > for separate programs such as python-updater, perl-cleaner.
> > >
> > > The idea is to use a number to identify compatibility. Those versions
> > > which have the same number are expected to work well with other
> > > installed programs. If the number is different, then all other depend
> > > programs should be re-emerged.
> >
> > I assume there is no such thing as a kernel-updater (at least there isn't
> > one on my system). Your suggestion would make it easier to write one. But,
> > since kernel symbol versioning is precise enough to detect ABI
> > compatibility between versions, isn't the problem largely solved in the
> > kernel? I'm not sure that such a script is needed other than to save you
> > from having to figure out what pkg contains the module that just broke?
> >
> > There is also the problem that some portion of gentoo users roll their own
> > kernels, without telling portage.
> >
> > So, if you wanted to write a kernel-updater, wouldn't it be better to
> > exhaustively check the module symbols against the Module.symvers file?
> > (There is also the vermagic field in modinfo, not sure where that comes
> > from. I'm assuming CONFIG_MODVERSIONS, but those not using that hopefully
> > know what they are doing anyway :-).
> >
> > >  When portage merge a package (A) to system, it allow the package to
> > > specify a number (or string, whatever) to specify a compatibility
> > > number. When other packages (e.g B) that depend on package A are
> > > merged, it will keep A's special number in /var/db/pkg. If package A
> > > is re-emerged and break compatibility, this number should change.
> > > Otherwise, this number remains the same.
> > > Those program like perl-cleaner, python-updater may benefit from these
> > > numbers. It may check for numbers stored in /var/db/pkg's B and number
> > > in A's. If these numbers differ, B should be re-emerged.
> > > As for python, all python 2.4's number should be "2.4" and python
> > > 2.2.x  should be "2.2". Perl is similar. As for kernel, the number
> > > should be timestamp.
> >
> > The rebuilder/module ebuilds don't need to know that version x is
> > compatible with version x+n if the rebuilder is only ever run upon ebuild
> > advise after an upgrade.
> >
> > Consequently, you could just have portage record the actual versions of
> > the deps that were in use at the time a pkg was emerged (if it doesn't
> > already?). Then the rebuilder just has to look for any kernel modules that
> > were emerged while depending on a kernel older than the present one. (I
> > don't know if this would help with package.provided kernels and
> > !CONFIG_MODVERSIONS kernels, it might.)
> >
> > IMHO an exhaustive search is still the best approach.
> >
> > (But don't ask me what you do if the module is not slotted and the
> > original kernel is still installed as a fall back...)
> >
> > -f
> >
> > > Suppose i can feed a kernel ebuild with a config file :), then everytime
> > > i change config, re-emege kernel, compatibility number should change.
> > > That's the signature for, say, kernel-updater to update all kernel
> > > packages.
> > >
> > > To implement this feature, ebuild should provide a function, say,
> > > pkg_version() that return compatibility number. Portage should store
> > > this number in /var/db/pkg. Also, when emerge an ebuild, portage
> > > should store current compatibility numbers of all direct dependencies.
> > > The rest of work is left to perl-cleaner, python-updater,
> > > kernel-updater ..., checking compatibility numbers and update packages
> > > appropriately.
> > >
> > > How about this?
> > > --
> > > Bi Cờ Lao
> > >
> > >
> >
> 
> 
> --
> Bi Cờ Lao
> 
> 

Re: [gentoo-portage-dev] idea: strict run-time dependencies to remove perl-cleaner, python-updater...

[Petteri Räty]

[-- Attachment #1: Type: text/plain, Size: 583 bytes --]

Finn Thain wrote:
> On Wed, 21 Sep 2005, pclouds wrote:
> 
> 
> I assume there is no such thing as a kernel-updater (at least there isn't 
> one on my system). Your suggestion would make it easier to write one. But, 
> since kernel symbol versioning is precise enough to detect ABI 
> compatibility between versions, isn't the problem largely solved in the 
> kernel? I'm not sure that such a script is needed other than to save you 
> from having to figure out what pkg contains the module that just broke?
> 

It is called module-rebuild.

Regards,
Petteri Räty


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]