[gentoo-dev] RFC: fsmove to profiles/updates

[gentoo-dev] RFC: fsmove to profiles/updates

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 965 bytes --]

Hello,

Right now, every time a bigger bunch of stuff (installed by various
packages) needs to be moved around the filesystem, we have a lot of
work to handle it somehow. And finally, users end up having to either
rebuild a lot of packages to get the files in the new locations, or
we do ugly things to move those files for them.

I believe we should consider implementing something simpler. Thus,
I propose introducing the following new command to profiles/updates:

    fsmove <old-location> <new-location>

which -- at the moment of update -- will cause all PM-owned files
in the old-location to be moved to the new one (recursively), updating
the vdb as necessary.

What remains to be solved/decided:

1. How to treat non-owned files? (leave them there, refuse to proceed
with updates?)

2. How to handle relevant required updates? (packages which
actually *have* to be updated before moving files)

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] RFC: fsmove to profiles/updates

[Kent Fredric]

On 7 August 2012 19:52, Michał Górny <mgorny@gentoo.org> wrote:
> Hello,
>
> Right now, every time a bigger bunch of stuff (installed by various
> packages) needs to be moved around the filesystem, we have a lot of
> work to handle it somehow. And finally, users end up having to either
> rebuild a lot of packages to get the files in the new locations, or
> we do ugly things to move those files for them.
>
> I believe we should consider implementing something simpler. Thus,
> I propose introducing the following new command to profiles/updates:
>
>     fsmove <old-location> <new-location>
>
> which -- at the moment of update -- will cause all PM-owned files
> in the old-location to be moved to the new one (recursively), updating
> the vdb as necessary.
>
> What remains to be solved/decided:
>
> 1. How to treat non-owned files? (leave them there, refuse to proceed
> with updates?)
>
> 2. How to handle relevant required updates? (packages which
> actually *have* to be updated before moving files)
>

I suggest, that due to the volatility of such actions, a user should
have to approve each bulk move before it is done, to avoid breaking
things.

Sort of like etc-update:

An update file is added to the repository
PMS's detect the new update, and detect the update has not been
performed, and starts notifying the user that pending updates are
needed.
User performs action(s) when ready via some client ( eselect ? PMS
specific? ~~ )

Additionally, move batches could have annotations preceding them
indicating either instructional ( einfo ) or automated ( like emerge
--config ) to handle things like "you'll want to close postgresql
before you do this or you'll get database corruption" .

I guess what I'm saying basically, is a hybrid concept, sort-of like
eselect news , except with executable behaviour attached.


At least, thats my 2c.

-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

Re: [gentoo-dev] RFC: fsmove to profiles/updates

[Peter Stuge]

Kent Fredric wrote:
> I suggest, that due to the volatility of such actions, a user should
> have to approve each bulk move before it is done, to avoid breaking
> things.

Further thoughts about this:

* The move is needed for some reason.

* The person running emerge will in the common case not know the
  details; so they are in a bad position to make any decision on
  the matter.

* There will without a doubt be cases when things break regardless of
  how clever the users are.

Rather than adding a prompt for the user to have to care about
(everyone will answer yes all the time or no all the time anyway)
I suggest that the action be made easy to undo, so that when
something breaks it is possible and indeed easy to roll it back.

Not so easy to say what else must be rolled back together with the
fsmove!

Personally I hate eselect news, I would much like to disable it. I
prefer not adding more of the same. If an action is neccessary then
go ahead and do it automatically, but make it easy to undo, and undo
automatically on failure, as well as allow me to undo when I find a
problem.


//Peter

Re: [gentoo-dev] RFC: fsmove to profiles/updates

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 620 bytes --]

On Tue, 7 Aug 2012 15:10:35 +0200
Peter Stuge <peter@stuge.se> wrote:

> Rather than adding a prompt for the user to have to care about
> (everyone will answer yes all the time or no all the time anyway)
> I suggest that the action be made easy to undo, so that when
> something breaks it is possible and indeed easy to roll it back.
> 
> Not so easy to say what else must be rolled back together with the
> fsmove!

I don't think that's possible. Much like with other kinds of updates,
the packages in the tree would be updated to install in the new
location anyway.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] RFC: fsmove to profiles/updates

[Rich Freeman]

On Tue, Aug 7, 2012 at 9:58 AM, Michał Górny <mgorny@gentoo.org> wrote:
>
> I don't think that's possible. Much like with other kinds of updates,
> the packages in the tree would be updated to install in the new
> location anyway.
>

If I were faced with doing this manually I know the first thing I'd do
is run quickpkg on the affected packages.  Maybe something could be
done with that (though quickpkg is not part of @system).

However, in general big moves like this are never going to be easy to
recover from.  If you have sed scripts cleaning up config files or
whatever who knows what the previous values were.

I think any kind of large-scale directory moves are going to be risky
on a distro like Gentoo.  We should probably give them careful thought
before implementing them.  This isn't something like Ubuntu where you
practically wipe and re-install all of /usr a few times a year from
what amounts to a bunch of tarballs.

Rich

Re: [gentoo-dev] RFC: fsmove to profiles/updates

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 907 bytes --]

On Tue, 7 Aug 2012 10:48:01 -0400
Rich Freeman <rich0@gentoo.org> wrote:

> I think any kind of large-scale directory moves are going to be risky
> on a distro like Gentoo.  We should probably give them careful thought
> before implementing them.  This isn't something like Ubuntu where you
> practically wipe and re-install all of /usr a few times a year from
> what amounts to a bunch of tarballs.

Yes, they are risky and complex. And many things which possibly could
help us, simply don't work on Gentoo. Sometimes because of PMS
stupidity, sometimes because we feel like ugly hacks done by users
should simply work.

We already give a lot of thought and care to handle the moves but we
still miss some essential tool which could help us handle them better.
Most importantly, which would allow us to avoid forcing users to have
half-moved system.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] RFC: fsmove to profiles/updates

[Kent Fredric]

On 8 August 2012 01:58, Michał Górny <mgorny@gentoo.org> wrote:
> I don't think that's possible. Much like with other kinds of updates,
> the packages in the tree would be updated to install in the new
> location anyway.

Sure, but the question is "when does this happen". Users are expecting
such changes when they emerge a new package, but if you're on a system
that has versions pinned, you're not expecting magical changes to
happen during emerge sync

I'd hope at the very least there was a FEATURES= option to disable
automatic fs moves.

I can understand how most people will probably want to "just let moves
happen", but I still think you should still have a way to disable this
for people who have higher security concerns.

Some moves will need checks done to see if they can be done safely or
not, and some moves will require updating files in /etc/ to make them
work, so moving the files but *not* changing /etc/* forcibly could
easily lead to a broken system .

And this is especially the case if you're trying to move dirs which
contain a mix of user and installed content. ( ie: /var/db/postgres/ )

Some will be able to be performed hands-free, and others will *need*
some user interaction to avoid a broken system.

> --
> Best regards,
> Michał Górny



-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz