[gentoo-dev] rfc: moving default location of portage tree

[gentoo-dev] rfc: moving default location of portage tree

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 172 bytes --]

All,

is there a tracker for when the portage tree can be moved out of
/usr/portage by default?

If not, what is the status of us being able to do this?

Thanks,

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Alec Warner]

[-- Attachment #1: Type: text/plain, Size: 1108 bytes --]

On Mon, Jul 9, 2018 at 1:21 PM, William Hubbs <williamh@gentoo.org> wrote:

> All,
>
> is there a tracker for when the portage tree can be moved out of
> /usr/portage by default?


I suspect the answer is 'whenever' but that mostly depends on
implementation and what you want to accomplish.

Do you want:

 - All hosts everywhere to move from $CURRENT to $NEW?
 - Only new installs to move from $CURRENT to $NEW?


>
> If not, what is the status of us being able to do this?


The former is probably 3 times easier than the latter.
 - Get testers to move their tree and report issues[0].
 - Change the stage3 defaults to be the new location.
 - Explicitly do nothing else.

New installs will get the new location, old installs will get the old
location.

The latter is harder (one must design and execute a migration for existing
installs) and I suspect the value of such a migration is low and the risk
high.
Is there any advantage to migrating existing installs?

[0] A number of people already point PORTDIR at some other location and
appear to operate without major issues.


>
> Thanks,
>
> William
>
>

[-- Attachment #2: Type: text/html, Size: 1913 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Dennis Schridde]

[-- Attachment #1: Type: text/plain, Size: 309 bytes --]

On Monday, 9 July 2018 19:26:54 CEST Alec Warner wrote:
> [0] A number of people already point PORTDIR at some other location and
> appear to operate without major issues.

I do have it in /var/cache/portage/gentoo (alongside /var/cache/portage/
{distfiles,packages,local} and that works quite well.

--Dennis

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Mon, Jul 9, 2018 at 1:26 PM Alec Warner <antarus@gentoo.org> wrote:
>
> The former is probably 3 times easier than the latter.
>  - Get testers to move their tree and report issues[0].
>  - Change the stage3 defaults to be the new location.
>  - Explicitly do nothing else.
>
> New installs will get the new location, old installs will get the old location.
>
> [0] A number of people already point PORTDIR at some other location and appear to operate without major issues.
>

IMO it would be best to just fix this for new installs, and post a
news item about it with some instructions for users to migrate.  There
really isn't much to it though.  I'd think you could just move it at
the filesystem level and then change the pointers.  Or you can just
change the pointers and do a sync to pull down a fresh copy and clean
up later.

I'm not sure where we default distfiles to these days but that should
also go outside of /usr.  It should also not be a subdirectory of
PORTDIR.  Ideally you should just be able to rm -r $PORTDIR and then
do a sync and get it right back.

(Also, your email got former/latter swapped around.)

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 311 bytes --]

>>>>> On Mon, 9 Jul 2018, William Hubbs wrote:

> is there a tracker for when the portage tree can be moved out of
> /usr/portage by default?

> If not, what is the status of us being able to do this?

Please remind me, what was the plan for the new location?
Somewhere under /var/db or /var/lib, IIRC?

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Mon, Jul 9, 2018 at 1:40 PM Ulrich Mueller <ulm@gentoo.org> wrote:
>
> >>>>> On Mon, 9 Jul 2018, William Hubbs wrote:
>
> > is there a tracker for when the portage tree can be moved out of
> > /usr/portage by default?
>
> > If not, what is the status of us being able to do this?
>
> Please remind me, what was the plan for the new location?
> Somewhere under /var/db or /var/lib, IIRC?
>

I'd also consider /var/cache here as well.  FHS specifically suggests
using it for web caches and the like (let's set aside the issue with
making that global), though for the most part it is more metadata
caching.  A key principle is that it can be wiped without loss of
data, and I think that is generally true for the repository since it
can be synced.

Stuff in /var/lib can't be deleted without some kind of loss of
application state.  /var/db isn't in FHS, and I note that even mysql
sticks its stuff in /var/lib.

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[Johannes Huber]

[-- Attachment #1.1: Type: text/plain, Size: 1253 bytes --]



Am 09.07.2018 um 20:05 schrieb Rich Freeman:
> On Mon, Jul 9, 2018 at 1:40 PM Ulrich Mueller <ulm@gentoo.org> wrote:
>>
>>>>>>> On Mon, 9 Jul 2018, William Hubbs wrote:
>>
>>> is there a tracker for when the portage tree can be moved out of
>>> /usr/portage by default?
>>
>>> If not, what is the status of us being able to do this?
>>
>> Please remind me, what was the plan for the new location?
>> Somewhere under /var/db or /var/lib, IIRC?
>>
> 
> I'd also consider /var/cache here as well.  FHS specifically suggests
> using it for web caches and the like (let's set aside the issue with
> making that global), though for the most part it is more metadata
> caching.  A key principle is that it can be wiped without loss of
> data, and I think that is generally true for the repository since it
> can be synced.
> 
> Stuff in /var/lib can't be deleted without some kind of loss of
> application state.  /var/db isn't in FHS, and I note that even mysql
> sticks its stuff in /var/lib.
> 

Imho it would make sense to split up portage files with this change.
Move the tree (ebuilds, profiles etc) to /var/lib/... and the metadata
cache to /var/db as it can be regenerated out of the tree.

Best regards,
Johannes


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Mon, Jul 9, 2018 at 2:11 PM Johannes Huber <johu@gentoo.org> wrote:
>
> Am 09.07.2018 um 20:05 schrieb Rich Freeman:
> > On Mon, Jul 9, 2018 at 1:40 PM Ulrich Mueller <ulm@gentoo.org> wrote:
> >>
> >>>>>>> On Mon, 9 Jul 2018, William Hubbs wrote:
> >>
> >>> is there a tracker for when the portage tree can be moved out of
> >>> /usr/portage by default?
> >>
> >>> If not, what is the status of us being able to do this?
> >>
> >> Please remind me, what was the plan for the new location?
> >> Somewhere under /var/db or /var/lib, IIRC?
> >>
> >
> > I'd also consider /var/cache here as well.  FHS specifically suggests
> > using it for web caches and the like (let's set aside the issue with
> > making that global), though for the most part it is more metadata
> > caching.  A key principle is that it can be wiped without loss of
> > data, and I think that is generally true for the repository since it
> > can be synced.
> >
> > Stuff in /var/lib can't be deleted without some kind of loss of
> > application state.  /var/db isn't in FHS, and I note that even mysql
> > sticks its stuff in /var/lib.
> >
>
> Imho it would make sense to split up portage files with this change.
> Move the tree (ebuilds, profiles etc) to /var/lib/... and the metadata
> cache to /var/db as it can be regenerated out of the tree.
>

Are you talking about the metadata that gets synced as part of the
repository?  Conceptually I like the idea of splitting it out, but IMO
the whole repository is really just one big cache, so keeping it
together since it always has to be consistent isn't a huge problem.

If you're talking about the stuff in /var/cache/edb, then that should
be separate from the repository, but should still be in cache.

I'd probably create /var/cache/portage, with subdirectories for
repositories (with a subdir for each one synced by portage), edb,
distfiles, and binary packages.
/var/cache/portage/repos/main
/var/cache/portage/repos/my-favorite-overlay
/var/cache/portage/distfiles
/var/cache/portage/edb

The stuff in /var/db/pkg should probably go in /var/lib/portage/pkg or
something like that, at least long-term.

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 776 bytes --]

>>>>> On Mon, 9 Jul 2018, Rich Freeman wrote:

> I'd also consider /var/cache here as well.  FHS specifically suggests
> using it for web caches and the like (let's set aside the issue with
> making that global), though for the most part it is more metadata
> caching.  A key principle is that it can be wiped without loss of
> data, and I think that is generally true for the repository since it
> can be synced.

I don't think that criterium is fulfilled, because you cannot easily
restore the previous state after it's been wiped. At least not when
syncing from a rsync mirror (which may have been updated in the mean
time).

Also Portage doesn't treat it likea a cache, i.e. it doesn't start to
fetch ebuilds from remote if it doesn't find them in the local tree.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1332 bytes --]

On Mon, Jul 09, 2018 at 08:36:33PM +0200, Ulrich Mueller wrote:
> >>>>> On Mon, 9 Jul 2018, Rich Freeman wrote:
> 
> > I'd also consider /var/cache here as well.  FHS specifically suggests
> > using it for web caches and the like (let's set aside the issue with
> > making that global), though for the most part it is more metadata
> > caching.  A key principle is that it can be wiped without loss of
> > data, and I think that is generally true for the repository since it
> > can be synced.
> 
> I don't think that criterium is fulfilled, because you cannot easily
> restore the previous state after it's been wiped. At least not when
> syncing from a rsync mirror (which may have been updated in the mean
> time).
 
The criteria for /var/cache do not require being able to restore the
exact previous state; they just require that the application be able to
regenerate or restore the data , so they are definitely fulfilled.[1].

> Also Portage doesn't treat it likea a cache, i.e. it doesn't start to
> fetch ebuilds from remote if it doesn't find them in the local tree.

There is no definition of how a cache should be treated in fhs, so I
don't see this as an argument against /var/cache either.

William

[1] https://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#varcacheApplicationCacheData

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Zac Medico]

[-- Attachment #1.1: Type: text/plain, Size: 1231 bytes --]

On 07/09/2018 01:00 PM, William Hubbs wrote:
> On Mon, Jul 09, 2018 at 08:36:33PM +0200, Ulrich Mueller wrote:
>>>>>>> On Mon, 9 Jul 2018, Rich Freeman wrote:
>>
>>> I'd also consider /var/cache here as well.  FHS specifically suggests
>>> using it for web caches and the like (let's set aside the issue with
>>> making that global), though for the most part it is more metadata
>>> caching.  A key principle is that it can be wiped without loss of
>>> data, and I think that is generally true for the repository since it
>>> can be synced.
>>
>> I don't think that criterium is fulfilled, because you cannot easily
>> restore the previous state after it's been wiped. At least not when
>> syncing from a rsync mirror (which may have been updated in the mean
>> time).
>  
> The criteria for /var/cache do not require being able to restore the
> exact previous state; they just require that the application be able to
> regenerate or restore the data , so they are definitely fulfilled.[1].

If it's an rsync tree then we cannot restore the precise state, for
example you might not be able to rebuild one of your installed packages
if the corresponding ebuild has been removed upstream.
-- 
Thanks,
Zac


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Zac Medico]

[-- Attachment #1.1: Type: text/plain, Size: 1505 bytes --]

On 07/09/2018 01:07 PM, Zac Medico wrote:
> On 07/09/2018 01:00 PM, William Hubbs wrote:
>> On Mon, Jul 09, 2018 at 08:36:33PM +0200, Ulrich Mueller wrote:
>>>>>>>> On Mon, 9 Jul 2018, Rich Freeman wrote:
>>>
>>>> I'd also consider /var/cache here as well.  FHS specifically suggests
>>>> using it for web caches and the like (let's set aside the issue with
>>>> making that global), though for the most part it is more metadata
>>>> caching.  A key principle is that it can be wiped without loss of
>>>> data, and I think that is generally true for the repository since it
>>>> can be synced.
>>>
>>> I don't think that criterium is fulfilled, because you cannot easily
>>> restore the previous state after it's been wiped. At least not when
>>> syncing from a rsync mirror (which may have been updated in the mean
>>> time).
>>  
>> The criteria for /var/cache do not require being able to restore the
>> exact previous state; they just require that the application be able to
>> regenerate or restore the data , so they are definitely fulfilled.[1].
> 
> If it's an rsync tree then we cannot restore the precise state, for
> example you might not be able to rebuild one of your installed packages
> if the corresponding ebuild has been removed upstream.

Whoops I didn't mean to simply repeat what Ulrich said. My point is that
the spirit of the FHS might be that "nothing is lost", but you certainly
can lose some valuable state if it's an rsync tree.
-- 
Thanks,
Zac


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 618 bytes --]

W dniu pon, 09.07.2018 o godzinie 12∶21 -0500, użytkownik William Hubbs
napisał:
> All,
> 
> is there a tracker for when the portage tree can be moved out of
> /usr/portage by default?
> 
> If not, what is the status of us being able to do this?

sys-apps/portage-mgorny has already done that.  The defaults locations
have been changed to:

  DISTDIR="/var/cache/portage/distfiles"
  PKGDIR="/var/cache/portage/packages"
  RPMDIR="/var/cache/portage/rpm"

Plus repositories are in /var/db/repos/<name>.  This is also the layout
used by eselect-repository.

-- 
Best regards,
Michał Górny

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 583 bytes --]

On Mon, Jul 09, 2018 at 08:43:31PM +0200, Michał Górny wrote:
> sys-apps/portage-mgorny has already done that.  The defaults locations
> have been changed to:
> 
>   DISTDIR="/var/cache/portage/distfiles"
>   PKGDIR="/var/cache/portage/packages"
>   RPMDIR="/var/cache/portage/rpm"
> 
> Plus repositories are in /var/db/repos/<name>.  This is also the layout
> used by eselect-repository.

I like this idea, but slightly different; I think we should stay out of
/var/db. We don't want folks to go messing around in there and nuke
/var/db/pkg by mistake.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1031 bytes --]

W dniu pon, 09.07.2018 o godzinie 15∶11 -0500, użytkownik William Hubbs
napisał:
> On Mon, Jul 09, 2018 at 08:43:31PM +0200, Michał Górny wrote:
> > sys-apps/portage-mgorny has already done that.  The defaults locations
> > have been changed to:
> > 
> >   DISTDIR="/var/cache/portage/distfiles"
> >   PKGDIR="/var/cache/portage/packages"
> >   RPMDIR="/var/cache/portage/rpm"
> > 
> > Plus repositories are in /var/db/repos/<name>.  This is also the layout
> > used by eselect-repository.
> 
> I like this idea, but slightly different; I think we should stay out of
> /var/db. We don't want folks to go messing around in there and nuke
> /var/db/pkg by mistake.
> 

Following that reasoning, we shouldn't use /var at all because people
might 'go messing around in there and nuke /var/* by mistake'.  Or any
directory.  Our only hope is Windows where we can create P:\ and not
worry that people might 'go messing around in there and nuke the system
by mistake'.

-- 
Best regards,
Michał Górny

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Mon, Jul 9, 2018 at 4:13 PM Michał Górny <mgorny@gentoo.org> wrote:
>
> W dniu pon, 09.07.2018 o godzinie 15∶11 -0500, użytkownik William Hubbs
> napisał:
> > On Mon, Jul 09, 2018 at 08:43:31PM +0200, Michał Górny wrote:
> > > sys-apps/portage-mgorny has already done that.  The defaults locations
> > > have been changed to:
> > >
> > >   DISTDIR="/var/cache/portage/distfiles"
> > >   PKGDIR="/var/cache/portage/packages"
> > >   RPMDIR="/var/cache/portage/rpm"
> > >
> > > Plus repositories are in /var/db/repos/<name>.  This is also the layout
> > > used by eselect-repository.
> >
> > I like this idea, but slightly different; I think we should stay out of
> > /var/db. We don't want folks to go messing around in there and nuke
> > /var/db/pkg by mistake.
> >
>
> Following that reasoning, we shouldn't use /var at all because people
> might 'go messing around in there and nuke /var/* by mistake'.  Or any
> directory.  Our only hope is Windows where we can create P:\ and not
> worry that people might 'go messing around in there and nuke the system
> by mistake'.
>

++

Though I do prefer /var/lib or /var/cache over /var/db, simply because
/var/lib is actually in FHS.

That said, all my comments should be taken as suggestions.  I don't
really have a huge concern with most of these proposals.  Any of them
are better than /usr, with distfiles being stacked inside the repo
(ugh!).


-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1425 bytes --]

On Mon, Jul 09, 2018 at 04:53:43PM -0400, Rich Freeman wrote:
> On Mon, Jul 9, 2018 at 4:13 PM Michał Górny <mgorny@gentoo.org> wrote:
> >
> > W dniu pon, 09.07.2018 o godzinie 15∶11 -0500, użytkownik William Hubbs
> > napisał:
> > > On Mon, Jul 09, 2018 at 08:43:31PM +0200, Michał Górny wrote:
> > > > sys-apps/portage-mgorny has already done that.  The defaults locations
> > > > have been changed to:
> > > >
> > > >   DISTDIR="/var/cache/portage/distfiles"
> > > >   PKGDIR="/var/cache/portage/packages"
> > > >   RPMDIR="/var/cache/portage/rpm"
> > > >
> > > > Plus repositories are in /var/db/repos/<name>.  This is also the layout
> > > > used by eselect-repository.
> > >
> > > I like this idea, but slightly different; I think we should stay out of
> > > /var/db. We don't want folks to go messing around in there and nuke
> > > /var/db/pkg by mistake.
> > >
> >
> > Following that reasoning, we shouldn't use /var at all because people
> > might 'go messing around in there and nuke /var/* by mistake'.  Or any
> > directory.  Our only hope is Windows where we can create P:\ and not
> > worry that people might 'go messing around in there and nuke the system
> > by mistake'.
> >
> 
> ++
> 
> Though I do prefer /var/lib or /var/cache over /var/db, simply because
> /var/lib is actually in FHS.

Agreed, /var/db I guess is a Gentoo invention of some kind?

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Kristian Fiskerstrand]

[-- Attachment #1.1: Type: text/plain, Size: 720 bytes --]

On 07/09/2018 11:14 PM, William Hubbs wrote:
>> Though I do prefer /var/lib or /var/cache over /var/db, simply because
>> /var/lib is actually in FHS.
> Agreed, /var/db I guess is a Gentoo invention of some kind?

well, for a gentoo-based PMS that might not be a bad thing.. but I'd say
cache is out of the question, whether it is /var/lib or /var/db doesn't
matter too much to me, but it needs to be announced properly ahead of
time to adjust LVM2 volumes etc etc if impacting existing systems... I'd
mostly argue any such change should only affect new systems

-- 
Kristian Fiskerstrand
OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Mon, Jul 9, 2018 at 5:34 PM Kristian Fiskerstrand <k_f@gentoo.org> wrote:
>
> I'd mostly argue any such change should only affect new systems
>

++

If a user wants to migrate it is pretty easy to do.  Update the
setting and do an mv, or don't do an mv in which case it will just
regenerate.  I think /var/db/pkg is the only thing that is
particularly sensitive there (if users lose that then they have a mess
- probably recoverable if they do an emerge -e world and then go
hunting for orphans).

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[Zac Medico]

[-- Attachment #1.1: Type: text/plain, Size: 508 bytes --]

On 07/09/2018 02:34 PM, Kristian Fiskerstrand wrote:
> 
> I'd mostly argue any such change should only affect new systems
>

Yes, changing defaults for existing systems would be annoying.

My recommendation is to have catalyst set the new defaults in the stage
tarballs.

When sys-apps/portage changes its internal defaults, I'd like for the
upgrade process to call a tool that generates configuration files when
necessary to ensure that the existing paths remain constant.
-- 
Thanks,
Zac


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[M. J. Everitt]

[-- Attachment #1.1: Type: text/plain, Size: 701 bytes --]

On 09/07/18 23:12, Zac Medico wrote:
> On 07/09/2018 02:34 PM, Kristian Fiskerstrand wrote:
>> I'd mostly argue any such change should only affect new systems
>>
> Yes, changing defaults for existing systems would be annoying.
>
> My recommendation is to have catalyst set the new defaults in the stage
> tarballs.
>
> When sys-apps/portage changes its internal defaults, I'd like for the
> upgrade process to call a tool that generates configuration files when
> necessary to ensure that the existing paths remain constant.
I think it should be possible for RelEng to make a start on catalyst
updates - is there anything that would inhibit going ahead with this,
potentially?

MJE


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Zac Medico]

[-- Attachment #1.1: Type: text/plain, Size: 869 bytes --]

On 07/09/2018 03:27 PM, M. J. Everitt wrote:
> On 09/07/18 23:12, Zac Medico wrote:
>> On 07/09/2018 02:34 PM, Kristian Fiskerstrand wrote:
>>> I'd mostly argue any such change should only affect new systems
>>>
>> Yes, changing defaults for existing systems would be annoying.
>>
>> My recommendation is to have catalyst set the new defaults in the stage
>> tarballs.
>>
>> When sys-apps/portage changes its internal defaults, I'd like for the
>> upgrade process to call a tool that generates configuration files when
>> necessary to ensure that the existing paths remain constant.
> I think it should be possible for RelEng to make a start on catalyst
> updates - is there anything that would inhibit going ahead with this,
> potentially?

No, nothing. Whatever catalyst puts it the default config will become
our new default.
-- 
Thanks,
Zac


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1121 bytes --]

On Mon, Jul 09, 2018 at 03:54:35PM -0700, Zac Medico wrote:
> On 07/09/2018 03:27 PM, M. J. Everitt wrote:
> > On 09/07/18 23:12, Zac Medico wrote:
> >> On 07/09/2018 02:34 PM, Kristian Fiskerstrand wrote:
> >>> I'd mostly argue any such change should only affect new systems
> >>>
> >> Yes, changing defaults for existing systems would be annoying.
> >>
> >> My recommendation is to have catalyst set the new defaults in the stage
> >> tarballs.
> >>
> >> When sys-apps/portage changes its internal defaults, I'd like for the
> >> upgrade process to call a tool that generates configuration files when
> >> necessary to ensure that the existing paths remain constant.
> > I think it should be possible for RelEng to make a start on catalyst
> > updates - is there anything that would inhibit going ahead with this,
> > potentially?
> 
> No, nothing. Whatever catalyst puts it the default config will become
> our new default.

I would still like to see notice about what the new defaults are and how
to migrate current systems to them.


Thanks,

William

> -- 
> Thanks,
> Zac
> 




[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[M. J. Everitt]

[-- Attachment #1.1: Type: text/plain, Size: 1605 bytes --]

On 10/07/18 21:09, William Hubbs wrote:
> On Mon, Jul 09, 2018 at 03:54:35PM -0700, Zac Medico wrote:
>> On 07/09/2018 03:27 PM, M. J. Everitt wrote:
>>> On 09/07/18 23:12, Zac Medico wrote:
>>>> On 07/09/2018 02:34 PM, Kristian Fiskerstrand wrote:
>>>>> I'd mostly argue any such change should only affect new systems
>>>>>
>>>> Yes, changing defaults for existing systems would be annoying.
>>>>
>>>> My recommendation is to have catalyst set the new defaults in the stage
>>>> tarballs.
>>>>
>>>> When sys-apps/portage changes its internal defaults, I'd like for the
>>>> upgrade process to call a tool that generates configuration files when
>>>> necessary to ensure that the existing paths remain constant.
>>> I think it should be possible for RelEng to make a start on catalyst
>>> updates - is there anything that would inhibit going ahead with this,
>>> potentially?
>> No, nothing. Whatever catalyst puts it the default config will become
>> our new default.
> I would still like to see notice about what the new defaults are and how
> to migrate current systems to them.
>
>
> Thanks,
>
> William
>
>> -- 
>> Thanks,
>> Zac
>>
>
>
I'd like to propose that further to the discussion here on the -dev
mailing list, the Council discuss and make a firm proposal on the new
default paths, and then RelEng can make the appropriate updates to the
catalyst builds. A news item can be compiled, with an appropriate wiki
article perhaps on migration strategy (I may volunteer to format such a
page with some appropriate guidance).
Regards,
Michael / veremitz.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Jory A. Pratt]

[-- Attachment #1.1: Type: text/plain, Size: 2387 bytes --]

On 07/10/18 16:35, M. J. Everitt wrote:
> On 10/07/18 21:09, William Hubbs wrote:
>> On Mon, Jul 09, 2018 at 03:54:35PM -0700, Zac Medico wrote:
>>> On 07/09/2018 03:27 PM, M. J. Everitt wrote:
>>>> On 09/07/18 23:12, Zac Medico wrote:
>>>>> On 07/09/2018 02:34 PM, Kristian Fiskerstrand wrote:
>>>>>> I'd mostly argue any such change should only affect new systems
>>>>>>
>>>>> Yes, changing defaults for existing systems would be annoying.
>>>>>
>>>>> My recommendation is to have catalyst set the new defaults in the stage
>>>>> tarballs.
>>>>>
>>>>> When sys-apps/portage changes its internal defaults, I'd like for the
>>>>> upgrade process to call a tool that generates configuration files when
>>>>> necessary to ensure that the existing paths remain constant.
>>>> I think it should be possible for RelEng to make a start on catalyst
>>>> updates - is there anything that would inhibit going ahead with this,
>>>> potentially?
>>> No, nothing. Whatever catalyst puts it the default config will become
>>> our new default.
>> I would still like to see notice about what the new defaults are and how
>> to migrate current systems to them.
>>
>>
>> Thanks,
>>
>> William
>>
>>> -- 
>>> Thanks,
>>> Zac
>>>
>>
>>
> I'd like to propose that further to the discussion here on the -dev
> mailing list, the Council discuss and make a firm proposal on the new
> default paths, and then RelEng can make the appropriate updates to the
> catalyst builds. A news item can be compiled, with an appropriate wiki
> article perhaps on migration strategy (I may volunteer to format such a
> page with some appropriate guidance).
> Regards,
> Michael / veremitz.
> 
This is a mess, many systems are setup with portage already on a
seperate partition for reasons. What advantage does it provide to move
the tree now after all these years? I have seen nothing more then lets
do this cause I like the ideal lately and it is getting old, there is no
benefit that would justify moving the tree or many other changes that
are being made in Gentoo lately.



-- 
=======================================================
Jory A. Pratt
Gentoo Linux Developer [Mozilla Lead]
E-Mail    : anarchy@gentoo.org
GnuPG FP  : D4AC 8D63 0B16 F7C9 08E9  B909 A0CC C3BA B4D0 88B4
GnuPG ID  : B4D088B4
=======================================================


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Gordon Pettey]

On Wed, Jul 11, 2018 at 2:29 AM, Jory A. Pratt <anarchy@gentoo.org> wrote:
> This is a mess, many systems are setup with portage already on a
> seperate partition for reasons. What advantage does it provide to move
> the tree now after all these years? I have seen nothing more then lets
> do this cause I like the ideal lately and it is getting old, there is no
> benefit that would justify moving the tree or many other changes that
> are being made in Gentoo lately.

1. If you're able to mount /usr/portage from another filesystem, why
would you think it wouldn't work in with /var/cache/portage?
1a. If your system is already installed, why do you think this even
affects you? Did you read?
2. Pretty sure following FHS more closely is something most people
would see as a benefit.

Re: [gentoo-dev] rfc: moving default location of portage tree

[Raymond Jennings]

As long as an announcement is made in advance (perhaps as a NEWS item)
and portage itself is prepared to do an in-place migration if
necessary, I think things will be fine.

I do think it would be a wise idea to "grandfather" the current layout
for awhile.

On Wed, Jul 11, 2018 at 6:24 AM Gordon Pettey <petteyg359@gmail.com> wrote:
> On Wed, Jul 11, 2018 at 2:29 AM, Jory A. Pratt <anarchy@gentoo.org> wrote:
> > This is a mess, many systems are setup with portage already on a
> > seperate partition for reasons. What advantage does it provide to move
> > the tree now after all these years? I have seen nothing more then lets
> > do this cause I like the ideal lately and it is getting old, there is no
> > benefit that would justify moving the tree or many other changes that
> > are being made in Gentoo lately.
>
> 1. If you're able to mount /usr/portage from another filesystem, why
> would you think it wouldn't work in with /var/cache/portage?

> 1a. If your system is already installed, why do you think this even
> affects you? Did you read?

> 2. Pretty sure following FHS more closely is something most people
> would see as a benefit.

I agree on this point, and I always found /usr/portage to be...well, strange.



For me, though, the most important issue is giving end users advanced
notice and making sure nothing breaks.

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Wed, Jul 11, 2018 at 11:36 AM Raymond Jennings <shentino@gmail.com> wrote:
>
> I do think it would be a wise idea to "grandfather" the current layout
> for awhile.
>

I don't see why we would ever stop supporting it, at least in general.
Maybe if some day somebody had a solution for a read-only /usr with
signature checking that might require portage to be mounted elsewhere,
but I don't ever see that becoming the default.

Portage just looks for the repository where you tell it to.  If you
tell it that the repository is in /var, it will use it.  If you put it
in /tmp, that's fine too.

This is just about the default, which should follow FHS.  The case of
separate mounts is exactly why /usr is a bad spot - the access
patterns for something like the repository have far more in common
with /var than /usr.

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[Alec Warner]

[-- Attachment #1: Type: text/plain, Size: 1193 bytes --]

On Wed, Jul 11, 2018 at 11:56 AM, Rich Freeman <rich0@gentoo.org> wrote:

> On Wed, Jul 11, 2018 at 11:36 AM Raymond Jennings <shentino@gmail.com>
> wrote:
> >
> > I do think it would be a wise idea to "grandfather" the current layout
> > for awhile.
> >
>
> I don't see why we would ever stop supporting it, at least in general.
> Maybe if some day somebody had a solution for a read-only /usr with
> signature checking that might require portage to be mounted elsewhere,
> but I don't ever see that becoming the default.
>
> Portage just looks for the repository where you tell it to.  If you
> tell it that the repository is in /var, it will use it.  If you put it
> in /tmp, that's fine too.
>

+1 to this. The challenge (in moving it) is that its been "/usr/portage"
for a long time so many tools
may have hard coded this location; as opposed to querying portage for where
the tree is, e.g.:

PORTDIR=$(portageq get_repo_path / gentoo)

-A


> This is just about the default, which should follow FHS.  The case of
> separate mounts is exactly why /usr is a bad spot - the access
> patterns for something like the repository have far more in common
> with /var than /usr.
>
-- 
> Rich
>
>

[-- Attachment #2: Type: text/html, Size: 2051 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Nils Freydank]

[-- Attachment #1: Type: text/plain, Size: 786 bytes --]

Am Mittwoch, 11. Juli 2018, 18:19:39 CEST schrieb Alec Warner:
> [...]
> 
> +1 to this. The challenge (in moving it) is that its been "/usr/portage"
> for a long time so many tools
> may have hard coded this location; as opposed to querying portage for where
> the tree is, e.g.:
> 
> PORTDIR=$(portageq get_repo_path / gentoo)
> 
> -A
Some people including myself moved the tree to /var by variable definitions 
(and not wild mounting) a while ago. This configuration *is* supported for a 
while now but not the default and if tools break they have to be fixed anyway. 

(Side note: At least most of the common tools like gentoolkit parts or repoman 
work on my machines with the moved tree.)

- Nils

-- 
GPG fingerprint: '766B 8122 1342 6912 3401 492A 8B54 D7A3 FF3C DB17'
Holgersson

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Richard Yao]

> On Jul 11, 2018, at 11:56 AM, Rich Freeman <rich0@gentoo.org> wrote:
> 
>> On Wed, Jul 11, 2018 at 11:36 AM Raymond Jennings <shentino@gmail.com> wrote:
>> 
>> I do think it would be a wise idea to "grandfather" the current layout
>> for awhile.
>> 
> 
> I don't see why we would ever stop supporting it, at least in general.
> Maybe if some day somebody had a solution for a read-only /usr with
> signature checking that might require portage to be mounted elsewhere,
> but I don't ever see that becoming the default.
> 
> Portage just looks for the repository where you tell it to.  If you
> tell it that the repository is in /var, it will use it.  If you put it
> in /tmp, that's fine too.
> 
> This is just about the default, which should follow FHS.  The case of
> separate mounts is exactly why /usr is a bad spot - the access
> patterns for something like the repository have far more in common
> with /var than /usr.

On my system, /usr/portage is a separate mountpoint. There is no need to have on,h top level directories be separate mountpoints.
> 
> -- 
> Rich
> 

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Wed, Jul 11, 2018 at 4:34 PM Richard Yao <ryao@gentoo.org> wrote:
>
> On my system, /usr/portage is a separate mountpoint. There is no need to have on,h top level directories be separate mountpoints.

It makes sense to follow FHS.  Sure, I can work around poor designs by
sticking mount points all over the place, or manually setting my
config to put stuff in sane locations.  It makes more sense to put all
the volatile stuff in /var, than to mix it up all over the place and
get users to set up separate mountpoints to make up for it.

If somebody is doing a new Gentoo install, why would they want to put
the repository in /usr, and nest a few GB of distfiles inside of the
repo?  Why should that be the place we direct them?  There is no
history for them.  A brand new install should put things in the most
logical place.

By all means let existing users decide whether to move stuff.  I'm
sure we have plenty of users with make.conf in /etc/.

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[Richard Yao]

> On Jul 11, 2018, at 4:43 PM, Rich Freeman <rich0@gentoo.org> wrote:
> 
>> On Wed, Jul 11, 2018 at 4:34 PM Richard Yao <ryao@gentoo.org> wrote:
>> 
>> On my system, /usr/portage is a separate mountpoint. There is no need to have on,h top level directories be separate mountpoints.
> 
> It makes sense to follow FHS.  Sure, I can work around poor designs by
> sticking mount points all over the place, or manually setting my
> config to put stuff in sane locations.  It makes more sense to put all
> the volatile stuff in /var, than to mix it up all over the place and
> get users to set up separate mountpoints to make up for it.

Is it a violation of the FHS? /usr is for readonly data and the portage tree is generally readonly, except when being updated. The same is true of everything else in /usr.

I am confused as to how we only now realized it was a FHS violation when it has been there for ~15 years. I was under the impression that /usr was the correct place for it.
> 
> If somebody is doing a new Gentoo install, why would they want to put
> the repository in /usr, and nest a few GB of distfiles inside of the
> repo?  Why should that be the place we direct them?  There is no
> history for them.  A brand new install should put things in the most
> logical place.
> 
> By all means let existing users decide whether to move stuff.  I'm
> sure we have plenty of users with make.conf in /etc/.
> 
> -- 
> Rich
> 

Re: [gentoo-dev] rfc: moving default location of portage tree

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1556 bytes --]

W dniu śro, 11.07.2018 o godzinie 18∶11 -0400, użytkownik Richard Yao
napisał:
> > On Jul 11, 2018, at 4:43 PM, Rich Freeman <rich0@gentoo.org> wrote:
> > 
> > > On Wed, Jul 11, 2018 at 4:34 PM Richard Yao <ryao@gentoo.org> wrote:
> > > 
> > > On my system, /usr/portage is a separate mountpoint. There is no need to have on,h top level directories be separate mountpoints.
> > 
> > It makes sense to follow FHS.  Sure, I can work around poor designs by
> > sticking mount points all over the place, or manually setting my
> > config to put stuff in sane locations.  It makes more sense to put all
> > the volatile stuff in /var, than to mix it up all over the place and
> > get users to set up separate mountpoints to make up for it.
> 
> Is it a violation of the FHS? /usr is for readonly data and the portage tree is generally readonly, except when being updated. The same is true of everything else in /usr.
> 
> I am confused as to how we only now realized it was a FHS violation when it has been there for ~15 years. I was under the impression that /usr was the correct place for it.
> > 

And we're back to the usual Gentoo argument of 'it was like this for
N years'.  So FYI, something 'being there for ~15 years' doesn't make it
right.  It only means that:

a. Gentoo devs were wrong 15 years ago.

b. Gentoo devs are still wrong today.

c. Gentoo devs can't manage to make such a simple change because they're
too concerned about hurting somebody's feelings about a path.

-- 
Best regards,
Michał Górny

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Richard Yao]

> On Jul 11, 2018, at 6:23 PM, Michał Górny <mgorny@gentoo.org> wrote:
> 
> W dniu śro, 11.07.2018 o godzinie 18∶11 -0400, użytkownik Richard Yao
> napisał:
>>>> On Jul 11, 2018, at 4:43 PM, Rich Freeman <rich0@gentoo.org> wrote:
>>>> 
>>>> On Wed, Jul 11, 2018 at 4:34 PM Richard Yao <ryao@gentoo.org> wrote:
>>>> 
>>>> On my system, /usr/portage is a separate mountpoint. There is no need to have on,h top level directories be separate mountpoints.
>>> 
>>> It makes sense to follow FHS.  Sure, I can work around poor designs by
>>> sticking mount points all over the place, or manually setting my
>>> config to put stuff in sane locations.  It makes more sense to put all
>>> the volatile stuff in /var, than to mix it up all over the place and
>>> get users to set up separate mountpoints to make up for it.
>> 
>> Is it a violation of the FHS? /usr is for readonly data and the portage tree is generally readonly, except when being updated. The same is true of everything else in /usr.
>> 
>> I am confused as to how we only now realized it was a FHS violation when it has been there for ~15 years. I was under the impression that /usr was the correct place for it.
>>> 
> 
> And we're back to the usual Gentoo argument of 'it was like this for
> N years'.  So FYI, something 'being there for ~15 years' doesn't make it
> right.  It only means that:
> 
> a. Gentoo devs were wrong 15 years ago.
> 
> b. Gentoo devs are still wrong today.
> 
> c. Gentoo devs can't manage to make such a simple change because they're
> too concerned about hurting somebody's feelings about a path.
This does not answer my question. Is it really a FHS violation? The contents of /usr changes when doing updates using the system package manager. When not doing updates, it really is readonly and the FHS says that /usr is for readonly things. I do not see how it is different from anything else in /usr.

I have been thinking that having it there was compliant for years and honestly, I don’t see how it is not complaint. Saying it is not compliant is not an explanation.
> 
> -- 
> Best regards,
> Michał Górny

Re: [gentoo-dev] rfc: moving default location of portage tree

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2457 bytes --]

W dniu śro, 11.07.2018 o godzinie 18∶26 -0400, użytkownik Richard Yao
napisał:
> > On Jul 11, 2018, at 6:23 PM, Michał Górny <mgorny@gentoo.org> wrote:
> > 
> > W dniu śro, 11.07.2018 o godzinie 18∶11 -0400, użytkownik Richard Yao
> > napisał:
> > > > > On Jul 11, 2018, at 4:43 PM, Rich Freeman <rich0@gentoo.org> wrote:
> > > > > 
> > > > > On Wed, Jul 11, 2018 at 4:34 PM Richard Yao <ryao@gentoo.org> wrote:
> > > > > 
> > > > > On my system, /usr/portage is a separate mountpoint. There is no need to have on,h top level directories be separate mountpoints.
> > > > 
> > > > It makes sense to follow FHS.  Sure, I can work around poor designs by
> > > > sticking mount points all over the place, or manually setting my
> > > > config to put stuff in sane locations.  It makes more sense to put all
> > > > the volatile stuff in /var, than to mix it up all over the place and
> > > > get users to set up separate mountpoints to make up for it.
> > > 
> > > Is it a violation of the FHS? /usr is for readonly data and the portage tree is generally readonly, except when being updated. The same is true of everything else in /usr.
> > > 
> > > I am confused as to how we only now realized it was a FHS violation when it has been there for ~15 years. I was under the impression that /usr was the correct place for it.
> > > > 
> > 
> > And we're back to the usual Gentoo argument of 'it was like this for
> > N years'.  So FYI, something 'being there for ~15 years' doesn't make it
> > right.  It only means that:
> > 
> > a. Gentoo devs were wrong 15 years ago.
> > 
> > b. Gentoo devs are still wrong today.
> > 
> > c. Gentoo devs can't manage to make such a simple change because they're
> > too concerned about hurting somebody's feelings about a path.
> 
> This does not answer my question. Is it really a FHS violation? The contents of /usr changes when doing updates using the system package manager. When not doing updates, it really is readonly and the FHS says that /usr is for readonly things. I do not see how it is different from anything else in /usr.
> 

You are bending the definition to the limit.

1. Repository updates can be done as unprivileged user (and it's
generally insane to --sync as root when you can do it unprivileged!).

2. Package managers can update repository cache while *not* performing
system updates.  This is writing.

-- 
Best regards,
Michał Górny

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 510 bytes --]

[Please fix your mailer. Your message has a broken "References" header.]

>>>>> On Wed, 11 Jul 2018, Richard Yao wrote:

> This does not answer my question. Is it really a FHS violation? The
> contents of /usr changes when doing updates using the system package
> manager. When not doing updates, it really is readonly and the FHS
> says that /usr is for readonly things. I do not see how it is
> different from anything else in /usr.

What about a system that is building binpkgs? Or an rsync mirror?

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Dennis Schridde]

[-- Attachment #1: Type: text/plain, Size: 655 bytes --]

On Thursday, 12 July 2018 07:21:20 CEST Ulrich Mueller wrote:
> >>>>> On Wed, 11 Jul 2018, Richard Yao wrote:
> > This does not answer my question. Is it really a FHS violation? The
> > contents of /usr changes when doing updates using the system package
> > manager. When not doing updates, it really is readonly and the FHS
> > says that /usr is for readonly things. I do not see how it is
> > different from anything else in /usr.
> 
> What about a system that is building binpkgs? Or an rsync mirror?

There are systems building software for other systems using $ROOT, too.  They 
need to download into distfiles and sometimes they build binpkgs, too.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Wed, Jul 11, 2018 at 6:11 PM Richard Yao <ryao@gentoo.org> wrote:
>
> Is it a violation of the FHS? /usr is for readonly data and the portage tree is generally readonly, except when being updated. The same is true of everything else in /usr.
>

It is application metadata.  It belongs in /var.  No other packages
write to /usr when they're doing internal updates.  Obviously you need
a writable /usr to actually install package changes, but that
shouldn't be necessary just to sync the repository.

I was asking around and it seems like most distros stick their
repositories in /var/lib.  I can't imagine that too many would have
even considered sticking them in /usr.

> I am confused as to how we only now realized it was a FHS violation when it has been there for ~15 years. I was under the impression that /usr was the correct place for it.

It has certainly been pointed out in the past.  Nothing was changed
for the same reason that nothing will probably be changed this time -
people don't like change and the people who know better just slowly
patch around Gentoo's oddities.  Somebody was just posting a manifesto
about deploying more experimental technologies, and here we can't move
a repository out of /usr.

And if nothing else, can we at least move /usr/portage/distfiles
someplace else?  Surely you have to agree that this doesn't belong in
usr, or nested in the middle of a repository?

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[Richard Yao]

> On Jul 11, 2018, at 6:24 PM, Rich Freeman <rich0@gentoo.org> wrote:
> 
>> On Wed, Jul 11, 2018 at 6:11 PM Richard Yao <ryao@gentoo.org> wrote:
>> 
>> Is it a violation of the FHS? /usr is for readonly data and the portage tree is generally readonly, except when being updated. The same is true of everything else in /usr.
>> 
> 
> It is application metadata.  It belongs in /var.  No other packages
> write to /usr when they're doing internal updates.  Obviously you need
> a writable /usr to actually install package changes, but that
> shouldn't be necessary just to sync the repository.
> 
> I was asking around and it seems like most distros stick their
> repositories in /var/lib.  I can't imagine that too many would have
> even considered sticking them in /usr.
I would consider the package manager to be special in that it is a step of the system update process, but I agree that it could be nicer to have in /var.

We have a problem using /var/lib because /var/lib/portage is already in use. I guess /var/portage is not  a terrible choice.
> 
>> I am confused as to how we only now realized it was a FHS violation when it has been there for ~15 years. I was under the impression that /usr was the correct place for it.
> 
> It has certainly been pointed out in the past.  Nothing was changed
> for the same reason that nothing will probably be changed this time -
> people don't like change and the people who know better just slowly
> patch around Gentoo's oddities.  Somebody was just posting a manifesto
> about deploying more experimental technologies, and here we can't move
> a repository out of /usr.
> 
> And if nothing else, can we at least move /usr/portage/distfiles
> someplace else?  Surely you have to agree that this doesn't belong in
> usr, or nested in the middle of a repository?
Well, if it is changed during system updates, then the same logic applies, but quite honestly, I never liked having the distfiles or packages directories there. You can have unnecessary headaches when mounting inside a mount point other than /.
> 
> -- 
> Rich
> 

Re: [gentoo-dev] rfc: moving default location of portage tree

[Vadim A. Misbakh-Soloviov]

> I guess /var/portage is not  a terrible choice.

Well, I double that.
I've already use the following structure:
|- /var/portage/
|	|- repos
|	|	|- gentoo
|	|	|- reponame1
|	|	|- reponame2
|	|- distfiles
|	|	|- ...
|	|	|- ...
|	|- packages
|	|	|- ...
|	|	|- ...
|	|- meta
|		|- layman
|			|- ... # (used to be used for layman stuff until eselect-repository)


And I think it's pretty fine and usefull, especially because distfiles/
packages does not belong to one specific repo (gentoo) as it does with /usr/
portage.

Re: [gentoo-dev] rfc: moving default location of portage tree

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1853 bytes --]

On Wed, Jul 11, 2018 at 06:24:20PM -0400, Rich Freeman wrote:
> On Wed, Jul 11, 2018 at 6:11 PM Richard Yao <ryao@gentoo.org> wrote:
> >
> > Is it a violation of the FHS? /usr is for readonly data and the portage tree is generally readonly, except when being updated. The same is true of everything else in /usr.
> >
> 
> It is application metadata.  It belongs in /var.  No other packages
> write to /usr when they're doing internal updates.  Obviously you need
> a writable /usr to actually install package changes, but that
> shouldn't be necessary just to sync the repository.
> 
> I was asking around and it seems like most distros stick their
> repositories in /var/lib.  I can't imagine that too many would have
> even considered sticking them in /usr.

That is the other part of this debate, some are saying /var/lib, and
others are saying /var/db.

 It turns out that /var/db is much more common than I thought it was
 (it exists in all *bsd variants at least), so that could be an argument
 for putting the repos in there.

> > I am confused as to how we only now realized it was a FHS violation when it has been there for ~15 years. I was under the impression that /usr was the correct place for it.
> 
> It has certainly been pointed out in the past.  Nothing was changed
> for the same reason that nothing will probably be changed this time -
> people don't like change and the people who know better just slowly
> patch around Gentoo's oddities.  Somebody was just posting a manifesto
> about deploying more experimental technologies, and here we can't move
> a repository out of /usr.

Another reason this couldn't be changed in the past was catalyst had a
lot of hard coded references to /usr/portage. This has been fixe. in
catalyst-3 and I understand that releng is now using catalyst-3.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Wed, Jul 11, 2018 at 11:16 PM William Hubbs <williamh@gentoo.org> wrote:
>
> That is the other part of this debate, some are saying /var/lib, and
> others are saying /var/db.
>
>  It turns out that /var/db is much more common than I thought it was
>  (it exists in all *bsd variants at least), so that could be an argument
>  for putting the repos in there.
>

FreeBSD does not put the package repository in /var/db.

Portage cloned many of the file paths from FreeBSD along with the
name/concept.  FreeBSD puts their package repository in /usr/ports, so
I'm not sure I'd hold them out as a great example of FHS.

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[Raymond Jennings]

Just for the record, but would putting a setting inside
/etc/portage/make.conf be the appropriate way to handle this?

Re: [gentoo-dev] rfc: moving default location of portage tree

[Alec Warner]

[-- Attachment #1: Type: text/plain, Size: 352 bytes --]

On Thu, Jul 12, 2018 at 1:43 PM, Raymond Jennings <shentino@gmail.com>
wrote:

> Just for the record, but would putting a setting inside
> /etc/portage/make.conf be the appropriate way to handle this?
>
>
The settings already exist (and have existed for 10 years.) This bikeshed
discussion is literally trying to decide what the default should be.

-A

[-- Attachment #2: Type: text/html, Size: 780 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Raymond Jennings]

In that case, I vote for /var/cache/portage, since that's literally
what purpose it serves.  Namely, the cache of the gentoo infra's
current copy of teh portage tree.

On Thu, Jul 12, 2018 at 11:00 AM Alec Warner <antarus@gentoo.org> wrote:
>
> On Thu, Jul 12, 2018 at 1:43 PM, Raymond Jennings <shentino@gmail.com> wrote:
>>
>> Just for the record, but would putting a setting inside
>> /etc/portage/make.conf be the appropriate way to handle this?
>>
>
> The settings already exist (and have existed for 10 years.) This bikeshed discussion is literally trying to decide what the default should be.
>
> -A
>

Re: [gentoo-dev] rfc: moving default location of portage tree

[konsolebox]

[-- Attachment #1: Type: text/plain, Size: 793 bytes --]

I have /var/lib/gentoo/portage defined in repos.conf/gentoo.conf.

On Fri, Jul 13, 2018, 2:50 AM Raymond Jennings <shentino@gmail.com> wrote:

> In that case, I vote for /var/cache/portage, since that's literally
> what purpose it serves.  Namely, the cache of the gentoo infra's
> current copy of teh portage tree.
>
> On Thu, Jul 12, 2018 at 11:00 AM Alec Warner <antarus@gentoo.org> wrote:
> >
> > On Thu, Jul 12, 2018 at 1:43 PM, Raymond Jennings <shentino@gmail.com>
> wrote:
> >>
> >> Just for the record, but would putting a setting inside
> >> /etc/portage/make.conf be the appropriate way to handle this?
> >>
> >
> > The settings already exist (and have existed for 10 years.) This
> bikeshed discussion is literally trying to decide what the default should
> be.
> >
> > -A
> >
>
>

[-- Attachment #2: Type: text/html, Size: 1306 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Thu, Jul 12, 2018 at 3:34 PM konsolebox <konsolebox@gmail.com> wrote:
>
> I have /var/lib/gentoo/portage defined in repos.conf/gentoo.conf.
>

Regardless of the base directory location, I might suggest a path
dedicated to repositories, of which the main gentoo repo is just an
initial one, and overlays could be placed as additional directories
inside.

/var/lib/portage/repositories/main
/var/lib/gentoo/repositories/main
/var/cache/gentoo/repos/main
/var/cache/portage/repositories/gentoo

(Something along those lines.)

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[Brian Dolbec]

On Thu, 12 Jul 2018 11:49:37 -0700
Raymond Jennings <shentino@gmail.com> wrote:

> In that case, I vote for /var/cache/portage, since that's literally
> what purpose it serves.  Namely, the cache of the gentoo infra's
> current copy of teh portage tree.
> 
> On Thu, Jul 12, 2018 at 11:00 AM Alec Warner <antarus@gentoo.org>
> wrote:
> >
> > On Thu, Jul 12, 2018 at 1:43 PM, Raymond Jennings
> > <shentino@gmail.com> wrote:  
> >>
> >> Just for the record, but would putting a setting inside
> >> /etc/portage/make.conf be the appropriate way to handle this?
> >>  
> >
> > The settings already exist (and have existed for 10 years.) This
> > bikeshed discussion is literally trying to decide what the default
> > should be.
> >
> > -A
> >  
> 

This is not a personal attack against you.  Just picked this one to say
something again...


PLEASE, PLEASE stop calling it the "portage" tree.  The repo name is
"gentoo".  "portage is the default package manager, but not the only
choice.  Far too often, it takes awhile to figure out what someone is
trying to say because of that confusion between the tree and the
package manager.

PLUS, it has been decided already long ago that the directory name
should reflect the repository name.  We have been enforcing that rule
for overlays for a long time.  It has just been taking a long time to
get our tooling in order so that we can change our own to follow that
rule.

So, "portage" should not be a directory name in the new default path.

-- 
Brian Dolbec <dolsen>

Re: [gentoo-dev] rfc: moving default location of portage tree

[Rich Freeman]

On Thu, Jul 12, 2018 at 3:47 PM Brian Dolbec <dolsen@gentoo.org> wrote:
>
> So, "portage" should not be a directory name in the new default path.
>

Well, in my examples I proposed it as that is the software that
created the path, but then again in the spirit of PMS portage isn't
the only PM.

So:
/var/lib/repos/gentoo ?

-- 
Rich

Re: [gentoo-dev] rfc: moving default location of portage tree

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 608 bytes --]

W dniu czw, 12.07.2018 o godzinie 15∶51 -0400, użytkownik Rich Freeman
napisał:
> On Thu, Jul 12, 2018 at 3:47 PM Brian Dolbec <dolsen@gentoo.org> wrote:
> > 
> > So, "portage" should not be a directory name in the new default path.
> > 
> 
> Well, in my examples I proposed it as that is the software that
> created the path, but then again in the spirit of PMS portage isn't
> the only PM.
> 
> So:
> /var/lib/repos/gentoo ?
> 

Subdirectories of /var/lib should be named after the tool/package name. 
There's no tool or package called 'repos'.

-- 
Best regards,
Michał Górny

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1225 bytes --]

On Thu, Jul 12, 2018 at 10:13:57PM +0200, Michał Górny wrote:
> W dniu czw, 12.07.2018 o godzinie 15∶51 -0400, użytkownik Rich Freeman
> napisał:
> > On Thu, Jul 12, 2018 at 3:47 PM Brian Dolbec <dolsen@gentoo.org> wrote:
> > > 
> > > So, "portage" should not be a directory name in the new default path.
> > > 
> > 
> > Well, in my examples I proposed it as that is the software that
> > created the path, but then again in the spirit of PMS portage isn't
> > the only PM.
> > 
> > So:
> > /var/lib/repos/gentoo ?
> > 
> 
> Subdirectories of /var/lib should be named after the tool/package name. 
> There's no tool or package called 'repos'.

Technically mgorny is correct here. FHS requires that everything under
/var/lib be under a directory for the package or for the distro [1].
Note the comment about packaging support in 5.8.1.

Based on that this is my thought:

* /var/lib/portage is for portage specific stuff -- maybe even /var/db/pkg
in the future should go to /var/lib/portage/pkg.

* /var/lib/gentoo, on the other hand, could be where repos, distfiles
and binpkgs go.
William

[1]
http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#varlibVariableStateInformation

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[konsolebox]

On Fri, Jul 13, 2018 at 5:12 AM, William Hubbs <williamh@gentoo.org> wrote:
> On Thu, Jul 12, 2018 at 10:13:57PM +0200, Michał Górny wrote:
>> W dniu czw, 12.07.2018 o godzinie 15∶51 -0400, użytkownik Rich Freeman
>> napisał:
>> > On Thu, Jul 12, 2018 at 3:47 PM Brian Dolbec <dolsen@gentoo.org> wrote:
>> > >
>> > > So, "portage" should not be a directory name in the new default path.
>> > >
>> >
>> > Well, in my examples I proposed it as that is the software that
>> > created the path, but then again in the spirit of PMS portage isn't
>> > the only PM.
>> >
>> > So:
>> > /var/lib/repos/gentoo ?
>> >
>>
>> Subdirectories of /var/lib should be named after the tool/package name.
>> There's no tool or package called 'repos'.
>
> Technically mgorny is correct here. FHS requires that everything under
> /var/lib be under a directory for the package or for the distro [1].
> Note the comment about packaging support in 5.8.1.
>
> Based on that this is my thought:
>
> * /var/lib/portage is for portage specific stuff -- maybe even /var/db/pkg
> in the future should go to /var/lib/portage/pkg.
>
> * /var/lib/gentoo, on the other hand, could be where repos, distfiles
> and binpkgs go.
> William
>
> [1]
> http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#varlibVariableStateInformation

/var/lib/gentoo/portage then. (Or /var/lib/gentoo/repos/gentoo if you
care about PMS diplomacy.)

People can just move it somewhere and/or use symbolic links if they
want to use a different path.

Besides having /var/lib/gentoo/portage being set as "PORTDIR", I also
have DISTDIR=/var/lib/gentoo/distfiles
and PKGDIR=/var/lib/gentoo/packages.

-- 
konsolebox

Re: [gentoo-dev] rfc: moving default location of portage tree

[Raymond Jennings]

On Thu, Jul 12, 2018 at 12:47 PM Brian Dolbec <dolsen@gentoo.org> wrote:
>
> On Thu, 12 Jul 2018 11:49:37 -0700
> Raymond Jennings <shentino@gmail.com> wrote:
>
> > In that case, I vote for /var/cache/portage, since that's literally
> > what purpose it serves.  Namely, the cache of the gentoo infra's
> > current copy of teh portage tree.
> >
> > On Thu, Jul 12, 2018 at 11:00 AM Alec Warner <antarus@gentoo.org>
> > wrote:
> > >
> > > On Thu, Jul 12, 2018 at 1:43 PM, Raymond Jennings
> > > <shentino@gmail.com> wrote:
> > >>
> > >> Just for the record, but would putting a setting inside
> > >> /etc/portage/make.conf be the appropriate way to handle this?
> > >>
> > >
> > > The settings already exist (and have existed for 10 years.) This
> > > bikeshed discussion is literally trying to decide what the default
> > > should be.
> > >
> > > -A
> > >
> >
>
> This is not a personal attack against you.  Just picked this one to say
> something again...
>
>
> PLEASE, PLEASE stop calling it the "portage" tree.  The repo name is
> "gentoo".  "portage is the default package manager, but not the only
> choice.  Far too often, it takes awhile to figure out what someone is
> trying to say because of that confusion between the tree and the
> package manager.

Point of order:

http://distfiles.gentoo.org/snapshots and numerous pieces of
documentation call it "portage"

The confusion is ingrained by documentation.

> PLUS, it has been decided already long ago that the directory name
> should reflect the repository name.  We have been enforcing that rule
> for overlays for a long time.  It has just been taking a long time to
> get our tooling in order so that we can change our own to follow that
> rule.
>
> So, "portage" should not be a directory name in the new default path.
>
> --
> Brian Dolbec <dolsen>
>
>

Re: [gentoo-dev] rfc: moving default location of portage tree

[Brian Dolbec]

On Thu, 12 Jul 2018 17:35:41 -0700
Raymond Jennings <shentino@gmail.com> wrote:

> On Thu, Jul 12, 2018 at 12:47 PM Brian Dolbec <dolsen@gentoo.org>
> wrote:
> >
> > On Thu, 12 Jul 2018 11:49:37 -0700
> > Raymond Jennings <shentino@gmail.com> wrote:
> >  
> > > In that case, I vote for /var/cache/portage, since that's
> > > literally what purpose it serves.  Namely, the cache of the
> > > gentoo infra's current copy of teh portage tree.
> > >
> > > On Thu, Jul 12, 2018 at 11:00 AM Alec Warner <antarus@gentoo.org>
> > > wrote:  
> > > >
> > > > On Thu, Jul 12, 2018 at 1:43 PM, Raymond Jennings
> > > > <shentino@gmail.com> wrote:  
> > > >>
> > > >> Just for the record, but would putting a setting inside
> > > >> /etc/portage/make.conf be the appropriate way to handle this?
> > > >>  
> > > >
> > > > The settings already exist (and have existed for 10 years.) This
> > > > bikeshed discussion is literally trying to decide what the
> > > > default should be.
> > > >
> > > > -A
> > > >  
> > >  
> >
> > This is not a personal attack against you.  Just picked this one to
> > say something again...
> >
> >
> > PLEASE, PLEASE stop calling it the "portage" tree.  The repo name is
> > "gentoo".  "portage is the default package manager, but not the only
> > choice.  Far too often, it takes awhile to figure out what someone
> > is trying to say because of that confusion between the tree and the
> > package manager.  
> 
> Point of order:
> 
> http://distfiles.gentoo.org/snapshots and numerous pieces of
> documentation call it "portage"
> 
> The confusion is ingrained by documentation.
> 

Yes, it is, and well we can't very well change the documentation until
we can get an end to this new default path bikeshed.  Council will
need to make the decision (soon I hope)...  Then we make all the changes
necessary.


-- 
Brian Dolbec <dolsen>

Re: [gentoo-dev] rfc: moving default location of portage tree

[konsolebox]

On Fri, Jul 13, 2018 at 3:47 AM, Brian Dolbec <dolsen@gentoo.org> wrote:
> On Thu, 12 Jul 2018 11:49:37 -0700
> Raymond Jennings <shentino@gmail.com> wrote:
>
>> In that case, I vote for /var/cache/portage, since that's literally
>> what purpose it serves.  Namely, the cache of the gentoo infra's
>> current copy of teh portage tree.
>>
>> On Thu, Jul 12, 2018 at 11:00 AM Alec Warner <antarus@gentoo.org>
>> wrote:
>> >
>> > On Thu, Jul 12, 2018 at 1:43 PM, Raymond Jennings
>> > <shentino@gmail.com> wrote:
>> >>
>> >> Just for the record, but would putting a setting inside
>> >> /etc/portage/make.conf be the appropriate way to handle this?
>> >>
>> >
>> > The settings already exist (and have existed for 10 years.) This
>> > bikeshed discussion is literally trying to decide what the default
>> > should be.
>> >
>> > -A
>> >
>>
>
> This is not a personal attack against you.  Just picked this one to say
> something again...
>
>
> PLEASE, PLEASE stop calling it the "portage" tree.  The repo name is
> "gentoo".  "portage is the default package manager, but not the only
> choice.  Far too often, it takes awhile to figure out what someone is
> trying to say because of that confusion between the tree and the
> package manager.
>
> PLUS, it has been decided already long ago that the directory name
> should reflect the repository name.  We have been enforcing that rule
> for overlays for a long time.  It has just been taking a long time to
> get our tooling in order so that we can change our own to follow that
> rule.
>
> So, "portage" should not be a directory name in the new default path.
>
> --
> Brian Dolbec <dolsen>

I don't mind calling ::gentoo as Gentoo's official ebuild repository,
but it also has been "a portage tree", and "the portage tree" by
default context. If you imply that people should change convention to
something more PMS friendly, be explicit, and perhaps make it
official, and the let them decide for themselves. Be fair at reminding
that it has been there, but it's better be changed for PMS's sake.
Don't make it look like the usage has always been wrong.

-- 
konsolebox

Re: [gentoo-dev] rfc: moving default location of portage tree

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 838 bytes --]

>>>>> On Fri, 13 Jul 2018, konsolebox  wrote:

> I don't mind calling ::gentoo as Gentoo's official ebuild repository,
> but it also has been "a portage tree", and "the portage tree" by
> default context. If you imply that people should change convention to
> something more PMS friendly, be explicit, and perhaps make it
> official, and the let them decide for themselves. Be fair at reminding
> that it has been there, but it's better be changed for PMS's sake.
> Don't make it look like the usage has always been wrong.

You may be surprised, but the word "Gentoo" doesn't even occur in the
main part (chapters 2 to 15) of the PMS document, except for one place
referring to "Gentoo's Catalyst tool".

Calling it "Gentoo repository" instead of "Portage tree" is purely a
matter of distro policy and has nothing to do with PMS.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[konsolebox]

On Fri, Jul 13, 2018 at 5:15 PM, Ulrich Mueller <ulm@gentoo.org> wrote:
>>>>>> On Fri, 13 Jul 2018, konsolebox  wrote:
>
>> I don't mind calling ::gentoo as Gentoo's official ebuild repository,
>> but it also has been "a portage tree", and "the portage tree" by
>> default context. If you imply that people should change convention to
>> something more PMS friendly, be explicit, and perhaps make it
>> official, and the let them decide for themselves. Be fair at reminding
>> that it has been there, but it's better be changed for PMS's sake.
>> Don't make it look like the usage has always been wrong.
>
> You may be surprised, but the word "Gentoo" doesn't even occur in the
> main part (chapters 2 to 15) of the PMS document, except for one place
> referring to "Gentoo's Catalyst tool".
>
> Calling it "Gentoo repository" instead of "Portage tree" is purely a
> matter of distro policy and has nothing to do with PMS.

PMS mentions "ebuild repository", hence "Gentoo's official ebuild
repository" vs. "Gentoo Portage Tree"; vs. "Which Portage tree do you
use?".

-- 
konsolebox

Re: [gentoo-dev] rfc: moving default location of portage tree

[Richard Yao]

[-- Attachment #1.1: Type: text/plain, Size: 2316 bytes --]

On 07/11/2018 03:29 AM, Jory A. Pratt wrote:
> On 07/10/18 16:35, M. J. Everitt wrote:
>> On 10/07/18 21:09, William Hubbs wrote:
>>> On Mon, Jul 09, 2018 at 03:54:35PM -0700, Zac Medico wrote:
>>>> On 07/09/2018 03:27 PM, M. J. Everitt wrote:
>>>>> On 09/07/18 23:12, Zac Medico wrote:
>>>>>> On 07/09/2018 02:34 PM, Kristian Fiskerstrand wrote:
>>>>>>> I'd mostly argue any such change should only affect new systems
>>>>>>>
>>>>>> Yes, changing defaults for existing systems would be annoying.
>>>>>>
>>>>>> My recommendation is to have catalyst set the new defaults in the stage
>>>>>> tarballs.
>>>>>>
>>>>>> When sys-apps/portage changes its internal defaults, I'd like for the
>>>>>> upgrade process to call a tool that generates configuration files when
>>>>>> necessary to ensure that the existing paths remain constant.
>>>>> I think it should be possible for RelEng to make a start on catalyst
>>>>> updates - is there anything that would inhibit going ahead with this,
>>>>> potentially?
>>>> No, nothing. Whatever catalyst puts it the default config will become
>>>> our new default.
>>> I would still like to see notice about what the new defaults are and how
>>> to migrate current systems to them.
>>>
>>>
>>> Thanks,
>>>
>>> William
>>>
>>>> -- 
>>>> Thanks,
>>>> Zac
>>>>
>>>
>>>
>> I'd like to propose that further to the discussion here on the -dev
>> mailing list, the Council discuss and make a firm proposal on the new
>> default paths, and then RelEng can make the appropriate updates to the
>> catalyst builds. A news item can be compiled, with an appropriate wiki
>> article perhaps on migration strategy (I may volunteer to format such a
>> page with some appropriate guidance).
>> Regards,
>> Michael / veremitz.
>>
> This is a mess, many systems are setup with portage already on a
> seperate partition for reasons. What advantage does it provide to move
> the tree now after all these years? I have seen nothing more then lets
> do this cause I like the ideal lately and it is getting old, there is no
> benefit that would justify moving the tree or many other changes that
> are being made in Gentoo lately.

People who want to move it could just set PORTDIR in make.conf. I don't
see any reason to move it either.
> 
> 
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 2773 bytes --]

On Wed, Jul 11, 2018 at 04:25:20PM -0400, Richard Yao wrote:
> On 07/11/2018 03:29 AM, Jory A. Pratt wrote:
> > On 07/10/18 16:35, M. J. Everitt wrote:
> >> On 10/07/18 21:09, William Hubbs wrote:
> >>> On Mon, Jul 09, 2018 at 03:54:35PM -0700, Zac Medico wrote:
> >>>> On 07/09/2018 03:27 PM, M. J. Everitt wrote:
> >>>>> On 09/07/18 23:12, Zac Medico wrote:
> >>>>>> On 07/09/2018 02:34 PM, Kristian Fiskerstrand wrote:
> >>>>>>> I'd mostly argue any such change should only affect new systems
> >>>>>>>
> >>>>>> Yes, changing defaults for existing systems would be annoying.
> >>>>>>
> >>>>>> My recommendation is to have catalyst set the new defaults in the stage
> >>>>>> tarballs.
> >>>>>>
> >>>>>> When sys-apps/portage changes its internal defaults, I'd like for the
> >>>>>> upgrade process to call a tool that generates configuration files when
> >>>>>> necessary to ensure that the existing paths remain constant.
> >>>>> I think it should be possible for RelEng to make a start on catalyst
> >>>>> updates - is there anything that would inhibit going ahead with this,
> >>>>> potentially?
> >>>> No, nothing. Whatever catalyst puts it the default config will become
> >>>> our new default.
> >>> I would still like to see notice about what the new defaults are and how
> >>> to migrate current systems to them.
> >>>
> >>>
> >>> Thanks,
> >>>
> >>> William
> >>>
> >>>> -- 
> >>>> Thanks,
> >>>> Zac
> >>>>
> >>>
> >>>
> >> I'd like to propose that further to the discussion here on the -dev
> >> mailing list, the Council discuss and make a firm proposal on the new
> >> default paths, and then RelEng can make the appropriate updates to the
> >> catalyst builds. A news item can be compiled, with an appropriate wiki
> >> article perhaps on migration strategy (I may volunteer to format such a
> >> page with some appropriate guidance).
> >> Regards,
> >> Michael / veremitz.
> >>
> > This is a mess, many systems are setup with portage already on a
> > seperate partition for reasons. What advantage does it provide to move
> > the tree now after all these years? I have seen nothing more then lets
> > do this cause I like the ideal lately and it is getting old, there is no
> > benefit that would justify moving the tree or many other changes that
> > are being made in Gentoo lately.
> 
> People who want to move it could just set PORTDIR in make.conf. I don't
> see any reason to move it either.

Actually, I believe that PORTDIR is becoming a thing of the past.

Also, the default definitely should not be on /usr per fhs. This would
allow /usr to be mounted read only.
This doesn't affect things like the example above where /usr/portage is
a mount point.

> > 
> > 
> > 
> 
> 




[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Richard Yao]

> On Jul 11, 2018, at 4:42 PM, William Hubbs <williamh@gentoo.org> wrote:
> 
>> On Wed, Jul 11, 2018 at 04:25:20PM -0400, Richard Yao wrote:
>>> On 07/11/2018 03:29 AM, Jory A. Pratt wrote:
>>>> On 07/10/18 16:35, M. J. Everitt wrote:
>>>>> On 10/07/18 21:09, William Hubbs wrote:
>>>>>> On Mon, Jul 09, 2018 at 03:54:35PM -0700, Zac Medico wrote:
>>>>>>> On 07/09/2018 03:27 PM, M. J. Everitt wrote:
>>>>>>>> On 09/07/18 23:12, Zac Medico wrote:
>>>>>>>>> On 07/09/2018 02:34 PM, Kristian Fiskerstrand wrote:
>>>>>>>>> I'd mostly argue any such change should only affect new systems
>>>>>>>>> 
>>>>>>>> Yes, changing defaults for existing systems would be annoying.
>>>>>>>> 
>>>>>>>> My recommendation is to have catalyst set the new defaults in the stage
>>>>>>>> tarballs.
>>>>>>>> 
>>>>>>>> When sys-apps/portage changes its internal defaults, I'd like for the
>>>>>>>> upgrade process to call a tool that generates configuration files when
>>>>>>>> necessary to ensure that the existing paths remain constant.
>>>>>>> I think it should be possible for RelEng to make a start on catalyst
>>>>>>> updates - is there anything that would inhibit going ahead with this,
>>>>>>> potentially?
>>>>>> No, nothing. Whatever catalyst puts it the default config will become
>>>>>> our new default.
>>>>> I would still like to see notice about what the new defaults are and how
>>>>> to migrate current systems to them.
>>>>> 
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> William
>>>>> 
>>>>>> -- 
>>>>>> Thanks,
>>>>>> Zac
>>>>>> 
>>>>> 
>>>>> 
>>>> I'd like to propose that further to the discussion here on the -dev
>>>> mailing list, the Council discuss and make a firm proposal on the new
>>>> default paths, and then RelEng can make the appropriate updates to the
>>>> catalyst builds. A news item can be compiled, with an appropriate wiki
>>>> article perhaps on migration strategy (I may volunteer to format such a
>>>> page with some appropriate guidance).
>>>> Regards,
>>>> Michael / veremitz.
>>>> 
>>> This is a mess, many systems are setup with portage already on a
>>> seperate partition for reasons. What advantage does it provide to move
>>> the tree now after all these years? I have seen nothing more then lets
>>> do this cause I like the ideal lately and it is getting old, there is no
>>> benefit that would justify moving the tree or many other changes that
>>> are being made in Gentoo lately.
>> 
>> People who want to move it could just set PORTDIR in make.conf. I don't
>> see any reason to move it either.
> 
> Actually, I believe that PORTDIR is becoming a thing of the past.
I used to use it 5 years ago. If it does not work due to regressions, we should fix that.
> 
> Also, the default definitely should not be on /usr per fhs. This would
> allow /usr to be mounted read only.
> This doesn't affect things like the example above where /usr/portage is
> a mount point.
> 
>>> 
>>> 
>>> 
>> 
>> 
> 
> 
> 

Re: [gentoo-dev] rfc: moving default location of portage tree

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 312 bytes --]

>>>>> On Mon, 9 Jul 2018, William Hubbs wrote:

> On Mon, Jul 09, 2018 at 04:53:43PM -0400, Rich Freeman wrote:
>> Though I do prefer /var/lib or /var/cache over /var/db, simply
>> because /var/lib is actually in FHS.

> Agreed, /var/db I guess is a Gentoo invention of some kind?

No, it exists in FreeBSD too.

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 519 bytes --]

>>>>> On Tue, 10 Jul 2018, Ulrich Mueller wrote:

>>>>> On Mon, 9 Jul 2018, William Hubbs wrote:
>> Agreed, /var/db I guess is a Gentoo invention of some kind?

> No, it exists in FreeBSD too.

As was pointed out to me, it exists in all three BSD variants [1,2,3].
Its purpose there is for "miscellaneous automatically generated
system-specific database files".

Ulrich

[1] https://www.freebsd.org/cgi/man.cgi?hier%287%29
[2] http://netbsd.gw.com/cgi-bin/man-cgi?hier+7+NetBSD-current
[3] https://man.openbsd.org/hier

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Brian Dolbec]

[-- Attachment #1: Type: text/plain, Size: 947 bytes --]

On Mon, 9 Jul 2018 12:21:36 -0500
William Hubbs <williamh@gentoo.org> wrote:

> All,
> 
> is there a tracker for when the portage tree can be moved out of
> /usr/portage by default?
> 
> If not, what is the status of us being able to do this?
> 
> Thanks,
> 
> William
> 

I don't recall a tracker bug ever being created.

It required the stages be generated by catalyst-3 which can be
configured to any location for the tree defaults.  catalyst-2 had paths
hard-coded all over the place, plus used those paths as keys in
python dictionaries...

I believe all stages are now built with catalyst-3 for all arches, but
I don't know about some of the lesser used arches as some of those are
older dates.

That and a portage release with the new default location set in it's
backup configs.

So, it should be ready to convert if the minor arches stage are being
generated with catalyst-3

-- 
Brian Dolbec <dolsen>


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1186 bytes --]

On Fri, Jul 27, 2018 at 10:32:17AM +0200, Ulrich Mueller wrote:
> So, considering all the feedback from mailing list and IRC:
> 
>    /usr/portage           -> /var/db/repos/gentoo
>    /usr/portage/distfiles -> /var/cache{,/gentoo}/distfiles
>    /usr/portage/packages  -> /var/cache{,/gentoo}/binpkgs
> 
> Open question: Should we have the additional "gentoo" path component
> for the ones in /var/cache? The tradeoff is between a path that is
> easier to type, or slightly easier usage if someone wants to NFS mount
> distfiles and binpkgs.

Section 5.5.2 describes the directory structure of /var/cache. These
paths are all optional [1].

/var/cache/fonts
/var/cache/man
/var/cache/www
/var/cache/<package>

Gentoo isn't a package, so I don't think /var/cache/gentoo/* is
appropriate. Here is my proposal:

/usr/portage           -> /var/db/repos/gentoo
/usr/portage/distfiles -> /var/cache/portage/distfiles
/usr/portage/packages  -> /var/cache/portage/binpkgs

I'm not 100% comfortable with /var/db, but I don't have any better
suggestion either.

William

[1] http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#varcacheApplicationCacheData

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 2928 bytes --]

[Moving the thread back from -project to -dev.]

>>>>> On Fri, 13 Jul 2018, Ulrich Mueller wrote:

>>>>> On Fri, 13 Jul 2018, Rich Freeman wrote:
>> Well, /var/lib/<something> is 3 right there.  If 5 is no good then you
>> only have one left.  We could just make it /var/lib/repos which seems
>> non-compliant.

> FHS 3.0 says: "An application (or a group of inter-related
> applications) must use a subdirectory of /var/lib for its data".
> Certainly /var/lib/repos is a subdirectory of /var/lib? So why would
> it be non-compliant? And if it was, do we care about non-compliance at
> the third directory level? The important part is that we move it out
> of /usr, and IMHO we should care to get the /var/{lib,cache,db} part
> somewhat right.

In the same section 5.8.1 (emphasis is mine):
"This hierarchy holds state information pertaining to an application
or the system. State information is data that programs modify while
they run, and that _pertains_to_one_specific_host._ Users must never
need to modify files in /var/lib to configure a package's operation,
and _the_specific_file_hierarchy_ used to store the data _must_not_be_
_exposed_ to regular users."

"Pertains to one specific host" doesn't seem to apply to the Gentoo
repository though. Also, there is that strange requirement that the
file hierarchy must not be exposed to users. At least for the
make.profile link we rely on a well defined hierarchy, and certainly
expose it to users. The same is true for license information in
/usr/portage/licenses. So I would conclude that using /var/lib does
not comply with the FHS.

It was mentioned that all three directories (ebuild repository, binary
packages, distfiles) have some characteristics of a cache. However, I
think this is much more true for distfiles than for the other two,
which cannot be easily restored to a previous state.

I therefore suggest the following scheme:

/usr/portage -> /var/db/repos/gentoo

This follows the existing usage in eselect-repositories. Note that
/var/db is not specified by the FHS (though it is mentioned in a
footnote [1]) but used by all current BSD variants. Plus, we already
have /var/db/pkgs and (as pointed out by antarus) we won't get rid of
it anytime soon.

/usr/portage/packages -> /var/db/binpkgs

This keeps binary packages along with ebuilds. As suggested by mgorny,
rename the "packages" directory to "binpkgs", because it is more
specific and avoids confusion with "pkgs".

/usr/portage/distfiles -> /var/cache/distfiles

Omitting the "portage" subdirectory level here, as distfiles aren't
specific to any package manager. We could use an intermediate
"package-manager" directory level, but then again, it would have only
one single subdirectory. The FHS isn't very specific about /var/cache
but mentions /var/cache/fonts as an example which appears to be
similar.

Ulrich

[1] https://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#ftn.idm236086558032

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Chí-Thanh Christopher Nguyễn]

Ulrich Mueller schrieb:
> Users must never
> need to modify files in /var/lib to configure a package's operation,
> and _the_specific_file_hierarchy_ used to store the data _must_not_be_
> _exposed_ to regular users."

One small note, while it is never needed to modify, skel.ebuild would then be 
a file that is meant to be accessed by users in /var/lib if your proposal is 
realized.


Best regards,
Chí-Thanh Christopher Nguyễn

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 1789 bytes --]

>>>>> On Thu, 19 Jul 2018, Chí-Thanh Christopher Nguyễn wrote:

>> Users must never need to modify files in /var/lib to configure a
>> package's operation, and _the_specific_file_hierarchy_ used to
>> store the data _must_not_be_ _exposed_ to regular users."

> One small note, while it is never needed to modify, skel.ebuild
> would then be a file that is meant to be accessed by users in
> /var/lib if your proposal is realized.

That's one of the reasons why the proposal prefers /var/db. The other
reason is existing usage in eselect-repository.

>>>>> On Thu, 19 Jul 2018, Ulrich Mueller wrote:

> In my understanding, a cache is typically an open collection of items.
> Some subset of them can be deleted without much negative consequence,
> and there may also be surplus items that are no longer necessary and
> will be expired at some later time in order to reclaim disk space.

> Nothing of this is true for an ebuild repository, which is a closed
> collection of files: A single file cannot be discarded without
> invalidating the whole repository. Also there cannot be any stray
> files which would be expired later. Same as above, a single stray file
> will invalidate all.

> (A collection of binary packages may qualify as a cache though, by
> this definition.)

So, considering all the feedback from mailing list and IRC:

   /usr/portage           -> /var/db/repos/gentoo
   /usr/portage/distfiles -> /var/cache{,/gentoo}/distfiles
   /usr/portage/packages  -> /var/cache{,/gentoo}/binpkgs

Open question: Should we have the additional "gentoo" path component
for the ones in /var/cache? The tradeoff is between a path that is
easier to type, or slightly easier usage if someone wants to NFS mount
distfiles and binpkgs.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Corentin “Nado” Pazdera]

July 27, 2018 4:07 PM, "William Hubbs" <williamh@gentoo.org> wrote:

> On Fri, Jul 27, 2018 at 10:32:17AM +0200, Ulrich Mueller wrote:
> 
>> So, considering all the feedback from mailing list and IRC:
>> 
>> /usr/portage -> /var/db/repos/gentoo
>> /usr/portage/distfiles -> /var/cache{,/gentoo}/distfiles
>> /usr/portage/packages -> /var/cache{,/gentoo}/binpkgs
>> 
>> Open question: Should we have the additional "gentoo" path component
>> for the ones in /var/cache? The tradeoff is between a path that is
>> easier to type, or slightly easier usage if someone wants to NFS mount
>> distfiles and binpkgs.
> 
> Section 5.5.2 describes the directory structure of /var/cache. These
> paths are all optional [1].
> 
> /var/cache/fonts
> /var/cache/man
> /var/cache/www
> /var/cache/<package>
> 
> Gentoo isn't a package, so I don't think /var/cache/gentoo/* is
> appropriate. Here is my proposal:
> 
> /usr/portage -> /var/db/repos/gentoo
> /usr/portage/distfiles -> /var/cache/portage/distfiles
> /usr/portage/packages -> /var/cache/portage/binpkgs
> 
> I'm not 100% comfortable with /var/db, but I don't have any better
> suggestion either.
> 
> William
> 
> [1] http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#varcacheApplicationCacheData

From the same source
"No other requirements are made on the data format of the cache directories."
And as you have quoted it, everything under /var/cache is optional.

So anything which doesn't conflict with another package seems fine according to FHS.

--
Corentin “Nado” Pazdera

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 1407 bytes --]

>>>>> On Fri, 27 Jul 2018, Corentin “Nado” Pazdera wrote:

> July 27, 2018 4:07 PM, "William Hubbs" <williamh@gentoo.org> wrote:

>> Section 5.5.2 describes the directory structure of /var/cache.
>> These paths are all optional [1].
>> 
>> /var/cache/fonts
>> /var/cache/man
>> /var/cache/www
>> /var/cache/<package>
>> 
>> Gentoo isn't a package, so I don't think /var/cache/gentoo/* is
>> appropriate. Here is my proposal:
>> 
>> /usr/portage -> /var/db/repos/gentoo
>> /usr/portage/distfiles -> /var/cache/portage/distfiles
>> /usr/portage/packages -> /var/cache/portage/binpkgs
>> 
>> I'm not 100% comfortable with /var/db, but I don't have any better
>> suggestion either.
>> 
>> [1] http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#varcacheApplicationCacheData

> From the same source
> "No other requirements are made on the data format of the cache
> directories."
> And as you have quoted it, everything under /var/cache is optional.

> So anything which doesn't conflict with another package seems fine
> according to FHS.

That's how I would read it, too. We could of course invent a package
name (like "package-manager" for virtual/package-manager) but it seems
cumbersome, and I don't see any benefit of it.

There also is /var/cache/fonts, so the FHS itself lists an example of
a directory that's not named after a specific package.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)

[Brian Dolbec]

[-- Attachment #1: Type: text/plain, Size: 1838 bytes --]

On Fri, 27 Jul 2018 16:31:15 +0200
Ulrich Mueller <ulm@gentoo.org> wrote:

> >>>>> On Fri, 27 Jul 2018, Corentin “Nado” Pazdera wrote:  
> 
> > July 27, 2018 4:07 PM, "William Hubbs" <williamh@gentoo.org>
> > wrote:  
> 
> >> Section 5.5.2 describes the directory structure of /var/cache.
> >> These paths are all optional [1].
> >> 
> >> /var/cache/fonts
> >> /var/cache/man
> >> /var/cache/www
> >> /var/cache/<package>
> >> 
> >> Gentoo isn't a package, so I don't think /var/cache/gentoo/* is
> >> appropriate. Here is my proposal:
> >> 
> >> /usr/portage -> /var/db/repos/gentoo
> >> /usr/portage/distfiles -> /var/cache/portage/distfiles
> >> /usr/portage/packages -> /var/cache/portage/binpkgs
> >> 
> >> I'm not 100% comfortable with /var/db, but I don't have any better
> >> suggestion either.
> >> 
> >> [1]
> >> http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#varcacheApplicationCacheData  
> 
> > From the same source
> > "No other requirements are made on the data format of the cache
> > directories."
> > And as you have quoted it, everything under /var/cache is
> > optional.  
> 
> > So anything which doesn't conflict with another package seems fine
> > according to FHS.  
> 
> That's how I would read it, too. We could of course invent a package
> name (like "package-manager" for virtual/package-manager) but it seems
> cumbersome, and I don't see any benefit of it.
> 
> There also is /var/cache/fonts, so the FHS itself lists an example of
> a directory that's not named after a specific package.
> 
> Ulrich

/var/db/repos/gentoo
/var/cache/distfiles
/var/cache/binpkgs

Works for me, just please keep "portage" out of it, after all distfiles
are not restricted to portage use only, and neither are binpkgs.  There
is alternate binpkg installers.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] rfc: moving default location of portage tree

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 1356 bytes --]

>>>>> On Fri, 27 Jul 2018, Brian Dolbec wrote:

> On Fri, 27 Jul 2018 16:31:15 +0200
> Ulrich Mueller <ulm@gentoo.org> wrote:

>> >>>>> On Fri, 27 Jul 2018, Corentin “Nado” Pazdera wrote:
>>
>> > From the same source
>> > "No other requirements are made on the data format of the cache
>> > directories."
>> > And as you have quoted it, everything under /var/cache is
>> > optional.
>>
>> > So anything which doesn't conflict with another package seems
>> > fine according to FHS.
>>
>> That's how I would read it, too. We could of course invent a
>> package name (like "package-manager" for virtual/package-manager)
>> but it seems cumbersome, and I don't see any benefit of it.
>>
>> There also is /var/cache/fonts, so the FHS itself lists an example
>> of a directory that's not named after a specific package.

> /var/db/repos/gentoo
> /var/cache/distfiles
> /var/cache/binpkgs

For the record, these three paths have been approved in today's
Council meeting.

> Works for me, just please keep "portage" out of it, after all
> distfiles are not restricted to portage use only, and neither are
> binpkgs. There is alternate binpkg installers.

By another Council vote, snapshot names should be changed from
portage-YYYYMMDD.tar.{bz2,xz} to gentoo-YYYYMMDD.tar.{bz2,xz}.

So, no "portage" any more. ;)

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]