[gentoo-dev] Moving our/portage stuff to var

[gentoo-dev] Moving our/portage stuff to var

[Tomáš Chvátal]

Currently we put portage into /usr/portage and all related stuff is to
be in the subfolders there (distfiles, binpkg).

I've always myself override these defaults in make.conf to point for
/var/portage/ (not /var/lib because I never bothered enough how to
make world and config files to be put elsewhere :P).

The only reason why we have this currently in usr is that bsd ports
put their stuff in there and I suppose Daniel just did the same.

With respect to reality how stuff is done in the linux land all the
variable data should be in /var so we should adjust and move it in
there too.

What would  you think?

Cheers

Tom

Re: [gentoo-dev] Moving our/portage stuff to var

[Diego Elio Pettenò]

On 17/12/2012 11:19, Tomáš Chvátal wrote:
> 
> I've always myself override these defaults in make.conf to point for
> /var/portage/ (not /var/lib because I never bothered enough how to
> make world and config files to be put elsewhere :P).

I would say let's work on that so that portage can keep them there.
Although I'm more for /var/cache/portage myself, as both distfiles and
tree can be re-generated.

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/

Re: [gentoo-dev] Moving our/portage stuff to var

[Tomáš Chvátal]

2012/12/17 Diego Elio Pettenò <flameeyes@flameeyes.eu>:
> On 17/12/2012 11:19, Tomáš Chvátal wrote:
>>
>> I've always myself override these defaults in make.conf to point for
>> /var/portage/ (not /var/lib because I never bothered enough how to
>> make world and config files to be put elsewhere :P).
>
> I would say let's work on that so that portage can keep them there.
> Although I'm more for /var/cache/portage myself, as both distfiles and
> tree can be re-generated.
>
Thats right, it should be even better location. :-)

Re: [gentoo-dev] Moving our/portage stuff to var

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 586 bytes --]

On Mon, 17 Dec 2012 11:23:00 +0100
Diego Elio Pettenò <flameeyes@flameeyes.eu> wrote:

> On 17/12/2012 11:19, Tomáš Chvátal wrote:
> > 
> > I've always myself override these defaults in make.conf to point for
> > /var/portage/ (not /var/lib because I never bothered enough how to
> > make world and config files to be put elsewhere :P).
> 
> I would say let's work on that so that portage can keep them there.
> Although I'm more for /var/cache/portage myself, as both distfiles and
> tree can be re-generated.

+1 on /var/cache.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Ben de Groot]

[-- Attachment #1: Type: text/plain, Size: 789 bytes --]

On 17 December 2012 18:27, Tomáš Chvátal <tomas.chvatal@gmail.com> wrote:

> 2012/12/17 Diego Elio Pettenò <flameeyes@flameeyes.eu>:
> > On 17/12/2012 11:19, Tomáš Chvátal wrote:
> >>
> >> I've always myself override these defaults in make.conf to point for
> >> /var/portage/ (not /var/lib because I never bothered enough how to
> >> make world and config files to be put elsewhere :P).
> >
> > I would say let's work on that so that portage can keep them there.
> > Although I'm more for /var/cache/portage myself, as both distfiles and
> > tree can be re-generated.
> >
> Thats right, it should be even better location. :-)
>
>
I prefer some location in /var/ as well.

-- 
Cheers,

Ben | yngwin
Gentoo developer
Gentoo Qt project lead, Gentoo Wiki admin

[-- Attachment #2: Type: text/html, Size: 1244 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[justin]

[-- Attachment #1: Type: text/plain, Size: 758 bytes --]

On 17/12/12 11:23, Diego Elio Pettenò wrote:
> On 17/12/2012 11:19, Tomáš Chvátal wrote:
>>
>> I've always myself override these defaults in make.conf to point for
>> /var/portage/ (not /var/lib because I never bothered enough how to
>> make world and config files to be put elsewhere :P).
> 
> I would say let's work on that so that portage can keep them there.
> Although I'm more for /var/cache/portage myself, as both distfiles and
> tree can be re-generated.
> 

fetch-restricted files are to be considered critical here. Do we want to
force the user to keep them twice? So an additional location which is
not a "cache"?
Of course PORTAGE_RO_DISTDIRS and friends are nice here, but they are
not part of a default setup.

justin


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 261 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Luca Barbato]

On 12/17/12 11:30 AM, Michał Górny wrote:
> On Mon, 17 Dec 2012 11:23:00 +0100
> Diego Elio Pettenò <flameeyes@flameeyes.eu> wrote:
>
>> On 17/12/2012 11:19, Tomáš Chvátal wrote:
>>>
>>> I've always myself override these defaults in make.conf to point for
>>> /var/portage/ (not /var/lib because I never bothered enough how to
>>> make world and config files to be put elsewhere :P).
>>
>> I would say let's work on that so that portage can keep them there.
>> Although I'm more for /var/cache/portage myself, as both distfiles and
>> tree can be re-generated.
>
> +1 on /var/cache.
>
Agreed.

Bonus points if we consider suggesting to move it on a dedicated file 
system ^^;

lu

Re: [gentoo-dev] Moving our/portage stuff to var

[Diego Elio Pettenò]

[-- Attachment #1: Type: text/plain, Size: 549 bytes --]

On 17/12/2012 12:06, justin wrote:
> fetch-restricted files are to be considered critical here. Do we want to
> force the user to keep them twice? So an additional location which is
> not a "cache"?
> Of course PORTAGE_RO_DISTDIRS and friends are nice here, but they are
> not part of a default setup.

I would still think they are cache. I can re-download Oracle's JRE
binaries; Portage's copy is a cache because I don't need to back it up.

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 553 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Dirkjan Ochtman]

On Mon, Dec 17, 2012 at 11:23 AM, Diego Elio Pettenò
<flameeyes@flameeyes.eu> wrote:
> I would say let's work on that so that portage can keep them there.
> Although I'm more for /var/cache/portage myself, as both distfiles and
> tree can be re-generated.

+1.

Cheers,

Dirkjan

Re: [gentoo-dev] Moving our/portage stuff to var

[Derek Dai]

[-- Attachment #1: Type: text/plain, Size: 731 bytes --]

+1 /var/cache

Derek Dai



On Mon, Dec 17, 2012 at 6:30 PM, Michał Górny <mgorny@gentoo.org> wrote:

> On Mon, 17 Dec 2012 11:23:00 +0100
> Diego Elio Pettenò <flameeyes@flameeyes.eu> wrote:
>
> > On 17/12/2012 11:19, Tomáš Chvátal wrote:
> > >
> > > I've always myself override these defaults in make.conf to point for
> > > /var/portage/ (not /var/lib because I never bothered enough how to
> > > make world and config files to be put elsewhere :P).
> >
> > I would say let's work on that so that portage can keep them there.
> > Although I'm more for /var/cache/portage myself, as both distfiles and
> > tree can be re-generated.
>
> +1 on /var/cache.
>
> --
> Best regards,
> Michał Górny
>

[-- Attachment #2: Type: text/html, Size: 1220 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Markos Chandras]

On 17 December 2012 10:30, Michał Górny <mgorny@gentoo.org> wrote:
> On Mon, 17 Dec 2012 11:23:00 +0100
> Diego Elio Pettenò <flameeyes@flameeyes.eu> wrote:
>
>> On 17/12/2012 11:19, Tomáš Chvátal wrote:
>> >
>> > I've always myself override these defaults in make.conf to point for
>> > /var/portage/ (not /var/lib because I never bothered enough how to
>> > make world and config files to be put elsewhere :P).
>>
>> I would say let's work on that so that portage can keep them there.
>> Although I'm more for /var/cache/portage myself, as both distfiles and
>> tree can be re-generated.
>
> +1 on /var/cache.
>
> --
> Best regards,
> Michał Górny

+1 sounds good to me.

-- 
Regards,
Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2

Re: [gentoo-dev] Moving our/portage stuff to var

[George Shapovalov]

On Monday 17 December 2012 11:19:20 Tomáš Chvátal wrote:
> I've always myself override these defaults in make.conf to point for
> /var/portage/ (not /var/lib because I never bothered enough how to
> make world and config files to be put elsewhere :P).
Finally!
And, while we are at it, lets more distfiles out of portage. That is the only 
(prominent) dir inside that has drastically different storage requirements. 
Having portage on a separate partition requires now changing defaults, bind 
mounts or symlinks. It is better to have it done right from the onset then to 
workaround every time.

Oh, and +1 on /var/cache/portage for the location.

George

Re: [gentoo-dev] Moving our/portage stuff to var

[Philipp Riegger]

On 17.12.2012 11:23, Diego Elio Pettenò wrote:
> On 17/12/2012 11:19, Tomáš Chvátal wrote:
>>
>> I've always myself override these defaults in make.conf to point for
>> /var/portage/ (not /var/lib because I never bothered enough how to
>> make world and config files to be put elsewhere :P).
>
> I would say let's work on that so that portage can keep them there.
> Although I'm more for /var/cache/portage myself, as both distfiles and
> tree can be re-generated.
>

With this change distfiles and packages could be moved to a directory 
which is not a subdir of portage. Something like

/var/cache/{portage,distfiles,packages}

or

/var/cache/portage/{tree,distfiles,packages}

since the file types and storage requirements are so different. At least 
I prefer not to have too many filesystems mounted inside each other.

Philipp

Re: [gentoo-dev] Moving our/portage stuff to var

[justin]

[-- Attachment #1: Type: text/plain, Size: 838 bytes --]

On 17/12/12 12:17, Diego Elio Pettenò wrote:
> On 17/12/2012 12:06, justin wrote:
>> fetch-restricted files are to be considered critical here. Do we want to
>> force the user to keep them twice? So an additional location which is
>> not a "cache"?
>> Of course PORTAGE_RO_DISTDIRS and friends are nice here, but they are
>> not part of a default setup.
> 
> I would still think they are cache. I can re-download Oracle's JRE
> binaries; Portage's copy is a cache because I don't need to back it up.
> 

I am more thinking about packages which are not as easy accessible as
JRE. There are a couple sci packages which are distributed on request by
mail other inconvenient methods. Sometimes even not by your own, but by
your PI or other seniors. And even sometimes upstreams doesn't
distribute them anymore at all.




[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 261 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Ulrich Mueller]

>>>>> On Mon, 17 Dec 2012, Michał Górny wrote:

> On Mon, 17 Dec 2012 11:23:00 +0100
> Diego Elio Pettenò <flameeyes@flameeyes.eu> wrote:

>> I would say let's work on that so that portage can keep them there.
>> Although I'm more for /var/cache/portage myself, as both distfiles and
>> tree can be re-generated.

> +1 on /var/cache.

If we change the location, can we then move distfiles to some place
outside of the tree? Something like:

   /var/cache/portage
   /var/cache/distfiles

Rationale for this is that the tree with many small files and
distfiles (which tend to be large) have very different parameters for
filesystem optimisation, when one of them (or both) are kept on a
separate partition.

Ulrich

Re: [gentoo-dev] Moving our/portage stuff to var

[Diego Elio Pettenò]

On 17/12/2012 13:42, Ulrich Mueller wrote:
> If we change the location, can we then move distfiles to some place
> outside of the tree? Something like:
> 
>    /var/cache/portage
>    /var/cache/distfiles

What I do on my systems is

/var/cache/portage/tree
/var/cache/portage/distfiles

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/

Re: [gentoo-dev] Moving our/portage stuff to var

[Diego Elio Pettenò]

[-- Attachment #1: Type: text/plain, Size: 616 bytes --]

On 17/12/2012 13:39, justin wrote:
> I am more thinking about packages which are not as easy accessible as
> JRE. There are a couple sci packages which are distributed on request by
> mail other inconvenient methods. Sometimes even not by your own, but by
> your PI or other seniors. And even sometimes upstreams doesn't
> distribute them anymore at all.

In which case you should keep them somewhere safer anyway, which is what
I do for that kind of files.

Remember that eclean acts on distfiles as well.

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 553 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Tiziano Müller]

Am Montag, den 17.12.2012, 11:19 +0100 schrieb Tomáš Chvátal:
> Currently we put portage into /usr/portage and all related stuff is to
> be in the subfolders there (distfiles, binpkg).
> 
> I've always myself override these defaults in make.conf to point for
> /var/portage/ (not /var/lib because I never bothered enough how to
> make world and config files to be put elsewhere :P).

I always move the stuff as well:

* /var/cache/distfiles
* /var/cache/packages ... may not be the best choice since they can't
always be regenerated
* /var/db/repositories/portage
* /var/db/repositories/... (other portage repositories)
* /var/db/paludis/repositories/... (for paludis-specific repositories,
like layman)

Tiziano

Re: [gentoo-dev] Moving our/portage stuff to var

[Kevin Chadwick]

On Mon, 17 Dec 2012 11:19:20 +0100
Tomáš Chvátal <tomas.chvatal@gmail.com> wrote:

> The only reason why we have this currently in usr is that bsd ports
> put their stuff in there and I suppose Daniel just did the same.

>> +1 on /var/cache.
>>  
>>> Agreed.

>>>Bonus points if we consider suggesting to move it on a dedicated
>>>file system ^^;

/var sounds right but if /usr is still huge it may annoy some (like
apt can) with smaller drives who now need lots of free space for new
programs in both /usr and /var. Of course there is LVM.

On OpenBSD the Auto partition map suggests /usr/ports /usr/src as
seperate partitions as long as you have a fair amount of space. It
possibly even suggests a seperate obj partition. The benefit being you
can mkfs/newfs much quicker than deleting many many files. Security (DAC
permission avoidance) and nuking more than what you wanted obviously
needs consideration for that kind of function. 

So it's probably a user exercise?

Re: [gentoo-dev] Moving our/portage stuff to var

[Diego Elio Pettenò]

On 17/12/2012 14:40, Kevin Chadwick wrote:
> /var sounds right but if /usr is still huge it may annoy some (like
> apt can) with smaller drives who now need lots of free space for new
> programs in both /usr and /var. Of course there is LVM.

Changing our defaults is unlikely to force users to change their settings.

> On OpenBSD the Auto partition map suggests /usr/ports /usr/src as
> seperate partitions as long as you have a fair amount of space. It
> possibly even suggests a seperate obj partition. The benefit being you
> can mkfs/newfs much quicker than deleting many many files. Security (DAC
> permission avoidance) and nuking more than what you wanted obviously
> needs consideration for that kind of function. 

Honestly I would never take what OpenBSD does to face value.

But in general this is a call for users — myself I have been keeping
them split on the tinderbox host but merged into the rootfs for the laptops.

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/

Re: [gentoo-dev] Moving our/portage stuff to var

[Rich Freeman]

On Mon, Dec 17, 2012 at 8:40 AM, Kevin Chadwick <ma1l1ists@yahoo.co.uk> wrote:
> So it's probably a user exercise?

It already is a user exercise.  A stage3 doesn't even contain the
/usr/portage directory - you manually create it per the handbook (or
more likely let tar/etc do it for you.

I also would like to see distfiles moved.  Ideally the package tree
should be a perfect copy of what is on the rsync mirrors.  It seems a
bit odd to stick other stuff in there, which needs special treatment
as a result.

To the extent that this isn't already supported, portage should simply
let you set the location in make.conf.

I'd also suggest at least considering how paludis handles this.  They
just have a directory containing config file per repository, with a
priority setting.  The portage tree is just another overlay, which is
a good way to handle it.  The sync mechanism handles the main tree
identically to overlays as a result, though you can specify what to
sync.

Rich

Re: [gentoo-dev] Moving our/portage stuff to var

[Diego Elio Pettenò]

On 17/12/2012 14:49, Rich Freeman wrote:
> I'd also suggest at least considering how paludis handles this.  They
> just have a directory containing config file per repository, with a
> priority setting.  The portage tree is just another overlay, which is
> a good way to handle it.  The sync mechanism handles the main tree
> identically to overlays as a result, though you can specify what to
> sync.

Let's not conflate two completely different changes with two completely
different work required to deal with them.

Changing our default locations, and the documentation is quick. Changing
the way the syncing/handling is done is a nightmare.

If we conflate the issue, we can stop discussing as we're never ever
going to go anywhere.

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/

Re: [gentoo-dev] Moving our/portage stuff to var

[Marc Schiffbauer]

[-- Attachment #1: Type: text/plain, Size: 1033 bytes --]

Am Montag, 17. Dezember 2012, 11:23:00 schrieb Diego Elio Pettenò:
> On 17/12/2012 11:19, Tomáš Chvátal wrote:
> > I've always myself override these defaults in make.conf to point for
> > /var/portage/ (not /var/lib because I never bothered enough how to
> > make world and config files to be put elsewhere :P).
> 
> I would say let's work on that so that portage can keep them there.
> Although I'm more for /var/cache/portage myself, as both distfiles and
> tree can be re-generated.

What about setups where portage tree is mounted via NFS to reduce traffic and 
disk space?

FHS states[1] that /var/cache is *locally* generated. Which aould not be the 
case for such setups...

I prefer /var as well, but I am not such if /var/cache would be the right 
place...


[1]
From FHS 2.3:
"/var/cache is intended for cached data from applications. Such data is 
locally generated as a result of time-consuming I/O or calculation."

-Marc
-- 
0x35A64134 - 8AAC 5F46 83B4 DB70 8317  3723 296C 6CCA 35A6 4134

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Diego Elio Pettenò]

[-- Attachment #1: Type: text/plain, Size: 841 bytes --]

On 17/12/2012 15:49, Marc Schiffbauer wrote:
> What about setups where portage tree is mounted via NFS to reduce traffic and 
> disk space?

Since nothing in Gentoo and/or other distributions _enforces_ FHS,
you're allowed to do as you prefer....

> FHS states[1] that /var/cache is *locally* generated. Which aould not be the 
> case for such setups...

Again, you can do as you prefer. Do you wish to violate FHS to just keep
in the usual place? You can. You want to move it somewhere else to
adhere to FHS? You can.

> I prefer /var as well, but I am not such if /var/cache would be the right 
> place...

Any other suggestions on where to place it? And please don't say
/var/lib because that would usually be backed up.

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 553 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Marc Schiffbauer]

[-- Attachment #1: Type: text/plain, Size: 1606 bytes --]

Am Montag, 17. Dezember 2012, 15:56:11 schrieb Diego Elio Pettenò:
> On 17/12/2012 15:49, Marc Schiffbauer wrote:
> > What about setups where portage tree is mounted via NFS to reduce traffic
> > and disk space?
> 
> Since nothing in Gentoo and/or other distributions _enforces_ FHS,
> you're allowed to do as you prefer....
> 
> > FHS states[1] that /var/cache is *locally* generated. Which aould not be
> > the case for such setups...
> 
> Again, you can do as you prefer. Do you wish to violate FHS to just keep
> in the usual place? You can. You want to move it somewhere else to
> adhere to FHS? You can.
> 
> > I prefer /var as well, but I am not such if /var/cache would be the right
> > place...
> 
> Any other suggestions on where to place it? And please don't say
> /var/lib because that would usually be backed up.

FHS also states:

"[...] Other portions may be shared [between systems], notably /var/mail, 
/var/cache/man, /var/cache/fonts, and /var/spool/news."

So I think this might indeed be interpreted like that /var/cache/portage would 
be perfectly ok.

Another place I could imagine is /var/portage because of its fundamental 
importance in gentoo.

FHS about that: "Applications must generally not add directories to the top 
level of /var. Such directories should only be added if they have some system-
wide implication, and in consultation with the FHS mailing list."

I think this would also be ok because portage can be counted as "system-wide 
implication" ... 

-Marc
-- 
0x35A64134 - 8AAC 5F46 83B4 DB70 8317  3723 296C 6CCA 35A6 4134

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Anthony G. Basile]

On 12/17/2012 05:30 AM, Michał Górny wrote:
> On Mon, 17 Dec 2012 11:23:00 +0100
> Diego Elio Pettenò<flameeyes@flameeyes.eu>  wrote:
>
>> On 17/12/2012 11:19, Tomáš Chvátal wrote:
>>> I've always myself override these defaults in make.conf to point for
>>> /var/portage/ (not /var/lib because I never bothered enough how to
>>> make world and config files to be put elsewhere :P).
>> I would say let's work on that so that portage can keep them there.
>> Although I'm more for /var/cache/portage myself, as both distfiles and
>> tree can be re-generated.
> +1 on /var/cache.
>
+1 on the idea of moving portage and +1 on the idea of /var/cache

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 8040 5A4D 8709 21B1 1A88  33CE 979C AF40 D045 5535
GnuPG ID  : D0455535

Re: [gentoo-dev] Moving our/portage stuff to var

[Brian Dolbec]

[-- Attachment #1: Type: text/plain, Size: 1191 bytes --]

On Mon, 2012-12-17 at 11:23 +0100, Diego Elio Pettenò wrote:
> On 17/12/2012 11:19, Tomáš Chvátal wrote:
> > 
> > I've always myself override these defaults in make.conf to point for
> > /var/portage/ (not /var/lib because I never bothered enough how to
> > make world and config files to be put elsewhere :P).
> 
> I would say let's work on that so that portage can keep them there.
> Although I'm more for /var/cache/portage myself, as both distfiles and
> tree can be re-generated.
> 

I've been pushing for the portage tree to move somewhere in /var for
awhile.

Also I think the tree and layman overlays should be under the same
master directory, then portage, pkgcore,... could just quickly scan the
master directory & sub directories to auto-add the valid overlays there.
No need to add them to PORTDIR_OVERLAY

something like

/var/cache/repositories/
/var/cache/repositories/gentoo
/var/cache/repositories/x11    <== overlay
/var/cache/repositories/local    <== overlay
/var/cache/repositories/distfiles    <== move it out of the tree dir.
/var/cache/repositories/packages    <== move it out of the tree dir.

-- 
Brian Dolbec <dolsen@gentoo.org>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Brian Dolbec]

[-- Attachment #1: Type: text/plain, Size: 758 bytes --]

On Mon, 2012-12-17 at 13:47 +0100, Diego Elio Pettenò wrote:
> On 17/12/2012 13:39, justin wrote:
> > I am more thinking about packages which are not as easy accessible as
> > JRE. There are a couple sci packages which are distributed on request by
> > mail other inconvenient methods. Sometimes even not by your own, but by
> > your PI or other seniors. And even sometimes upstreams doesn't
> > distribute them anymore at all.
> 
> In which case you should keep them somewhere safer anyway, which is what
> I do for that kind of files.
> 
> Remember that eclean acts on distfiles as well.
> 

which is why eclean has a config file where you can add files/pkgs,
patterns to exclude from being cleaned.
-- 
Brian Dolbec <dolsen@gentoo.org>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Brian Dolbec]

[-- Attachment #1: Type: text/plain, Size: 1120 bytes --]

On Mon, 2012-12-17 at 15:56 +0100, Diego Elio Pettenò wrote:
> On 17/12/2012 15:49, Marc Schiffbauer wrote:
> > What about setups where portage tree is mounted via NFS to reduce traffic and 
> > disk space?
> 
> Since nothing in Gentoo and/or other distributions _enforces_ FHS,
> you're allowed to do as you prefer....
> 
> > FHS states[1] that /var/cache is *locally* generated. Which aould not be the 
> > case for such setups...
> 
> Again, you can do as you prefer. Do you wish to violate FHS to just keep
> in the usual place? You can. You want to move it somewhere else to
> adhere to FHS? You can.
> 
> > I prefer /var as well, but I am not such if /var/cache would be the right 
> > place...
> 
> Any other suggestions on where to place it? And please don't say
> /var/lib because that would usually be backed up.
> 

then /var/repositories/  similar to my previous reply.  It is very clear
by the name what it's purpose is.  Also name the portage tree dir gentoo
like it's repo_name and all but one of the layman overlays available to
install.
-- 
Brian Dolbec <dolsen@gentoo.org>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Diego Elio Pettenò]

[-- Attachment #1: Type: text/plain, Size: 910 bytes --]

On 17/12/2012 16:51, Brian Dolbec wrote:
>> > 
> then /var/repositories/  similar to my previous reply.  It is very clear
> by the name what it's purpose is.  Also name the portage tree dir gentoo
> like it's repo_name and all but one of the layman overlays available to
> install.

Erm, why should we invent something new on top of var which is something
that about everywhere it's suggested NOT to do?

Using /var/lib clearly spells "back this up yourself".
Using /var/cache clearly spells "can be lost without consequences".
Using /var/tmp clearly spells "things will be deleted".

/var/repositories sounds just like "NIH", seriously.

Somebody already proposed "/var/db" — that probably makes more sense if
you want to go that route, although I wouldn't put distfiles or packages
there.

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 553 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Brian Dolbec]

[-- Attachment #1: Type: text/plain, Size: 1572 bytes --]

On Mon, 2012-12-17 at 15:02 +0100, Diego Elio Pettenò wrote:
> On 17/12/2012 14:49, Rich Freeman wrote:
> > I'd also suggest at least considering how paludis handles this.  They
> > just have a directory containing config file per repository, with a
> > priority setting.  The portage tree is just another overlay, which is
> > a good way to handle it.  The sync mechanism handles the main tree
> > identically to overlays as a result, though you can specify what to
> > sync.
> 
> Let's not conflate two completely different changes with two completely
> different work required to deal with them.
> 
> Changing our default locations, and the documentation is quick. Changing
> the way the syncing/handling is done is a nightmare.
> 
> If we conflate the issue, we can stop discussing as we're never ever
> going to go anywhere.
> 

I agree.  

But for the purpose of answering Rich, layman is also capable of syncing
the portage tree, although, not using the round robin and other user
sync variations portage is capable of.  Once I finish the python 3
compatibility code changes.  I will re-visit getting layman's api use
into portage, pkgcore for them to use to sync the overlays from within
the package manager.  I have already demonstrated how easy it is for
portage to run layman's api.  An in tree example is app-portage/esearch
which adding the -l parameter will use the layman api if available or
fall back to running layman in a subprocess to sync the overlays as well
as the portage tree.
-- 
Brian Dolbec <dolsen@gentoo.org>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 245 bytes --]

On Mon, 17 Dec 2012 15:56:11 +0100
Diego Elio Pettenò <flameeyes@flameeyes.eu> wrote:
> Any other suggestions on where to place it? And please don't say
> /var/lib because that would usually be backed up.

/var/db

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

["Paweł Hajdan, Jr."]

[-- Attachment #1: Type: text/plain, Size: 351 bytes --]

On 12/17/12 2:19 AM, Tomáš Chvátal wrote:
> With respect to reality how stuff is done in the linux land all the
> variable data should be in /var so we should adjust and move it in
> there too.
> 
> What would  you think?

Fully seconded.

+1 to /var/cache/portage and having distfiles outside of the portage
tree directory.

Paweł


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 203 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Ultrabug]

On 17/12/2012 13:15, Dirkjan Ochtman wrote:
> On Mon, Dec 17, 2012 at 11:23 AM, Diego Elio Pettenò
> <flameeyes@flameeyes.eu> wrote:
>> I would say let's work on that so that portage can keep them there.
>> Although I'm more for /var/cache/portage myself, as both distfiles and
>> tree can be re-generated.
>
> +1.
>
> Cheers,
>
> Dirkjan
>

+1

Ultra

Re: [gentoo-dev] Moving our/portage stuff to var

[Zac Medico]

On 12/17/2012 02:19 AM, Tomáš Chvátal wrote:
> Currently we put portage into /usr/portage and all related stuff is to
> be in the subfolders there (distfiles, binpkg).
> 
> I've always myself override these defaults in make.conf to point for
> /var/portage/ (not /var/lib because I never bothered enough how to
> make world and config files to be put elsewhere :P).
> 
> The only reason why we have this currently in usr is that bsd ports
> put their stuff in there and I suppose Daniel just did the same.
> 
> With respect to reality how stuff is done in the linux land all the
> variable data should be in /var so we should adjust and move it in
> there too.
> 
> What would  you think?

I like the idea. As noted in bug #378603 [1], I'd like the portage
ebuild to ensure that the locations don't unexpectedly change for
existing installs.

Will it break catalyst? If so, we might begin the migration by fixing
catalyst and having to set the new default locations in the make.conf
that it generates.

[1] https://bugs.gentoo.org/show_bug.cgi?id=378603#c1
-- 
Thanks,
Zac

Re: [gentoo-dev] Moving our/portage stuff to var

[Zac Medico]

On 12/17/2012 02:19 AM, Tomáš Chvátal wrote:
> Currently we put portage into /usr/portage and all related stuff is to
> be in the subfolders there (distfiles, binpkg).
> 
> I've always myself override these defaults in make.conf to point for
> /var/portage/ (not /var/lib because I never bothered enough how to
> make world and config files to be put elsewhere :P).
> 
> The only reason why we have this currently in usr is that bsd ports
> put their stuff in there and I suppose Daniel just did the same.
> 
> With respect to reality how stuff is done in the linux land all the
> variable data should be in /var so we should adjust and move it in
> there too.
> 
> What would  you think?

I like the idea. As noted in bug #378603 [1], I'd like the portage
ebuild to ensure that the locations don't unexpectedly change for
existing installs.

Will it break catalyst? If so, we might begin the migration by fixing
catalyst and having to set the new default locations in the make.conf
that it generates.

[1] https://bugs.gentoo.org/show_bug.cgi?id=378603#c1
-- 
Thanks,
Zac

Re: [gentoo-dev] Moving our/portage stuff to var

[Rick "Zero_Chaos" Farina]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/18/2012 01:38 PM, Zac Medico wrote:
> On 12/17/2012 02:19 AM, Tomáš Chvátal wrote:
>> Currently we put portage into /usr/portage and all related stuff is to
>> be in the subfolders there (distfiles, binpkg).
>>
>> I've always myself override these defaults in make.conf to point for
>> /var/portage/ (not /var/lib because I never bothered enough how to
>> make world and config files to be put elsewhere :P).
>>
>> The only reason why we have this currently in usr is that bsd ports
>> put their stuff in there and I suppose Daniel just did the same.
>>
>> With respect to reality how stuff is done in the linux land all the
>> variable data should be in /var so we should adjust and move it in
>> there too.
>>
>> What would  you think?
> 
> I like the idea. As noted in bug #378603 [1], I'd like the portage
> ebuild to ensure that the locations don't unexpectedly change for
> existing installs.
> 
> Will it break catalyst? If so, we might begin the migration by fixing
> catalyst and having to set the new default locations in the make.conf
> that it generates.
> 
> [1] https://bugs.gentoo.org/show_bug.cgi?id=378603#c1
> 
Yes, it will break catalyst.  However, if the folder that the portage
tree goes in is still something/something/portage then it shouldn't be
hard to fix.

- -ZC
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iQIcBAEBAgAGBQJQ0MjSAAoJEKXdFCfdEflKTZ4P/2h5iKIGbNWhxAGkWEDRez31
YiCVUf5lJeDGAYpJ0hM20aWebzrR+HITqcKENKfGwerr+2/VeKVKsyYR8M1tNbQs
9jCNTMBfoYOEm8ElOu0Vzny5uWV3LflP0BMsnOjBDUpxw9ESXuz+54tkknsyEaLD
yBoVoe4zj4GfY/B3aXO08gqig/w7sGR3iLPyhy3iJimznK/qMszyMJ9LzCFVSmwz
zXVMoaM1VKTwVI7E/CZmxBWIF4KVoQ1fl4xJoqt4l82HK/5IDoRLu+s5c6rJLDhp
SPDsFQa+XtdHOZvaXhn+15zAcWFJm8UBdXtnfi8tJmpoukEJr3BscC9DRUCdRzbR
mzE0ptqDayoJ8tKuxUR+ppU987PPIAJ03h/X3yTWsIc7QPB1TtFI5W6mk4/iVPSy
FF9LrcxYvAM/+Cqu+LlhKIapqK+XAA7nw6txpEjiw3HcTZ+0O/GGHTlKM0gxbx0i
kMGe00zwodoN7DwGYenmulDk3eX71moZ/ioXMMNisDAazmcbOKOEXC/pmuUk342H
9oUtgYCH/PJGjdZHAlQnBwyfIP2YCNRZEnL6lN5t6V+p0XPzMB97tJCYsB2qEifx
m1qOPZhQX5dXovKJZyrs93SbpkUhMF8TXi6IsosVtAQFldRd2LJOawRBEkZzF2Yn
mn0gQl0ORyorOMEYsxbL
=lZrh
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Moving our/portage stuff to var

[Rick "Zero_Chaos" Farina]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/18/2012 02:49 PM, Rick "Zero_Chaos" Farina wrote:
> On 12/18/2012 01:38 PM, Zac Medico wrote:
>> On 12/17/2012 02:19 AM, Tomáš Chvátal wrote:
>>> Currently we put portage into /usr/portage and all related stuff is to
>>> be in the subfolders there (distfiles, binpkg).
>>>
>>> I've always myself override these defaults in make.conf to point for
>>> /var/portage/ (not /var/lib because I never bothered enough how to
>>> make world and config files to be put elsewhere :P).
>>>
>>> The only reason why we have this currently in usr is that bsd ports
>>> put their stuff in there and I suppose Daniel just did the same.
>>>
>>> With respect to reality how stuff is done in the linux land all the
>>> variable data should be in /var so we should adjust and move it in
>>> there too.
>>>
>>> What would  you think?
> 
>> I like the idea. As noted in bug #378603 [1], I'd like the portage
>> ebuild to ensure that the locations don't unexpectedly change for
>> existing installs.
> 
>> Will it break catalyst? If so, we might begin the migration by fixing
>> catalyst and having to set the new default locations in the make.conf
>> that it generates.
> 
>> [1] https://bugs.gentoo.org/show_bug.cgi?id=378603#c1
> 
> Yes, it will break catalyst.  However, if the folder that the portage
> tree goes in is still something/something/portage then it shouldn't be
> hard to fix.
> 
It should probably be mentioned (since most of us don't use the
snapshots every day) that the snapshots actually contain a folder called
"portage" in the tarball.  Not that it would be impossible to change,
but if we migrate from /usr/portage to /var/whatever/portage then the
changes are trivial, if we migrate to /var/repositories/gentoo that
makes things like unpacking the snapshots significantly non-trivial.

I really don't care what everyone wants to do here (although I'm
generally for sticking closer to FHS), but I warn that if the path
doesn't end in "portage" the changes are going to be significantly
non-trivial.

Thanks,
Zero

> -ZC
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iQIcBAEBAgAGBQJQ0MwqAAoJEKXdFCfdEflKF0YQAJ1mBkAPgDcIdXL+2eIdc5GO
yk7t2qSOtSaU159C8d2xI2AdB3b5jm5AXWrrvpAovNIAMTN/fFa781oKQVkFprsS
AZsT3UFluLep3ngLVTvpv8nry2t0AhT+/k1u1GAeurnTT5b4GDGXu7EF+QHzeaLC
+8VF8/N0TvRp+PwZ9lEJGXd+RVXxQ6AJXo1XsIbgKnA/L5mCKCfYn3o+9oQCqHbW
fPVeGyqEAKVYFG8rNUHwxPHYUvxhIC7xxuBl81ErXv5VqhcgNNjX4SMnKXeT3Z/V
FRxtnLJMz69mlX9C+CMf9boFVfveMjGj7pZOzopJGGhJM3d8Q/rAbgHejEmtrlAd
n9FBLc7ihGGBGGcTHrmhG+1DalmY/c2WaR63NfR+Le5mIpPpC0mibhK5TYeWRAcF
zFJhjmLwBfyAlBLnQk6gd4x/c8jMH68wgvRqPNMJbWETkT3tntgl7cUcoesv6TdT
fGNuISGCqMSEHHTKxX9XvqBl5EDSNEIS4I7rbFwO3qtuHuR2+88H506E9RQU4Xdd
+/bV6EW3Z7s/9hp7LhvK5CqEXAfJwN278aJOKAuyNX/YmwKa1VXy/ZDMXEUrTa6x
r+LcgyNHUkY3Sr4OFR7yoIY/oH4FrP1Y5H45GQUNje/XeL7bnLoN9h4QfeIwwt/L
6KZZtj3hmJa7I/m5hBwv
=VeR7
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Moving our/portage stuff to var

[Florian D.]

Am Mon, 17 Dec 2012 13:45:35 +0100
schrieb Diego Elio Pettenò <flameeyes@flameeyes.eu>:

> On 17/12/2012 13:42, Ulrich Mueller wrote:
> > If we change the location, can we then move distfiles to some place
> > outside of the tree? Something like:
> > 
> >    /var/cache/portage
> >    /var/cache/distfiles
> 
> What I do on my systems is
> 
> /var/cache/portage/tree
> /var/cache/portage/distfiles
> 

+1 for moving the location

my suggestion:

/var/gentoo/distfiles
/var/gentoo/packages
/var/gentoo/repositories

/var/gentoo/repositories/layman
/var/gentoo/repositories/portage
.
.
this could be extended, like:
/var/gentoo/repositories/sth_paludis_related

It would be nice, if you'd adopted my scheme, then I don't need to change anything.. ;) 

Re: [gentoo-dev] Moving our/portage stuff to var

[Diego Elio Pettenò]

[-- Attachment #1: Type: text/plain, Size: 1204 bytes --]

No /var/gentoo. No /var/repositories.

/var/db/gentoo, /var/db/repositories, /var/cache/portage ... as long as Zac
is fine with one whatever, but let's not invent any new top-level.

Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/



On Tue, Dec 18, 2012 at 10:20 PM, Florian D. <flockmock@gmail.com> wrote:

> Am Mon, 17 Dec 2012 13:45:35 +0100
> schrieb Diego Elio Pettenò <flameeyes@flameeyes.eu>:
>
> > On 17/12/2012 13:42, Ulrich Mueller wrote:
> > > If we change the location, can we then move distfiles to some place
> > > outside of the tree? Something like:
> > >
> > >    /var/cache/portage
> > >    /var/cache/distfiles
> >
> > What I do on my systems is
> >
> > /var/cache/portage/tree
> > /var/cache/portage/distfiles
> >
>
> +1 for moving the location
>
> my suggestion:
>
> /var/gentoo/distfiles
> /var/gentoo/packages
> /var/gentoo/repositories
>
> /var/gentoo/repositories/layman
> /var/gentoo/repositories/portage
> .
> .
> this could be extended, like:
> /var/gentoo/repositories/sth_paludis_related
>
> It would be nice, if you'd adopted my scheme, then I don't need to change
> anything.. ;)
>
>

[-- Attachment #2: Type: text/html, Size: 1857 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Zac Medico]

On 12/18/2012 01:33 PM, Diego Elio Pettenò wrote:
> No /var/gentoo. No /var/repositories.
> 
> /var/db/gentoo, /var/db/repositories, /var/cache/portage ... as long as
> Zac is fine with one whatever, but let's not invent any new top-level.

Yeah, /var/db or /var/cache sounds good to me.

I would encourage all those interested to coordinate with the catalyst
developers to have the new defaults written in the make.conf that it
generates. Having the new defaults explicitly recorded there does not
require any changes to portage (though I will bring portage's internal
defaults into sync as soon as possible). Also, I think it might help
avoid confusion for users if they are able to see the new defaults
explicitly recorded in make.conf (as opposed to
/usr/share/portage/config/make.globals which is somewhat obscure).
-- 
Thanks,
Zac

Re: [gentoo-dev] Moving our/portage stuff to var

[Joshua Saddler]

[-- Attachment #1: Type: text/plain, Size: 1281 bytes --]

On Mon, 17 Dec 2012 11:19:20 +0100
Tomáš Chvátal <tomas.chvatal@gmail.com> wrote:

> Currently we put portage into /usr/portage and all related stuff is to
> be in the subfolders there (distfiles, binpkg).
> 
> I've always myself override these defaults in make.conf to point for
> /var/portage/ (not /var/lib because I never bothered enough how to
> make world and config files to be put elsewhere :P).
> 
> The only reason why we have this currently in usr is that bsd ports
> put their stuff in there and I suppose Daniel just did the same.
> 
> With respect to reality how stuff is done in the linux land all the
> variable data should be in /var so we should adjust and move it in
> there too.
> 
> What would you think?

do it. stick it somewhere in /var. i have a small SLC SSD just for /var and /usr/portage partitions, since those consistently incur high writes. dropping to just one partition for all that i/o would be real nice.

if this proposed change is made, please make sure to contact the GDP. while we don't update things like manpages or elog announcements, we would have a ton of stuff to fix in gentoo.org/doc/en/ . also, make sure stuff is sorted out on the catalyst/releng end well in advance, so users aren't stuck with bad stages.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Marc Schiffbauer]

[-- Attachment #1: Type: text/plain, Size: 419 bytes --]

Am Dienstag, 18. Dezember 2012, 22:33:06 schrieb Diego Elio Pettenò:
> No /var/gentoo. No /var/repositories.
> 
> /var/db/gentoo, /var/db/repositories, /var/cache/portage ... as long as Zac
> is fine with one whatever, but let's not invent any new top-level.

Why not? From a FHS pov it seems ok...

I would suggest /var/portage ...

-Marc
-- 
0x35A64134 - 8AAC 5F46 83B4 DB70 8317  3723 296C 6CCA 35A6 4134

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Diego Elio Pettenò]

[-- Attachment #1: Type: text/plain, Size: 766 bytes --]

On 19/12/2012 13:44, Marc Schiffbauer wrote:
> Why not? From a FHS pov it seems ok...
> 
> I would suggest /var/portage ...

Seriously, mine is going to be a huge veto here with as much power I can
put.

There is a _reason_ why stuff is added to /var/lib instead of having

/var/postgres
/var/mysql
/var/foobar
/var/wtf
/var/wth
/var/imtired

...

As I said on other messages before (which you probably missed since you
ask "Why not?"), putting it in /var/lib or /var/db or /var/cache makes
it explicit how you should handle its backup.

/var/portage ? I have to look it up manually.

Let's to things properly instead of how it looks cooler.

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 553 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Vadim A. Misbakh-Soloviov]

[-- Attachment #1: Type: text/plain, Size: 2287 bytes --]

But, anyway, I think, /var/repositories/gentoo is very very nice idea ;)

19.12.2012 03:03, Rick "Zero_Chaos" Farina пишет:
> On 12/18/2012 02:49 PM, Rick "Zero_Chaos" Farina wrote:
>> On 12/18/2012 01:38 PM, Zac Medico wrote:
>>> On 12/17/2012 02:19 AM, Tomáš Chvátal wrote:
>>>> Currently we put portage into /usr/portage and all related stuff is to
>>>> be in the subfolders there (distfiles, binpkg).
>>>>
>>>> I've always myself override these defaults in make.conf to point for
>>>> /var/portage/ (not /var/lib because I never bothered enough how to
>>>> make world and config files to be put elsewhere :P).
>>>>
>>>> The only reason why we have this currently in usr is that bsd ports
>>>> put their stuff in there and I suppose Daniel just did the same.
>>>>
>>>> With respect to reality how stuff is done in the linux land all the
>>>> variable data should be in /var so we should adjust and move it in
>>>> there too.
>>>>
>>>> What would  you think?
> 
>>> I like the idea. As noted in bug #378603 [1], I'd like the portage
>>> ebuild to ensure that the locations don't unexpectedly change for
>>> existing installs.
> 
>>> Will it break catalyst? If so, we might begin the migration by fixing
>>> catalyst and having to set the new default locations in the make.conf
>>> that it generates.
> 
>>> [1] https://bugs.gentoo.org/show_bug.cgi?id=378603#c1
> 
>> Yes, it will break catalyst.  However, if the folder that the portage
>> tree goes in is still something/something/portage then it shouldn't be
>> hard to fix.
> 
> It should probably be mentioned (since most of us don't use the
> snapshots every day) that the snapshots actually contain a folder called
> "portage" in the tarball.  Not that it would be impossible to change,
> but if we migrate from /usr/portage to /var/whatever/portage then the
> changes are trivial, if we migrate to /var/repositories/gentoo that
> makes things like unpacking the snapshots significantly non-trivial.
> 
> I really don't care what everyone wants to do here (although I'm
> generally for sticking closer to FHS), but I warn that if the path
> doesn't end in "portage" the changes are going to be significantly
> non-trivial.
> 
> Thanks,
> Zero
> 
>> -ZC
> 
> 
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 899 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Diego Elio Pettenò]

[-- Attachment #1: Type: text/plain, Size: 377 bytes --]

On 19/12/2012 14:03, Vadim A. Misbakh-Soloviov wrote:
> But, anyway, I think, /var/repositories/gentoo is very very nice idea ;)

I'm going to repeat myself until this is shot down entirely.

We're not going to create a new top-level directory in /var. Get over
it. Stop.

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 553 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Ulrich Mueller]

>>>>> On Wed, 19 Dec 2012, Diego Elio Pettenò wrote:

> On 19/12/2012 13:44, Marc Schiffbauer wrote:
>> I would suggest /var/portage ...

> Seriously, mine is going to be a huge veto here with as much power I
> can put.

Why? The portage tree is of central importance for Gentoo, so IMHO a
second-level directory would be acceptable for it. Besides, it
currently is in /usr/portage, so it wouldn't be new but would only
move from /usr to /var.

> There is a _reason_ why stuff is added to /var/lib instead of having

> /var/postgres
> /var/mysql
> /var/foobar
> /var/wtf
> /var/wth
> /var/imtired

> ...

I don't understand how this is related to the discussion. None of the
above have any relevance for Gentoo that would be comparable to
Portage.

This doesn't mean that /var/portage is the only possible choice. But
IMHO it's better than some of the other suggestions that I've seen
here, like /var/cache/portage/repositories/gentoo/tree and so on.

> As I said on other messages before (which you probably missed since
> you ask "Why not?"), putting it in /var/lib or /var/db or /var/cache
> makes it explicit how you should handle its backup.

Yes, these are certainly fine, as long as we don't add additional
useless subdirectory levels.

> /var/portage ? I have to look it up manually.

Please, stay serious. ;-)

Ulrich

Re: [gentoo-dev] Moving our/portage stuff to var

[Diego Elio Pettenò]

On 19/12/2012 14:43, Ulrich Mueller wrote:
> Why? The portage tree is of central importance for Gentoo, so IMHO a
> second-level directory would be acceptable for it. Besides, it
> currently is in /usr/portage, so it wouldn't be new but would only
> move from /usr to /var.

I'm irked enough by /usr/portage that using that as a reason is just
going to make me feel even more strongly that it should not be /var/portage.

> I don't understand how this is related to the discussion. None of the
> above have any relevance for Gentoo that would be comparable to
> Portage.

See above.

> This doesn't mean that /var/portage is the only possible choice. But
> IMHO it's better than some of the other suggestions that I've seen
> here, like /var/cache/portage/repositories/gentoo/tree and so on.

I'm not arguing that it should go 5 levels deep. But two or three deep
is fine for me. Is it going to be /var/db/portage/master ? Fine. Is it
going to be /var/cache/portage/tree? Fine. /var/cache/portage +
/var/cache/distfiles ? Fine.

Just mv /usr/portage /var/portage ? FFS no. Among other things, as many
said before, we should really take distfiles out of the tree itself, and
packages the same. And I don't want /var/packages or /var/distfiles at all.

>> /var/portage ? I have to look it up manually.
> 
> Please, stay serious. ;-)

I am serious. If it's my first time backing up a system, and I encounter
a directory "/var/portage", it doesn't make it clear what it contains.
Is it re-generable? Should it be backed up entirely?

That's why my suggestion is to use /var/cache: it makes it clear that
there is no definitive reason to back it up (as Justin said there is an
issue with distfiles you can't re-download but that's a different story
I'd say — maybe setting a default read-only distdir for said packages
might make sense, but I don't want to get there at all).

Also, I usually keep /var/cache in a more "unsafe" disk — I don't care
if I lose cache because the drive dies, while /var/lib is fully backed
up. I don't usually split /var/db but I can see what people were saying
about having different allocation requirements for the tree compared to
distfiles, and I guess that if we put the tree there we could gain
something even for /var/db/pkg by splitting it.

Tree hierarchies are there to make things more easily organized, not
just to look nice.

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/

Re: [gentoo-dev] Moving our/portage stuff to var

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 19/12/12 08:56 AM, Diego Elio Pettenò wrote:
> 
> That's why my suggestion is to use /var/cache: it makes it clear
> that there is no definitive reason to back it up (as Justin said
> there is an issue with distfiles you can't re-download but that's a
> different story I'd say — maybe setting a default read-only distdir
> for said packages might make sense, but I don't want to get there
> at all).


In terms of the fact that a current copy of the portage tree is always
available, no it isn't necessary to back it up.  However, if one isn't
constantly maintaining their system via -uDN and doing say, updates on
a monthly cycle (ie, production systems), then it is very useful to
maintain the same portage tree snapshot as the system's last -uDN ...
 As such I would argue that it is worthwhile to back it up.
Similarly, 'packages' should probably stay synchronized with the tree.

So in terms of the above, would that mean /var/lib is a better fit?
or would that mean /var/cache and it is up to the user to add their
own backup of /var/cache/portage ?


Distfiles, imo, are definitely just cache and can be discarded at any
time.  There are issues if one has a very old tree that some distfiles
disappear from the mirrors (especially gentoo patchset tarballs) but
such is life -- personally I'd like to see all such files stored on a
dev's webspace in perpetuity so that SRC_URI could grab it from there
after it's dropped from the mirrors.  As for "special" distfiles
(fetch-restricted etc), these would need to be downloaded manually
anyways and if they are of value they should be backed up elsewhere
(ie, not rely on the distfiles dir to keep them).


Either of i.e. /var/cache/{distfiles,packages,portage} or
/var/cache/portage/{distfiles,packages,tree} works for me; i can see
the extra directory level keeping all portage bits together as looking
nicer for the end user but meh.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlDRy8kACgkQ2ugaI38ACPAiLQD9HeENg+cPrkQcHhPF54h1AaPG
hvTvaq4GaghMNXCKV7sBAKz8cKR6LD8grvuTnftWVJiRYYbhYM+HANTaE5xWs6f+
=WPmW
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Moving our/portage stuff to var

[Diego Elio Pettenò]

[-- Attachment #1: Type: text/plain, Size: 453 bytes --]

On 19/12/2012 15:14, Ian Stakenvicius wrote:
> So in terms of the above, would that mean /var/lib is a better fit?
> or would that mean /var/cache and it is up to the user to add their
> own backup of /var/cache/portage ?

I would say it's up to the user. When I do that kind of setup I actually
use a single rsync source and tar it up from there.

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 553 bytes --]

[gentoo-dev] Re: Moving our/portage stuff to var

[Duncan]

Ulrich Mueller posted on Wed, 19 Dec 2012 14:43:56 +0100 as excerpted:

>>>>>> On Wed, 19 Dec 2012, Diego Elio Pettenò wrote:
> 
>> On 19/12/2012 13:44, Marc Schiffbauer wrote:
>>> I would suggest /var/portage ...
> 
>> Seriously, mine is going to be a huge veto here with as much power I
>> can put.
> 
> Why? The portage tree is of central importance for Gentoo, so IMHO a
> second-level directory would be acceptable for it. Besides, it currently
> is in /usr/portage, so it wouldn't be new but would only move from /usr
> to /var.
> 
>> There is a _reason_ why stuff is added to /var/lib instead of having
> 
>> /var/postgres /var/mysql /var/foobar /var/wtf /var/wth /var/imtired
> 
>> ...
> 
> I don't understand how this is related to the discussion. None of the
> above have any relevance for Gentoo that would be comparable to Portage.

Seriously, people, we're talking about a user-level default, not 
something there's no user-level way to change.  A ...

>> huge veto here with as much power I can put

... seems rather strong for a simple default, for something already 
DESIGNED for the user to put wherever they want!


FWIW, here, my tree's fully resolved to /usr/src/portage, but:

PORTDIR=/p
DISTDIR=${PORTDIR}/src
PKGDIR=/m/pw

/p -> /usr/src/portage

/p because I wanted as short a path as possible.  DISTDIR is set inside 
it, but to a customized srcdir name as makes more sense to me as well.  
My layman toplevel is inside as well, as $PORTDIR/layman.  (Yes, rsync-
exclude is set appropriately too.)

/usr/src is a dedicated partition containing PORTDIR, the kernel git tree 
and a local automated kernel-patches dir (similar to /etc/portage/patches 
but for the kernel), and the system ccache.  Actually, it contains a 
second system ccache as well, for the 32-bit chroot that's the build-
image for my netbook.

PKGDIR, /m/pw, is its own dedicated partition, with /mnt being a symlink 
to /m (short paths), and pw abreviating pkg-workstation.  There's another 
dedicated partition, /m/nr/pn, that's the PKGDIR for my netbook's build-
image (nr indicating netbook root, pn indicating pkg-netbook).

The gentoo tree and overlays, along with DISTDIR, are common to both the 
workstation and the netbook, and directly net-downloadable, so keeping src 
aka DISTDIR and the layman tree inside PORTDIR, on the /usr/src 
partition, makes sense to me.  But I keep separate PKGDIRs in dedicated 
partitions so I can have backup partitions for them, thus keeping 
installable binpkgs for both the workstation and the netbook, even if the 
working partitions get fscked.


The point of all this being, these settings are DESIGNED to be set by the 
user, who can have an as crazy-for-others-but-makes-sense-to-the-user 
configuration as they want!

Based on that, yes, the defaults have some level of symbolic importance, 
but even if they're something like
/some/crazy/nth/level/weirdness/subdir/portage by default, it's *JUST* 
the defaults, and if users feel strongly about it at all, they can change 
them.  IMO there's thus no reason to feel so strongly that one issues a...

>>huge veto here with as much power I can put


In fact, it could be argued that...

/some/crazy/nth/level/weirdness/subdir/portage

... *should* be the default, precisely to MAKE that point, that it's just 
a default for something intended to be set by the user.

Not that /I'd/ argue for it, especially since the gentoo way is 
customizable configuration but sane "just works" defaults where possible 
and I'd hardly call that sane, but it /would/ make the point.


Anyway, yes, getting the default out of /usr/portage into /var somewhere 
seems reasonable, and if I were to choose, I'd choose 
/var/cache/portage as that seems to me to be the clearest labeled intent, 
but it's a default INTENDED to be set to something else, should a user 
feel strongly about it, and as such, the default really isn't that big of 
a deal.  Yes let's move it into /var, but /var/db/portage vs. 
/var/cache/portage vs. /var/portage ... if people have strong feelings 
about it, they'll move it anyway, so /whatever/ the default, even 
something as insane as the path suggested to make the point above, it's 
simply not worth having a coronary (or incinerating opposition with a 
flaming stare) over.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Moving our/portage stuff to var

[Brian Dolbec]

[-- Attachment #1: Type: text/plain, Size: 2573 bytes --]

On Tue, 2012-12-18 at 23:16 -0800, Joshua Saddler wrote:
> On Mon, 17 Dec 2012 11:19:20 +0100
> Tomáš Chvátal <tomas.chvatal@gmail.com> wrote:
> 
> > Currently we put portage into /usr/portage and all related stuff is to
> > be in the subfolders there (distfiles, binpkg).
> > 
> > I've always myself override these defaults in make.conf to point for
> > /var/portage/ (not /var/lib because I never bothered enough how to
> > make world and config files to be put elsewhere :P).
> > 
> > The only reason why we have this currently in usr is that bsd ports
> > put their stuff in there and I suppose Daniel just did the same.
> > 
> > With respect to reality how stuff is done in the linux land all the
> > variable data should be in /var so we should adjust and move it in
> > there too.
> > 
> > What would you think?
> 
> do it. stick it somewhere in /var. i have a small SLC SSD just for /var and 
> /usr/portage partitions, since those consistently incur high writes. dropping 
> to just one partition for all that i/o would be real nice.
> 
> if this proposed change is made, please make sure to contact the GDP. while we 
> don't update things like manpages or elog announcements, we would have a ton 
> of stuff to fix in gentoo.org/doc/en/ . also, make sure stuff is sorted out on 
> the catalyst/releng end well in advance, so users aren't stuck with bad stages.

Yes, Catalyst and the portage defaults will be changed in a co-ordinated
manner.  I've started poking around catalyst, but it has paths
hard-coded nearly everywhere, despite it having passed config variables
around most places.  So, It is going to need code cleanup first.  It
will take me some time to get familiar enough with the code before I
make many changes to clean it up.

For the documentation, primarily the install handbook, perhaps it would
be better to mention both locations, at least mention the old location
so existing and old users won't be thrown for a loop.  It will also need
some attention in the forums where tons of threads will be
referencing /usr/portage.      Perhaps the docs team could start
preparing the docs changes in a manner that will be easy to
search/replace with the final correct locations when that decision is
made.

And YES Diego, it won't be /var/portage or /var/repositories,  we heard
you. 

From my rough tracking, I believe somewhere under /var/cache was
majority vote.  Anyway, once catalyst is ready it will be easy to set it
to whatever is finally decided.
-- 
Brian Dolbec <dolsen@gentoo.org>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Diego Elio Pettenò]

[-- Attachment #1: Type: text/plain, Size: 259 bytes --]

On 19/12/2012 16:54, Brian Dolbec wrote:
> 
> And YES Diego, it won't be /var/portage or /var/repositories,  we heard
> you. 

Thanks, it's appreciated :)

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 553 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Marc Schiffbauer]

[-- Attachment #1: Type: text/plain, Size: 851 bytes --]

Am Mittwoch, 19. Dezember 2012, 14:43:56 schrieb Ulrich Mueller:
> >>>>> On Wed, 19 Dec 2012, Diego Elio Pettenò wrote:
> > On 19/12/2012 13:44, Marc Schiffbauer wrote:
> >> I would suggest /var/portage ...
> > 
> > Seriously, mine is going to be a huge veto here with as much power I
> > can put.
> 
> Why? The portage tree is of central importance for Gentoo, so IMHO a
> second-level directory would be acceptable for it. Besides, it
> currently is in /usr/portage, so it wouldn't be new but would only
> move from /usr to /var.

+1

FHS 2.3: "Applications must generally not add directories to the top level of 
/var. Such directories should only be added __if they have some system-wide 
implication__, and in consultation with the FHS mailing list."

-Marc
-- 
0x35A64134 - 8AAC 5F46 83B4 DB70 8317  3723 296C 6CCA 35A6 4134

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Diego Elio Pettenò]

[-- Attachment #1: Type: text/plain, Size: 700 bytes --]

On 19/12/2012 17:19, Marc Schiffbauer wrote:
> FHS 2.3: "Applications must generally not add directories to the top level of 
> /var. Such directories should only be added __if they have some system-wide 
> implication__, and in consultation with the FHS mailing list."

Since you like adding emphasis:

FHS 2.3: "Applications must generally not add directories to the top
level of /var. Such directories should only be added if they have some
system-wide implication, *and in consultation with the FHS mailing list.*"

It's always nice to cherry-pick half a phrase from a standard, right?

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 553 bytes --]

Re: [gentoo-dev] Re: Moving our/portage stuff to var

[George Shapovalov]

On Wednesday 19 December 2012 15:52:28 Duncan wrote:
> >> There is a _reason_ why stuff is added to /var/lib instead of having
> >> 
> >> /var/postgres /var/mysql /var/foobar /var/wtf /var/wth /var/imtired
> >> 
> >> ...
> > 
> > I don't understand how this is related to the discussion. None of the
> > above have any relevance for Gentoo that would be comparable to Portage.
> 
> Seriously, people, we're talking about a user-level default, not
> something there's no user-level way to change.  A ...
Um, no. Dont forget the "educate by example" aspect.
Seriously, how many people are going to read the fine print rather than just 
skimming over and going with the defaults? Oh, sure, for those who know all 
the fine details of FS layout and who manage to notice this is off and care to 
change it they totally are able to do so. But most of the users (who even 
notice)? It is going to be - "devs set it so, it must be right" and then we 
have to combat to set things right again?

Oh, and clearly I am here with Diego: /var/portage or /var/gentoo is plain 
wrong.

George

Re: [gentoo-dev] Moving our/portage stuff to var

[Alan McKinnon]

On Wed, 19 Dec 2012 14:56:44 +0100
Diego Elio Pettenò <flameeyes@flameeyes.eu> wrote:

> Just mv /usr/portage /var/portage ? FFS no. Among other things, as
> many said before, we should really take distfiles out of the tree
> itself, and packages the same. And I don't want /var/packages
> or /var/distfiles at all.

If we are going to move distfiles out of the tree into, what are the
odds of getting /some/path/portage/local to move somewhere else too?

That one has irked me for ages, its the one thing left on my systems
that stops the local tree dir being an exact replica of the upstream
master.

-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-dev] Moving our/portage stuff to var

[Zac Medico]

On 12/19/2012 02:01 PM, Alan McKinnon wrote:
> On Wed, 19 Dec 2012 14:56:44 +0100
> Diego Elio Pettenò <flameeyes@flameeyes.eu> wrote:
> 
>> Just mv /usr/portage /var/portage ? FFS no. Among other things, as
>> many said before, we should really take distfiles out of the tree
>> itself, and packages the same. And I don't want /var/packages
>> or /var/distfiles at all.
> 
> If we are going to move distfiles out of the tree into, what are the
> odds of getting /some/path/portage/local to move somewhere else too?

What program uses this "local" directory? It's not used directly by
portage itself, though portage has an exclude for it in the default
PORTAGE_RSYNC_OPTS setting (in /usr/share/portage/config/make.globals).

> That one has irked me for ages, its the one thing left on my systems
> that stops the local tree dir being an exact replica of the upstream
> master.

For portage's defaults, I won't settle for anything less than having
them all refer to separate directories which are *not* nested within one
other. These are the current default settings which violate my requirements:

	PORTDIR=/usr/portage
	DISTDIR=${PORTDIR}/distfiles
	PKGDIR=${PORTDIR}/packages
	RPMDIR=${PORTDIR}/rpm
-- 
Thanks,
Zac

Re: [gentoo-dev] Moving our/portage stuff to var

[Mike Gilbert]

On Wed, Dec 19, 2012 at 5:19 PM, Zac Medico <zmedico@gentoo.org> wrote:
> On 12/19/2012 02:01 PM, Alan McKinnon wrote:
>> On Wed, 19 Dec 2012 14:56:44 +0100
>> Diego Elio Pettenņ <flameeyes@flameeyes.eu> wrote:
>>
>>> Just mv /usr/portage /var/portage ? FFS no. Among other things, as
>>> many said before, we should really take distfiles out of the tree
>>> itself, and packages the same. And I don't want /var/packages
>>> or /var/distfiles at all.
>>
>> If we are going to move distfiles out of the tree into, what are the
>> odds of getting /some/path/portage/local to move somewhere else too?
>
> What program uses this "local" directory? It's not used directly by
> portage itself, though portage has an exclude for it in the default
> PORTAGE_RSYNC_OPTS setting (in /usr/share/portage/config/make.globals).
>

layman used to use /usr/portage/local for storing overlays. See code
listing 2.2.

http://www.gentoo.org/proj/en/overlays/userguide.xml

Re: [gentoo-dev] Moving our/portage stuff to var

[Marc Schiffbauer]

Am 19.12.2012 17:25, schrieb Diego Elio Pettenò:
> On 19/12/2012 17:19, Marc Schiffbauer wrote:
>> FHS 2.3: "Applications must generally not add directories to the top 
>> level of
>> /var. Such directories should only be added __if they have some 
>> system-wide
>> implication__, and in consultation with the FHS mailing list."
> 
> Since you like adding emphasis:
> 
> FHS 2.3: "Applications must generally not add directories to the top
> level of /var. Such directories should only be added if they have some
> system-wide implication, *and in consultation with the FHS mailing 
> list.*"
> 
> It's always nice to cherry-pick half a phrase from a standard, right?

No. That would be the next step. But I do not see any problem here. Do 
you?

-- 
8AAC 5F46 83B4 DB70 8317  3723 296C 6CCA 35A6 4134

Re: [gentoo-dev] Moving our/portage stuff to var

[Brian Dolbec]

[-- Attachment #1: Type: text/plain, Size: 1973 bytes --]

On Wed, 2012-12-19 at 17:33 -0500, Mike Gilbert wrote:
> On Wed, Dec 19, 2012 at 5:19 PM, Zac Medico <zmedico@gentoo.org> wrote:
> > On 12/19/2012 02:01 PM, Alan McKinnon wrote:
> >> On Wed, 19 Dec 2012 14:56:44 +0100
> >> Diego Elio Pettenņ <flameeyes@flameeyes.eu> wrote:
> >>
> >>> Just mv /usr/portage /var/portage ? FFS no. Among other things, as
> >>> many said before, we should really take distfiles out of the tree
> >>> itself, and packages the same. And I don't want /var/packages
> >>> or /var/distfiles at all.
> >>
> >> If we are going to move distfiles out of the tree into, what are the
> >> odds of getting /some/path/portage/local to move somewhere else too?

I am already making this configurable in catalyst which builds the
stages and install media.  It is my intention to move it to the new
repositories location alongside the gentoo tree directory and any layman
installed overlays.

> >
> > What program uses this "local" directory? It's not used directly by
> > portage itself, though portage has an exclude for it in the default
> > PORTAGE_RSYNC_OPTS setting (in /usr/share/portage/config/make.globals).
> >
> 
> layman used to use /usr/portage/local for storing overlays. See code
> listing 2.2.
> 
> http://www.gentoo.org/proj/en/overlays/userguide.xml
> 

I plan to migrate layman to the new portage/catalyst defaults when they
are done.  The current default is /var/lib/layman


As I will not be the sole dictator as to what these new defaults will
be.  I am working on the code to make it easy to set to whatever the
final decision is made to be.

just to clarify, I'm voting for...

/var/cache/distfiles
/var/cache/packages
/var/cache/repositories/
/var/cache/repositories/gentoo   <== the main portage tree
/var/cache/repositories/local    <== the new location for a local overlay
/var/cache/repositories/some-overlay  <== layman installed overlay


-- 
Brian Dolbec <dolsen@gentoo.org>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Ulrich Mueller]

>>>>> On Wed, 19 Dec 2012, Brian Dolbec wrote:

> just to clarify, I'm voting for...

> /var/cache/distfiles
> /var/cache/packages

Fine.

> /var/cache/repositories/
> /var/cache/repositories/gentoo   <== the main portage tree
> /var/cache/repositories/local    <== the new location for a local overlay
> /var/cache/repositories/some-overlay  <== layman installed overlay

So the Portage tree would move from the second level (/usr/portage) to
the fourth level? IMHO this is unacceptable. Make it /var/cache/gentoo
or /var/cache/portage; the /var/cache directory really isn't so
overpopulated that there's a need for hiding things in subdirs.

Also I wonder if local overlays should be in /var/cache? It might not
always be possible to restore them.

Ulrich

[gentoo-dev] Re: Moving our/portage stuff to var

[Duncan]

Ulrich Mueller posted on Thu, 20 Dec 2012 09:11:39 +0100 as excerpted:

> Also I wonder if local overlays should be in /var/cache? It might not
> always be possible to restore them.

Good point.  My local overlay's in /usr/local/ (which is a dedicated 
partition I actually keep several backups of, including my netbook's 
copy), for exactly that reason.

(Actually, /usr/local is a symlink, to a shorter path that's the actual 
partition, /l, and both PORTDIR and my local overlay are simply "p" (/p 
being a symlink to the gentoo tree in /usr/src), so I configure and 
access the overlay as /l/p, and simply /p for the gentoo tree, but the /l 
partition is also reachable via /usr/local symlink.  But however it's 
identified, the partition reachable as either /l or /usr/local contains 
the local overlay, and is backed up multiple ways including on an 
entirely separate machine, which its own copy of the same /l aka /usr/
local.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Moving our/portage stuff to var

[George Shapovalov]

On Thursday 20 December 2012 09:11:39 Ulrich Mueller wrote:
> > /var/cache/repositories/local    <== the new location for a local overlay
> 
> Also I wonder if local overlays should be in /var/cache? It might not
> always be possible to restore them.
So, this is the present /usr/local/portage? What's wrong with it?
According to FHS, /usr/local/ is reserved for local admin meddling, which fits 
the bill rather nicely fo local overlay. This is the one location that was 
actually making sense from the beginning.  

Now, trying to recall its history, I seem to remember that it was not there 
from the onset, but rather added when we first had this idea of overlays, and, 
faintly (remember), even having some discussion with some technical arguments. 
I guess this is how it ended up in the place that makes sense :).

George 

Re: [gentoo-dev] Moving our/portage stuff to var

[Alan McKinnon]

On Wed, 19 Dec 2012 14:19:52 -0800
Zac Medico <zmedico@gentoo.org> wrote:

> On 12/19/2012 02:01 PM, Alan McKinnon wrote:
> > On Wed, 19 Dec 2012 14:56:44 +0100
> > Diego Elio Pettenò <flameeyes@flameeyes.eu> wrote:
> > 
> >> Just mv /usr/portage /var/portage ? FFS no. Among other things, as
> >> many said before, we should really take distfiles out of the tree
> >> itself, and packages the same. And I don't want /var/packages
> >> or /var/distfiles at all.
> > 
> > If we are going to move distfiles out of the tree into, what are the
> > odds of getting /some/path/portage/local to move somewhere else too?
> 
> What program uses this "local" directory? It's not used directly by
> portage itself, though portage has an exclude for it in the default
> PORTAGE_RSYNC_OPTS setting
> (in /usr/share/portage/config/make.globals).

It goes back a long time, and is basically a poor man's local overlay
without having to use layman. As I understand it, portage will treat
the directory like any other when looking for ebuilds and resolving
deps, but exclude it from a sync.

> 
> > That one has irked me for ages, its the one thing left on my systems
> > that stops the local tree dir being an exact replica of the upstream
> > master.
> 
> For portage's defaults, I won't settle for anything less than having
> them all refer to separate directories which are *not* nested within
> one other. These are the current default settings which violate my
> requirements:
> 
> 	PORTDIR=/usr/portage
> 	DISTDIR=${PORTDIR}/distfiles
> 	PKGDIR=${PORTDIR}/packages
> 	RPMDIR=${PORTDIR}/rpm

/usr/portage/local has the taste feel and smell of a hacky workaround:
shove a directory in the tree and exclude it from sync.

I suspect the best solution all round is to move all support for local
overlays into layman. I'd be happy with that. Probably make the portage
code cleaner too.



-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-dev] Moving our/portage stuff to var

[Rich Freeman]

On Thu, Dec 20, 2012 at 4:25 AM, George Shapovalov <george@gentoo.org> wrote:
> On Thursday 20 December 2012 09:11:39 Ulrich Mueller wrote:
>> > /var/cache/repositories/local    <== the new location for a local overlay
>>
>> Also I wonder if local overlays should be in /var/cache? It might not
>> always be possible to restore them.
> So, this is the present /usr/local/portage? What's wrong with it?
> According to FHS, /usr/local/ is reserved for local admin meddling, which fits
> the bill rather nicely fo local overlay. This is the one location that was
> actually making sense from the beginning.

I actually like the /var/cache/repositories approach.  You can always
add a symlink to it if you want to for convenience (as I already do
for /var/lib/portage/world).  There is really nothing special about
/usr/portage, other than it being the lowest-priority overlay.

As far as the local one goes - again this is configurable in
/etc/make.conf, and we don't really need to pre-create that directory
either.

Rich

Re: [gentoo-dev] Moving our/portage stuff to var

[Ulrich Mueller]

>>>>> On Thu, 20 Dec 2012, Rich Freeman wrote:

> I actually like the /var/cache/repositories approach.  You can always
> add a symlink to it if you want to for convenience (as I already do
> for /var/lib/portage/world).  There is really nothing special about
> /usr/portage, other than it being the lowest-priority overlay.

There's also nothing special about Earth amongst other planets,
so please add "Earth / Solar System" when addressing your letters.

SCNR,
Ulrich

Re: [gentoo-dev] Moving our/portage stuff to var

[Graham Murray]

Zac Medico <zmedico@gentoo.org> writes:

> On 12/19/2012 02:01 PM, Alan McKinnon wrote:
>> If we are going to move distfiles out of the tree into, what are the
>> odds of getting /some/path/portage/local to move somewhere else too?
>
> What program uses this "local" directory? It's not used directly by
> portage itself, though portage has an exclude for it in the default
> PORTAGE_RSYNC_OPTS setting (in /usr/share/portage/config/make.globals).

It is useful for 'site' local ebuilds. It allows a 'master' repository
to sync with the main Gentoo one without disturbing local changes but
allow other systems on-site to fetch the modified tree with a simple
'emerge --sync'.

Re: [gentoo-dev] Moving our/portage stuff to var

[Brian Dolbec]

[-- Attachment #1: Type: text/plain, Size: 2267 bytes --]

On Thu, 2012-12-20 at 09:11 +0100, Ulrich Mueller wrote:
> >>>>> On Wed, 19 Dec 2012, Brian Dolbec wrote:
> 
> > just to clarify, I'm voting for...
> 
> > /var/cache/distfiles
> > /var/cache/packages
> 
> Fine.
> 
> > /var/cache/repositories/
> > /var/cache/repositories/gentoo   <== the main portage tree
> > /var/cache/repositories/local    <== the new location for a local overlay
> > /var/cache/repositories/some-overlay  <== layman installed overlay
> 
> So the Portage tree would move from the second level (/usr/portage) to
> the fourth level? IMHO this is unacceptable. Make it /var/cache/gentoo
> or /var/cache/portage; the /var/cache directory really isn't so
> overpopulated that there's a need for hiding things in subdirs.
> 
> Also I wonder if local overlays should be in /var/cache? It might not
> always be possible to restore them.
> 
> Ulrich
> 

My idea for having all repos under one directory is to make it easier
for a pkg manager to simply scan the directory to know all installed
overlays.  Currently each one has to be listed in a configured variable
in make.conf.  So if you wanted your local overlay somewhere else, then
a symlink would work (provided the PM can/will autoscan repos), or add
it to the PORTDIR_OVERLAY variable (current behavior).  I don't
otherwise have a strong desire for it to be there.

If and only if the tree and all overlays (not other directories) are not
under one directory, then an autoscan cannot easily happen.

For an example of what I am referring to.  In layman-2.0.0 I have added
a /etc/layman/overlays directory.  On start it will scan it for any
*.xml files and add those specifications to the overlays variable.
Layman-1.4 and previous required you add each entry to that overlays
variable in layman.cfg.  Essentially this makes the overlays directory a
plug-in directory, so if you want to install an overlay not listed in
the main repositories list, just download the xml spec to 

/etc/layman/overlays/${meaningful-name}.xml  
then layman -f (adds it to it's cache)

no editing required :)

If repositories is too long a name, repos is shorter and still
meaningful, although still puts them at a 4th level.

-- 
Brian Dolbec <dolsen@gentoo.org>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 20/12/12 10:37 AM, Brian Dolbec wrote:
>> 
>>> /var/cache/repositories/ /var/cache/repositories/gentoo   <==
>>> the main portage tree /var/cache/repositories/local    <== the
>>> new location for a local overlay 
>>> /var/cache/repositories/some-overlay  <== layman installed
>>> overlay
> 
> My idea for having all repos under one directory is to make it
> easier for a pkg manager to simply scan the directory to know all
> installed overlays.  Currently each one has to be listed in a
> configured variable in make.conf.  So if you wanted your local
> overlay somewhere else, then a symlink would work (provided the PM
> can/will autoscan repos), or add it to the PORTDIR_OVERLAY variable
> (current behavior).  I don't otherwise have a strong desire for it
> to be there.
> 
> If and only if the tree and all overlays (not other directories)
> are not under one directory, then an autoscan cannot easily
> happen.
> 


You could do this while not having the portage tree be in that
directory.  IE, portage goes in /var/cache/portage , and all the
overlays go into /var/cache/repositories.

The tree is separate enough IMO that autoscan can still happen easily,
and also I believe that it can be assumed that the tree is in place.
For instance, if the tree's location is defined to be elsewhere, it
isn't done so via PORTDIR_OVERLAYS but rather PORTDIR.


On an unrelated note, I would never treat my "local" overlays as
cache.  Ebuilds that (as a user) I wrote and installed by hand are not
likely to be kept in a repository someplace, but rather the overlay
dir would most likely be it's only location.  IIRC the reason for
/usr/portage/local/ was to have a path within the portage tree that
rsync wouldn't kill; given that what you're suggesting is already not
under the proposed portage tree location, emerge --sync couldn't touch
it, and so I don't see a need at all to provide a 'local' repository
destination by default.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlDTNlQACgkQ2ugaI38ACPCxQAEApT/7CaIbuVTwnDQk93hhDjGu
mXKPdCJg4h1iMECtdoABAJj2601LuRPUKFJ+BJa/FqrdRTsjSpBRiEd8pvO2042P
=W3T9
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Moving our/portage stuff to var

[Michael Mol]

On Thu, Dec 20, 2012 at 11:01 AM, Ian Stakenvicius <axs@gentoo.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On 20/12/12 10:37 AM, Brian Dolbec wrote:
>>>
>>>> /var/cache/repositories/ /var/cache/repositories/gentoo   <==
>>>> the main portage tree /var/cache/repositories/local    <== the
>>>> new location for a local overlay
>>>> /var/cache/repositories/some-overlay  <== layman installed
>>>> overlay
>>
>> My idea for having all repos under one directory is to make it
>> easier for a pkg manager to simply scan the directory to know all
>> installed overlays.  Currently each one has to be listed in a
>> configured variable in make.conf.  So if you wanted your local
>> overlay somewhere else, then a symlink would work (provided the PM
>> can/will autoscan repos), or add it to the PORTDIR_OVERLAY variable
>> (current behavior).  I don't otherwise have a strong desire for it
>> to be there.
>>
>> If and only if the tree and all overlays (not other directories)
>> are not under one directory, then an autoscan cannot easily
>> happen.
>>
>
>
> You could do this while not having the portage tree be in that
> directory.  IE, portage goes in /var/cache/portage , and all the
> overlays go into /var/cache/repositories.
>
> The tree is separate enough IMO that autoscan can still happen easily,
> and also I believe that it can be assumed that the tree is in place.
> For instance, if the tree's location is defined to be elsewhere, it
> isn't done so via PORTDIR_OVERLAYS but rather PORTDIR.
>
>
> On an unrelated note, I would never treat my "local" overlays as
> cache.  Ebuilds that (as a user) I wrote and installed by hand are not
> likely to be kept in a repository someplace, but rather the overlay
> dir would most likely be it's only location.  IIRC the reason for
> /usr/portage/local/ was to have a path within the portage tree that
> rsync wouldn't kill; given that what you're suggesting is already not
> under the proposed portage tree location, emerge --sync couldn't touch
> it, and so I don't see a need at all to provide a 'local' repository
> destination by default.
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.19 (GNU/Linux)
>
> iF4EAREIAAYFAlDTNlQACgkQ2ugaI38ACPCxQAEApT/7CaIbuVTwnDQk93hhDjGu
> mXKPdCJg4h1iMECtdoABAJj2601LuRPUKFJ+BJa/FqrdRTsjSpBRiEd8pvO2042P
> =W3T9
> -----END PGP SIGNATURE-----
>

It's sounding like the nearly the optimal solution would be:

/var/cache/portage/distfiles
/var/cache/portage/repositories/gentoo
/var/cache/portage/repositories/{sunrise,kde,gnome,whatever,layman,grabs}
/var/db/portage/repositories/{non-cache,repo,names,go,here}

Clearly, some data in question needs to be treated as persistent, and
others can be treated as cache. So it should probably be divided up
that way. The placement of tree and overlays as subfolders of the same
folder strikes me as appropriate, too.

The only thing I can't see an elegant workaround for are how to avoid
or handle repo name collisions between
/var/cache/portage/repositories/* and /var/db/portage/repositories/*

--
:wq

Re: [gentoo-dev] Moving our/portage stuff to var

[Diego Elio Pettenò]

[-- Attachment #1: Type: text/plain, Size: 363 bytes --]

On 20/12/2012 17:16, Michael Mol wrote:
> 
> /var/cache/portage/distfiles
> /var/cache/portage/repositories/gentoo
> /var/cache/portage/repositories/{sunrise,kde,gnome,whatever,layman,grabs}
> /var/db/portage/repositories/{non-cache,repo,names,go,here}

+1

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 553 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Rich Freeman]

On Thu, Dec 20, 2012 at 11:19 AM, Diego Elio Pettenò
<flameeyes@flameeyes.eu> wrote:
> On 20/12/2012 17:16, Michael Mol wrote:
>>
>> /var/cache/portage/distfiles
>> /var/cache/portage/repositories/gentoo
>> /var/cache/portage/repositories/{sunrise,kde,gnome,whatever,layman,grabs}
>> /var/db/portage/repositories/{non-cache,repo,names,go,here}
>
> +1

Agreed.

Look, we're not going to find any place that makes everybody happy.
This one seems to be logical from a design standpoint.  As long as we
make sure that everything is set in configuration then individuals can
move it wherever they want to.  Symlinks also work.

Rich

Re: [gentoo-dev] Moving our/portage stuff to var

[Rick "Zero_Chaos" Farina]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/20/2012 11:16 AM, Michael Mol wrote:
> On Thu, Dec 20, 2012 at 11:01 AM, Ian Stakenvicius <axs@gentoo.org> wrote:
> On 20/12/12 10:37 AM, Brian Dolbec wrote:
>>>>>
>>>>>> /var/cache/repositories/ /var/cache/repositories/gentoo   <==
>>>>>> the main portage tree /var/cache/repositories/local    <== the
>>>>>> new location for a local overlay
>>>>>> /var/cache/repositories/some-overlay  <== layman installed
>>>>>> overlay
>>>>
>>>> My idea for having all repos under one directory is to make it
>>>> easier for a pkg manager to simply scan the directory to know all
>>>> installed overlays.  Currently each one has to be listed in a
>>>> configured variable in make.conf.  So if you wanted your local
>>>> overlay somewhere else, then a symlink would work (provided the PM
>>>> can/will autoscan repos), or add it to the PORTDIR_OVERLAY variable
>>>> (current behavior).  I don't otherwise have a strong desire for it
>>>> to be there.
>>>>
>>>> If and only if the tree and all overlays (not other directories)
>>>> are not under one directory, then an autoscan cannot easily
>>>> happen.
>>>>
> 
> 
> You could do this while not having the portage tree be in that
> directory.  IE, portage goes in /var/cache/portage , and all the
> overlays go into /var/cache/repositories.
> 
> The tree is separate enough IMO that autoscan can still happen easily,
> and also I believe that it can be assumed that the tree is in place.
> For instance, if the tree's location is defined to be elsewhere, it
> isn't done so via PORTDIR_OVERLAYS but rather PORTDIR.
> 
> 
> On an unrelated note, I would never treat my "local" overlays as
> cache.  Ebuilds that (as a user) I wrote and installed by hand are not
> likely to be kept in a repository someplace, but rather the overlay
> dir would most likely be it's only location.  IIRC the reason for
> /usr/portage/local/ was to have a path within the portage tree that
> rsync wouldn't kill; given that what you're suggesting is already not
> under the proposed portage tree location, emerge --sync couldn't touch
> it, and so I don't see a need at all to provide a 'local' repository
> destination by default.
> 
>>
> 
> It's sounding like the nearly the optimal solution would be:
> 
> /var/cache/portage/distfiles
> /var/cache/portage/repositories/gentoo
> /var/cache/portage/repositories/{sunrise,kde,gnome,whatever,layman,grabs}
> /var/db/portage/repositories/{non-cache,repo,names,go,here}

Not to oversimplify but why exactly can't we leave /usr/local/portage
where it is? I'm not going to want to cd
/var/db/portage/repositories/local every time I want to edit a local
ebuild...

- -ZC
> 
> Clearly, some data in question needs to be treated as persistent, and
> others can be treated as cache. So it should probably be divided up
> that way. The placement of tree and overlays as subfolders of the same
> folder strikes me as appropriate, too.
> 
> The only thing I can't see an elegant workaround for are how to avoid
> or handle repo name collisions between
> /var/cache/portage/repositories/* and /var/db/portage/repositories/*
> 
> --
> :wq
> 
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iQIcBAEBAgAGBQJQ0zvlAAoJEKXdFCfdEflKfiEP/0uLujqLlv1DVydqj3xXZUVq
t/c5mDsg3iJgt5T7Hm+ER949r2GUqju4veed4JQWFlVSaOoLEViL1Me/jPco5fC8
v064ktt2hOLPb+tR2IWaK3tR8i+LhcFEcIyANhl62ENPWgvOAR6V0KNFuudQLicS
QUFaJYKZkkYuPSTTqPld3QXzFwH1X6RCQaOtjCOqZKAZr9iW8HRNTTLpoa4bSMgr
VBswHyH+q0C9TzIVv5u8G8s8cYNdqHf1wrSTeMjq961tVzF3Tno5s1zk1MOyQ7cQ
MkQCxiMAum0d9PX87UkPuvHKgLdZ7e+tW26B9bS3M9yGu66lsHB7+sOTxFAJ9kqp
YKLuO2XPmpIMyDNc/5rQtTl5ygA9CmqSpUZEjMgwvCmemOHO3CsPXXQxfq6Ze7kK
/aNfCHJVEP/x8bY7PdWoexaScW/Qnqrqm6R+GCd6B3LGmTinGaDWYzJj+pAkpGUn
OAHcxATC9gX3AZr9atTFHRaPkD3L3FdYothVDZq5DDkW2qAmuhbqaEhzytDI3GLl
R+MEWYcMqvNLV5eYlNPe4OOaYfFTr/1mP0k/3ixjxJFwMDXxmIaJGTKoMOyoPu3O
diZlo0m2EPCH7Ggl9Fh0xf4P/wDSQB0AkfyldQhnVNbR5DRcTrkU6IfJBVLdcJ40
XljVqWuG27XmMXwjLGgV
=NLJg
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Moving our/portage stuff to var

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 20/12/12 11:25 AM, Rick "Zero_Chaos" Farina wrote:
> On 12/20/2012 11:16 AM, Michael Mol wrote:
>> On Thu, Dec 20, 2012 at 11:01 AM, Ian Stakenvicius
>> <axs@gentoo.org> wrote: On 20/12/12 10:37 AM, Brian Dolbec
>> wrote:
>>>>>> 
>>>>>>> /var/cache/repositories/ /var/cache/repositories/gentoo
>>>>>>> <== the main portage tree /var/cache/repositories/local
>>>>>>> <== the new location for a local overlay 
>>>>>>> /var/cache/repositories/some-overlay  <== layman
>>>>>>> installed overlay
>>>>> 
>>>>> My idea for having all repos under one directory is to make
>>>>> it easier for a pkg manager to simply scan the directory to
>>>>> know all installed overlays.  Currently each one has to be
>>>>> listed in a configured variable in make.conf.  So if you
>>>>> wanted your local overlay somewhere else, then a symlink
>>>>> would work (provided the PM can/will autoscan repos), or
>>>>> add it to the PORTDIR_OVERLAY variable (current behavior).
>>>>> I don't otherwise have a strong desire for it to be there.
>>>>> 
>>>>> If and only if the tree and all overlays (not other
>>>>> directories) are not under one directory, then an autoscan
>>>>> cannot easily happen.
>>>>> 
> 
> 
>> You could do this while not having the portage tree be in that 
>> directory.  IE, portage goes in /var/cache/portage , and all the 
>> overlays go into /var/cache/repositories.
> 
>> The tree is separate enough IMO that autoscan can still happen
>> easily, and also I believe that it can be assumed that the tree
>> is in place. For instance, if the tree's location is defined to
>> be elsewhere, it isn't done so via PORTDIR_OVERLAYS but rather
>> PORTDIR.
> 
> 
>> On an unrelated note, I would never treat my "local" overlays as 
>> cache.  Ebuilds that (as a user) I wrote and installed by hand
>> are not likely to be kept in a repository someplace, but rather
>> the overlay dir would most likely be it's only location.  IIRC
>> the reason for /usr/portage/local/ was to have a path within the
>> portage tree that rsync wouldn't kill; given that what you're
>> suggesting is already not under the proposed portage tree
>> location, emerge --sync couldn't touch it, and so I don't see a
>> need at all to provide a 'local' repository destination by
>> default.
> 
>>> 
> 
>> It's sounding like the nearly the optimal solution would be:
> 
>> /var/cache/portage/distfiles 
>> /var/cache/portage/repositories/gentoo 
>> /var/cache/portage/repositories/{sunrise,kde,gnome,whatever,layman,grabs}
>>
>> 
/var/db/portage/repositories/{non-cache,repo,names,go,here}
> 
> Not to oversimplify but why exactly can't we leave
> /usr/local/portage where it is? I'm not going to want to cd 
> /var/db/portage/repositories/local every time I want to edit a
> local ebuild...
> 
> -ZC
> 

IMO local user overlays would always end up being defined and placed
wherever the user decides them to be -- ie, /usr/local/portage as Rick
so nicely pointed out.

It may be a nice idea to try and enforce a structure to it/them, but
since PORTDIR_OVERLAY can be defined to include any path at all I
don't really see a point to it.


..I'm still with ulm about the tree not being in the repositories
subdir though.  If I wanted to nuke all the overlays installed, "rm
- -Rf /var/cache/portage/repos/*" is very easy.  If the tree is also in
that dir then it becomes less easy:  "find /var/cache/portage/repos
- -maxdepth 1 -mindepth 1 -not -name gentoo -exec rm -Rf {} \+"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlDTPlUACgkQ2ugaI38ACPBtPAD8DJ6BPjL6xY8alB5pbo7vQ5kb
dzRO9Z32F3r84RyVccABAIEu+k+ztw0ipoCwhmLlBHyiU6aEOsExixNvnMMLLu9X
=2gWT
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Moving our/portage stuff to var

[Ulrich Mueller]

>>>>> On Thu, 20 Dec 2012, Diego Elio Pettenò wrote:

>> /var/cache/portage/distfiles
>> /var/cache/portage/repositories/gentoo
>> /var/cache/portage/repositories/{sunrise,kde,gnome,whatever,layman,grabs}
>> /var/db/portage/repositories/{non-cache,repo,names,go,here}

> +1

-1

The subdirs are too deeply nested. (Ebuilds would be at the eighth
level then...)

Let's put the tree in /var/cache/portage please, and distfiles in
/var/cache/distfiles. Layman overlays can stay where they are, or move
to /var/cache/layman.

Ulrich

Re: [gentoo-dev] Moving our/portage stuff to var

[Michael Mol]

On Thu, Dec 20, 2012 at 12:05 PM, Ulrich Mueller <ulm@gentoo.org> wrote:
>>>>>> On Thu, 20 Dec 2012, Diego Elio Pettenň wrote:
>
>>> /var/cache/portage/distfiles
>>> /var/cache/portage/repositories/gentoo
>>> /var/cache/portage/repositories/{sunrise,kde,gnome,whatever,layman,grabs}
>>> /var/db/portage/repositories/{non-cache,repo,names,go,here}
>
>> +1
>
> -1
>
> The subdirs are too deeply nested. (Ebuilds would be at the eighth
> level then...)

Maybe I missed something...but what's wrong with that?

>
> Let's put the tree in /var/cache/portage please, and distfiles in
> /var/cache/distfiles. Layman overlays can stay where they are, or move
> to /var/cache/layman.

--
:wq

Re: [gentoo-dev] Moving our/portage stuff to var

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1355 bytes --]

On Thu, Dec 20, 2012 at 06:58:11AM -0500, Rich Freeman wrote:
> On Thu, Dec 20, 2012 at 4:25 AM, George Shapovalov <george@gentoo.org> wrote:
> > On Thursday 20 December 2012 09:11:39 Ulrich Mueller wrote:
> >> > /var/cache/repositories/local    <== the new location for a local overlay
> >>
> >> Also I wonder if local overlays should be in /var/cache? It might not
> >> always be possible to restore them.
> > So, this is the present /usr/local/portage? What's wrong with it?
> > According to FHS, /usr/local/ is reserved for local admin meddling, which fits
> > the bill rather nicely fo local overlay. This is the one location that was
> > actually making sense from the beginning.
> 
> I actually like the /var/cache/repositories approach.  You can always
> add a symlink to it if you want to for convenience (as I already do
> for /var/lib/portage/world).  There is really nothing special about
> /usr/portage, other than it being the lowest-priority overlay.
 
 If we do this, I don't like the name repositories -- what kind of
 repositories? should I put git repositories in there?

> As far as the local one goes - again this is configurable in
> /etc/make.conf, and we don't really need to pre-create that directory
> either.

Right, there is nothing to pre-create for this; it should be left to the
user.

William


[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Ulrich Mueller]

>>>>> On Thu, 20 Dec 2012, Michael Mol wrote:

>>>> /var/cache/portage/distfiles
>>>> /var/cache/portage/repositories/gentoo
>>>> /var/cache/portage/repositories/{sunrise,kde,gnome,whatever,layman,grabs}
>>>> /var/db/portage/repositories/{non-cache,repo,names,go,here}

>> -1
>> 
>> The subdirs are too deeply nested. (Ebuilds would be at the eighth
>> level then...)

> Maybe I missed something...but what's wrong with that?

There's no good reason for nesting it so deeply. As it is proposed
above, /var/cache/portage would contain only two subdirectories, and
/var/cache/portage/repositories only a single "gentoo" on a stable
system. Also /var/cache itself isn't overpopulated; I count about ten
entries on my systems.

We should go with a shorter (easier to remember, easier to type) path
and move things at least one level up. Two would be even better.

>> Let's put the tree in /var/cache/portage please, and distfiles in
>> /var/cache/distfiles. Layman overlays can stay where they are, or
>> move to /var/cache/layman.

Ulrich

Re: [gentoo-dev] Moving our/portage stuff to var

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 396 bytes --]

On Thu, 20 Dec 2012 07:37:52 -0800
Brian Dolbec <dolsen@gentoo.org> wrote:
> My idea for having all repos under one directory is to make it easier
> for a pkg manager to simply scan the directory to know all installed
> overlays.

That's going to cause trouble, unless we start forcing overlays to
contain enough information for a PM to configure them properly...

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[George Shapovalov]

On Thursday 20 December 2012 13:36:27 Alan McKinnon wrote:
> > What program uses this "local" directory? It's not used directly by
> > portage itself, though portage has an exclude for it in the default
> > PORTAGE_RSYNC_OPTS setting
> > (in /usr/share/portage/config/make.globals).
> 
> It goes back a long time, and is basically a poor man's local overlay
> without having to use layman. As I understand it, portage will treat
> the directory like any other when looking for ebuilds and resolving
> deps, but exclude it from a sync.
Nope, he means /usr/portage/local, not /usr/local/portage. It is rarely (if 
ever, nowadays) present, as I understand, but you can find it mentioned in 
that config file. It may be just a legacy definition by now, looking at how 
only layman was mentioned with relation to it and even that one appears not to 
use it any more.

BTW, /usr/local/portage is hardly ""poor man's". Where are you going to store 
your local changes that are of interest only to you and not present in any 
other overlays? (like, you want to keep some old version of some package after 
it has been cleaned, or your personal mods). The location even accords to FHS, 
which is, apparently, a rarity :).

George

Re: [gentoo-dev] Moving our/portage stuff to var

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 662 bytes --]

On Thu, 20 Dec 2012 18:44:06 +0100
Ulrich Mueller <ulm@gentoo.org> wrote:
> There's no good reason for nesting it so deeply. As it is proposed
> above, /var/cache/portage would contain only two subdirectories, and
> /var/cache/portage/repositories only a single "gentoo" on a stable
> system. Also /var/cache itself isn't overpopulated; I count about ten
> entries on my systems.

You really think most people don't have to use overlays?

> We should go with a shorter (easier to remember, easier to type) path
> and move things at least one level up. Two would be even better.

You shouldn't ever be typing that path in...

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Moving our/portage stuff to var

[Markos Chandras]

On 20 December 2012 17:44, Ulrich Mueller <ulm@gentoo.org> wrote:
>>>>>> On Thu, 20 Dec 2012, Michael Mol wrote:
>
>>>>> /var/cache/portage/distfiles
>>>>> /var/cache/portage/repositories/gentoo
>>>>> /var/cache/portage/repositories/{sunrise,kde,gnome,whatever,layman,grabs}
>>>>> /var/db/portage/repositories/{non-cache,repo,names,go,here}
>
>>> -1
>>>
>>> The subdirs are too deeply nested. (Ebuilds would be at the eighth
>>> level then...)
>
>> Maybe I missed something...but what's wrong with that?
>
> There's no good reason for nesting it so deeply. As it is proposed
> above, /var/cache/portage would contain only two subdirectories, and
> /var/cache/portage/repositories only a single "gentoo" on a stable
> system. Also /var/cache itself isn't overpopulated; I count about ten
> entries on my systems.
>
> We should go with a shorter (easier to remember, easier to type) path
> and move things at least one level up. Two would be even better.
>
>>> Let's put the tree in /var/cache/portage please, and distfiles in
>>> /var/cache/distfiles. Layman overlays can stay where they are, or
>>> move to /var/cache/layman.
>
> Ulrich
>

Yeah +1 to that. Makes more sense to me

-- 
Regards,
Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2

Re: [gentoo-dev] Moving our/portage stuff to var

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 20/12/12 01:18 PM, George Shapovalov wrote:
> On Thursday 20 December 2012 13:36:27 Alan McKinnon wrote:
>>> What program uses this "local" directory? It's not used
>>> directly by portage itself, though portage has an exclude for
>>> it in the default PORTAGE_RSYNC_OPTS setting (in
>>> /usr/share/portage/config/make.globals).
>> 
>> It goes back a long time, and is basically a poor man's local
>> overlay without having to use layman. As I understand it, portage
>> will treat the directory like any other when looking for ebuilds
>> and resolving deps, but exclude it from a sync.
> Nope, he means /usr/portage/local, not /usr/local/portage.

Alan's description *was* for /usr/portage/local

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlDTVxYACgkQ2ugaI38ACPAEnQD/fo3/VbYD32yZMUuaEB0zZHES
71qGChzegSgxLqV01FoA/i583ha2AX+xLdw9/tyC7HUkzQ+9jXbqWKoT4Jay6bHc
=dt7h
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Moving our/portage stuff to var

[Michael Mol]

On Thu, Dec 20, 2012 at 12:31 PM, William Hubbs <williamh@gentoo.org> wrote:
> On Thu, Dec 20, 2012 at 06:58:11AM -0500, Rich Freeman wrote:
>> On Thu, Dec 20, 2012 at 4:25 AM, George Shapovalov <george@gentoo.org> wrote:
>> > On Thursday 20 December 2012 09:11:39 Ulrich Mueller wrote:
>> >> > /var/cache/repositories/local    <== the new location for a local overlay
>> >>
>> >> Also I wonder if local overlays should be in /var/cache? It might not
>> >> always be possible to restore them.
>> > So, this is the present /usr/local/portage? What's wrong with it?
>> > According to FHS, /usr/local/ is reserved for local admin meddling, which fits
>> > the bill rather nicely fo local overlay. This is the one location that was
>> > actually making sense from the beginning.
>>
>> I actually like the /var/cache/repositories approach.  You can always
>> add a symlink to it if you want to for convenience (as I already do
>> for /var/lib/portage/world).  There is really nothing special about
>> /usr/portage, other than it being the lowest-priority overlay.
>
>  If we do this, I don't like the name repositories -- what kind of
>  repositories? should I put git repositories in there?

I was pondering this, too. How about 'pms', for trees and overlays?

--
:wq

Re: [gentoo-dev] Moving our/portage stuff to var

[Dale]

Ciaran McCreesh wrote:
> On Thu, 20 Dec 2012 18:44:06 +0100
> Ulrich Mueller <ulm@gentoo.org> wrote:
>> There's no good reason for nesting it so deeply. As it is proposed
>> above, /var/cache/portage would contain only two subdirectories, and
>> /var/cache/portage/repositories only a single "gentoo" on a stable
>> system. Also /var/cache itself isn't overpopulated; I count about ten
>> entries on my systems.
> You really think most people don't have to use overlays?
>
>> We should go with a shorter (easier to remember, easier to type) path
>> and move things at least one level up. Two would be even better.
> You shouldn't ever be typing that path in...
>


I was thinking tab completion myself.  Just hit a couple letters and hit
tab.  That tab key types much faster and more accurately than I can.  ;-) 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Re: [gentoo-dev] Moving our/portage stuff to var

[Ulrich Mueller]

>>>>> On Thu, 20 Dec 2012, Ciaran McCreesh wrote:

>> We should go with a shorter (easier to remember, easier to type) path
>> and move things at least one level up. Two would be even better.

> You shouldn't ever be typing that path in...

Ebuilds tell users to do so:

pkg_nofetch() {
    einfo "Please download ${foo} and place it in ${DISTDIR}"
}

Re: [gentoo-dev] Moving our/portage stuff to var

[George Shapovalov]

On Thursday 20 December 2012 13:21:11 Ian Stakenvicius wrote:
> > Nope, he means /usr/portage/local, not /usr/local/portage.
> 
> Alan's description *was* for /usr/portage/local
Really? It matches /usr/local/portage pretty well. How did it come around 
then? We had /usr/local/portage for ages for storing local changes..

Re: [gentoo-dev] Moving our/portage stuff to var

[Zac Medico]

On 12/20/2012 06:12 AM, Graham Murray wrote:
> Zac Medico <zmedico@gentoo.org> writes:
> 
>> On 12/19/2012 02:01 PM, Alan McKinnon wrote:
>>> If we are going to move distfiles out of the tree into, what are the
>>> odds of getting /some/path/portage/local to move somewhere else too?
>>
>> What program uses this "local" directory? It's not used directly by
>> portage itself, though portage has an exclude for it in the default
>> PORTAGE_RSYNC_OPTS setting (in /usr/share/portage/config/make.globals).
> 
> It is useful for 'site' local ebuilds. It allows a 'master' repository
> to sync with the main Gentoo one without disturbing local changes but
> allow other systems on-site to fetch the modified tree with a simple
> 'emerge --sync'.

This usage is slightly annoying, because it tends to give people the
impression that it's safe to store random things inside $PORTDIR, while
it's somewhat fragile given that it relies on special rsync options.
Occasionally, we get bug reports from people who have lost files because
of this sort of confusion:

  https://bugs.gentoo.org/show_bug.cgi?id=131030
  https://bugs.gentoo.org/show_bug.cgi?id=392565
-- 
Thanks,
Zac

Re: [gentoo-dev] Moving our/portage stuff to var

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 20/12/12 01:55 PM, George Shapovalov wrote:
> On Thursday 20 December 2012 13:21:11 Ian Stakenvicius wrote:
>>> Nope, he means /usr/portage/local, not /usr/local/portage.
>> 
>> Alan's description *was* for /usr/portage/local

> Really? It matches /usr/local/portage pretty well. How did it come
> around then? We had /usr/local/portage for ages for storing local
> changes..
> 

/usr/local/portage has always been a convention or recommendation;
it's not a directory that portage (package or tree) ever created,
enforced, or did anything in particular to support.

/usr/portage/local/ came around (i think -- i was around at this time
but was not a dev and was not privy to decision making) so that
locally modified ebuilds could be stored and distributed (ie via
netmount or manual rsync) along with the rest of the portage tree
without worries of the changes being wiped out on the next --sync.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlDTZhEACgkQ2ugaI38ACPBEXwD+IuFOgsHcQDNaqUCUfSZW53ca
7gsST6Prls/7rPmpGqcBAKnnUIH48UPcDYrwexlNbmPzRN9CjYaeR36/2qo/hC47
=r5C9
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Moving our/portage stuff to var

[Zac Medico]

On 12/20/2012 03:36 AM, Alan McKinnon wrote:
> On Wed, 19 Dec 2012 14:19:52 -0800
> Zac Medico <zmedico@gentoo.org> wrote:
> 
>> On 12/19/2012 02:01 PM, Alan McKinnon wrote:
>>> On Wed, 19 Dec 2012 14:56:44 +0100
>>> Diego Elio Pettenò <flameeyes@flameeyes.eu> wrote:
>>>
>>>> Just mv /usr/portage /var/portage ? FFS no. Among other things, as
>>>> many said before, we should really take distfiles out of the tree
>>>> itself, and packages the same. And I don't want /var/packages
>>>> or /var/distfiles at all.
>>>
>>> If we are going to move distfiles out of the tree into, what are the
>>> odds of getting /some/path/portage/local to move somewhere else too?
>>
>> What program uses this "local" directory? It's not used directly by
>> portage itself, though portage has an exclude for it in the default
>> PORTAGE_RSYNC_OPTS setting
>> (in /usr/share/portage/config/make.globals).
> 
> It goes back a long time, and is basically a poor man's local overlay
> without having to use layman. As I understand it, portage will treat
> the directory like any other when looking for ebuilds and resolving
> deps, but exclude it from a sync.

Portage doesn't have any special handling for this directory, aside from
the exclude in the default PORTAGE_RSYNC_OPTS setting. I would not
encourage people to use this directory for anything, because it tends to
give people the impression that it's safe to store random things inside
$PORTDIR, while it's somewhat fragile given that it relies on special
rsync options. Occasionally, we get bug reports from people who have
lost files because of this sort of confusion:

  https://bugs.gentoo.org/show_bug.cgi?id=131030
  https://bugs.gentoo.org/show_bug.cgi?id=392565

>>
>>> That one has irked me for ages, its the one thing left on my systems
>>> that stops the local tree dir being an exact replica of the upstream
>>> master.
>>
>> For portage's defaults, I won't settle for anything less than having
>> them all refer to separate directories which are *not* nested within
>> one other. These are the current default settings which violate my
>> requirements:
>>
>> 	PORTDIR=/usr/portage
>> 	DISTDIR=${PORTDIR}/distfiles
>> 	PKGDIR=${PORTDIR}/packages
>> 	RPMDIR=${PORTDIR}/rpm
> 
> /usr/portage/local has the taste feel and smell of a hacky workaround:
> shove a directory in the tree and exclude it from sync.

Right.

> I suspect the best solution all round is to move all support for local
> overlays into layman. I'd be happy with that. Probably make the portage
> code cleaner too.

As mentioned, portage doesn't have any special handling for this
directory (aside from the rsync exclude).
-- 
Thanks,
Zac

Re: [gentoo-dev] Moving our/portage stuff to var

[Alan McKinnon]

On Thu, 20 Dec 2012 19:18:56 +0100
George Shapovalov <george@gentoo.org> wrote:

> > It goes back a long time, and is basically a poor man's local
> > overlay without having to use layman. As I understand it, portage
> > will treat the directory like any other when looking for ebuilds
> > and resolving deps, but exclude it from a sync.  
> Nope, he means /usr/portage/local, not /usr/local/portage. It is
> rarely (if ever, nowadays) present, as I understand, but you can find
> it mentioned in that config file. It may be just a legacy definition
> by now, looking at how only layman was mentioned with relation to it
> and even that one appears not to use it any more.

I thought about this some more, and now realise I've been using
essentially the same set of config files since about 2004. I've never
lost them and always had a current copy so each time I build a new host
I just copy, tweak CFLAGS, maybe MAKEOPTS, and let 'er rip.

My "local" is probably years out of date. Serves me right for not
reading 50 screens of man page with every new host :-)

This sub-thread is probably just noise, sorry for that.

-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-dev] Moving our/portage stuff to var

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 708 bytes --]

On Thu, 20 Dec 2012 19:44:36 +0100
Ulrich Mueller <ulm@gentoo.org> wrote:
> >>>>> On Thu, 20 Dec 2012, Ciaran McCreesh wrote:
> >> We should go with a shorter (easier to remember, easier to type)
> >> path and move things at least one level up. Two would be even
> >> better.
> 
> > You shouldn't ever be typing that path in...
> 
> Ebuilds tell users to do so:
> 
> pkg_nofetch() {
>     einfo "Please download ${foo} and place it in ${DISTDIR}"
> }
> 

I believe Unix has a facility for taking bits of text that are on the
screen and copying them without the need to type the whole thing in.
But in any case, DISTDIR shouldn't be under the repository dir at all.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]