[gentoo-dev] Is /var/cache the right place for repositories?

[gentoo-dev] Is /var/cache the right place for repositories?

[Ulrich Mueller]

The FHS says:

   /var/cache is intended for cached data from applications. Such data
   is locally generated as a result of time-consuming I/O or
   calculation. The application must be able to regenerate or restore
   the data.

Now I wonder: After removal of e.g. the Portage tree from a system, it
is generally not possible to restore it. (It can be refetched, but not
to its previous state.)

Same is true for distfiles, at least to some degree. They may have
vanished upstream or from mirrors.

Maybe /var/lib would be a better choice? It would also take care of
the issue with fetch-restricted files.

Ulrich

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 20/12/12 12:27 PM, Ulrich Mueller wrote:
> The FHS says:
> 
> /var/cache is intended for cached data from applications. Such
> data is locally generated as a result of time-consuming I/O or 
> calculation. The application must be able to regenerate or restore 
> the data.
> 
> Now I wonder: After removal of e.g. the Portage tree from a system,
> it is generally not possible to restore it. (It can be refetched,
> but not to its previous state.)
> 
> Same is true for distfiles, at least to some degree. They may have 
> vanished upstream or from mirrors.
> 
> Maybe /var/lib would be a better choice? It would also take care
> of the issue with fetch-restricted files.
> 

I had asked more or less the same thing a few days ago.  The cases
where this would matter are few, however, and those users that need
the state preserved could ensure it by including these specific paths
in their backups and/or ensuring any cache-cleaner scripts (and AFAIK
there aren't any that wouldn't be custom-installed) do not remove them.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlDTUkgACgkQ2ugaI38ACPDgAwEAhFH3Jc8E56WfePr3W1396+Jk
65q7X8eEwNAYr8eJLwQA/1Xi7E42004M3frMDCDDBVZeD1EYmKkvXA8POhQUZc36
=JMod
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Ulrich Mueller]

>>>>> On Thu, 20 Dec 2012, Ian Stakenvicius wrote:

> On 20/12/12 12:27 PM, Ulrich Mueller wrote:
>> The FHS says:
>> 
>> /var/cache is intended for cached data from applications. Such
>> data is locally generated as a result of time-consuming I/O or 
>> calculation. The application must be able to regenerate or restore 
>> the data.
>> 
>> Now I wonder: After removal of e.g. the Portage tree from a system,
>> it is generally not possible to restore it. (It can be refetched,
>> but not to its previous state.)
>> 
>> Same is true for distfiles, at least to some degree. They may have 
>> vanished upstream or from mirrors.
>> 
>> Maybe /var/lib would be a better choice? It would also take care
>> of the issue with fetch-restricted files.

> I had asked more or less the same thing a few days ago.  The cases
> where this would matter are few, however, and those users that need
> the state preserved could ensure it by including these specific paths
> in their backups and/or ensuring any cache-cleaner scripts (and AFAIK
> there aren't any that wouldn't be custom-installed) do not remove them.

What about /usr/portage/licenses, for example? Some of the licenses
are required to be present on the system if the corresponding software
is installed. So users cannot legally remove them.

Should we really put them under /var/cache which suggests that
everything in there can be wiped?

Ulrich

[gentoo-dev] Keeping licenses around

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 20/12/12 01:12 PM, Ulrich Mueller wrote:
> 
> What about /usr/portage/licenses, for example? Some of the
> licenses are required to be present on the system if the
> corresponding software is installed. So users cannot legally remove
> them.
> 

...  well, along those lines, the list of licenses are not pertinent
to the system unless the software that relates to them is installed;
maybe emerge should automatically during the merge phase ensure the
license files are copied to the main system in say /var/lib/licenses
or similar?

The system can exist with /usr/portage not installed at any given
time, now (I have systems that NFS-mount it, but I only bother to do
so when i'm going to emerge something).  If licenses not existing is
going to be a problem then we should resolve this no matter what.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlDTVrYACgkQ2ugaI38ACPAHbwEAhXRKaWP4zFH6fRmWMcXJjUJk
diGd9Dmwqe7EeRnSNZ0A/1En77FU6mI11FpVQZ5IMp16uSBPSbPoIePjbbz2PI0E
=R/VK
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Keeping licenses around

[Diego Elio Pettenò]

[-- Attachment #1: Type: text/plain, Size: 558 bytes --]

On 20/12/2012 19:19, Ian Stakenvicius wrote:
> ...  well, along those lines, the list of licenses are not pertinent
> to the system unless the software that relates to them is installed;
> maybe emerge should automatically during the merge phase ensure the
> license files are copied to the main system in say /var/lib/licenses
> or similar?

Why not in their own /var/db/pkg then? And +1 on this, it's something I
was wondering myself some time ago.

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 553 bytes --]

Re: [gentoo-dev] Keeping licenses around

[Rich Freeman]

On Thu, Dec 20, 2012 at 1:19 PM, Ian Stakenvicius <axs@gentoo.org> wrote:
> On 20/12/12 01:12 PM, Ulrich Mueller wrote:
>>
>> What about /usr/portage/licenses, for example? Some of the
>> licenses are required to be present on the system if the
>> corresponding software is installed. So users cannot legally remove
>> them.

Perhaps a better way to phrase that is that some of the licenses claim
that they are required to be present on the system if the
corresponding software is installed.

Licenses can't make you do things - only laws can make you do things.
Licenses just give you the right to "break" a law.

I think that this is legally very dubious.  If people want to save
copies of license files they can of course do so, but I'm not sure it
is really worth a lot of effort to automate it.  But, if somebody
wants to stick it in /var/db/pkg or whatever I guess it is just a few
thousand inodes...

Rich

Re: [gentoo-dev] Keeping licenses around

[Rick "Zero_Chaos" Farina]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/20/2012 03:25 PM, Rich Freeman wrote:
> On Thu, Dec 20, 2012 at 1:19 PM, Ian Stakenvicius <axs@gentoo.org> wrote:
>> On 20/12/12 01:12 PM, Ulrich Mueller wrote:
>>>
>>> What about /usr/portage/licenses, for example? Some of the
>>> licenses are required to be present on the system if the
>>> corresponding software is installed. So users cannot legally remove
>>> them.
> 
> Perhaps a better way to phrase that is that some of the licenses claim
> that they are required to be present on the system if the
> corresponding software is installed.
> 
> Licenses can't make you do things - only laws can make you do things.
> Licenses just give you the right to "break" a law.
> 
> I think that this is legally very dubious.  If people want to save
> copies of license files they can of course do so, but I'm not sure it
> is really worth a lot of effort to automate it.  But, if somebody
> wants to stick it in /var/db/pkg or whatever I guess it is just a few
> thousand inodes...
I'm pretty happy with only one copy of every license on my system...

- -ZC
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iQIcBAEBAgAGBQJQ03niAAoJEKXdFCfdEflKR3YP/30k+BOSTshaEmEJLgFhPxlI
VcRpEXoRWr1UDDbTr8xNqYcPenPvslus5TpaBLMr4DQRYM2mKcMCcOSGGMr5kWzp
wvEvfj+1EAUUu6xL2BgeMbLbRGyfBv+GwmPI1L+wvWgZhfo3tdGSimGZwGxA1mMI
dx3Oi15d4TZsQGO4wvrgaCcbNaMDFEF6D65CewmJiPCmLljAZDMN3lxejr8bbGgp
SjJOrlxAZckZnE6aXKtmZHeXu5YzjT2iVEat0ADjad0eR69/AaRfbLyd75Rp7rkL
W7zKHnZv2d3P4UGi1vDM6EXPlJHil6F1ruUU6nvRHnyn89ndIF1xwIIKpuellWWD
+11FXIJIWpF5+V2jWLPdZtxtp21Mr7Sphsr8s162ryHh7qWHzkCaaia8laX+EbwP
k4IEfjP3wSWo71rKOxJsAMoJb3FPum4cM/GLh8eedIZMabODUL43B5uk4jkigZki
/g1zwLCBXzbOjjGn1ngVCZKLWLKWh3qTo7VDBEa320L8UEKqxASehywd8TeUzWIb
o7roPyau9LsHrHwuLi3stVsBsx8P+zN8s0+AFJSjKNbfWPY4+lcZjgFd8oceEAGk
rWnBt2RJkjy6ODn1jVSAQpRrUqyiKonpk8FHDrk3g4e2NgZf4duIy2BOoLe5SiUi
ao+vj/d1mfZVvhe8U/9F
=Gc3+
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Keeping licenses around

[Zac Medico]

On 12/20/2012 10:19 AM, Ian Stakenvicius wrote:
> On 20/12/12 01:12 PM, Ulrich Mueller wrote:
> 
>> What about /usr/portage/licenses, for example? Some of the
>> licenses are required to be present on the system if the
>> corresponding software is installed. So users cannot legally remove
>> them.
> 
> 
> ...  well, along those lines, the list of licenses are not pertinent
> to the system unless the software that relates to them is installed;
> maybe emerge should automatically during the merge phase ensure the
> license files are copied to the main system in say /var/lib/licenses
> or similar?

If only a small subset of licenses require it, then maybe we should just
use dodoc on those licenses that require it. Saving all licenses could
be overkill.
-- 
Thanks,
Zac

Re: [gentoo-dev] Keeping licenses around

[Rich Freeman]

On Thu, Dec 20, 2012 at 3:46 PM, Zac Medico <zmedico@gentoo.org> wrote:
>
> If only a small subset of licenses require it, then maybe we should just
> use dodoc on those licenses that require it. Saving all licenses could
> be overkill.

Seems like a reasonable compromise.  I would think such licenses would
be rare.  I still think it is extralegal to try to require this, but
some would rather not take risks and I can appreciate that...

Rich

Re: [gentoo-dev] Keeping licenses around

[Ulrich Mueller]

>>>>> On Thu, 20 Dec 2012, Rich Freeman wrote:

>> If only a small subset of licenses require it, then maybe we should just
>> use dodoc on those licenses that require it. Saving all licenses could
>> be overkill.

> Seems like a reasonable compromise.  I would think such licenses would
> be rare.  I still think it is extralegal to try to require this, but
> some would rather not take risks and I can appreciate that...

The current solution is to install such licenses when USE="bindist"
is set. When installing from source, one can normally assume that
${PORTDIR}/licenses exists.

Ulrich

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Chí-Thanh Christopher Nguyễn]

Ulrich Mueller schrieb:
>> Now I wonder: After removal of e.g. the Portage tree from a system,
>> it is generally not possible to restore it. (It can be refetched,
>> but not to its previous state.)

Is it required that the _exact_ _same_ _data_ will be regenerated? This
is not the case with most users of /var/cache (like ccache for example).
They only regenerate what is needed so the application continues to work
properly. The ebuilds that are needed for portage functioning are saved
to /var/db/pkg already.

squid cache would be another example, or just about every other Linux
distro's package manager.

> What about /usr/portage/licenses, for example? Some of the licenses
> are required to be present on the system if the corresponding software
> is installed. So users cannot legally remove them.

They are not required for functioning of the system, and a sync will
restore them.

> Should we really put them under /var/cache which suggests that
> everything in there can be wiped?

Yes.


Best regards,
Chí-Thanh Christopher Nguyễn

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Alexandre Rostovtsev]

On Thu, 2012-12-20 at 18:27 +0100, Ulrich Mueller wrote:
> The FHS says:
> 
>    /var/cache is intended for cached data from applications. Such data
>    is locally generated as a result of time-consuming I/O or
>    calculation. The application must be able to regenerate or restore
>    the data.
> 
> Now I wonder: After removal of e.g. the Portage tree from a system, it
> is generally not possible to restore it. (It can be refetched, but not
> to its previous state.)
> 
> Same is true for distfiles, at least to some degree. They may have
> vanished upstream or from mirrors.
> 
> Maybe /var/lib would be a better choice? It would also take care of
> the issue with fetch-restricted files.

Due to fetch-restricted files, /var/lib does make sense for distfiles.
And of course /var/lib should be used for the default personal overlay
(currently in /usr/local/portage).

But I think that the main portage and overlay checkouts are already
cache-like in the sense that any manual user changes are automatically
overwritten by "emerge --sync" / "layman -S", which the users are
supposed to run on a sufficiently regular basis. So /var/cache does seem
like a reasonable place for them.

[gentoo-dev] Re: Is /var/cache the right place for repositories?

[Duncan]

Alexandre Rostovtsev posted on Thu, 20 Dec 2012 13:10:38 -0500 as
excerpted:

> On Thu, 2012-12-20 at 18:27 +0100, Ulrich Mueller wrote:
>> The FHS says:
>> 
>>    /var/cache is intended for cached data from applications. Such data
>>    is locally generated as a result of time-consuming I/O or
>>    calculation. The application must be able to regenerate or restore
>>    the data.
>> 
>> Now I wonder: After removal of e.g. the Portage tree from a system, it
>> is generally not possible to restore it. (It can be refetched, but not
>> to its previous state.)
>> 
>> Same is true for distfiles

> But I think that the main portage and overlay checkouts are already
> cache-like in the sense that any manual user changes are automatically
> overwritten by "emerge --sync" / "layman -S", which the users are
> supposed to run on a sufficiently regular basis. So /var/cache does seem
> like a reasonable place for them.

I'd been wondering about the point others have made about "locally 
generated", vs "Internet downloaded".

However, upon rereading the above FHS quote, it hit me -- "from 
applications... locally generated... as a result of time-consuming I/O" 
is actually pretty explicit.  I believe the emphasis has been on "locally 
generated", and the point that it explicitly includes "as a result of 
time-consuming I/O" in the definition of "locally generated" has been 
missed entirely.  I know I missed it.  But, if internet downloads 
triggered by running a local app don't qualify as "generated as a result 
of time-consuming I/O", what other I/O-basis generated files DO qualify 
as cache?  That seems to pretty explicitly include Internet downloads in 
the definition, to me!

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

[gentoo-dev] Re: Is /var/cache the right place for repositories?

[Ulrich Mueller]

>>>>> On Fri, 21 Dec 2012, Duncan  wrote:

>>> The FHS says:
>>> 
>>>    /var/cache is intended for cached data from applications. Such
>>>    data is locally generated as a result of time-consuming I/O or
>>>    calculation. The application must be able to regenerate or
>>>    restore the data.

> I'd been wondering about the point others have made about "locally
> generated", vs "Internet downloaded".

> However, upon rereading the above FHS quote, it hit me -- "from
> applications... locally generated... as a result of time-consuming
> I/O" is actually pretty explicit. I believe the emphasis has been on
> "locally generated", and the point that it explicitly includes "as a
> result of time-consuming I/O" in the definition of "locally
> generated" has been missed entirely. I know I missed it. But, if
> internet downloads triggered by running a local app don't qualify as
> "generated as a result of time-consuming I/O", what other I/O-basis
> generated files DO qualify as cache? That seems to pretty explicitly
> include Internet downloads in the definition, to me!

While this might be true for distfiles, Portage doesn't use the local
copy of the tree as a cache. A normal emerge command doesn't trigger a
download of the tree. And if the tree was removed, emerge doesn't
work.

Ulrich

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 807 bytes --]

On Thu, 20 Dec 2012 18:27:26 +0100
Ulrich Mueller <ulm@gentoo.org> wrote:
> The FHS says:
> 
>    /var/cache is intended for cached data from applications. Such data
>    is locally generated as a result of time-consuming I/O or
>    calculation. The application must be able to regenerate or restore
>    the data.
> 
> Now I wonder: After removal of e.g. the Portage tree from a system, it
> is generally not possible to restore it. (It can be refetched, but not
> to its previous state.)
> 
> Same is true for distfiles, at least to some degree. They may have
> vanished upstream or from mirrors.
> 
> Maybe /var/lib would be a better choice? It would also take care of
> the issue with fetch-restricted files.

The tree is a database. It belongs in /var/db/.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Rick "Zero_Chaos" Farina]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/20/2012 01:14 PM, Ciaran McCreesh wrote:
> On Thu, 20 Dec 2012 18:27:26 +0100
> Ulrich Mueller <ulm@gentoo.org> wrote:
>> The FHS says:
>>
>>    /var/cache is intended for cached data from applications. Such data
>>    is locally generated as a result of time-consuming I/O or
>>    calculation. The application must be able to regenerate or restore
>>    the data.
>>
>> Now I wonder: After removal of e.g. the Portage tree from a system, it
>> is generally not possible to restore it. (It can be refetched, but not
>> to its previous state.)
>>
>> Same is true for distfiles, at least to some degree. They may have
>> vanished upstream or from mirrors.
>>
>> Maybe /var/lib would be a better choice? It would also take care of
>> the issue with fetch-restricted files.
> 
> The tree is a database. It belongs in /var/db/.
> 
+1
and another +1 from my other personality, so +2 from us

- -ZC
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iQIcBAEBAgAGBQJQ03oTAAoJEKXdFCfdEflKKLYQAIJRRlt3xnh30rxbVi3T7Iuy
tcx853ngIoyhc8fuhygDdsWPaIwa2WqOBuM6SuxiRlj3BDuL3sgSpMX3bWaL2ukK
VXRJKZ2zVBOdc/KfbBvkWjN3P9nlOMMV8HGtGgwlniLF6BeeKlI2SqI8tWRX5bBN
eUDbb1I0gSV1Tr1F3s9j5GSjd+kq17DloeXD4W0/uOEnN9Q+dvh8zMRA208ZY7qj
B0f7TMgkh1ryy1OPSeXDlwgIjJbzgHWoDXN7RPiN/WF/E4msea/oq3tgz6TdTtdN
sg+9VqqVXmJwHeMSpiQbR13JJ/I1pt+y9j8KgkUU5SGedyNYrzvaaSsy+ENdmVU2
JyEX5U+dFIgEo3uZX55fStGJyMWDoV5PmDkSUzDNyC2t1kgROYRh8JokGukqY3tL
VKeFKwqysRmJfeLuFYbgsXezrQQrGD1PWsPt9yLoS5hFER89+A95d8f00tTv8mCe
KAihOScCwtxl1PHfkOwB4BZJwdilwXBoKaAgT91Ul43GGabxYOMpCKfq2w9dzDT/
oMWPUrwIgJT/kdmBm2+TaLfwEfVHUCOhPaD5vqBeYJ2MJhZWr23VRlR5YUEf7E9W
S+yBSiLkr2ZCVv3sZ80tRezN6FVZRKSJjIeZ11u7AWXN3hxtbvidJV+yL416/nJr
4rD8jfLJHHOvVaiX36JH
=MnSP
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Luca Barbato]

On 12/20/2012 07:14 PM, Ciaran McCreesh wrote:
> The tree is a database. It belongs in /var/db/.
> 

That's a good point.

lu

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Sebastian Pipping]

On 20.12.2012 19:14, Ciaran McCreesh wrote:
> The tree is a database. It belongs in /var/db/.

I don't see /var/db in the latest release of the Filesystem Hierarchy
Standard:

  http://www.pathname.com/fhs/pub/fhs-2.3.html#THEVARHIERARCHY

I would prefer something that blends with FHS.

Best,



Sebastian

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Ulrich Mueller]

>>>>> On Mon, 24 Dec 2012, Sebastian Pipping wrote:

> On 20.12.2012 19:14, Ciaran McCreesh wrote:
>> The tree is a database. It belongs in /var/db/.

> I don't see /var/db in the latest release of the Filesystem
> Hierarchy Standard:

>   http://www.pathname.com/fhs/pub/fhs-2.3.html#THEVARHIERARCHY

Wrong standard to choose from. ;-) /var/db/ is already required by the
PMS for /var/db/pkg/.

> I would prefer something that blends with FHS.

Is this important for a Gentoo specific directory?

   /var/db/portage/         PORTDIR
   /var/db/layman/          layman storage
   /var/db/pkg/             VDB (no change)
   /usr/local/portage/      local overlays (no change)
   /var/cache/distfiles/    DISTDIR
   /var/cache/packages/     PKGDIR

Alternatively, the last two could be under
/var/cache/portage/{distfiles,packages}/.

Ulrich

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Michael Mol]

On Mon, Dec 24, 2012 at 4:08 AM, Ulrich Mueller <ulm@gentoo.org> wrote:
>>>>>> On Mon, 24 Dec 2012, Sebastian Pipping wrote:
>
>> On 20.12.2012 19:14, Ciaran McCreesh wrote:
>>> The tree is a database. It belongs in /var/db/.
>
>> I don't see /var/db in the latest release of the Filesystem
>> Hierarchy Standard:
>
>>   http://www.pathname.com/fhs/pub/fhs-2.3.html#THEVARHIERARCHY
>
> Wrong standard to choose from. ;-) /var/db/ is already required by the
> PMS for /var/db/pkg/.
>
>> I would prefer something that blends with FHS.
>
> Is this important for a Gentoo specific directory?
>
>    /var/db/portage/         PORTDIR
>    /var/db/layman/          layman storage
>    /var/db/pkg/             VDB (no change)
>    /usr/local/portage/      local overlays (no change)
>    /var/cache/distfiles/    DISTDIR
>    /var/cache/packages/     PKGDIR
>
> Alternatively, the last two could be under
> /var/cache/portage/{distfiles,packages}/.

Query that's been percolating in my mind...how much of this is
specific to Gentoo, and how much has strong overlap with closely
related distros like Sabayon and Funtoo?

--
:wq

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Diego Elio Pettenò]

On 24/12/2012 10:08, Ulrich Mueller wrote:
>    /var/cache/packages/     PKGDIR

Maybe /var/spool/binpkgs ?

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Ulrich Mueller]

>>>>> On Mon, 24 Dec 2012, Diego Elio Pettenò wrote:

>> /var/cache/packages/ PKGDIR

> Maybe /var/spool/binpkgs ?

This doesn't look right to me. /var/spool contains things like printer
queues or outgoing mail that are typically deleted after processing.

Ulrich

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Diego Elio Pettenò]

On 24/12/2012 14:32, Ulrich Mueller wrote:
> This doesn't look right to me. /var/spool contains things like printer
> queues or outgoing mail that are typically deleted after processing.

Not sure how /var/cache fits for binpkgs though, tbh.

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Ulrich Mueller]

>>>>> On Mon, 24 Dec 2012, Diego Elio Pettenò wrote:

> On 24/12/2012 14:32, Ulrich Mueller wrote:
>> This doesn't look right to me. /var/spool contains things like
>> printer queues or outgoing mail that are typically deleted after
>> processing.

> Not sure how /var/cache fits for binpkgs though, tbh.

Why not? Because they are distributed to other systems?

/var/lib then? (Though FHS acolytes would probably put them in /srv ...)

Ulrich

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Diego Elio Pettenò]

On 24/12/2012 16:43, Ulrich Mueller wrote:
> Why not? Because they are distributed to other systems?

More because they can be used as a backup themselves, if I want to keep
older versions available.

> /var/lib then? 

Fine by me.

> (Though FHS acolytes would probably put them in /srv ...)

Let's not get on with /srv right now please.

-- 
Diego Elio Pettenò — Flameeyes
flameeyes@flameeyes.eu — http://blog.flameeyes.eu/

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Ulrich Mueller]

>>>>> On Mon, 24 Dec 2012, Diego Elio Pettenò wrote:

> On 24/12/2012 16:43, Ulrich Mueller wrote:
>> Why not? Because they are distributed to other systems?

> More because they can be used as a backup themselves, if I want to
> keep older versions available.

This is a valid argument, of course.

>> /var/lib then?

> Fine by me.

>> (Though FHS acolytes would probably put them in /srv ...)

Insert a smiley of your choice here. ;-)

> Let's not get on with /srv right now please.

Ulrich

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Rick "Zero_Chaos" Farina]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/24/2012 09:00 AM, Diego Elio Pettenò wrote:
> On 24/12/2012 14:32, Ulrich Mueller wrote:
>> This doesn't look right to me. /var/spool contains things like printer
>> queues or outgoing mail that are typically deleted after processing.
> 
> Not sure how /var/cache fits for binpkgs though, tbh.
> 
"Application cache data. Such data are locally generated as a result of
time-consuming I/O or calculation. The application must be able to
regenerate or restore the data. The cached files can be deleted without
loss of data."

No sure how it doesn't...

Binpackages are really essentially cache created by portage through
time-consuming I/O and calculation (compiling) and can easily be
regenerated locally.  Plus, you can delete all of this and the system is
still functional.

- -ZC
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iQIcBAEBAgAGBQJQ2H5HAAoJEKXdFCfdEflKoPEP/14QYZIF/mbquRFkiCnp5KCN
s11qw4He6yEsgnjvMKA1CCWZ0R85G/wfVnj0DpcK83zXrP9Znbrk4Yatue/7KXaZ
I/sDg5Woo1FT6Mb9EY7hgpawIS25+6xA3eRsPKIlPzBG+i43ZTM4JhDLJeTs1VSH
APhkH0EXiA0H8ngCTgP9ReDXoi8KqPbMYGe/t3NVL4KalPdkDsjHeqfUG95C660f
TM34UvOGBA4HpySmH+FRdsUxV+9tJtdOZFjSm/oQX5IZrzLQA5lOSHe6t8sQJnsk
/b6TYncolfVpUED6y/8072S4GL+mEucf8NFIyMClpDymfILS7zFR0hEawm+UrLjm
O2/0ivPHQQA/P4uwTDQzJ1KqHZAgN0lDgbSZYZ5290whypSyJoGIKfVIvSI/qjFR
JOy5pCMkY9oClOqZB6s32WowKCzPipT7MPvBgotPuBoHaaMJOeW53FJadi/VEyGc
qL6Uv6jn0WKJJpGrONm7LwXnYB8kVzOmqVLpGEIO1mqEX9QL71qsq/Fw1pAyqqB5
NSq1dDbKye9C7nH1xSmhzgGFTs3V+IHKAV2iwjeElhZJF/Iv2+nj/6gONpNI7279
x1Zbi7i3JM1z4EMSaV+Nt60endPeB4KnDFoPXlRLZTlyR2qcLVNVr+qAIWG3m+mM
QqQCREx2n/KV/hFUUh5U
=7lrq
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Michael Hampicke]

Am 24.12.2012 17:09, schrieb Rick "Zero_Chaos" Farina:
> On 12/24/2012 09:00 AM, Diego Elio Pettenò wrote:
>> On 24/12/2012 14:32, Ulrich Mueller wrote:
>>> This doesn't look right to me. /var/spool contains things like printer
>>> queues or outgoing mail that are typically deleted after processing.
> 
>> Not sure how /var/cache fits for binpkgs though, tbh.
> 
> "Application cache data. Such data are locally generated as a result of
> time-consuming I/O or calculation. The application must be able to
> regenerate or restore the data. The cached files can be deleted without
> loss of data."
> 
> No sure how it doesn't...
> 
> Binpackages are really essentially cache created by portage through
> time-consuming I/O and calculation (compiling) and can easily be
> regenerated locally.  Plus, you can delete all of this and the system is
> still functional.

Not that I am opposed to keep binpackages in /var/cache - but people on
this thread have brought up lot's of reasons why for certain aspects not
to keep certain data in certain places.
This just hit my mind: can binpackages easily be regenerated locally if
their ebuilds are not in portage anymore?
I mean they can: grab the ebuild, compile it with the ebuild command,
there you go, but isn't that also true for the whole portage tree?

[gentoo-dev] Re: Is /var/cache the right place for repositories?

[Duncan]

Michael Hampicke posted on Tue, 25 Dec 2012 10:09:15 +0100 as excerpted:

> Am 24.12.2012 17:09, schrieb Rick "Zero_Chaos" Farina:
>> On 12/24/2012 09:00 AM, Diego Elio Pettenò wrote:

>>> Not sure how /var/cache fits for binpkgs though, tbh.
>> 
>> No sure how it doesn't...
>> 
>> Binpackages are really essentially cache created by portage through
>> time-consuming I/O and calculation (compiling) and can easily be
>> regenerated locally.  Plus, you can delete all of this and the system
>> is still functional.
> 
> Not that I am opposed to keep binpackages in /var/cache - but people on
> this thread have brought up lot's of reasons why for certain aspects not
> to keep certain data in certain places.

Also, consider what happens if gcc or the like breaks.  Normally those 
with FEATURES=binpkg can still revert to their last known working binpkg, 
and this has long been listed as one of the reasons people should 
consider enabling binpkgs.  But if it's gone due to "cache cleanup" and 
gcc is broken...

A system reinstall from binpkgs sure speeds things up if you fatfinger an 
rm --recursive or some such, as well.  Basically, you're installing a 
custom bindistro in that case, making PKGDIR more a binpkg repository 
than a simple cache of individual packages.  It is for this reason I keep 
my binpkgs on a dedicated partition, and back it up, something I do NOT 
do with the gentoo ebuild tree, the kernel tree, or ccache, which to me 
ARE caches, while my binpkg dir isn't.

But I set the vars myself so what the defaults are isn't a big deal, here.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Peter Stuge]

Michael Hampicke wrote:
> can binpackages easily be regenerated locally if their ebuilds are
> not in portage anymore?

If the package is still installed it is very easy with quickpkg.


//Peter

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Michael Mol]

On Mon, Dec 24, 2012 at 8:32 AM, Ulrich Mueller <ulm@gentoo.org> wrote:
>>>>>> On Mon, 24 Dec 2012, Diego Elio Pettenň wrote:
>
>>> /var/cache/packages/ PKGDIR
>
>> Maybe /var/spool/binpkgs ?
>
> This doesn't look right to me. /var/spool contains things like printer
> queues or outgoing mail that are typically deleted after processing.

Then treat it like garbage collection. Some maintenance action could
go through and remove the files which aren't fetch-restricted. Portage
could do this at the end of its cycle, or it could be set up as a cron
job, or it could require a manual maintenance step.


--
:wq

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 677 bytes --]

On Mon, 24 Dec 2012 10:08:13 +0100
Ulrich Mueller <ulm@gentoo.org> wrote:

> >>>>> On Mon, 24 Dec 2012, Sebastian Pipping wrote:
> 
> > On 20.12.2012 19:14, Ciaran McCreesh wrote:
> >> The tree is a database. It belongs in /var/db/.
> 
> > I don't see /var/db in the latest release of the Filesystem
> > Hierarchy Standard:
> 
> >   http://www.pathname.com/fhs/pub/fhs-2.3.html#THEVARHIERARCHY
> 
> Wrong standard to choose from. ;-) /var/db/ is already required by the
> PMS for /var/db/pkg/.

Incorrect. The PMS specifies vdb as being 'unspecified'. The fact that
it provides a path there doesn't seem really relevant.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Rick "Zero_Chaos" Farina]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/24/2012 04:08 AM, Ulrich Mueller wrote:
>>>>>> On Mon, 24 Dec 2012, Sebastian Pipping wrote:
> 
>> On 20.12.2012 19:14, Ciaran McCreesh wrote:
>>> The tree is a database. It belongs in /var/db/.
> 
>> I don't see /var/db in the latest release of the Filesystem
>> Hierarchy Standard:
> 
>>   http://www.pathname.com/fhs/pub/fhs-2.3.html#THEVARHIERARCHY
> 
> Wrong standard to choose from. ;-) /var/db/ is already required by the
> PMS for /var/db/pkg/.
> 
>> I would prefer something that blends with FHS.
> 
> Is this important for a Gentoo specific directory?
> 
>    /var/db/portage/         PORTDIR
>    /var/db/layman/          layman storage
>    /var/db/pkg/             VDB (no change)
>    /usr/local/portage/      local overlays (no change)
>    /var/cache/distfiles/    DISTDIR
>    /var/cache/packages/     PKGDIR
> 
> Alternatively, the last two could be under
> /var/cache/portage/{distfiles,packages}/.
> 
I am not 100% on this, but I think this is my first +1 for this thread.

+1

I really like this layout, it almost makes sense.
I won't be bike shedding on this topic (really don't care that much),
but I do really like this layout.

- -ZC
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iQIcBAEBAgAGBQJQ2H3/AAoJEKXdFCfdEflKJMwQALbugkzciQpw3KTr32Dwl2p2
TFMRClKv4W006SXjbxLciQg+hLDMPBOIjbCl5UtpWcFSsCWWUXGBJIX7A6m7TZ6N
lj4VGjEDBjCKzkp3XypoRXL1XIiuqQpxv3FAqpFbhLczXRP+oQoP3ZmdbDZF4Dky
oJf/Pttl4bD8CA6cWZ6tXDvnrZ2w4cJYm/AnuOaCahSM/3MqscWq884lnucbT6Xs
IBa6DhNV/iqAXTQ5v/54p6izl6EbV/UJEzFjSVOsPAgmCwVjsc1ZFkZi2BAlt8iv
f+8j0SGHRrUXk22nOIe1bwdg7CTpn0cjrYPTjG+sWcx+tEgNzF7xkLLWgeSj4+jL
kY7KXvfsmVyamAybySGJNWIjv8n97YkJTy8bT7caIoCB8h0oJvrC2eNRJFISuEjv
DpKvql1nNyJJ1/k2aUoBLiUjLpSIGeZ0607W4woTM0mrEo8RGvXGV87y8Y4jGML1
2ks87XcEb/jBPVxCodITwWyB9/aqzC4K0K5rLj5xqIDdeoxb2A8HVefbUEY2mcD/
cFXTl7hnX9KdNl4+VrNSVvNNVR+pZIZz8lT8wiu4wqVwm+CjaY+YPMuGy3ps2cmo
Pq3/HbSSQwhP6bEZfZ5md8dZ2p2LSW9xJzhbxmuFCUrLxDAbZTsjDKeJy0q3aHNG
Xi+Z+m8PqCDotRD63PWR
=lGRB
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 460 bytes --]

On Mon, 24 Dec 2012 03:17:06 +0100
Sebastian Pipping <sping@gentoo.org> wrote:
> On 20.12.2012 19:14, Ciaran McCreesh wrote:
> > The tree is a database. It belongs in /var/db/.
> 
> I don't see /var/db in the latest release of the Filesystem Hierarchy
> Standard:
> 
>   http://www.pathname.com/fhs/pub/fhs-2.3.html#THEVARHIERARCHY
> 
> I would prefer something that blends with FHS.

That's ok, Gentoo doesn't follow FHS.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[vivo75]

Il 24/12/2012 10:11, Ciaran McCreesh ha scritto:
> On Mon, 24 Dec 2012 03:17:06 +0100
> Sebastian Pipping <sping@gentoo.org> wrote:
>> On 20.12.2012 19:14, Ciaran McCreesh wrote:
>>> The tree is a database. It belongs in /var/db/.
yes and no,
yes it contain data and executable needed to update gentoo system, in a 
hierarchical and relational form
no, it's a cache of a remote database generally mantained from others.

Actually also the difference in importance between /var/db/pkg and 
/????/ebuild_tree is very high.
Loose the pkg db and your best plan is to re-emerge the entire world, 
provided you still have a copy of /var/lib/portage/world (or 
equivalent), loose the latter and have a laugh.
To put those in the same category seem risky

Not that I personally care since everything gentoo related is kept in /g 
on my systems, also this for various reason mainly because it's 
something used to mantain a system and if maintainaince is not needed 
it's very easy this way to remove.

>> I don't see /var/db in the latest release of the Filesystem Hierarchy
>> Standard:
>>
>>    http://www.pathname.com/fhs/pub/fhs-2.3.html#THEVARHIERARCHY
>>
>> I would prefer something that blends with FHS.
> That's ok, Gentoo doesn't follow FHS.
>
And it's ok to "prefere" to stay near a standard and use it as a 
guideline, for various reason, less difference with others and because a 
bunch of people has already toughted on it, to name just two.
Raising to "MUST blend" would be indeed not beneficial.

Regards,
Francesco Riosa

Re: [gentoo-dev] Is /var/cache the right place for repositories?

[Sebastian Pipping]

On 20.12.2012 18:27, Ulrich Mueller wrote:
> Now I wonder: After removal of e.g. the Portage tree from a system, it
> is generally not possible to restore it. (It can be refetched, but not
> to its previous state.)
> 
> Same is true for distfiles, at least to some degree. They may have
> vanished upstream or from mirrors.
> 
> Maybe /var/lib would be a better choice? It would also take care of
> the issue with fetch-restricted files.

Thanks for bringing it up.  What you address above is the exact reason
why Layman's home was moved to /var/lib/layman/ eventually.  It has a
cache aspect, bit it's not a true cache.

Best,



Sebastian