[gentoo-user] tools to clean up /usr/portage/packages?

[gentoo-user] tools to clean up /usr/portage/packages?

[Mark Knecht]

Are there any tools that will:

1) Ensure that for every installed packages there is a corresponding
tbz2 file in /usr/portage/packages?

2) Remove any older versions in /usr/portage/packages prior to me
running a backup?

Thanks,
Mark

[gentoo-user] Re: tools to clean up /usr/portage/packages?

[Nikos Chantziaras]

On 28/02/12 00:41, Mark Knecht wrote:
> Are there any tools that will:
>
> 1) Ensure that for every installed packages there is a corresponding
> tbz2 file in /usr/portage/packages?
>
> 2) Remove any older versions in /usr/portage/packages prior to me
> running a backup?

I think app-portage/gentoolkit can help with its "eclean" tool 
(specifically, "eclean-pkg").

"man eclean" should get you started.

Re: [gentoo-user] Re: tools to clean up /usr/portage/packages?

[Mark Knecht]

On Mon, Feb 27, 2012 at 2:49 PM, Nikos Chantziaras <realnc@arcor.de> wrote:
> On 28/02/12 00:41, Mark Knecht wrote:
>>
>> Are there any tools that will:
>>
>> 1) Ensure that for every installed packages there is a corresponding
>> tbz2 file in /usr/portage/packages?
>>
>> 2) Remove any older versions in /usr/portage/packages prior to me
>> running a backup?
>
>
> I think app-portage/gentoolkit can help with its "eclean" tool
> (specifically, "eclean-pkg").
>
> "man eclean" should get you started.
>
>

Thanks Nikos. That does seem to get me into the right space. I'm not
sure I'm understanding all the counts yet though. emerge suggest 992
packages. My simple count of existing packages is 994. eclean-pkg
wants to remove only 1. I had expected it to remove 2.

What might I be doing wrong here?

Also, this is raising an additional question for me. What's the
difference between buildpkg and buildsyspkg. man emerge doesn't talk
about the latter as best I can tell.

Thanks,
Mark

c2stable ~ # emerge -pve @world
<SNIP>

Total: 992 packages (992 reinstalls), Size of downloads: 0 kB
Fetch Restriction: 1 package
c2stable ~ #

c2stable ~ # ls -alR /usr/portage/packages | grep tbz2 | wc
    994    8946   66857
c2stable ~ #

c2stable ~ # eclean-pkg -p
 * Building file list for packages cleaning...
 * Here are the binary packages that would be deleted:
 [   23.1 K ] media-libs/libdiscid-0.1.1
 ===========
 [   23.1 K ] Total space from 1 files would be freed in the packages directory
c2stable ~ #

Re: [gentoo-user] Re: tools to clean up /usr/portage/packages?

[Paul Hartman]

On Mon, Feb 27, 2012 at 4:49 PM, Nikos Chantziaras <realnc@arcor.de> wrote:
> On 28/02/12 00:41, Mark Knecht wrote:
>>
>> Are there any tools that will:
>>
>> 1) Ensure that for every installed packages there is a corresponding
>> tbz2 file in /usr/portage/packages?
>>
>> 2) Remove any older versions in /usr/portage/packages prior to me
>> running a backup?
>
>
> I think app-portage/gentoolkit can help with its "eclean" tool
> (specifically, "eclean-pkg").
>
> "man eclean" should get you started.

And as an example of savings... I run eclean once in a while, but not
automated. I just ran it and got these results:

[   14.8 G ] Total space from 1673 files were freed in the distfiles directory

I guess I should use it more frequently. ;)

Re: [gentoo-user] Re: tools to clean up /usr/portage/packages?

[Mark Knecht]

On Mon, Feb 27, 2012 at 3:18 PM, Paul Hartman
<paul.hartman+gentoo@gmail.com> wrote:
> On Mon, Feb 27, 2012 at 4:49 PM, Nikos Chantziaras <realnc@arcor.de> wrote:
>> On 28/02/12 00:41, Mark Knecht wrote:
>>>
>>> Are there any tools that will:
>>>
>>> 1) Ensure that for every installed packages there is a corresponding
>>> tbz2 file in /usr/portage/packages?
>>>
>>> 2) Remove any older versions in /usr/portage/packages prior to me
>>> running a backup?
>>
>>
>> I think app-portage/gentoolkit can help with its "eclean" tool
>> (specifically, "eclean-pkg").
>>
>> "man eclean" should get you started.
>
> And as an example of savings... I run eclean once in a while, but not
> automated. I just ran it and got these results:
>
> [   14.8 G ] Total space from 1673 files were freed in the distfiles directory
>
> I guess I should use it more frequently. ;)
>

15GB is a nice clean up!

I don't think I'd want to run it automatically, at least not often. If
it automatically deleted things that work in favor of newly built but
untested packages that would defeat the purpose in my mind.

As basically nothing but a home user I'm trying after 12 years to
piece together some sort of a backup strategy here, including how to
do a restore if a drive died, etc. I'll ask some questions about that
later, but likely it should be it's own thread.

Cheers,
Mark

- Mark

Re: [gentoo-user] Re: tools to clean up /usr/portage/packages?

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 457 bytes --]

On Mon, 27 Feb 2012 15:15:18 -0800, Mark Knecht wrote:

> Also, this is raising an additional question for me. What's the
> difference between buildpkg and buildsyspkg. man emerge doesn't talk
> about the latter as best I can tell.

buildpkg builds packages for all installs, buildsyspkg only for packages
in the system set. man make.conf explains the FEATURES options.


-- 
Neil Bothwick

Wow! That lightning sounds clo..zzzzit!" NO CARRIER

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Re: tools to clean up /usr/portage/packages?

[Mark Knecht]

On Mon, Feb 27, 2012 at 3:42 PM, Neil Bothwick <neil@digimed.co.uk> wrote:
> On Mon, 27 Feb 2012 15:15:18 -0800, Mark Knecht wrote:
>
>> Also, this is raising an additional question for me. What's the
>> difference between buildpkg and buildsyspkg. man emerge doesn't talk
>> about the latter as best I can tell.
>
> buildpkg builds packages for all installs, buildsyspkg only for packages
> in the system set. man make.conf explains the FEATURES options.
>
>
> --
> Neil Bothwick

Thanks Neil!

- Mark

Re: [gentoo-user] Re: tools to clean up /usr/portage/packages?

[Paul Hartman]

On Mon, Feb 27, 2012 at 5:30 PM, Mark Knecht <markknecht@gmail.com> wrote:
> On Mon, Feb 27, 2012 at 3:18 PM, Paul Hartman
> <paul.hartman+gentoo@gmail.com> wrote:
>> On Mon, Feb 27, 2012 at 4:49 PM, Nikos Chantziaras <realnc@arcor.de> wrote:
>>> On 28/02/12 00:41, Mark Knecht wrote:
>>>>
>>>> Are there any tools that will:
>>>>
>>>> 1) Ensure that for every installed packages there is a corresponding
>>>> tbz2 file in /usr/portage/packages?
>>>>
>>>> 2) Remove any older versions in /usr/portage/packages prior to me
>>>> running a backup?
>>>
>>>
>>> I think app-portage/gentoolkit can help with its "eclean" tool
>>> (specifically, "eclean-pkg").
>>>
>>> "man eclean" should get you started.
>>
>> And as an example of savings... I run eclean once in a while, but not
>> automated. I just ran it and got these results:
>>
>> [   14.8 G ] Total space from 1673 files were freed in the distfiles directory
>>
>> I guess I should use it more frequently. ;)
>>
>
> 15GB is a nice clean up!
>
> I don't think I'd want to run it automatically, at least not often. If
> it automatically deleted things that work in favor of newly built but
> untested packages that would defeat the purpose in my mind.
>
> As basically nothing but a home user I'm trying after 12 years to
> piece together some sort of a backup strategy here, including how to
> do a restore if a drive died, etc. I'll ask some questions about that
> later, but likely it should be it's own thread.
>
> Cheers,
> Mark

You can probably just exclude /usr/portage from your backup entirely,
since it'll be restored with an emerge --sync (or webrsync) and any
distfiles can be downloaded again if they are needed.

Re: [gentoo-user] Re: tools to clean up /usr/portage/packages?

[Mark Knecht]

On Mon, Feb 27, 2012 at 3:48 PM, Paul Hartman
<paul.hartman+gentoo@gmail.com> wrote:
> On Mon, Feb 27, 2012 at 5:30 PM, Mark Knecht <markknecht@gmail.com> wrote:
>> On Mon, Feb 27, 2012 at 3:18 PM, Paul Hartman
>> <paul.hartman+gentoo@gmail.com> wrote:
>>> On Mon, Feb 27, 2012 at 4:49 PM, Nikos Chantziaras <realnc@arcor.de> wrote:
>>>> On 28/02/12 00:41, Mark Knecht wrote:
>>>>>
>>>>> Are there any tools that will:
>>>>>
>>>>> 1) Ensure that for every installed packages there is a corresponding
>>>>> tbz2 file in /usr/portage/packages?
>>>>>
>>>>> 2) Remove any older versions in /usr/portage/packages prior to me
>>>>> running a backup?
>>>>
>>>>
>>>> I think app-portage/gentoolkit can help with its "eclean" tool
>>>> (specifically, "eclean-pkg").
>>>>
>>>> "man eclean" should get you started.
>>>
>>> And as an example of savings... I run eclean once in a while, but not
>>> automated. I just ran it and got these results:
>>>
>>> [   14.8 G ] Total space from 1673 files were freed in the distfiles directory
>>>
>>> I guess I should use it more frequently. ;)
>>>
>>
>> 15GB is a nice clean up!
>>
>> I don't think I'd want to run it automatically, at least not often. If
>> it automatically deleted things that work in favor of newly built but
>> untested packages that would defeat the purpose in my mind.
>>
>> As basically nothing but a home user I'm trying after 12 years to
>> piece together some sort of a backup strategy here, including how to
>> do a restore if a drive died, etc. I'll ask some questions about that
>> later, but likely it should be it's own thread.
>>
>> Cheers,
>> Mark
>
> You can probably just exclude /usr/portage from your backup entirely,
> since it'll be restored with an emerge --sync (or webrsync) and any
> distfiles can be downloaded again if they are needed.
>

Agreed.

My server has about 400GB to back up. Roughly 360GB is virtual
machines which get backed up daily already so I have that handled.

Of the other 40GB it seems that (excluding portage, /var and a few
other things) I need to back up about 24GB which I think can be backed
up live. I'm not really worried about restoring the exact state of the
machine in one pass. This isn't a business, etc. I just want to get
back fairly quickly to where I was before the presumed failure. I
figure if I get:

/home
/boot
/usr/src
/etc
/var/lib/portage

and maybe one or two more, then a restore would hopefully be something
like doing a quick install as per the Gentoo docs and then laying this
stuff on top and doing an emerge -ke @world.

Or at least that's what I'm trying to puzzle together.

I'm planning on trying it with an additional hard drive as a test.
I'll have to modify fstab as the main system is a 5 drive RAID6
monster and for testing I just want a single drive to verify that it
works.

QUESTION: As for ensuring that every package actually has a
corresponding tbz2 file in the packages directory, would

emerge -ek @world

install everything from packages except in the case of something not
existing in which case it would build and store it?

Thanks,
Mark

Re: [gentoo-user] Re: tools to clean up /usr/portage/packages?

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 612 bytes --]

On Mon, 27 Feb 2012 16:45:16 -0800, Mark Knecht wrote:

> QUESTION: As for ensuring that every package actually has a
> corresponding tbz2 file in the packages directory, would
> 
> emerge -ek @world
> 
> install everything from packages except in the case of something not
> existing in which case it would build and store it?


Yes. And you can check that everything has a package by adding -p to that
command and grepping for ebuild. You could probably also use -eKp and
check for an error.


-- 
Neil Bothwick

C&W music backward: get yer dog, wife, job, truck, kids, and sobriety
back.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Re: tools to clean up /usr/portage/packages?

[Mark Knecht]

On Tue, Feb 28, 2012 at 2:40 AM, Neil Bothwick <neil@digimed.co.uk> wrote:
> On Mon, 27 Feb 2012 16:45:16 -0800, Mark Knecht wrote:
>
>> QUESTION: As for ensuring that every package actually has a
>> corresponding tbz2 file in the packages directory, would
>>
>> emerge -ek @world
>>
>> install everything from packages except in the case of something not
>> existing in which case it would build and store it?
>
>
> Yes. And you can check that everything has a package by adding -p to that
> command and grepping for ebuild. You could probably also use -eKp and
> check for an error.
>
>
> --
> Neil Bothwick

Yep, I especially like the -epK version. Very clear what's missing.

Thanks!

Re: [gentoo-user] Re: tools to clean up /usr/portage/packages?

[Mark Knecht]

On Mon, Feb 27, 2012 at 3:18 PM, Paul Hartman
<paul.hartman+gentoo@gmail.com> wrote:
> On Mon, Feb 27, 2012 at 4:49 PM, Nikos Chantziaras <realnc@arcor.de> wrote:
>> On 28/02/12 00:41, Mark Knecht wrote:
>>>
>>> Are there any tools that will:
>>>
>>> 1) Ensure that for every installed packages there is a corresponding
>>> tbz2 file in /usr/portage/packages?
>>>
>>> 2) Remove any older versions in /usr/portage/packages prior to me
>>> running a backup?
>>
>>
>> I think app-portage/gentoolkit can help with its "eclean" tool
>> (specifically, "eclean-pkg").
>>
>> "man eclean" should get you started.
>
> And as an example of savings... I run eclean once in a while, but not
> automated. I just ran it and got these results:
>
> [   14.8 G ] Total space from 1673 files were freed in the distfiles directory
>
> I guess I should use it more frequently. ;)
>

Hey Paul,
   I have absolutely no recollection of turning on buildpkg on my
wife's machine, but doing updates tonight I noticed it was there and
did a quick cleanup. While I didn't beat you, I came closer than I
would have guessed just an hour ago! ;-)

Cheers,
Mark

k2 ~ # eclean-pkg -d
<SNIP>
 ===========
 [   12.1 G ] Total space from 3793 files were freed in the packages directory
k2 ~ #