[gentoo-portage-dev] Portage questions

[gentoo-portage-dev] Portage questions

[Joakim Tjernlund]

1)
  Is there a way to generate a snapshot of an installed portage VDB and then later
  compare that snapshot against the current VDB and generate a list of
  added/updated packages?

2)
  Currently we generate a tar file with binary pkgs containing all updated pkgs,
  unpack the tar file and emerge the binary pkgs with emerge --usepkgonly ...
  into a new ROOT. This works but that misses any updates to the profile(both gentoo and our own).

  To fix that we include a copy of both profiles in the tar file and copy our
  updated profiles into the new ROOT profiles before merging. This feels a bit
  clumsy and I wonder if there is a better way? 

 Jocke

[gentoo-portage-dev] Re: Portage questions

[Duncan]

Joakim Tjernlund posted on Wed, 16 Sep 2015 13:13:40 +0000 as excerpted:

>   Is there a way to generate a snapshot of an installed portage VDB and
>   then later compare that snapshot against the current VDB and generate
>   a list of added/updated packages?

That one is either relatively simple, or I'm not understanding your 
question.

Portage's installed package database, vdb, is located at /var/db/pkg/.  
It's organized as a category/package-version tree, much like the normal 
gentoo packages tree, except that the package dir names have the version 
appended (and of course there's no profile/metadata/etc subdirs).  Most 
files in the individual pkg-ver dirs are plain text, tho the environment 
file is compressed (bz2 here, tho it's possible that's configurable, IDK).

So a snapshot of vdb should be as simple as tarballing /var/db/pkg.  You 
should then be able to untar it somewhere and do a recursive diff or 
whatever, to compare the freshly unarchived version against the existing 
one and get your list of added/updated packages based on the diff.

The other question I didn't (as a user not a dev) understand.  I'd need a 
fuller explanation (it feels like I came in half way thru the story and 
missed something critical, which has me wondering if I'm missing 
something on the first one too, since it seemed so simple, thus the 
remark above to that effect), but it's possible a dev will understand 
better and be able to answer.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-portage-dev] Portage questions

[Zac Medico]

On 09/16/2015 06:13 AM, Joakim Tjernlund wrote:
> 1)
>   Is there a way to generate a snapshot of an installed portage VDB and then later
>   compare that snapshot against the current VDB and generate a list of
>   added/updated packages?

This command will generate a flat file that is useful for comparisons
with diff -u:

    qlist -IvCURS | LC_ALL=C sort

> 2)
>   Currently we generate a tar file with binary pkgs containing all updated pkgs,
>   unpack the tar file and emerge the binary pkgs with emerge --usepkgonly ...
>   into a new ROOT. This works but that misses any updates to the profile(both gentoo and our own).
> 
>   To fix that we include a copy of both profiles in the tar file and copy our
>   updated profiles into the new ROOT profiles before merging. This feels a bit
>   clumsy and I wonder if there is a better way? 
> 
>  Jocke
> 

Like Duncan said, I feel like we're missing some context here. I wonder
why you use binary packages instead of a full system image, considering
that it's a "new ROOT."
-- 
Thanks,
Zac

Re: [gentoo-portage-dev] Portage questions

[Joakim Tjernlund]

On Sun, 2015-09-20 at 09:47 -0700, Zac Medico wrote:
> On 09/16/2015 06:13 AM, Joakim Tjernlund wrote:
> > 1)
> >   Is there a way to generate a snapshot of an installed portage VDB and then later
> >   compare that snapshot against the current VDB and generate a list of
> >   added/updated packages?
> 
> This command will generate a flat file that is useful for comparisons
> with diff -u:
> 
>     qlist -IvCURS | LC_ALL=C sort

I was hoping for something akin to "emerge -pNDu world" but pointing emerge
to another VDB. That would make it easier to see what has changed between then and now.

> 
> > 2)
> >   Currently we generate a tar file with binary pkgs containing all updated pkgs,
> >   unpack the tar file and emerge the binary pkgs with emerge --usepkgonly ...
> >   into a new ROOT. This works but that misses any updates to the profile(both gentoo and our own).
> > 
> >   To fix that we include a copy of both profiles in the tar file and copy our
> >   updated profiles into the new ROOT profiles before merging. This feels a bit
> >   clumsy and I wonder if there is a better way? 
> > 
> >  Jocke
> > 
> 
> Like Duncan said, I feel like we're missing some context here. I wonder
> why you use binary packages instead of a full system image, considering
> that it's a "new ROOT."

Every update to ROOT will be too big if one have use a full system image at every update.
Instead we use BTRFS and create a snapshot of current / in a new BTRFS subvolume
and install the updates into this snapshot, then just change default subvolume in BTRFS and
reboot.
To transfer profile changes before merging binary pkgs one have to copy these into
the new ROOT.

 Jocke

Re: [gentoo-portage-dev] Portage questions

[Zac Medico]

On 09/21/2015 06:45 AM, Joakim Tjernlund wrote:
> On Sun, 2015-09-20 at 09:47 -0700, Zac Medico wrote:
>> On 09/16/2015 06:13 AM, Joakim Tjernlund wrote:
>>> 1)
>>>   Is there a way to generate a snapshot of an installed portage VDB and then later
>>>   compare that snapshot against the current VDB and generate a list of
>>>   added/updated packages?
>>
>> This command will generate a flat file that is useful for comparisons
>> with diff -u:
>>
>>     qlist -IvCURS | LC_ALL=C sort
> 
> I was hoping for something akin to "emerge -pNDu world" but pointing emerge
> to another VDB. That would make it easier to see what has changed between then and now.

You can export the EPREFIX variable in order to make emerge enter prefix
mode.

>>
>>> 2)
>>>   Currently we generate a tar file with binary pkgs containing all updated pkgs,
>>>   unpack the tar file and emerge the binary pkgs with emerge --usepkgonly ...
>>>   into a new ROOT. This works but that misses any updates to the profile(both gentoo and our own).
>>>
>>>   To fix that we include a copy of both profiles in the tar file and copy our
>>>   updated profiles into the new ROOT profiles before merging. This feels a bit
>>>   clumsy and I wonder if there is a better way? 
>>>
>>>  Jocke
>>>
>>
>> Like Duncan said, I feel like we're missing some context here. I wonder
>> why you use binary packages instead of a full system image, considering
>> that it's a "new ROOT."
> 
> Every update to ROOT will be too big if one have use a full system image at every update.
> Instead we use BTRFS and create a snapshot of current / in a new BTRFS subvolume
> and install the updates into this snapshot, then just change default subvolume in BTRFS and
> reboot.
> To transfer profile changes before merging binary pkgs one have to copy these into
> the new ROOT.
> 
>  Jocke
> 

I suppose that emerge --sync is what most people use for this. To
distribute your profiles, you might configure portage to sync from your
own rsync server.
-- 
Thanks,
Zac