Re: [gentoo-portage-dev] [PATCH] Log changes between vdb_metadata.pickle updates

[Zac Medico <zmedico@gentoo.org>]

On 11/07/2014 08:51 AM, Brian Dolbec wrote:
> On Fri,  7 Nov 2014 00:45:55 -0800
> Zac Medico <zmedico@gentoo.org> wrote:
> 
>> This adds add support to generate a vdb_metadata_delta.json file
>> which tracks package merges / unmerges that occur between updates to
>> vdb_metadata.pickle. IndexedVardb can use the delta together with
>> vdb_metadata.pickle to reconstruct a complete view of /var/db/pkg,
>> so that it can avoid expensive listdir calls in /var/db/pkg/*.
>> Note that vdb_metadata.pickle is only updated periodically, in
>> order to avoid excessive re-writes of a large file.
>>
>> In order to test the performance gains from this patch, you need to
>> generate /var/cache/edb/vdb_metadata_delta.json first, which will
>> happen automatically if you run 'emerge -p anything' with root
>> privileges.
>> ---
> 
> The following code I would like to see either as an independant class
> and file if possible, then just instantiated here in the main vardbapi.
> Looking over the code, I didn't see much use of other class functions.
> This class is already too large in many ways.

Yeah, I definitely want to split it out.

> Also is there a
> possibility this code could be re-used as a generic delta cache
> anywhere else?

Maybe. For example, the PreservedLibsRegistry and WorldSelectedSet
classes both have similarities in the way that encapsulate an on-disk
data store and manage concurrency. Maybe I'll create a helper class that
can be utilized by these classes to manage concurrency with on-disk data
stores.

> Another possibility is moving this code and the aux_cache code to
> another class that the vardbapi class also subclasses.  This would move
> all the cache code to a small class easily viewed, edited, maintained.

In this case, I think a helper class will work just fine, so there will
be no need for inheritance.
-- 
Thanks,
Zac