Re: [gentoo-portage-dev] Changing the VDB format

[Joshua Kinard <kumba@gentoo.org>]

On 4/11/2022 15:20, Sid Spry wrote:
> On Mon, Apr 11, 2022, at 3:02 PM, Joshua Kinard wrote:
>>
>> I think json is the best format for storing the data on-disk.  It's intended
>> to be a data serialization format to convert data from a non-specific memory
>> format to a storable on-disk format and back again, so this is a perfect use
>> for it.
> 
> Can we avoid adding another format? I find json very hard to edit by hand, it's
> good at storing lots of data in a quasi-textual format, but is strict enough to be
> obnoxious to work with.

I sympathize with you here on this, json is not a format geared for editing
by hand...  :: looks disapprovingly at net-misc/kea ::

> Can the files not be concatenated? Doing so is similar to the tar suggestion,
> but would keep everything very portage-like. Have the contents assigned to
> variables. I am betting someone tried this at the start but settled on the current
> scheme. Does anyone know why? (This would have to be done in bash syntax
> I assume.)
> 
> Alternatively, I think the tar suggestion is quite elegant. There's streaming
> decompressors you can use from python. It adds an extra step to modify but
> that could be handled transparently by a dev mode. In dev mode, leave the files
> after extraction and do not re-extract, for release mode replace the archive with
> what is on disk.

Out of curiosity, what are you doing that requires manual editing of the VDB
data?  That data isn't, in normal scenarios, supposed to be arbitrarily
edited.  Throwing out a good optimization for what sounds like a niche
corner-case doesn't seem like a great plan.  Given json's malleable format,
and especially Matt's example script, converting from json data to another
format that's more conducive to manual editing in rare circumstances, then
converting back, is not impossible.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic