[gentoo-user] attic

[gentoo-user] attic

[William Kenworthy]

Hi , I used to be able to get old ebuilds from "the attic" but I cant 
find it on google - is it still around?

* gentoo has moved dev-embedded/reedsolomon to dev-embedded/reedsolo 
(then removing the old ebuilds) breaking my homeassistant install 
easiest fix is a local copy until HA catches up.

BillK

Re: [gentoo-user] attic

[Michael]

[-- Attachment #1: Type: text/plain, Size: 455 bytes --]

On Sunday, 3 September 2023 07:49:36 BST William Kenworthy wrote:
> Hi , I used to be able to get old ebuilds from "the attic" but I cant
> find it on google - is it still around?

Perhaps have a look here at the archives?

https://gitweb.gentoo.org/


> * gentoo has moved dev-embedded/reedsolomon to dev-embedded/reedsolo
> (then removing the old ebuilds) breaking my homeassistant install
> easiest fix is a local copy until HA catches up.
> 
> BillK


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] attic

[Rich Freeman]

On Sun, Sep 3, 2023 at 4:44 AM Michael <confabulate@kintzios.com> wrote:
>
> On Sunday, 3 September 2023 07:49:36 BST William Kenworthy wrote:
> > Hi , I used to be able to get old ebuilds from "the attic" but I cant
> > find it on google - is it still around?
>
> Perhaps have a look here at the archives?
>
> https://gitweb.gentoo.org/

The archives will only contain data migrated from CVS - so only things
from more than a few years ago.

You want to look into the main repo for anything recently deleted.

> > * gentoo has moved dev-embedded/reedsolomon to dev-embedded/reedsolo
> > (then removing the old ebuilds) breaking my homeassistant install
> > easiest fix is a local copy until HA catches up.

Both CVS and git maintain a record of anything that has been deleted,
but they do it differently.  The attic directory in CVS contains
anything deleted from CVS.  In git you need to search the commit
history for these files.

This can be done via the website, though the search capability is a
little limited.  I ended up having to search from a local clone
because your package name contains an error and the web search found
nothing.

To find your file, go to:
https://gitweb.gentoo.org/repo/gentoo.git/
Go to the search box in the top right and search for:
dev-python/reedsolomon     (note that the package category is
different from what was in your email)
Find the commit one commit before the one that removed your package.
(ie one that contains your package in its most recent version)  If you
find the one that deleted your file, then just look at the parent in
the commit header and click on that to go back one version where it is
still present.
Click the tree hash to browse the historical version of the repository
that existed before your file was deleted.
For example, you can find v1.6.1 of that package at:
https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-python/reedsolomon/reedsolomon-1.6.1.ebuild?id=149a131188ebce76a87fd8363fb212f5f1620a02


If the search function is too limiting on the website, here is how to
do it from a local checkout.  This is what I ended up doing since you
had the wrong package name.

Note that the first step here requires a few minutes and a few GB of
space.  My search example is also a bit broader than it would have to
be, but you got the package category wrong and searching for
"dev-embedded/reedsolomon" turned up nothing.

git clone https://anongit.gentoo.org/git/repo/gentoo.git
cd gentoo
git log --all --full-history --raw --no-merges -- "**/reedsolomon/*"

Then browse through the history for the file you're interested in.
Suppose you want reedsolomon-1.6.1.ebuild.
Easiest way to do that is to find a commit just before it was deleted,
so just search the log for that filename.  Ignore the first commit
where it comes up, which is where the file was deleted (if you examine
that commit the file will be missing, since it was deleted).  Go to
the next one.
So reedsolomon-1.6.1.ebuild was deleted in commit
beedaf82bd7abd72a654e26627774aef38590149.  The next commit in the log
search is 149a131188ebce76a87fd8363fb212f5f1620a02,

git checkout 149a131188ebce76a87fd8363fb212f5f1620a02
cd dev-python/reedsolomon
cat reedsolomon-1.6.1.ebuild

You can also see this on the web interface at:
https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-python/reedsolomon/reedsolomon-1.6.1.ebuild?id=149a131188ebce76a87fd8363fb212f5f1620a02

The web git interface is capable of displaying past commits.  It just
can't search for wildcards/etc.

-- 
Rich

Re: [gentoo-user] attic

[William Kenworthy]

On 3/9/23 18:29, Rich Freeman wrote:
> On Sun, Sep 3, 2023 at 4:44 AM Michael <confabulate@kintzios.com> wrote:
>> On Sunday, 3 September 2023 07:49:36 BST William Kenworthy wrote:
>>> Hi , I used to be able to get old ebuilds from "the attic" but I cant
>>> find it on google - is it still around?
>> Perhaps have a look here at the archives?
>>
>> https://gitweb.gentoo.org/
> The archives will only contain data migrated from CVS - so only things
> from more than a few years ago.
>
> You want to look into the main repo for anything recently deleted.
>
>>> * gentoo has moved dev-embedded/reedsolomon to dev-embedded/reedsolo
>>> (then removing the old ebuilds) breaking my homeassistant install
>>> easiest fix is a local copy until HA catches up.
> Both CVS and git maintain a record of anything that has been deleted,
> but they do it differently.  The attic directory in CVS contains
> anything deleted from CVS.  In git you need to search the commit
> history for these files.
>
> This can be done via the website, though the search capability is a
> little limited.  I ended up having to search from a local clone
> because your package name contains an error and the web search found
> nothing.
>
> To find your file, go to:
> https://gitweb.gentoo.org/repo/gentoo.git/
> Go to the search box in the top right and search for:
> dev-python/reedsolomon     (note that the package category is
> different from what was in your email)
> Find the commit one commit before the one that removed your package.
> (ie one that contains your package in its most recent version)  If you
> find the one that deleted your file, then just look at the parent in
> the commit header and click on that to go back one version where it is
> still present.
> Click the tree hash to browse the historical version of the repository
> that existed before your file was deleted.
> For example, you can find v1.6.1 of that package at:
> https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-python/reedsolomon/reedsolomon-1.6.1.ebuild?id=149a131188ebce76a87fd8363fb212f5f1620a02
>
>
> If the search function is too limiting on the website, here is how to
> do it from a local checkout.  This is what I ended up doing since you
> had the wrong package name.
>
> Note that the first step here requires a few minutes and a few GB of
> space.  My search example is also a bit broader than it would have to
> be, but you got the package category wrong and searching for
> "dev-embedded/reedsolomon" turned up nothing.
>
> git clone https://anongit.gentoo.org/git/repo/gentoo.git
> cd gentoo
> git log --all --full-history --raw --no-merges -- "**/reedsolomon/*"
>
> Then browse through the history for the file you're interested in.
> Suppose you want reedsolomon-1.6.1.ebuild.
> Easiest way to do that is to find a commit just before it was deleted,
> so just search the log for that filename.  Ignore the first commit
> where it comes up, which is where the file was deleted (if you examine
> that commit the file will be missing, since it was deleted).  Go to
> the next one.
> So reedsolomon-1.6.1.ebuild was deleted in commit
> beedaf82bd7abd72a654e26627774aef38590149.  The next commit in the log
> search is 149a131188ebce76a87fd8363fb212f5f1620a02,
>
> git checkout 149a131188ebce76a87fd8363fb212f5f1620a02
> cd dev-python/reedsolomon
> cat reedsolomon-1.6.1.ebuild
>
> You can also see this on the web interface at:
> https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-python/reedsolomon/reedsolomon-1.6.1.ebuild?id=149a131188ebce76a87fd8363fb212f5f1620a02
>
> The web git interface is capable of displaying past commits.  It just
> can't search for wildcards/etc.
>
Thanks Rich,

unfortunately the web interface isn't helpful - I cant just navigate the 
tree to find commits - 
"https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-python/reedsolomon/" 
gives path not found - it looks like you have to know the commit first 
by downloading the git tree to search it - not friendly at all!

the wrong package category was due to trying to wrangle a few ebuilds 
with the same problem which I confused in the email.  I have now found 
the last of them in an overlay with the distfiles (a similar problem)!

This was all self caused - I had years of portage backups (started last 
time I had a major problem with finding ancient ebuilds) I lost when 
simplifying/re configuring my systems :(

BillK

[gentoo-user] Re: attic

[Nuno Silva]

On 2023-09-04, William Kenworthy wrote:

> On 3/9/23 18:29, Rich Freeman wrote:
>> On Sun, Sep 3, 2023 at 4:44 AM Michael <confabulate@kintzios.com> wrote:
>>> On Sunday, 3 September 2023 07:49:36 BST William Kenworthy wrote:
>>>> Hi , I used to be able to get old ebuilds from "the attic" but I cant
>>>> find it on google - is it still around?
>>> Perhaps have a look here at the archives?
>>>
>>> https://gitweb.gentoo.org/
>> The archives will only contain data migrated from CVS - so only things
>> from more than a few years ago.
>>
>> You want to look into the main repo for anything recently deleted.
[...]
>> This can be done via the website, though the search capability is a
>> little limited.  I ended up having to search from a local clone
>> because your package name contains an error and the web search found
>> nothing.
>>
>> To find your file, go to:
>> https://gitweb.gentoo.org/repo/gentoo.git/
>> Go to the search box in the top right and search for:
>> dev-python/reedsolomon     (note that the package category is
>> different from what was in your email)
>> Find the commit one commit before the one that removed your package.
>> (ie one that contains your package in its most recent version)  If you
>> find the one that deleted your file, then just look at the parent in
>> the commit header and click on that to go back one version where it is
>> still present.
>> Click the tree hash to browse the historical version of the repository
>> that existed before your file was deleted.
>> For example, you can find v1.6.1 of that package at:
>> https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-python/reedsolomon/reedsolomon-1.6.1.ebuild?id=149a131188ebce76a87fd8363fb212f5f1620a02
[...]
>> The web git interface is capable of displaying past commits.  It just
>> can't search for wildcards/etc.
>>
> Thanks Rich,
>
> unfortunately the web interface isn't helpful - I cant just navigate
> the tree to find commits - 
> "https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-python/reedsolomon/"
> gives path not found - it looks like you have to know the commit first
> by downloading the git tree to search it - not friendly at all!

With /log/ instead of /tree/ in the URL it at least shows the list of
commits. From a quick check, this seems to include the commit removing
the directory when it's removed instead of renamed, so hopefully it
helps too with retrieving older ebuilds?

(But note that Rich was suggesting using the *search* feature of the
gitweb interface, which, in this case, also finds the same topmost
commit if I search for "reedsolomon".)

-- 
Nuno Silva

Re: [gentoo-user] Re: attic

[William Kenworthy]

On 4/9/23 16:04, Nuno Silva wrote:
> On 2023-09-04, William Kenworthy wrote:
>
>> On 3/9/23 18:29, Rich Freeman wrote:
>>> On Sun, Sep 3, 2023 at 4:44 AM Michael <confabulate@kintzios.com> wrote:
>>>> On Sunday, 3 September 2023 07:49:36 BST William Kenworthy wrote:
>>>>> Hi , I used to be able to get old ebuilds from "the attic" but I cant
>>>>> find it on google - is it still around?
>>>> Perhaps have a look here at the archives?
>>>>
>>>> https://gitweb.gentoo.org/
>>> The archives will only contain data migrated from CVS - so only things
>>> from more than a few years ago.
>>>
>>> You want to look into the main repo for anything recently deleted.
> [...]
>>> This can be done via the website, though the search capability is a
>>> little limited.  I ended up having to search from a local clone
>>> because your package name contains an error and the web search found
>>> nothing.
>>>
>>> To find your file, go to:
>>> https://gitweb.gentoo.org/repo/gentoo.git/
>>> Go to the search box in the top right and search for:
>>> dev-python/reedsolomon     (note that the package category is
>>> different from what was in your email)
>>> Find the commit one commit before the one that removed your package.
>>> (ie one that contains your package in its most recent version)  If you
>>> find the one that deleted your file, then just look at the parent in
>>> the commit header and click on that to go back one version where it is
>>> still present.
>>> Click the tree hash to browse the historical version of the repository
>>> that existed before your file was deleted.
>>> For example, you can find v1.6.1 of that package at:
>>> https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-python/reedsolomon/reedsolomon-1.6.1.ebuild?id=149a131188ebce76a87fd8363fb212f5f1620a02
> [...]
>>> The web git interface is capable of displaying past commits.  It just
>>> can't search for wildcards/etc.
>>>
>> Thanks Rich,
>>
>> unfortunately the web interface isn't helpful - I cant just navigate
>> the tree to find commits -
>> "https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-python/reedsolomon/"
>> gives path not found - it looks like you have to know the commit first
>> by downloading the git tree to search it - not friendly at all!
> With /log/ instead of /tree/ in the URL it at least shows the list of
> commits. From a quick check, this seems to include the commit removing
> the directory when it's removed instead of renamed, so hopefully it
> helps too with retrieving older ebuilds?
>
> (But note that Rich was suggesting using the *search* feature of the
> gitweb interface, which, in this case, also finds the same topmost
> commit if I search for "reedsolomon".)
>
tkx, missed that!

BillK

Re: [gentoo-user] Re: attic

[Rich Freeman]

On Mon, Sep 4, 2023 at 4:38 AM William Kenworthy <billk@iinet.net.au> wrote:
>
> On 4/9/23 16:04, Nuno Silva wrote:
> >
> > (But note that Rich was suggesting using the *search* feature of the
> > gitweb interface, which, in this case, also finds the same topmost
> > commit if I search for "reedsolomon".)
> >
> tkx, missed that!

Note that in terms of indexing git and CVS have their pros and cons,
because they use different data structures.  I've heard the saying
that Git is a data structure masquerading as an SCM, and certainly the
inconsistencies in the command line operations bear that out.  Git
tends to be much more useful in general, but for things like finding
deleted files CVS was definitely more time-efficient.

The reason for this is that everything in git is reachable via
commits, and these are reachable from a head via a linked list.  The
most recent commit gives access to the current version of the
repository, and a pointer to the immediately previous commit(s).  To
find a deleted file, git must go to the most recent commit in whatever
branch you are searching, then descend its tree to look for the file.
If it is not found, it then goes to the previous commit and descends
that tree.  There are 745k commits in the active Gentoo repository.  I
think there are something like 2M of them in the historical one.  Each
commit is a random seek, and then each step down the directory tree to
find a file is another random seek.

In CVS everything is organized first by file, and then each file has
its own commit history.  So finding a file, deleted or otherwise, just
requires a seek for each level in the directory tree.  Then you can
directly read its history.

So finding an old deleted file in the gentoo git repo can require
millions of reads, while doing so in CVS only required about 3.  It is
no surprise that the web interfaces were designed to make that
operation much easier - if you do sufficiently complex searches in the
git web interface it will time you out to avoid bogging down the
server, which is why some searches may require you to clone the repo
and do it locally.

Now, if you want to find out what changed in a particular commit the
situation is reversed.  If you identify a commit in git and want to
see what changed, it can directly read the commit from disk using its
hash.  It then looks at the parent commit, then descends both trees
doing a diff at each level.  Since everything is content-hashed only
directory trees that contain differences need to be read.  If a commit
had changes to 50 files, it might only take 10 reads to figure out
which files changed, and then another 100 to compare the contents of
each file and generate diffs.  If you wanted to do that in CVS you'd
have to read every single file in the repository and read the
sequential history of each file to find any commits that have the same
time/author.  CVS commits also aren't atomic so ordering across files
might not be the same.

Git is a thing of beauty when you think about what it was designed to
do and how well-suited to this design its architecture is.  The same
can be said of several data-driven FOSS applications.  The right
algorithm can make a huge difference...

-- 
Rich

Re: [gentoo-user] Re: attic

[Alan McKinnon]

[-- Attachment #1: Type: text/plain, Size: 872 bytes --]

On Mon, Sep 4, 2023 at 2:36 PM Rich Freeman <rich0@gentoo.org> wrote:

> On Mon, Sep 4, 2023 at 4:38 AM William Kenworthy <billk@iinet.net.au>
> wrote:
> >
> > On 4/9/23 16:04, Nuno Silva wrote:
> > >
> > > (But note that Rich was suggesting using the *search* feature of the
> > > gitweb interface, which, in this case, also finds the same topmost
> > > commit if I search for "reedsolomon".)
> > >
> > tkx, missed that!
>
> Note that in terms of indexing git and CVS have their pros and cons,
> because they use different data structures.  I've heard the saying
> that Git is a data structure masquerading as an SCM, and certainly the
> inconsistencies in the command line operations bear that out.
>

I'd always heard that Git is a file system and all useful side effects are
pure luck

-- 
Alan McKinnon
alan dot mckinnon at gmail dot com

[-- Attachment #2: Type: text/html, Size: 1449 bytes --]