* Re: [gentoo-user] emerge --sync keeps failing
2021-08-02 12:01 ` Michael
@ 2021-08-02 13:23 ` Dale
2021-08-02 15:43 ` n952162
` (2 subsequent siblings)
3 siblings, 0 replies; 17+ messages in thread
From: Dale @ 2021-08-02 13:23 UTC (permalink / raw
To: gentoo-user
Michael wrote:
> On Monday, 2 August 2021 12:10:12 BST n952162 wrote:
>> On 8/1/21 8:32 PM, Michael Orlitzky wrote:
>>> On Sun, 2021-08-01 at 17:32 +0200, n952162 wrote:
>>>> * Verifying /var/db/repos/gentoo/.tmp-unverified-download-quarantine
>>>>
>>>> ...!!! Manifest verification failed:
>>>> Manifest mismatch for metadata/news/Manifest
>>>>
>>>> I've raised this question before and the only useful answer I got was to
>>>> keep trying
>>> On the off chance that something is screwy on the remote end, you can
>>> always use "emerge-webrsync" and delay the problem until next time.
>>>
>>> Otherwise, I would say check "dmesg" for disk errors, but if it's
>>> happening on two machines that's a lot less likely.
>> I have this problem every month. Why does it fail? Is it just a
>> timeout because my network is slow? Can that be tweaked?
> I get this problem over here, but on rare occasions. Leaving it for half a
> day usually fixes it. Have you tried a different rsync mirror? You can use
> 'mirrorselect -i -r' for this task.
This is a very good idea. I had to switch a few months ago. The one I
was using was either really busy or had other issues that slowed it to a
crawl. It reminded me of dial-up days. I might add, I found some that
didn't appear to be set up to sync with anymore. It appeared they
removed Gentoo files from the server. Could be a glitch so I didn't
report it.
I synced last night, about 12 hours ago, without any problems.
Dale
:-) :-)
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: [gentoo-user] emerge --sync keeps failing
2021-08-02 12:01 ` Michael
2021-08-02 13:23 ` Dale
@ 2021-08-02 15:43 ` n952162
2021-08-02 21:17 ` Neil Bothwick
2021-09-06 13:55 ` n952162
3 siblings, 0 replies; 17+ messages in thread
From: n952162 @ 2021-08-02 15:43 UTC (permalink / raw
To: gentoo-user
On 8/2/21 2:01 PM, Michael wrote:
> On Monday, 2 August 2021 12:10:12 BST n952162 wrote:
>> On 8/1/21 8:32 PM, Michael Orlitzky wrote:
>>> On Sun, 2021-08-01 at 17:32 +0200, n952162 wrote:
>>>> * Verifying /var/db/repos/gentoo/.tmp-unverified-download-quarantine
>>>>
>>>> ...!!! Manifest verification failed:
>>>> Manifest mismatch for metadata/news/Manifest
>>>>
>>>> I've raised this question before and the only useful answer I got was to
>>>> keep trying
>>> On the off chance that something is screwy on the remote end, you can
>>> always use "emerge-webrsync" and delay the problem until next time.
>>>
>>> Otherwise, I would say check "dmesg" for disk errors, but if it's
>>> happening on two machines that's a lot less likely.
>> I have this problem every month. Why does it fail? Is it just a
>> timeout because my network is slow? Can that be tweaked?
> I get this problem over here, but on rare occasions. Leaving it for half a
> day usually fixes it. Have you tried a different rsync mirror? You can use
> 'mirrorselect -i -r' for this task.
Ah, good idea.
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [gentoo-user] emerge --sync keeps failing
2021-08-02 12:01 ` Michael
2021-08-02 13:23 ` Dale
2021-08-02 15:43 ` n952162
@ 2021-08-02 21:17 ` Neil Bothwick
2021-08-02 22:42 ` Michael
2021-09-06 13:55 ` n952162
3 siblings, 1 reply; 17+ messages in thread
From: Neil Bothwick @ 2021-08-02 21:17 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: text/plain, Size: 651 bytes --]
On Mon, 02 Aug 2021 13:01:40 +0100, Michael wrote:
> > I have this problem every month. Why does it fail? Is it just a
> > timeout because my network is slow? Can that be tweaked?
>
> I get this problem over here, but on rare occasions. Leaving it for
> half a day usually fixes it. Have you tried a different rsync mirror?
> You can use 'mirrorselect -i -r' for this task.
I used to see this from time to time, but then I switched to using git
instead of rsync and haven't seen it again. There's the added bonus that
syncing is MUCH faster too.
--
Neil Bothwick
WinErr 019: User error - Not our fault. Is Not! Is Not!
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [gentoo-user] emerge --sync keeps failing
2021-08-02 21:17 ` Neil Bothwick
@ 2021-08-02 22:42 ` Michael
2021-08-03 1:28 ` John Covici
0 siblings, 1 reply; 17+ messages in thread
From: Michael @ 2021-08-02 22:42 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: text/plain, Size: 770 bytes --]
On Monday, 2 August 2021 22:17:35 BST Neil Bothwick wrote:
> On Mon, 02 Aug 2021 13:01:40 +0100, Michael wrote:
> > > I have this problem every month. Why does it fail? Is it just a
> > > timeout because my network is slow? Can that be tweaked?
> >
> > I get this problem over here, but on rare occasions. Leaving it for
> > half a day usually fixes it. Have you tried a different rsync mirror?
> > You can use 'mirrorselect -i -r' for this task.
>
> I used to see this from time to time, but then I switched to using git
> instead of rsync and haven't seen it again. There's the added bonus that
> syncing is MUCH faster too.
Last time I looked at git for portage it was not not using gpg to verify the
downloaded data - has this feature been added since then?
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [gentoo-user] emerge --sync keeps failing
2021-08-02 22:42 ` Michael
@ 2021-08-03 1:28 ` John Covici
2021-08-03 8:55 ` Michael
0 siblings, 1 reply; 17+ messages in thread
From: John Covici @ 2021-08-03 1:28 UTC (permalink / raw
To: gentoo-user
On Mon, 02 Aug 2021 18:42:29 -0400,
Michael wrote:
>
> [1 <text/plain; UTF-8 (7bit)>]
> On Monday, 2 August 2021 22:17:35 BST Neil Bothwick wrote:
> > On Mon, 02 Aug 2021 13:01:40 +0100, Michael wrote:
> > > > I have this problem every month. Why does it fail? Is it just a
> > > > timeout because my network is slow? Can that be tweaked?
> > >
> > > I get this problem over here, but on rare occasions. Leaving it for
> > > half a day usually fixes it. Have you tried a different rsync mirror?
> > > You can use 'mirrorselect -i -r' for this task.
> >
> > I used to see this from time to time, but then I switched to using git
> > instead of rsync and haven't seen it again. There's the added bonus that
> > syncing is MUCH faster too.
>
> Last time I looked at git for portage it was not not using gpg to verify the
> downloaded data - has this feature been added since then?
I don't know when you last looked, but I see it checking the signature
when downloading from git.
--
Your life is like a penny. You're going to lose it. The question is:
How do
you spend it?
John Covici wb2una
covici@ccs.covici.com
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: [gentoo-user] emerge --sync keeps failing
2021-08-03 1:28 ` John Covici
@ 2021-08-03 8:55 ` Michael
2021-08-03 10:49 ` Neil Bothwick
0 siblings, 1 reply; 17+ messages in thread
From: Michael @ 2021-08-03 8:55 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: text/plain, Size: 2023 bytes --]
On Tuesday, 3 August 2021 02:28:09 BST John Covici wrote:
> On Mon, 02 Aug 2021 18:42:29 -0400,
>
> Michael wrote:
> >
> > On Monday, 2 August 2021 22:17:35 BST Neil Bothwick wrote:
> > > On Mon, 02 Aug 2021 13:01:40 +0100, Michael wrote:
> > > > > I have this problem every month. Why does it fail? Is it just a
> > > > > timeout because my network is slow? Can that be tweaked?
> > > >
> > > > I get this problem over here, but on rare occasions. Leaving it for
> > > > half a day usually fixes it. Have you tried a different rsync mirror?
> > > > You can use 'mirrorselect -i -r' for this task.
> > >
> > > I used to see this from time to time, but then I switched to using git
> > > instead of rsync and haven't seen it again. There's the added bonus that
> > > syncing is MUCH faster too.
> >
> > Last time I looked at git for portage it was not not using gpg to verify
> > the downloaded data - has this feature been added since then?
>
> I don't know when you last looked, but I see it checking the signature
> when downloading from git.
It was some inordinate time ago - can't recall when exactly. It was after
portage started using gpg with rsync and quarantined any fetched files after
they were downloaded until their signatures were validated. As I understood
at the time the difference with using git was it would only check the top
commit in the repository had a valid signature, but not what eventually landed
at the local PC. Also, even to do this check it required you have enabled
"sync-git-verify-commit-signature = yes" - but according to 'man portage' it
defaults to "no".
Anyhow, I recall using git to sync a live ebuild -9999 from some repo and
portage started downloading gigabytes of cruft. Presumably whole decades of
old commits I didn't need or have space for. I subsequently discovered I had
to set "EGIT_CLONE_TYPE=shallow" in make.conf, but I can't find this variable
in the man page now. I suppose/hope portage using git will only download more
recent commits?
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [gentoo-user] emerge --sync keeps failing
2021-08-03 8:55 ` Michael
@ 2021-08-03 10:49 ` Neil Bothwick
2021-08-03 10:51 ` Michael
0 siblings, 1 reply; 17+ messages in thread
From: Neil Bothwick @ 2021-08-03 10:49 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: text/plain, Size: 803 bytes --]
On Tue, 03 Aug 2021 09:55:46 +0100, Michael wrote:
> Anyhow, I recall using git to sync a live ebuild -9999 from some repo
> and portage started downloading gigabytes of cruft. Presumably whole
> decades of old commits I didn't need or have space for. I subsequently
> discovered I had to set "EGIT_CLONE_TYPE=shallow" in make.conf, but I
> can't find this variable in the man page now. I suppose/hope portage
> using git will only download more recent commits?
It does, you get just the current state of the tree by default, it's just
orders of magnitude faster, even compared with using rsync with a local
mirror. And I don't have to worry abut syncing too often and upsetting
infra as I'm syncing from github.
--
Neil Bothwick
Talk is cheap because supply exceeds demand.
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [gentoo-user] emerge --sync keeps failing
2021-08-03 10:49 ` Neil Bothwick
@ 2021-08-03 10:51 ` Michael
0 siblings, 0 replies; 17+ messages in thread
From: Michael @ 2021-08-03 10:51 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: text/plain, Size: 883 bytes --]
On Tuesday, 3 August 2021 11:49:23 BST Neil Bothwick wrote:
> On Tue, 03 Aug 2021 09:55:46 +0100, Michael wrote:
> > Anyhow, I recall using git to sync a live ebuild -9999 from some repo
> > and portage started downloading gigabytes of cruft. Presumably whole
> > decades of old commits I didn't need or have space for. I subsequently
> > discovered I had to set "EGIT_CLONE_TYPE=shallow" in make.conf, but I
> > can't find this variable in the man page now. I suppose/hope portage
> > using git will only download more recent commits?
>
> It does, you get just the current state of the tree by default, it's just
> orders of magnitude faster, even compared with using rsync with a local
> mirror. And I don't have to worry abut syncing too often and upsetting
> infra as I'm syncing from github.
I see, thanks Neil. Does it now check sigs of downloaded data, like rsync
does?
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [gentoo-user] emerge --sync keeps failing
2021-08-02 12:01 ` Michael
` (2 preceding siblings ...)
2021-08-02 21:17 ` Neil Bothwick
@ 2021-09-06 13:55 ` n952162
3 siblings, 0 replies; 17+ messages in thread
From: n952162 @ 2021-09-06 13:55 UTC (permalink / raw
To: gentoo-user
On 8/2/21 2:01 PM, Michael wrote:
> On Monday, 2 August 2021 12:10:12 BST n952162 wrote:
>> On 8/1/21 8:32 PM, Michael Orlitzky wrote:
>>> On Sun, 2021-08-01 at 17:32 +0200, n952162 wrote:
>>>> * Verifying /var/db/repos/gentoo/.tmp-unverified-download-quarantine
>>>>
>>>> ...!!! Manifest verification failed:
>>>> Manifest mismatch for metadata/news/Manifest
>>>>
>>>> I've raised this question before and the only useful answer I got was to
>>>> keep trying
>>> On the off chance that something is screwy on the remote end, you can
>>> always use "emerge-webrsync" and delay the problem until next time.
>>>
>>> Otherwise, I would say check "dmesg" for disk errors, but if it's
>>> happening on two machines that's a lot less likely.
>> I have this problem every month. Why does it fail? Is it just a
>> timeout because my network is slow? Can that be tweaked?
> I get this problem over here, but on rare occasions. Leaving it for half a
> day usually fixes it. Have you tried a different rsync mirror? You can use
> 'mirrorselect -i -r' for this task.
Unfortunately, mirrorselect doesn't seem to work anymore:
configparser.MissingSectionHeaderError: File contains no section headers.
It seems to have lost track of whether it's editing make.conf or repos.conf.
I ran it with the -o option, to a tempfile and tried to put that into
/etc/portage/repos.conf, I think, but that didn't work either.
Next, I manually editted /usr/share/portage/config/repos.conf and put
that line in where the old sync-uri was defined. But that didn't work
either.
Now, I've created a new directory /etc/portage/repos.conf/ and moved the
file generate by the -o of mirrorselect into that as gentoo.conf and
added a "[gentoo]" section head. It MIGHT be working now.
^ permalink raw reply [flat|nested] 17+ messages in thread