From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 6E293139660 for ; Sat, 29 Oct 2016 10:11:28 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 3250321C082; Sat, 29 Oct 2016 10:11:22 +0000 (UTC) Received: from mail-qk0-f169.google.com (mail-qk0-f169.google.com [209.85.220.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id CD6CFE09FF for ; Sat, 29 Oct 2016 10:11:21 +0000 (UTC) Received: by mail-qk0-f169.google.com with SMTP id x11so65988473qka.1 for ; Sat, 29 Oct 2016 03:11:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to; bh=7E+sMggE+BCSjLuWHTW4OD+SgVLVzw7Sf8wSd7w/LyM=; b=VP5su/XUnI5HbwiReBO8X6aVunnI60dLC1SCV3+0FoHvWnFAZGB23305vQD0iymG3H spj33ZaRvTD2ciEQbDbN4ww6JYrsuQGnhVWFGKp3mXdOVj1AnpJChSXOJ3uU0UbN1NqZ 6ziPCQODX3TNG8cQox+I9GOEzxw44+7QtVVK+yhMmNsUSufCOfgXv29Fkg11Jv1HsK1S Y8Ru/3ytCUdg9oJCV9EcvrALRXgj6njcC6TgPLT5oPcMva78hU/CNjICiBNNvedbaZVF MZY9iBLRYQt06DWdidd0LIdEr+gSUb7U8yGbiVkJnZi/XkzzxiZt1ZzZ+YAzbzJDBWBq Sttw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to; bh=7E+sMggE+BCSjLuWHTW4OD+SgVLVzw7Sf8wSd7w/LyM=; b=IeZHmLiB612EfDCGrF1OXFt9V7QRbHNnhhK7EfWY/Gc7R3HVE1xNivauVM9zp5xLHL /tmHI6zhJPTCSoC4ivaEE4iA4hSP3LAD4l2eVZfoC5Ozo9npI/WrbzSnDosAdAsMzJaA sG0f4FEiUNIwmWEk0Q4q3zQhMyUUfGGWe3GR2U3QymwwxEvLSeXRYNUZDtjpouQGdr5V LTPligOJh9oten+q85dzJqV6RvzTHjkQnxmEGUGAkFfOtMSxT/AbOBkVdXaRm4lCX2tE 3XNhYifWchANwHrB9OEJ4SxOUViDs5lKa6/Dh7vOLHJv2JCsQGlAp0gXbLSkdW1RYtk3 MXsg== X-Gm-Message-State: ABUngvelJ+pjXuvvfIrn/WWNmoQyW3PAlQqgoTW5GqySmCKbjiDsaOCtPuuKRxbNENlqH1fUaT6vu9tk9uk06Q== X-Received: by 10.55.69.73 with SMTP id s70mr5088623qka.161.1477735880850; Sat, 29 Oct 2016 03:11:20 -0700 (PDT) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org MIME-Version: 1.0 Sender: freemanrich@gmail.com Received: by 10.140.89.83 with HTTP; Sat, 29 Oct 2016 03:11:20 -0700 (PDT) In-Reply-To: <20161029104236.3896aaf9@digimed.co.uk> References: <3085e41f-ea80-f807-0273-ee789c63c266@wht.com.au> <20161029104236.3896aaf9@digimed.co.uk> From: Rich Freeman Date: Sat, 29 Oct 2016 06:11:20 -0400 X-Google-Sender-Auth: DKkajfuHGh8pHueBn44Wz1nU7Qk Message-ID: Subject: Re: [gentoo-user] Possible heads up: Digest verification failed To: gentoo-user@lists.gentoo.org Content-Type: text/plain; charset=UTF-8 X-Archives-Salt: c2382d17-c0da-433a-a4d8-07eba54cb70e X-Archives-Hash: 73d861549b655c13a68e46bceb1c12f4 On Sat, Oct 29, 2016 at 5:42 AM, Neil Bothwick wrote: > On Fri, 28 Oct 2016 23:53:03 -0700, Daniel Campbell wrote: > >> > Anyone seeing this again? I've just sync'd to two servers in >> > Australia, and then, for the hell of it, one in Canada and am getting >> > it for: >> > >> > dev-libs/botan >> > app-arch/tar >> > media-video/libav >> > app-crypt/qca >> > net-print/cups-filters >> > >> > I suppose time will sort it out..... >> > >> > Andrew >> > >> This shouldn't happen unless the distfiles aren't found, or someone (a >> dev) didn't use repoman to commit, leaving an old Manifest around. > > It looks like repoman is the culprit > > https://bugs.gentoo.org/show_bug.cgi?id=598376 > This is probably not the issue here, since Gentoo uses thin manifests (there is nothing for repoman to update). The manifests that are causing the problem aren't created by regular Gentoo developers. They're created by a script that runs as a part of the rsync mirror process. This is a fairly well-known problem that has been around for over a year. You will only run into this problem if you use rsync to update your repository, since the problem is created when creating the master rsync mirror. The original git repository doesn't contain the error, and the git mirror on github doesn't mess with the Manifests. The issue apparently has to do with Changelog generation. In April the Council gave Infra the option to stop generating Changelogs, which would eliminate the problem. I suspect those maintaining the scripts prefer to keep them around, and I don't think anybody on the Council has access to change the scripts. I switched to git syncing eons ago, so I've never seen this bug. I recognize it has been a source of frustration for a lot of users, and a bit of frustration for the Council, since there doesn't seem to be a lot we can do to change it in practice. zlg is of course right that these kinds of problems can also be caused by maintainer failure to use repoman/etc or if an upstream distfile changes. If that is the problem then you'll see it no matter how you sync your repo. However, when you get a bunch of these after syncing it is almost always a result of the mirror creation process. I can't remember the last time I saw a manifest error (granted, I'm also using mgorny's stable mirror branch, which I think screens for these kinds of errors). While there can be some latency I do in general recommend syncing from https://github.com/gentoo-mirror/gentoo . This is a mirror of the Gentoo developer git repository with two changes: 1. Metadata is added to the mirror, which greatly speeds things up compared to using the raw git repository (you can do this yourself, it is one of the steps done by the rsync generation process as well, but this one is not buggy). 2. The default stable branch of this mirror screens for numerous issues before accepting commits. That means it is generally a little behind the main branch (at this moment the main branch is 2 minutes old, and the default stable branch is 20min old), but a lot of the really annoying issues that are caused by devs skipping repoman won't be seen. Now, if a maintainer breaks a package then this mirror will quickly get out of date until the problem is corrected, but Gentoo QA gets warnings when this is happening and usually the maintainer is being pestered or somebody else is fixing it. I suspect this process has probably reduced the error rate for everybody. I have seen this get a few days old though, which is something to keep in mind. It does not contain Changelogs, though if you use it you'll have a full history so you can just run git whatchanged to get something pretty close to a changelog. To use it just put this in /etc/portage/repos.conf/gentoo.conf: [DEFAULT] main-repo = gentoo [gentoo] location = /usr/portage sync-type = git sync-uri = https://github.com/gentoo-mirror/gentoo.git auto-sync = yes If you want to git-sync from some other mirror, just change the url accordingly. If you switch your mirror I suggest just renaming /usr/portage and letting portage re-create it. The other big benefit of git syncing is that if you sync every day it is a lot faster. If you sync less often it will become slower compared to rsync. git is much more efficient at finding what has changed, but rsync is not burdened with transferring a complete history. If you only sync once every few months rsync will be a lot faster. -- Rich