* Re: [gentoo-dev] Need GitHub snapshot hash verification failure samples
2017-07-05 19:48 [gentoo-dev] Need GitHub snapshot hash verification failure samples Michał Górny
2017-07-05 22:30 ` Vadim A. Misbakh-Soloviov
@ 2017-07-06 20:15 ` Sergei Trofimovich
1 sibling, 0 replies; 4+ messages in thread
From: Sergei Trofimovich @ 2017-07-06 20:15 UTC (permalink / raw
To: Michał Górny; +Cc: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 6451 bytes --]
On Wed, 05 Jul 2017 21:48:12 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> Hi, everyone.
>
> I've seen multiple bugs related to hash verification failures for GitHub
> snapshots lately. However, none of the maintainers have been so far able
> to provide me with a sample of the old and new snapshot for comparison,
> so we still have no clue what's happening exactly.
>
> if you see your package failing or get a report for it, then *please*
> save the original tarball before replacing it with the new one and send
> me both for comparison. Thank you.
Sounds easy to verify.
1. grab all the github tarballs (should be a better way to do it with proper USE expansiion):
$ egrep -R 'SRC_URI.*github.com' metadata/ | grep -o '[^/ ]*$' | sort -u > github_distfiles.list
2. grab all manifest files that look like defining these files and remove them locally:
$ git grep -l -F -f ./github_distfiles.list | grep -F /Manifest | xargs rm -v
3. Refetch distfiles from internets:
$ mkdir /tmp/fresh
$ GENTOO_MIRRORS= DISTDIR=/tmp/fresh repoman manifest
As a result each 'git diff' report is your potential candidate.
You have new file in /tmp/fresh/<file>
and old one on http://distfiles.gentoo.org/distfiles/<file>
A few samples:
--- a/app-admin/qtpass/Manifest
+++ b/app-admin/qtpass/Manifest
@@ -1,4 +1,4 @@
-DIST qtpass-1.0.5.tar.gz 636461 SHA256 0c07bd1eb9e5336c0225f891e5b9a9df103f218619cf7ec6311edf654e8db281
-DIST qtpass-1.1.0.tar.gz 671525 SHA256 60b458062f54184057e55dbd9c93958a8bf845244ffd70b9cb31bf58697f0dc6
+DIST qtpass-1.0.5.tar.gz 636457 SHA256 b9f1c1ecf4afbe716915792ff692e7114568de5bd8c47750d5c8404aa28699e7
+DIST qtpass-1.1.0.tar.gz 671537 SHA256 f2fff7922902c4c118e04164c078ca80e9a28221320b4253d3117d885e8417b6
diffoscope reports case change only in root dir name:
$ diffoscope old/qtpass-1.1.0.tar.gz new/qtpass-1.1.0.tar.gz
│ │ @@ -1,83 +1,83 @@
│ │ -drwxrwxr-x 0 root (0) root (0) 0 2016-01-25 09:58:18.000000 qtpass-1.1.0/
│ │ +drwxrwxr-x 0 root (0) root (0) 0 2016-01-25 09:58:18.000000 QtPass-1.1.0/
...
I guess somebody decided to rename github repo slightly.
Both files are at:
http://dev.gentoo.org/~slyfox/unstable_tarballs/old/qtpass-1.1.0.tar.gz
http://dev.gentoo.org/~slyfox/unstable_tarballs/new/qtpass-1.1.0.tar.gz
--- a/app-crypt/acme/Manifest
+++ b/app-crypt/acme/Manifest
@@ -1,3 +1,3 @@
DIST certbot-0.14.1.tar.gz 851705 SHA256 7992fced742649e7b7668e4db7685de12248a4ffba66810cb336e9b6412e3567
DIST certbot-0.15.0.tar.gz 942788 SHA256 87d306b1c013b472b8f548b38ccc476c125816435bb3b99e932fed09ac777296
-DIST letsencrypt-0.1.0.tar.gz 524821 SHA256 1c1ac7b41e5e0fc0e41a7ef159ac9147a4aafff54453d57b519eb05bf52ade14
+DIST letsencrypt-0.1.0.tar.gz 524854 SHA256 3ba1add217fc1665ad1d3c4812c0de60590f406cb83d6514332898ab60b26f62
$ diffoscope old/letsencrypt-0.1.0.tar.gz new/letsencrypt-0.1.0.tar.gz
│ │ @@ -1,579 +1,579 @@
│ │ -drwxrwxr-x 0 root (0) root (0) 0 2015-12-02 23:55:43.000000 letsencrypt-0.1.0/
│ │ +drwxrwxr-x 0 root (0) root (0) 0 2015-12-02 23:55:43.000000 certbot-0.1.0/
Same thing.
http://dev.gentoo.org/~slyfox/unstable_tarballs/old/letsencrypt-0.1.0.tar.gz
http://dev.gentoo.org/~slyfox/unstable_tarballs/new/letsencrypt-0.1.0.tar.gz
Zip file!
--- a/app-crypt/etcd-ca/Manifest
+++ b/app-crypt/etcd-ca/Manifest
@@ -1,2 +1,2 @@
-DIST etcd-ca-0_p20140903.zip 1178338 SHA256 5da9f7afad6dd373d96c5d36dd30e9f43cfc8fc2359bbf2d0c6a864fff139f81
+DIST etcd-ca-0_p20140903.zip 1178338 SHA256 7ef6b7f34324bd4b48b369990a7eb70e30809240f3c3d97b7d56d021af3f43f3
$ diffoscope old/etcd-ca-0_p20140903.zip new/etcd-ca-0_p20140903.zip
│ drwx--- 0.0 fat 0 bx stor 14-Sep-03 21:30 etcd-ca-812f3626796be16d9db052720ce9c54f5a40bb26/
│ --rw---- 0.0 fat 24 bx stor 14-Sep-03 21:30 etcd-ca-812f3626796be16d9db052720ce9c54f5a40bb26/.gitconfig
│ --rw---- 0.0 fat 3924 bx defN 14-Sep-03 21:30 etcd-ca-812f3626796be16d9db052720ce9c54f5a40bb26/CONTRIBUTING.md
│ +-rw---- 0.0 fat 24 tx stor 14-Sep-03 21:30 etcd-ca-812f3626796be16d9db052720ce9c54f5a40bb26/.gitconfig
│ +-rw---- 0.0 fat 3924 tx defN 14-Sep-03 21:30 etcd-ca-812f3626796be16d9db052720ce9c54f5a40bb26/CONTRIBUTING.md
Here contents didn't change but zip compressor decided to pick different file type (bx/tx is binary/text).
http://dev.gentoo.org/~slyfox/unstable_tarballs/old/etcd-ca-0_p20140903.zip
http://dev.gentoo.org/~slyfox/unstable_tarballs/new/etcd-ca-0_p20140903.zip
--- a/app-emacs/lua-mode/Manifest
+++ b/app-emacs/lua-mode/Manifest
@@ -1 +1 @@
-DIST lua-mode-20130419.tar.gz 26236 SHA256 75c1696421983fbb58946ea649d2917f0deefc8b4f1dbc16b819e0cd603e396a
+DIST lua-mode-20130419.tar.gz 26242 SHA256 7a5e1a21e53aeab6e7cad8c616f6b026fd32f414bc6a32371e04d4e7424800c7
This one is different. Tag expansion changed (on GitHub's side?):
$ diffoscope old/lua-mode-20130419.tar.gz new/lua-mode-20130419.tar.gz | lv
│ ├── lua-mode-rel-20130419/lua-mode.el
│ │ @@ -31,15 +31,15 @@
│ │ ;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
│ │ ;; MA 02110-1301, USA.
│ │
│ │ ;; Keywords: languages, processes, tools
│ │
│ │ ;; This field is expanded to commit SHA, date & associated heads/tags during
│ │ ;; archive creation.
│ │ -;; Revision: 040bc8f (Fri, 19 Apr 2013 11:27:32 +0400 (rel-20130419))
│ │ +;; Revision: 040bc8f (Fri, 19 Apr 2013 11:27:32 +0400 (tag: rel-20130419))
│ │ ;;
│ │
│ │ ;;; Commentary:
│ │
│ │ ;; Thanks to d87 <github.com/d87> for an idea of highlighting lua
│ │ ;; builtins/numbers
│ │
│ ╵
http://dev.gentoo.org/~slyfox/unstable_tarballs/old/lua-mode-20130419.tar.gz
http://dev.gentoo.org/~slyfox/unstable_tarballs/new/lua-mode-20130419.tar.gz
--- a/app-emulation/docker/Manifest
+++ b/app-emulation/docker/Manifest
@@ -1,3 +1,3 @@
-DIST docker-17.03.1.tar.gz 7773296 SHA256 a8f1eefadf3966885ad0579facfc2017cca7dd3a0b20d086dfd798168716cb83
+DIST docker-17.03.1.tar.gz 7773988 SHA256 411e32ee388ad6d99479b97a3937c851bd84dacf4267be9d5501665e468e148e
$ diffoscope old/docker-17.03.1.tar.gz new/docker-17.03.1.tar.gz
--
Sergei
[-- Attachment #2: Цифровая подпись OpenPGP --]
[-- Type: application/pgp-signature, Size: 195 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread