[gentoo-dev] [RFC] New Manifest hashes and how to enable them

[gentoo-dev] [RFC] New Manifest hashes and how to enable them

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 3947 bytes --]

Hi, everyone.

I'd like to open an early discussion and start planning transition to
an updated set of Manifest hashes.


Current state
=============

The current hash set includes the three following hashes:
- SHA256 (SHA2),
- SHA512 (SHA2),
- Whirlpool.

Of these three hashes, SHA256 is considered 'required' by Portage. All
current Manifests in Gentoo include it.

SHA512 and Whirlpool are included in the majority of Manifest.
The packages missing them are tracked in [1].

There are still some stale hashes present in some Manifests (e.g.
RMD160).


Supported hash algorithms in package managers
=============================================

The following hashes are supported by the stable version of Portage,
pkgcore and Paludis: MD5, SHA1, SHA256, SHA512, RMD160, WHIRLPOOL.

Support for MD5, SHA1, SHA256 and SHA512 in Portage and pkgcore is
provided unconditionally by all supported Python versions. For
WHIRLPOOL, a fallback implementation is always provided.

The following hashes are supported by the current git version of Portage
and are on the way of being integrated into pkgcore: BLAKE2B, BLAKE2S,
SHA3-256, SHA3-512. They are not currently supported by Paludis.

Support for those algorithms is guaranteed in Python 3.6+. Fallbacks for
older Python versions are supported -- using pygcrypt, pycryptodome
and pysha3 (the last one doesn't include BLAKE2) in Portage; and using
pycryptodome in pkgcore.

Additionally, the current git version of Portage supports
the STREEBOG256 and STREEBOG512 (GOST R 34.11-2012) algorithms. They can
be provided either using pygcrypt or pygost modules.

Of the listed fallbacks, the following limitations need to be noted:

A. pygcrypt requires old cffi, and therefore is not in ::gentoo. I have
contacted upstream and they are working on updating it.

B. pycryptodome blocks pycrypto, and some packages explicitly require
pycrypto (see tracker at [2]).

C. pygost is just horrible code-wise, and I've only added it so that we
can run tests without any special dependencies.


SHA256 deprecation
==================

I think the first reasonable change would be to deprecate SHA256. It is
pretty much the same algorithm as SHA512, except for different
parameters. It is weaker than SHA512, and SHA512 is supported on all
existing platforms anyway.

Therefore, I think the following procedure would apply (please correct
me if I'm wrong):

1. wait till all remaining blockers of [1] are fixed,

2. update the required hash from SHA256 to SHA512 in Portage,

3. wait till stable Portage carries the above change + (possibly some
upgrade time?),

4. remove SHA256 from list of hashes included in Manifests.


New hash set
============

After deprecating SHA256, the hash list would include only SHA512
and WHIRLPOOL. I think the first one should be kept as a portable
required hash, and I'm indifferent to keeping the second one.

Of the new hashes, I think it would be the most reasonable to add SHA3-
512. It is built-in since Python 3.6+, and multiple implementations are
available for older Python versions (cryptography [though I think it
doesn't work with current openssl], pycryptodome, pygcrypt, pysha3).

The alternative would be to use BLAKE2B hash (which provides similar
strength to SHA3-512). However, it provides less fallbacks (no support
in libgcypt nor cryptography).

For diversity, we could also consider including the Streebog hash
(a user has requested it already). However, it seems to be at SHA2 level
and does not have any reasonable implementation except for libgcrypt.

Therefore, my proposal would be to use the following set once their
support reaches the stable version of Portage:

  manifest-hashes = SHA512 SHA3-512 WHIRLPOOL


Your thoughts?



[1]:https://bugs.gentoo.org/612716
[2]:https://bugs.gentoo.org/611568

-- 
Best regards,
Michał Górny

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] [RFC] New Manifest hashes and how to enable them

[Matthias Maier]

[-- Attachment #1: Type: text/plain, Size: 745 bytes --]

>   manifest-hashes = SHA512 SHA3-512 WHIRLPOOL
>
> Your thoughts?

I just want to point out that according to GLEP 63 we only require pgp
signatures with at least sha-256 [1]. Further, our PGP signatures by the
release team are as well either SHA-256/SHA-512.

So using SHA3-512 (or whirlpool for that matter) is nice but it feels a
bit like overdoing it a bit. What about simply SHA512 and calling it a
day?

Further, it might be a good time to finally resolve the issue with our
rsync integrity for users. (What is the gain of using a secure hash
algorithm in the manifests if you can simply replace the manifest with a
MITM attack on the rsync update?)

Best,
Matthias

[1] https://wiki.gentoo.org/wiki/GLEP:63#Specifications_for_GnuPG_keys

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 850 bytes --]

Re: [gentoo-dev] [RFC] New Manifest hashes and how to enable them

[Vadim A. Misbakh-Soloviov]

Good idea, but all the time I read it from first mention until the end of your 
email, I asked myself: "Who the hell on the Earth need GOST-crypto crap in 
portage?".

The only purpose of this crypto algorythms is to use them in Russian 
government-related structures (includig schools, tho :-/ ) just because of 
"security through NIH-syndrome".

So, does it mean, gentoo was (or have plans to be) certified by FSB to be 
"Russian national OS", or how is it possible on Earth that GOST algos was 
implemented in portage? :)



So, excluding this moment, I voting up for your suggestion ;)

Re: [gentoo-dev] [RFC] New Manifest hashes and how to enable them

[Vadim A. Misbakh-Soloviov]

> What is the gain of using a secure hash
> algorithm in the manifests if you can simply replace the manifest with a
> MITM attack on the rsync update?

I'd say "the solution is to stop using rsync and use git" (there is git mirror 
with all the metadata), but...
Git does not support (correct me, if I'm wrong) resuming a fetch in case of 
fails (bad connection, slow connection, or the any other reason to stop it and 
continue later).

So... We either need GPG manifest signing enabled, or totally move to git and 
ignore all the users with bad internet connection and totally move portage to 
git (hint: we shouldn't), until we invent something else, that can solve all 
of that problems.

Re: [gentoo-dev] [RFC] New Manifest hashes and how to enable them

[David Seifert]

On Mon, 2017-04-03 at 19:09 +0200, Michał Górny wrote:
> Therefore, my proposal would be to use the following set once their
> support reaches the stable version of Portage:
> 
>   manifest-hashes = SHA512 SHA3-512 WHIRLPOOL
> 
> 
> Your thoughts?
> 
> 
> 
> [1]:https://bugs.gentoo.org/612716
> [2]:https://bugs.gentoo.org/611568
> 
> -- 
> Best regards,
> Michał Górny

+5 Let's not repeat the git mistake of delaying this until it's too
late.

Re: [gentoo-dev] [RFC] New Manifest hashes and how to enable them

[Robin H. Johnson]

On Tue, Apr 04, 2017 at 12:49:16AM +0700, Vadim A. Misbakh-Soloviov wrote:
> > What is the gain of using a secure hash
> > algorithm in the manifests if you can simply replace the manifest with a
> > MITM attack on the rsync update?
> I'd say "the solution is to stop using rsync and use git" (there is git mirror 
> with all the metadata), but...
> Git does not support (correct me, if I'm wrong) resuming a fetch in case of 
> fails (bad connection, slow connection, or the any other reason to stop it and 
> continue later).
Upstream is still working on resumable fetch, and if you need it
already, it can be had via git bundles.

-- 
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Trustee & Treasurer
E-Mail   : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

Re: [gentoo-dev] [RFC] New Manifest hashes and how to enable them

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 774 bytes --]

On wto, 2017-04-04 at 00:32 +0700, Vadim A. Misbakh-Soloviov wrote:
> Good idea, but all the time I read it from first mention until the end of your 
> email, I asked myself: "Who the hell on the Earth need GOST-crypto crap in 
> portage?".
> 
> The only purpose of this crypto algorythms is to use them in Russian 
> government-related structures (includig schools, tho :-/ ) just because of 
> "security through NIH-syndrome".
> 
> So, does it mean, gentoo was (or have plans to be) certified by FSB to be 
> "Russian national OS", or how is it possible on Earth that GOST algos was 
> implemented in portage? :)
> 

I think the point was more like it's unfair to give NSA backdoors
without giving one to KGB as well.

-- 
Best regards,
Michał Górny

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] [RFC] New Manifest hashes and how to enable them

[Dirkjan Ochtman]

On Mon, Apr 3, 2017 at 7:09 PM, Michał Górny <mgorny@gentoo.org> wrote:
> Your thoughts?

This seems pretty hasty.

First of all, SHA-256 should be safe for all intents and purposes, and
for the foreseeable future. This is nothing like Git's usage of SHA-1,
which was known to be on the way to brokenville for a long time. I
don't think there is a solid reason for deprecating it now.

Second, the amount of diversity proposed does not make sense. If
asked, I would propose we keep SHA-256 as one of the options and
additionally add a SHA3 variant and a BLAKE2 variant as other options.
This would provide more than enough diversity. Also totally agreed
with Vadim on the obscurity of the GOST algorithms.

But, this is the kind of thing where we really should get input from
the Security project, so we should get people like Hanno and Kristian
involved.

Third, I don't much trust the security record of the python libraries
mentioned. cryptography is the best Python library for crypto by far,
and I think we should use it exclusively for anything Python doesn't
provide in the stdlib. The PyCrypto security record is not exactly
stellar IIRC, and since pycryptodome is a fork of it, I don't trust it
that much, either.

But mainly, please, I think we should leave the security-sensitive
decisions to people with more security expertise.

Cheers,

Dirkjan

Re: [gentoo-dev] [RFC] New Manifest hashes and how to enable them

[Hanno Böck]

Hi,

On Mon, 3 Apr 2017 22:00:15 +0200
Dirkjan Ochtman <djc@gentoo.org> wrote:

> First of all, SHA-256 should be safe for all intents and purposes, and
> for the foreseeable future. This is nothing like Git's usage of SHA-1,
> which was known to be on the way to brokenville for a long time. I
> don't think there is a solid reason for deprecating it now.
> 
> Second, the amount of diversity proposed does not make sense. If
> asked, I would propose we keep SHA-256 as one of the options and
> additionally add a SHA3 variant and a BLAKE2 variant as other options.
> This would provide more than enough diversity. Also totally agreed
> with Vadim on the obscurity of the GOST algorithms.
> 
> But, this is the kind of thing where we really should get input from
> the Security project, so we should get people like Hanno and Kristian
> involved.

As you specifically asked for my opinion:
I think there's no reason to doubt the security of any of the sha2
hashes (including sha256), any of sha3 or blake2 for the forseeable
future. (That means counting in many decades - there isn't even a shred
of evidence sha256 is going to be broken any time soon.)
There's no point in deprecating anything.

I find it unnecessary to introduce additional complexity here and
adding obscurity algorithms like gost sounds really bizarre and
unnecessary. I'd recommend against introducing anything that
requires unusual dependencies.
If anything I'd propose to just change to a single hash functio

-- 
Hanno Böck
https://hboeck.de/

mail/jabber: hanno@hboeck.de
GPG: FE73757FA60E4E21B937579FA5880072BBB51E42

Re: [gentoo-dev] [RFC] New Manifest hashes and how to enable them

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 1053 bytes --]

>>>>> On Mon, 3 Apr 2017, Dirkjan Ochtman wrote:

> This seems pretty hasty.

> First of all, SHA-256 should be safe for all intents and purposes,
> and for the foreseeable future. This is nothing like Git's usage of
> SHA-1, which was known to be on the way to brokenville for a long
> time. I don't think there is a solid reason for deprecating it now.

Hasty? The plan has been outlined already 7 years ago, in GLEP 59 [1].

IIUC, it is enough to keep the strongest checksum of a given set,
namely SHA512 for SHA-2. The increase in security by keeping a second
weaker checksum of the same family (i.e. SHA256) is negligible.

So the only reason for keeping SHA256 was that old Portage versions
didn't support SHA512. However, by now SHA512 is supported since more
than 5 years, namely since Portage version 2.1.10.44 which went stable
on the last arch in February 2012.

I don't have a strong opinion if we should add SHA-3, but I don't
think that it is urgently needed.

Ulrich

[1] https://wiki.gentoo.org/wiki/GLEP:59#Checksum_depreciation_timing

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] [RFC] New Manifest hashes and how to enable them

[Kristian Fiskerstrand]

[Sent from my iPad, as it is not a secured device there are no cryptographic keys on this device, meaning this message is sent without an OpenPGP signature. In general you should *not* rely on any information sent over such an unsecure channel, if you find any information controversial or un-expected send a response and request a signed confirmation]

> On 3 Apr 2017, at 18:09, Michał Górny <mgorny@gentoo.org> wrote:
> 

> Therefore, my proposal would be to use the following set once their
> support reaches the stable version of Portage:
> 
>  manifest-hashes = SHA512 SHA3-512 WHIRLPOOL
> 
> 
> Your thoughts?
> 

SHA256 is perfectly fine to use from a security perspective, so no need to do anything from that point of view. The big difference between SHA256 and SHA512 is performance, you have significant gains of using sha256 on 32 bit architectures, whereby SHA512 is quite fine when having 64 bit registers. SHA512 is well-tested and already part of package managers etc, so I dont really have too strong opinions on making it mandatory and allow for sha256 to be replaced, as long as it is clear that it isn't required from a strict security view.

As for SHA3 introduction, how well tested is the implementation used by the package managers, what are performance metrics etc? We don't really need this atm, but nice to have it in the package managers as a backup if that was to change, but should not be required digest algo

(and yes, we really need to give Gentoo Keys all the help that we can in getting the OpenPGP signing ready, everything else is just bikeshedding until that is in place and it is a making me rather sad that we haven't managed to do this already)

Re: [gentoo-dev] [RFC] New Manifest hashes and how to enable them

[Chí-Thanh Christopher Nguyễn]

Michał Górny schrieb:
> I think the first reasonable change would be to deprecate SHA256. It is
> pretty much the same algorithm as SHA512, except for different
> parameters. It is weaker than SHA512, and SHA512 is supported on all
> existing platforms anyway.

I think there is nothing wrong or insecure with continuing to use 
SHA256, even though it is technically weaker than SHA512. If it is 
already included in all Manifests then keeping it as standard is 
preferable I think.

Some people consider having a second dissimilar algorithm at hand a good 
idea. I suggest SHA3 in that case.

manifest-hashes = SHA256 SHA3-256


Best regards,
Chí-Thanh Christopher Nguyễn