[gentoo-dev] [PATCH] install-qa-check.d: Support QA{,_STRICT}_INSTALL_PATHS variables (bug 670902)

[gentoo-dev] [PATCH] install-qa-check.d: Support QA{,_STRICT}_INSTALL_PATHS variables (bug 670902)

[Zac Medico]

The QA_INSTALL_PATHS variable exempts paths from "unexpected paths"
warnings generated by metadata/install-qa-check.d/08gentoo-paths.
If the QA_STRICT_INSTALL_PATHS variable is set then any exemptions
in QA_INSTALL_PATHS are ignored.

Since we probably don't want to whitelist /nix for all ebuilds,
this patch will allow us to whitelist it with a setting like this
in global scope of the nix ebuilds:

QA_INSTALL_PATHS=( /nix )

Bug: https://bugs.gentoo.org/670902
Signed-off-by: Zac Medico <zmedico@gentoo.org>
---
 metadata/install-qa-check.d/08gentoo-paths | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/metadata/install-qa-check.d/08gentoo-paths b/metadata/install-qa-check.d/08gentoo-paths
index 3ee887df08f..a3289b112dd 100644
--- a/metadata/install-qa-check.d/08gentoo-paths
+++ b/metadata/install-qa-check.d/08gentoo-paths
@@ -60,6 +60,27 @@ gentoo_path_check() {
 
 	${shopt_save}
 
+	if [[ ${#bad_paths[@]} -gt 0 && ${QA_INSTALL_PATHS} &&
+		${QA_STRICT_INSTALL_PATHS-unset} == unset ]]; then
+		local filtered_paths=()
+		local grep_args=()
+		local qa_install_paths
+		if [[ $(declare -p QA_INSTALL_PATHS) == "declare -a "* ]]; then
+			qa_install_paths=( "${QA_INSTALL_PATHS[@]}" )
+		else
+			set -f
+			qa_install_paths=( ${QA_INSTALL_PATHS} )
+			set +f
+		fi
+		for x in "${qa_install_paths[@]}"; do
+			grep_args+=( -e "^/${x#/}\$" )
+		done
+		while read -r -d ''; do
+			[[ ${REPLY} ]] && filtered_paths+=( "${REPLY}" )
+		done < <(printf -- '%s\0' "${bad_paths[@]}" | grep -zv "${grep_args[@]}")
+		bad_paths=( "${filtered_paths[@]}" )
+	fi
+
 	# report
 	# ------
 	if [[ -n ${bad_paths[@]} ]]; then
-- 
2.18.1

Re: [gentoo-dev] [PATCH] install-qa-check.d: Support QA{,_STRICT}_INSTALL_PATHS variables (bug 670902)

[Michael Orlitzky]

On 11/12/2018 03:33 PM, Zac Medico wrote:
> 
> QA_INSTALL_PATHS=( /nix )
> 

That really, really, really doesn't belong there.

Re: [gentoo-dev] [PATCH] install-qa-check.d: Support QA{,_STRICT}_INSTALL_PATHS variables (bug 670902)

[Zac Medico]

[-- Attachment #1.1: Type: text/plain, Size: 260 bytes --]

On 11/12/18 12:57 PM, Michael Orlitzky wrote:
> On 11/12/2018 03:33 PM, Zac Medico wrote:
>>
>> QA_INSTALL_PATHS=( /nix )
>>
> 
> That really, really, really doesn't belong there.

I'm open to suggestions for alternatives. Ideas?
-- 
Thanks,
Zac


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] [PATCH] install-qa-check.d: Support QA{,_STRICT}_INSTALL_PATHS variables (bug 670902)

[Michael Orlitzky]

On 11/12/2018 04:06 PM, Zac Medico wrote:
> On 11/12/18 12:57 PM, Michael Orlitzky wrote:
>> On 11/12/2018 03:33 PM, Zac Medico wrote:
>>>
>>> QA_INSTALL_PATHS=( /nix )
>>
>> That really, really, really doesn't belong there.
> 
> I'm open to suggestions for alternatives. Ideas?
> 

/var/lib/nix?

The idea being, to put it in the right place by default, and let people
override it with EXTRA_ECONF if they really want to download random
binaries from strangers and run them.

Re: [gentoo-dev] [PATCH] install-qa-check.d: Support QA{,_STRICT}_INSTALL_PATHS variables (bug 670902)

[Zac Medico]

[-- Attachment #1.1: Type: text/plain, Size: 1199 bytes --]

On 11/12/18 2:34 PM, Michael Orlitzky wrote:
> On 11/12/2018 04:06 PM, Zac Medico wrote:
>> On 11/12/18 12:57 PM, Michael Orlitzky wrote:
>>> On 11/12/2018 03:33 PM, Zac Medico wrote:
>>>>
>>>> QA_INSTALL_PATHS=( /nix )
>>>
>>> That really, really, really doesn't belong there.
>>
>> I'm open to suggestions for alternatives. Ideas?
>>
> 
> /var/lib/nix?
> 
> The idea being, to put it in the right place by default, and let people
> override it with EXTRA_ECONF if they really want to download random
> binaries from strangers and run them.

I recommend to add /nix to the whitelist because this is the default
location for all operating systems, as shown consistently throughout the
installation instructions found at
https://nixos.org/nix/manual/#chap-installation.

The nix manual also has this explicit warning in the "Building Nix from
Source" section found at https://nixos.org/nix/manual/#sec-building-source:

> Warning: It is best not to change the Nix store from its default,
since doing
> so makes it impossible to use pre-built binaries from the standard Nixpkgs
> channels — that is, all packages will need to be built from source.
-- 
Thanks,
Zac


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] [PATCH] install-qa-check.d: Support QA{,_STRICT}_INSTALL_PATHS variables (bug 670902)

[Michael Orlitzky]

On 11/12/2018 06:47 PM, Zac Medico wrote:
>>
>> The idea being, to put it in the right place by default, and let people
>> override it with EXTRA_ECONF if they really want to download random
>> binaries from strangers and run them.
> 
> I recommend to add /nix to the whitelist because this is the default
> location for all operating systems, as shown consistently throughout the
> installation instructions found at
> https://nixos.org/nix/manual/#chap-installation.

I mean... I know... my argument is not that they don't tell you to do
something dumb. If you really want the official experience, you can
close your eyes, cross your fingers, say a prayer to RMS, and then
follow their installation instructions:

 $ curl https://nixos.org/nix/install | sh

The fact that some people choose to use portage to install it probably
means that they were looking for something a little less yee-haw. We
trust the package manager to not let ebuilds do dumb things to our
systems: no surprise network access, no random filesystem reads/writes,
reliable uninstalls, things installed in sensible paths, etc. We
shouldn't make exceptions to those policies without a good reason.


> The nix manual also has this explicit warning in the "Building Nix from
> Source" section found at https://nixos.org/nix/manual/#sec-building-source:
> 
>> Warning: It is best not to change the Nix store from its default,
> since doing
>> so makes it impossible to use pre-built binaries from the standard Nixpkgs
>> channels — that is, all packages will need to be built from source.

Do I have to be that guy who suggests that if people don't want to build
from source, then maybe they took a wrong turn back at distrowatch?

You can override the nix store location with EXTRA_ECONF if you really
want to dump stuff in /nix. At which point the warning is just telling
you what's up: you're doing something wrong.

Re: [gentoo-dev] [PATCH] install-qa-check.d: Support QA{,_STRICT}_INSTALL_PATHS variables (bug 670902)

[Zac Medico]

[-- Attachment #1.1: Type: text/plain, Size: 2493 bytes --]

On 11/12/18 9:44 PM, Michael Orlitzky wrote:
> On 11/12/2018 06:47 PM, Zac Medico wrote:
>>>
>>> The idea being, to put it in the right place by default, and let people
>>> override it with EXTRA_ECONF if they really want to download random
>>> binaries from strangers and run them.
>>
>> I recommend to add /nix to the whitelist because this is the default
>> location for all operating systems, as shown consistently throughout the
>> installation instructions found at
>> https://nixos.org/nix/manual/#chap-installation.
> 
> I mean... I know... my argument is not that they don't tell you to do
> something dumb. If you really want the official experience, you can
> close your eyes, cross your fingers, say a prayer to RMS, and then
> follow their installation instructions:
> 
>  $ curl https://nixos.org/nix/install | sh
> 
> The fact that some people choose to use portage to install it probably
> means that they were looking for something a little less yee-haw.

The benefit of using the ebuild is the same as always, it allows for
automated upgrade, uninstall, etc.

> We
> trust the package manager to not let ebuilds do dumb things to our
> systems: no surprise network access, no random filesystem reads/writes,
> reliable uninstalls, things installed in sensible paths, etc. We
> shouldn't make exceptions to those policies without a good reason.

What does any of this have to do with the nix having a file store under
/nix?

>> The nix manual also has this explicit warning in the "Building Nix from
>> Source" section found at https://nixos.org/nix/manual/#sec-building-source:
>>
>>> Warning: It is best not to change the Nix store from its default,
>> since doing
>>> so makes it impossible to use pre-built binaries from the standard Nixpkgs
>>> channels — that is, all packages will need to be built from source.
> 
> Do I have to be that guy who suggests that if people don't want to build
> from source, then maybe they took a wrong turn back at distrowatch?

They're able to choose between building from source or using prebuilt
packages, and changing the location of the /nix file store is unnecessary.

> You can override the nix store location with EXTRA_ECONF if you really
> want to dump stuff in /nix. At which point the warning is just telling
> you what's up: you're doing something wrong.

What's inherently wrong about nix having a file store under /nix? Is
this purely about FHS?
-- 
Thanks,
Zac


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] [PATCH] install-qa-check.d: Support QA{,_STRICT}_INSTALL_PATHS variables (bug 670902)

[Michael Orlitzky]

On 11/13/2018 01:21 AM, Zac Medico wrote:
> 
> What's inherently wrong about nix having a file store under /nix? Is
> this purely about FHS?
> 

It goes against not only the FHS, but against our existing policies and
common sense. There's no reason to expect that path to even be writable.
And nix stores everything there, so if you have / mounted on a smaller
partition with e.g. /home and /var mounted elsewhere, then it's going to
fill the disk up with junk that emerge won't remove for you.

But it's also just absurd how much trouble we go to to wind up right
back where we started:

  * Spend years developing a FHS so that people can get the most out
    of their storage hardware and know where to find stuff on their
    computers.

  * Adopt a bunch of policies in Gentoo for where ebuilds should store
    things.

  * Add QA checks to ensure that packages abide by said policies.

  * Add package-manager-specific variables that sidestep the QA checks.

  * Begin adding exceptions to every package that puts things in the
    wrong place so that people don't see warnings about things being in
    the wrong place...

All to achieve the end result, that nothing is where it's supposed to be
and you can't plan anything.

I think I'm just tired. I'll stop.

Re: [gentoo-dev] [PATCH] install-qa-check.d: Support QA{,_STRICT}_INSTALL_PATHS variables (bug 670902)

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 777 bytes --]

On Tue, Nov 13, 2018 at 01:57:47AM -0500, Michael Orlitzky wrote:
> On 11/13/2018 01:21 AM, Zac Medico wrote:
> > 
> > What's inherently wrong about nix having a file store under /nix? Is
> > this purely about FHS?
> > 
> 
> It goes against not only the FHS, but against our existing policies and
> common sense. There's no reason to expect that path to even be writable.
> And nix stores everything there, so if you have / mounted on a smaller
> partition with e.g. /home and /var mounted elsewhere, then it's going to
> fill the disk up with junk that emerge won't remove for you.

I'm with Zac on this, we should install the way upstream directs, or if
we have that strong of an aversion to it, we should open a bug upstream
to fix it.

Thanks,

William

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]