[gentoo-dev] RFC: new category for container related packages, instead of app-emulation

[gentoo-dev] RFC: new category for container related packages, instead of app-emulation

[Georgy Yakovlev]

[-- Attachment #1: Type: text/plain, Size: 2723 bytes --]

Hi,

We've been collecting more and more container related packages in
 app-emulation/*

What do you think about finally moving those packages to separate category?

probably app-containers/

Here's the tentative list, most tools have description attached for easier
review, 36 candidates so far, there may be more around, I only looked at
app-emulation.

app-emulation/buildah - A tool that facilitates building OCI images
app-emulation/cadvisor - container analyzer
app-emulation/conmon - An OCI container runtime monitor
app-emulation/containerd - A daemon to control runC
app-emulation/containers-storage - containers/storage library
app-emulation/cri-o - OCI-based implementation of Kubernetes CRI 
app-emulation/cri-tools - CLI and validation tools for CRI
app-emulation/crun - OCI Container Runtime fully written in C
app-emulation/distrobuilder - System container image builder for LXC and LXD
app-emulation/docker - Main offender
app-emulation/docker-bench-security - Test for best docker practices
app-emulation/docker-cli - Main offender cli
app-emulation/docker-compose - Multi-container orchestration for Docker
app-emulation/docker-credential-helpers -
app-emulation/docker-gc - Docker garbage collection of containers and images
app-emulation/docker-proxy - Docker container networking
app-emulation/docker-registry -
app-emulation/docker-swarm -
app-emulation/flannel - An etcd backed network fabric for containers
app-emulation/go-secbench - run and evaluate the docker security benchmark
app-emulation/img - Standalone Dockerfile and OCI container image builder
app-emulation/k3d - creates k3s clusters in docker
app-emulation/kompose - Tool to move from docker-compose to Kubernetes
app-emulation/lxc - A userspace interface for the Linux kernel containment
app-emulation/lxc-templates -
app-emulation/lxd - Fast, dense and secure container management
app-emulation/nerdctl - Docker-compatible CLI for containerd
app-emulation/podman - Main offender killer
app-emulation/reg - Docker registry v2 command line client
app-emulation/runc - another docker thing
app-emulation/s6-overlay - an s6-based init system for containers
app-emulation/sen - Terminal User Interface for docker engine
app-emulation/skopeo - Utility for operations on container images/repositories
app-emulation/slirp4netns - User-mode networking for unpriv network namespaces
app-emulation/snapd - Service and tools for management of snap packages
app-emulation/umoci - Manipulation tool for OCI images

Those 4 are technically emulation related, so I'm not sure about category:
app-emulation/docker-machine
app-emulation/docker-machine-kvm
app-emulation/hyperd
app-emulation/runv

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 862 bytes --]

Re: [gentoo-dev] RFC: new category for container related packages, instead of app-emulation

[Aaron Bauman]

[-- Attachment #1: Type: text/plain, Size: 353 bytes --]

On Thu, Aug 05, 2021 at 02:44:40PM -0700, Georgy Yakovlev wrote:
> Hi,
> 
> We've been collecting more and more container related packages in
>  app-emulation/*
> 
> What do you think about finally moving those packages to separate category?
> 

This would be a more accurate categorization of the packages and I am in
favor of it.

-Aaron

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [gentoo-dev] RFC: new category for container related packages, instead of app-emulation

[Alec Warner]

On Thu, Aug 5, 2021 at 2:44 PM Georgy Yakovlev <gyakovlev@gentoo.org> wrote:
>
> Hi,
>
> We've been collecting more and more container related packages in
>  app-emulation/*
>
> What do you think about finally moving those packages to separate category?

As always my opinion is that:

(a) Categories were a design mistake.
(b) The mistake is hard to fix.
(c) It's basically low-value to try to 'correctly' categorize packages
because of A.
(d) Recategorizing means a bunch of stuff has to be updated.

Do people actually care what category things are in? I just use
--search or eix or whatever and the category is just this...bad
concept we attach to packages for silly historical reasons..

-A

> probably app-containers/
>
> Here's the tentative list, most tools have description attached for easier
> review, 36 candidates so far, there may be more around, I only looked at
> app-emulation.
>
> app-emulation/buildah - A tool that facilitates building OCI images
> app-emulation/cadvisor - container analyzer
> app-emulation/conmon - An OCI container runtime monitor
> app-emulation/containerd - A daemon to control runC
> app-emulation/containers-storage - containers/storage library
> app-emulation/cri-o - OCI-based implementation of Kubernetes CRI
> app-emulation/cri-tools - CLI and validation tools for CRI
> app-emulation/crun - OCI Container Runtime fully written in C
> app-emulation/distrobuilder - System container image builder for LXC and LXD
> app-emulation/docker - Main offender
> app-emulation/docker-bench-security - Test for best docker practices
> app-emulation/docker-cli - Main offender cli
> app-emulation/docker-compose - Multi-container orchestration for Docker
> app-emulation/docker-credential-helpers -
> app-emulation/docker-gc - Docker garbage collection of containers and images
> app-emulation/docker-proxy - Docker container networking
> app-emulation/docker-registry -
> app-emulation/docker-swarm -
> app-emulation/flannel - An etcd backed network fabric for containers
> app-emulation/go-secbench - run and evaluate the docker security benchmark
> app-emulation/img - Standalone Dockerfile and OCI container image builder
> app-emulation/k3d - creates k3s clusters in docker
> app-emulation/kompose - Tool to move from docker-compose to Kubernetes
> app-emulation/lxc - A userspace interface for the Linux kernel containment
> app-emulation/lxc-templates -
> app-emulation/lxd - Fast, dense and secure container management
> app-emulation/nerdctl - Docker-compatible CLI for containerd
> app-emulation/podman - Main offender killer
> app-emulation/reg - Docker registry v2 command line client
> app-emulation/runc - another docker thing
> app-emulation/s6-overlay - an s6-based init system for containers
> app-emulation/sen - Terminal User Interface for docker engine
> app-emulation/skopeo - Utility for operations on container images/repositories
> app-emulation/slirp4netns - User-mode networking for unpriv network namespaces
> app-emulation/snapd - Service and tools for management of snap packages
> app-emulation/umoci - Manipulation tool for OCI images
>
> Those 4 are technically emulation related, so I'm not sure about category:
> app-emulation/docker-machine
> app-emulation/docker-machine-kvm
> app-emulation/hyperd
> app-emulation/runv

Re: [gentoo-dev] RFC: new category for container related packages, instead of app-emulation

[Thomas Mueller]

> On Thu, Aug 5, 2021 at 2:44 PM Georgy Yakovlev <gyakovlev@gentoo.org> wrote:

> > We've been collecting more and more container related packages in
> >  app-emulation/*

> > What do you think about finally moving those packages to separate category?

> As always my opinion is that:
        
> (a) Categories were a design mistake.
> (b) The mistake is hard to fix.
> (c) It's basically low-value to try to 'correctly' categorize packages
because of A.
> (d) Recategorizing means a bunch of stuff has to be updated.

> Do people actually care what category things are in? I just use
> --search or eix or whatever and the category is just this...bad
> concept we attach to packages for silly historical reasons..

-A      

Gentoo portage has an awful lot of categories.

In contrast, Void Linux has all packages not subdivided into categories.

I counted for FreeBSD ports, 63 categories; pkgsrc, 48 categories (NetBSD, but ported to other mostly (quasi-)Unix OSes).

My count could have been just slightly off.

I count 165 for Gentoo, could be slightly off.

I count 61 for Arbor, from Exherbo.

I count 125 categories for Haiku ports, which is modeled after Gentoo but not nearly as many packages.

Tom

Re: [gentoo-dev] RFC: new category for container related packages, instead of app-emulation

[Florian Schmaus]

On 05.08.21 23:44, Georgy Yakovlev wrote:
> Hi,
> 
> We've been collecting more and more container related packages in
>  app-emulation/*
> 
> What do you think about finally moving those packages to separate category?
> 
> probably app-containers/

Categories are cheap, they help to organize the vast amount of packages
we have and the proposed changed categorize the packages more accurately
(to copy aaron's words). Therefore I like the proposal of a new
app-containers/ category.

- Flow

Re: [gentoo-dev] RFC: new category for container related packages, instead of app-emulation

[A Schenck]

On 8/5/21 5:57 PM, Alec Warner wrote:
> On Thu, Aug 5, 2021 at 2:44 PM Georgy Yakovlev <gyakovlev@gentoo.org> wrote:
>> Hi,
>>
>> We've been collecting more and more container related packages in
>>  app-emulation/*
>>
>> What do you think about finally moving those packages to separate category?
> As always my opinion is that:
>
> (a) Categories were a design mistake.
> (b) The mistake is hard to fix.
> (c) It's basically low-value to try to 'correctly' categorize packages
> because of A.
> (d) Recategorizing means a bunch of stuff has to be updated.
>
> Do people actually care what category things are in? I just use
> --search or eix or whatever and the category is just this...bad
> concept we attach to packages for silly historical reasons..

We particularly like categories, and especially the (mostly) two-level
concept in gentoo.  `emerge --search` works for some things, but often
returns extra junk to filter through, when we know a project name, it's
easier to just `ls /usr/portage/dev-ruby/` and see if there's a `grpc`
package in it (upstream gem seems to force `-j12` for native
compilation, wondered if gentoo packagers had fixed that problem). 
Also, the graphical portage interface we use (and barely maintain) has a
two-level category picker which even lets you select just the top-level
category and see what's under several of it's subcategories:

kuroo two-level category picker

-A


>
> -A
>
>> probably app-containers/
>>
>> Here's the tentative list, most tools have description attached for easier
>> review, 36 candidates so far, there may be more around, I only looked at
>> app-emulation.
>>
>> app-emulation/buildah - A tool that facilitates building OCI images
>> app-emulation/cadvisor - container analyzer
>> app-emulation/conmon - An OCI container runtime monitor
>> app-emulation/containerd - A daemon to control runC
>> app-emulation/containers-storage - containers/storage library
>> app-emulation/cri-o - OCI-based implementation of Kubernetes CRI
>> app-emulation/cri-tools - CLI and validation tools for CRI
>> app-emulation/crun - OCI Container Runtime fully written in C
>> app-emulation/distrobuilder - System container image builder for LXC and LXD
>> app-emulation/docker - Main offender
>> app-emulation/docker-bench-security - Test for best docker practices
>> app-emulation/docker-cli - Main offender cli
>> app-emulation/docker-compose - Multi-container orchestration for Docker
>> app-emulation/docker-credential-helpers -
>> app-emulation/docker-gc - Docker garbage collection of containers and images
>> app-emulation/docker-proxy - Docker container networking
>> app-emulation/docker-registry -
>> app-emulation/docker-swarm -
>> app-emulation/flannel - An etcd backed network fabric for containers
>> app-emulation/go-secbench - run and evaluate the docker security benchmark
>> app-emulation/img - Standalone Dockerfile and OCI container image builder
>> app-emulation/k3d - creates k3s clusters in docker
>> app-emulation/kompose - Tool to move from docker-compose to Kubernetes
>> app-emulation/lxc - A userspace interface for the Linux kernel containment
>> app-emulation/lxc-templates -
>> app-emulation/lxd - Fast, dense and secure container management
>> app-emulation/nerdctl - Docker-compatible CLI for containerd
>> app-emulation/podman - Main offender killer
>> app-emulation/reg - Docker registry v2 command line client
>> app-emulation/runc - another docker thing
>> app-emulation/s6-overlay - an s6-based init system for containers
>> app-emulation/sen - Terminal User Interface for docker engine
>> app-emulation/skopeo - Utility for operations on container images/repositories
>> app-emulation/slirp4netns - User-mode networking for unpriv network namespaces
>> app-emulation/snapd - Service and tools for management of snap packages
>> app-emulation/umoci - Manipulation tool for OCI images
>>
>> Those 4 are technically emulation related, so I'm not sure about category:
>> app-emulation/docker-machine
>> app-emulation/docker-machine-kvm
>> app-emulation/hyperd
>> app-emulation/runv

Re: [gentoo-dev] RFC: new category for container related packages, instead of app-emulation

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1526 bytes --]

On Thu, Aug 05, 2021 at 05:57:06PM -0700, Alec Warner wrote:
> On Thu, Aug 5, 2021 at 2:44 PM Georgy Yakovlev <gyakovlev@gentoo.org> wrote:
> >
> > Hi,
> >
> > We've been collecting more and more container related packages in
> >  app-emulation/*
> >
> > What do you think about finally moving those packages to separate category?
> 
> As always my opinion is that:
> 
> (a) Categories were a design mistake.
> (b) The mistake is hard to fix.
> (c) It's basically low-value to try to 'correctly' categorize packages
> because of A.

I disagree that categories are a mistake. For one thing, they help with
situations where different upstream packages have the same name (we have
two docker packages for example, both named docker by their upstreams),
and in my opinion they help keep the tree organized. It is nice to be
able to do something like `ls -d <category>` and check out somewhat
related groups of packages.

I will agree that C above is subjective.

> (d) Recategorizing means a bunch of stuff has to be updated.

All of the updates are much easier to do since we use git, so this isn't
a concern. You can set this up so it hits the tree all at once.

> Do people actually care what category things are in? I just use
> --search or eix or whatever and the category is just this...bad
> concept we attach to packages for silly historical reasons..

Yes, I think they are useful like I said above.

So add me to the list of folks supporting recatagorizing the packages.

William

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] RFC: new category for container related packages, instead of app-emulation

[Toralf Förster]

[-- Attachment #1.1: Type: text/plain, Size: 271 bytes --]

On 8/6/21 2:57 AM, Alec Warner wrote:
> As always my opinion is that:
> 
> (a) Categories were a design mistake.

Hhm, the concept of a directory+files structure is wide spreaded and 
many search algorithms benefit from it.

-- 
Toralf
PGP 23217DA7 9B888F45


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [gentoo-dev] RFC: new category for container related packages, instead of app-emulation

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 617 bytes --]

Am Donnerstag, 5. August 2021, 23:44:40 CEST schrieb Georgy Yakovlev:
> Hi,
> 
> We've been collecting more and more container related packages in
>  app-emulation/*
> 
> What do you think about finally moving those packages to separate category?
> 
> probably app-containers/
> 
> Here's the tentative list, most tools have description attached for easier
> review, 36 candidates so far, there may be more around, I only looked at
> app-emulation.

Sounds like a good idea! -a

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] RFC: new category for container related packages, instead of app-emulation

[Louis Sautier]

[-- Attachment #1.1: Type: text/plain, Size: 1054 bytes --]

On 06/08/2021 02:57, Alec Warner wrote:
> On Thu, Aug 5, 2021 at 2:44 PM Georgy Yakovlev <gyakovlev@gentoo.org> wrote:
>> Hi,
>>
>> We've been collecting more and more container related packages in
>>   app-emulation/*
>>
>> What do you think about finally moving those packages to separate category?
> As always my opinion is that:
>
> (a) Categories were a design mistake.
> (b) The mistake is hard to fix.
> (c) It's basically low-value to try to 'correctly' categorize packages
> because of A.
> (d) Recategorizing means a bunch of stuff has to be updated.
>
> Do people actually care what category things are in? I just use
> --search or eix or whatever and the category is just this...bad
> concept we attach to packages for silly historical reasons..
>
> -A

Hi,

I also care about categories: if I want to find a Python MySQL library, 
I'll run "eix -C dev-python mysql" (3 results) instead of a plain "eix 
mysql" (33 results). And, as William said, they really help with 
ambiguous package names such as docker.



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [gentoo-dev] RFC: new category for container related packages, instead of app-emulation

[Alec Warner]

On Fri, Aug 6, 2021 at 2:12 PM William Hubbs <williamh@gentoo.org> wrote:
>
> On Thu, Aug 05, 2021 at 05:57:06PM -0700, Alec Warner wrote:
> > On Thu, Aug 5, 2021 at 2:44 PM Georgy Yakovlev <gyakovlev@gentoo.org> wrote:
> > >
> > > Hi,
> > >
> > > We've been collecting more and more container related packages in
> > >  app-emulation/*
> > >
> > > What do you think about finally moving those packages to separate category?
> >
> > As always my opinion is that:
> >
> > (a) Categories were a design mistake.
> > (b) The mistake is hard to fix.
> > (c) It's basically low-value to try to 'correctly' categorize packages
> > because of A.
>
> I disagree that categories are a mistake. For one thing, they help with
> situations where different upstream packages have the same name (we have
> two docker packages for example, both named docker by their upstreams),
> and in my opinion they help keep the tree organized. It is nice to be
> able to do something like `ls -d <category>` and check out somewhat
> related groups of packages.

Sure, but you can build a much better mechanism to disambiguate
packages than a category[0].
Grouping packages together is again better served by a different
system. If you want to group packages using categories:

 - A package can only be in 1 group (its category).
 - It's hard to have custom groupings (basically requires I make an
overlay, move the package into a different group, and then do a
pkgmove in my overlay? Does this even work?)
 - Moving packages between groups is not a trivial operation.

>
> I will agree that C above is subjective.
>
> > (d) Recategorizing means a bunch of stuff has to be updated.
>
> All of the updates are much easier to do since we use git, so this isn't
> a concern. You can set this up so it hits the tree all at once.

I think you misunderstand the cost you are imposing on users. Updating
git is the easy part!

You add the pkgmove.
You rewrite some DEPEND atoms.
You bump.

But the impact on *downstream*.

Everyone else who uses those atoms also needs to update them.

For portage-visible names portage will update them (so e.g. it will do
the pkgmove in the VDB, in your binpkg repo, in your metadata, in
/etc/portage/*, world, sets, etc.)
For non-portage visible names you are stuck updating it yourself.

 - Overlays (all need to update their atoms)
 - Any kind of scripts that drive your systems (e.g. "install
foo/bar-123 needs to say 'baz/bar-123' instead")

So my point is that it isn't free. That doesn't mean don't do it; just
understand that it isn't a free operation; or even cheap!

-A

[0] When I say it's a design mistake I am basically talking about
encoding metadata in a name. It's a common design mistake because then
the metadata changes and so does the name. Very common with naming in
computer science. I'm all for names that provide unambiguous packages;
I just want to be able to reference a package in a stable way.

>
> > Do people actually care what category things are in? I just use
> > --search or eix or whatever and the category is just this...bad
> > concept we attach to packages for silly historical reasons..
>
> Yes, I think they are useful like I said above.
>
> So add me to the list of folks supporting recatagorizing the packages.
>
> William

Re: [gentoo-dev] RFC: new category for container related packages, instead of app-emulation

[Joshua Kinard]

On 8/5/2021 20:57, Alec Warner wrote:
> On Thu, Aug 5, 2021 at 2:44 PM Georgy Yakovlev <gyakovlev@gentoo.org> wrote:
>>
>> Hi,
>>
>> We've been collecting more and more container related packages in
>>  app-emulation/*
>>
>> What do you think about finally moving those packages to separate category?
> 
> As always my opinion is that:
> 
> (a) Categories were a design mistake.
> (b) The mistake is hard to fix.
> (c) It's basically low-value to try to 'correctly' categorize packages
> because of A.
> (d) Recategorizing means a bunch of stuff has to be updated.
> 
> Do people actually care what category things are in? I just use
> --search or eix or whatever and the category is just this...bad
> concept we attach to packages for silly historical reasons..
> 
> -A

Categories are there because Portage was originally inspired by FreeBSD's
Ports system, which also uses categories.  Though they have far, far fewer
categories than we do.  They also don't have the two-tier system of
"foo-bar/package".  Just "foo/package".  So not really a design mistake,
per-say, but rather an intentional part of the original design of Portage.

That said, other distros also use categories for their packaging systems,
they're just less apparent.  Run aptitude in Debian for a curses-based
interface to the apt system and you can browse their multi-level package
organization....which can be quite the rabbit hole at times.  Far easier to
just search for what you're looking for there.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-dev] RFC: new category for container related packages, instead of app-emulation

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 1302 bytes --]



> On 6 Aug 2021, at 23:27, Louis Sautier <sbraz@gentoo.org> wrote:
> 
> On 06/08/2021 02:57, Alec Warner wrote:
>> On Thu, Aug 5, 2021 at 2:44 PM Georgy Yakovlev <gyakovlev@gentoo.org> wrote:
>>> Hi,
>>> 
>>> We've been collecting more and more container related packages in
>>>  app-emulation/*
>>> 
>>> What do you think about finally moving those packages to separate category?
>> As always my opinion is that:
>> 
>> (a) Categories were a design mistake.
>> (b) The mistake is hard to fix.
>> (c) It's basically low-value to try to 'correctly' categorize packages
>> because of A.
>> (d) Recategorizing means a bunch of stuff has to be updated.
>> 
>> Do people actually care what category things are in? I just use
>> --search or eix or whatever and the category is just this...bad
>> concept we attach to packages for silly historical reasons..
>> 
>> -A
> 
> Hi,
> 
> I also care about categories: if I want to find a Python MySQL library, I'll run "eix -C dev-python mysql" (3 results) instead of a plain "eix mysql" (33 results). And, as William said, they really help with ambiguous package names such as docker.

This is definitely my approach too.

Anyway, I'm supportive of the new category. If it makes things clearer for people, why not?

best,
sam

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] RFC: new category for container related packages, instead of app-emulation

[Ionen Wolkens]

[-- Attachment #1: Type: text/plain, Size: 1747 bytes --]

On Sat, Aug 07, 2021 at 05:56:55AM +0100, Sam James wrote:
> 
> 
> > On 6 Aug 2021, at 23:27, Louis Sautier <sbraz@gentoo.org> wrote:
> > 
> > On 06/08/2021 02:57, Alec Warner wrote:
> >> On Thu, Aug 5, 2021 at 2:44 PM Georgy Yakovlev <gyakovlev@gentoo.org> wrote:
> >>> Hi,
> >>> 
> >>> We've been collecting more and more container related packages in
> >>>  app-emulation/*
> >>> 
> >>> What do you think about finally moving those packages to separate category?
> >> As always my opinion is that:
> >> 
> >> (a) Categories were a design mistake.
> >> (b) The mistake is hard to fix.
> >> (c) It's basically low-value to try to 'correctly' categorize packages
> >> because of A.
> >> (d) Recategorizing means a bunch of stuff has to be updated.
> >> 
> >> Do people actually care what category things are in? I just use
> >> --search or eix or whatever and the category is just this...bad
> >> concept we attach to packages for silly historical reasons..
> >> 
> >> -A
> > 
> > Hi,
> > 
> > I also care about categories: if I want to find a Python MySQL library, I'll run "eix -C dev-python mysql" (3 results) instead of a plain "eix mysql" (33 results). And, as William said, they really help with ambiguous package names such as docker.
> 
> This is definitely my approach too.
> 
> Anyway, I'm supportive of the new category. If it makes things clearer for people, why not?

How I feel as well.

While it wouldn't hurt to consider revisiting the system eventually
(mostly to avoid pkgmoves), I feel that's not a debate that needs to
come up every time consider new categories given they're still cheap
to add and a system change wouldn't happen overnight.

So I'd say just go for it.

-- 
ionen

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [gentoo-dev] RFC: new category for container related packages, instead of app-emulation

[Michał Górny]

On Thu, 2021-08-05 at 17:57 -0700, Alec Warner wrote:
> On Thu, Aug 5, 2021 at 2:44 PM Georgy Yakovlev <gyakovlev@gentoo.org> wrote:
> > 
> > Hi,
> > 
> > We've been collecting more and more container related packages in
> >  app-emulation/*
> > 
> > What do you think about finally moving those packages to separate category?
> 
> As always my opinion is that:
> 
> (a) Categories were a design mistake.
> (b) The mistake is hard to fix.
> (c) It's basically low-value to try to 'correctly' categorize packages
> because of A.
> (d) Recategorizing means a bunch of stuff has to be updated.
> 

Categories themselves were not a design mistake.  The design mistake is
using categories to permit conflicting package names.

Categories are convenient.  Sure, they're not perfect but they serve
their purpose to some degree and there's little harm in having them.
If you want to organize packages better, nobody's stopping you.  Until
you've got a better and widespread replacement, I don't see why people
shouldn't be using categories as they see fit.

The other part is something we could aim for fixing but so far most
developers seems to disagree with me, so there's no point in pursuing
that.

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] RFC: new category for container related packages, instead of app-emulation

[A Schenck]

On 8/6/21 11:58 AM, A Schenck wrote:
> On 8/5/21 5:57 PM, Alec Warner wrote:
>> On Thu, Aug 5, 2021 at 2:44 PM Georgy Yakovlev <gyakovlev@gentoo.org> wrote:
>>> Hi,
>>>
>>> We've been collecting more and more container related packages in
>>>  app-emulation/*
>>>
>>> What do you think about finally moving those packages to separate category?
>> As always my opinion is that:
>>
>> (a) Categories were a design mistake.
>> (b) The mistake is hard to fix.
>> (c) It's basically low-value to try to 'correctly' categorize packages
>> because of A.
>> (d) Recategorizing means a bunch of stuff has to be updated.
>>
>> Do people actually care what category things are in? I just use
>> --search or eix or whatever and the category is just this...bad
>> concept we attach to packages for silly historical reasons..
> We particularly like categories, and especially the (mostly) two-level
> concept in gentoo.  `emerge --search` works for some things, but often
> returns extra junk to filter through, when we know a project name, it's
> easier to just `ls /usr/portage/dev-ruby/` and see if there's a `grpc`
> package in it (upstream gem seems to force `-j12` for native
> compilation, wondered if gentoo packagers had fixed that problem). 
> Also, the graphical portage interface we use (and barely maintain) has a
> two-level category picker which even lets you select just the top-level
> category and see what's under several of it's subcategories:
>
> kuroo two-level category picker
Unsurprisingly the list dropped the image even though it was small. 
Here's an ancient screenshot of the two-level category-package view:
https://a.fsdn.com/con/app/proj/kuroo/screenshots/Kuroo4-packages.png
>
> -A
>
>
>> -A
>>
>>> probably app-containers/
>>>
>>> Here's the tentative list, most tools have description attached for easier
>>> review, 36 candidates so far, there may be more around, I only looked at
>>> app-emulation.
>>>
>>> app-emulation/buildah - A tool that facilitates building OCI images
>>> app-emulation/cadvisor - container analyzer
>>> app-emulation/conmon - An OCI container runtime monitor
>>> app-emulation/containerd - A daemon to control runC
>>> app-emulation/containers-storage - containers/storage library
>>> app-emulation/cri-o - OCI-based implementation of Kubernetes CRI
>>> app-emulation/cri-tools - CLI and validation tools for CRI
>>> app-emulation/crun - OCI Container Runtime fully written in C
>>> app-emulation/distrobuilder - System container image builder for LXC and LXD
>>> app-emulation/docker - Main offender
>>> app-emulation/docker-bench-security - Test for best docker practices
>>> app-emulation/docker-cli - Main offender cli
>>> app-emulation/docker-compose - Multi-container orchestration for Docker
>>> app-emulation/docker-credential-helpers -
>>> app-emulation/docker-gc - Docker garbage collection of containers and images
>>> app-emulation/docker-proxy - Docker container networking
>>> app-emulation/docker-registry -
>>> app-emulation/docker-swarm -
>>> app-emulation/flannel - An etcd backed network fabric for containers
>>> app-emulation/go-secbench - run and evaluate the docker security benchmark
>>> app-emulation/img - Standalone Dockerfile and OCI container image builder
>>> app-emulation/k3d - creates k3s clusters in docker
>>> app-emulation/kompose - Tool to move from docker-compose to Kubernetes
>>> app-emulation/lxc - A userspace interface for the Linux kernel containment
>>> app-emulation/lxc-templates -
>>> app-emulation/lxd - Fast, dense and secure container management
>>> app-emulation/nerdctl - Docker-compatible CLI for containerd
>>> app-emulation/podman - Main offender killer
>>> app-emulation/reg - Docker registry v2 command line client
>>> app-emulation/runc - another docker thing
>>> app-emulation/s6-overlay - an s6-based init system for containers
>>> app-emulation/sen - Terminal User Interface for docker engine
>>> app-emulation/skopeo - Utility for operations on container images/repositories
>>> app-emulation/slirp4netns - User-mode networking for unpriv network namespaces
>>> app-emulation/snapd - Service and tools for management of snap packages
>>> app-emulation/umoci - Manipulation tool for OCI images
>>>
>>> Those 4 are technically emulation related, so I'm not sure about category:
>>> app-emulation/docker-machine
>>> app-emulation/docker-machine-kvm
>>> app-emulation/hyperd
>>> app-emulation/runv

Re: [gentoo-dev] RFC: new category for container related packages, instead of app-emulation

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 1413 bytes --]

> 
> Categories themselves were not a design mistake.  The design mistake is
> using categories to permit conflicting package names.
> 
> Categories are convenient.  Sure, they're not perfect but they serve
> their purpose to some degree and there's little harm in having them.
> If you want to organize packages better, nobody's stopping you.  Until
> you've got a better and widespread replacement, I don't see why people
> shouldn't be using categories as they see fit.
> 
> The other part is something we could aim for fixing but so far most
> developers seems to disagree with me, so there's no point in pursuing
> that.
> 

So how about improving categories instead?

Rough idea: 
* introduce 1st, 2nd, 3rd class categories, categorized in metadata somewhere
* 1st class is what you want in your world file (and default setting)
* 2nd class is everything else
* 3rd class is stuff that should never be in your world file (but still can be, if you want)

Then tooling can (maybe output a warning but) default to highest class category if none is given.

Examples:
* 1st class: app-*, dev-lang, games-*, ...
* 2nd class: dev-haskell, dev-perl, dev-php, dev-python, dev-texlive, media-libs, net-libs, sci-libs, ...
* 3rd class: acct-*, perl-core, virtual


-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] RFC: new category for container related packages, instead of app-emulation

[Eray Aslan]

On Sat, Aug 07, 2021 at 01:17:00AM -0400, Ionen Wolkens wrote:
> On Sat, Aug 07, 2021 at 05:56:55AM +0100, Sam James wrote:
> > 
> > 
> > > On 6 Aug 2021, at 23:27, Louis Sautier <sbraz@gentoo.org> wrote:
> > > 
> > > On 06/08/2021 02:57, Alec Warner wrote:
> > >> Do people actually care what category things are in? I just use
> > >> --search or eix or whatever and the category is just this...bad
> > >> concept we attach to packages for silly historical reasons..
> > > 
> > > I also care about categories: if I want to find a Python MySQL library, I'll run "eix -C dev-python mysql" (3 results) instead of a plain "eix mysql" (33 results). And, as William said, they really help with ambiguous package names such as docker.
> > 
> > This is definitely my approach too.
> > 
> > Anyway, I'm supportive of the new category. If it makes things clearer for people, why not?
> 
> While it wouldn't hurt to consider revisiting the system eventually
> (mostly to avoid pkgmoves), I feel that's not a debate that needs to
> come up every time consider new categories given they're still cheap
> to add and a system change wouldn't happen overnight.

Categories are just one piece of metadata about the package and
incorporating - one somewhat arbitrary - metadata in a name is in
general not a good idea. I think that was all Alec was saying and I tend
to agree.

But it is not an easy fix so go ahead afaic.

-- 
Eray