[gentoo-user] Kernel config for Docker

[gentoo-user] Kernel config for Docker

[Peter Humphrey]

Afternoon all,

I'm trying to follow the wiki[1] to set up gentoo-sources-5.4.38, but the wiki 
seems to have been written for a different kernel version. Nothing daunted, I 
set as many parameters as I could, rebooted and ran /usr/share/docker/contrib/
check-config.sh. I got 9 things missing, thus:

$ /usr/share/docker/contrib/check-config.sh > /tmp/config-check
$ grep missing /tmp/config-check
- CONFIG_NF_NAT_IPV4: missing
- CONFIG_NF_NAT_NEEDED: missing
- CONFIG_IOSCHED_CFQ: missing
- CONFIG_CFQ_GROUP_IOSCHED: missing
      - CONFIG_INET_XFRM_MODE_TRANSPORT: missing
    - CONFIG_AUFS_FS: missing
    - /dev/zfs: missing
    - zfs command: missing
    - zpool command: missing

I can't find any of those. Any clues for the uninitiated?

1.  https://wiki.gentoo.org/wiki/Docker

-- 
Regards,
Peter.

Re: [gentoo-user] Kernel config for Docker

[Mark Knecht]

[-- Attachment #1: Type: text/plain, Size: 1457 bytes --]

On Sat, May 16, 2020 at 10:12 AM Peter Humphrey <peter@prh.myzen.co.uk>
wrote:
>
> Afternoon all,
>
> I'm trying to follow the wiki[1] to set up gentoo-sources-5.4.38, but the
wiki
> seems to have been written for a different kernel version. Nothing
daunted, I
> set as many parameters as I could, rebooted and ran
/usr/share/docker/contrib/
> check-config.sh. I got 9 things missing, thus:
>
> $ /usr/share/docker/contrib/check-config.sh > /tmp/config-check
> $ grep missing /tmp/config-check
> - CONFIG_NF_NAT_IPV4: missing
> - CONFIG_NF_NAT_NEEDED: missing
> - CONFIG_IOSCHED_CFQ: missing
> - CONFIG_CFQ_GROUP_IOSCHED: missing
>       - CONFIG_INET_XFRM_MODE_TRANSPORT: missing
>     - CONFIG_AUFS_FS: missing
>     - /dev/zfs: missing
>     - zfs command: missing
>     - zpool command: missing
>
> I can't find any of those. Any clues for the uninitiated?
>
> 1.  https://wiki.gentoo.org/wiki/Docker
>
> --
> Regards,
> Peter.

Hi Peter,
   Welcome to the world of Docker.

   Do you have an image you know you want to run? If so just try running it
and see if you get any complaints. There's a very simple Hello World image
and there are basic Ubuntu console level images that don't require much of
anything to get going.

   The NF_NAT stuff is networking. IOSCHED should be under the CPU
configuration area I think.

   I have /dev/zfs on my Kubuntu machine but I don't have zfs or zpool and
haven't ever had a problem with those.

Good lluck,
Mark

[-- Attachment #2: Type: text/html, Size: 1928 bytes --]

Re: [gentoo-user] Kernel config for Docker

[Victor Ivanov]

[-- Attachment #1.1: Type: text/plain, Size: 2306 bytes --]

Some of these are obsolete and have been removed in newer kernels.

NF_NAT ones have been superseded by / renamed to:

    CONFIG_NF_NAT
    CONFIG_IP_NF_NAT
    CONFIG_IP6_NF_NAT

I'm not sure about CONFIG_INET_XFRM_MODE_TRANSPORT but I believe it's
now incorporated into other XFRM config variables in newer kernels. I
will need to look at the kernel changelogs. But you should be fine
without it.

AUFS is not part of the main kernel tree, so the following option does
not apply:

    CONFIG_AUFS_FS

That said, Docker does _not_ require AUFS, but it does still support it
as an option. Docker docs, recommend using OverlayFS instead which is
also part of the main kernel tree.

As far as I recall there are patches and overlays that will allow you to
enable AUFS if needed. Debian, for example, does not include AUFS and
instead uses DKMS for AUFS support.

The CFQ I/O scheduler is obsolete and has been removed, so the following
do not exist and have no alternatives except for running an older kernel:

    CONFIG_IOSCHED_CFQ
    CONFIG_CFQ_GROUP_IOSCHED

Docker will run just fine with the BFQ scheduler.

Generally speaking the docker kernel validation script is fundamentally
flawed by design - it doesn't check the kernel options based on the
kernel version. Instead it uses as single global list of CONFIG options
and matches against that.

- Victor

On 16/05/2020 18:12, Peter Humphrey wrote:
> Afternoon all,
> 
> I'm trying to follow the wiki[1] to set up gentoo-sources-5.4.38, but the wiki 
> seems to have been written for a different kernel version. Nothing daunted, I 
> set as many parameters as I could, rebooted and ran /usr/share/docker/contrib/
> check-config.sh. I got 9 things missing, thus:
> 
> $ /usr/share/docker/contrib/check-config.sh > /tmp/config-check
> $ grep missing /tmp/config-check
> - CONFIG_NF_NAT_IPV4: missing
> - CONFIG_NF_NAT_NEEDED: missing
> - CONFIG_IOSCHED_CFQ: missing
> - CONFIG_CFQ_GROUP_IOSCHED: missing
>       - CONFIG_INET_XFRM_MODE_TRANSPORT: missing
>     - CONFIG_AUFS_FS: missing
>     - /dev/zfs: missing
>     - zfs command: missing
>     - zpool command: missing
> 
> I can't find any of those. Any clues for the uninitiated?
> 
> 1.  https://wiki.gentoo.org/wiki/Docker
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Kernel config for Docker

[Viktar Patotski]

[-- Attachment #1: Type: text/plain, Size: 2586 bytes --]

hey,

to solve this part:
- /dev/zfs: missing
    - zfs command: missing
    - zpool command: missing


Do `emerge sys-fs/zfs` and reboot.

Viktar

On Sat, May 16, 2020 at 8:05 PM Victor Ivanov <vic.m.ivanov@gmail.com>
wrote:

> Some of these are obsolete and have been removed in newer kernels.
>
> NF_NAT ones have been superseded by / renamed to:
>
>     CONFIG_NF_NAT
>     CONFIG_IP_NF_NAT
>     CONFIG_IP6_NF_NAT
>
> I'm not sure about CONFIG_INET_XFRM_MODE_TRANSPORT but I believe it's
> now incorporated into other XFRM config variables in newer kernels. I
> will need to look at the kernel changelogs. But you should be fine
> without it.
>
> AUFS is not part of the main kernel tree, so the following option does
> not apply:
>
>     CONFIG_AUFS_FS
>
> That said, Docker does _not_ require AUFS, but it does still support it
> as an option. Docker docs, recommend using OverlayFS instead which is
> also part of the main kernel tree.
>
> As far as I recall there are patches and overlays that will allow you to
> enable AUFS if needed. Debian, for example, does not include AUFS and
> instead uses DKMS for AUFS support.
>
> The CFQ I/O scheduler is obsolete and has been removed, so the following
> do not exist and have no alternatives except for running an older kernel:
>
>     CONFIG_IOSCHED_CFQ
>     CONFIG_CFQ_GROUP_IOSCHED
>
> Docker will run just fine with the BFQ scheduler.
>
> Generally speaking the docker kernel validation script is fundamentally
> flawed by design - it doesn't check the kernel options based on the
> kernel version. Instead it uses as single global list of CONFIG options
> and matches against that.
>
> - Victor
>
> On 16/05/2020 18:12, Peter Humphrey wrote:
> > Afternoon all,
> >
> > I'm trying to follow the wiki[1] to set up gentoo-sources-5.4.38, but
> the wiki
> > seems to have been written for a different kernel version. Nothing
> daunted, I
> > set as many parameters as I could, rebooted and ran
> /usr/share/docker/contrib/
> > check-config.sh. I got 9 things missing, thus:
> >
> > $ /usr/share/docker/contrib/check-config.sh > /tmp/config-check
> > $ grep missing /tmp/config-check
> > - CONFIG_NF_NAT_IPV4: missing
> > - CONFIG_NF_NAT_NEEDED: missing
> > - CONFIG_IOSCHED_CFQ: missing
> > - CONFIG_CFQ_GROUP_IOSCHED: missing
> >       - CONFIG_INET_XFRM_MODE_TRANSPORT: missing
> >     - CONFIG_AUFS_FS: missing
> >     - /dev/zfs: missing
> >     - zfs command: missing
> >     - zpool command: missing
> >
> > I can't find any of those. Any clues for the uninitiated?
> >
> > 1.  https://wiki.gentoo.org/wiki/Docker
> >
>
>

[-- Attachment #2: Type: text/html, Size: 3413 bytes --]

Re: [gentoo-user] Kernel config for Docker

[Andrew Udvare]

[-- Attachment #1.1: Type: text/plain, Size: 2638 bytes --]

On 16/05/2020 13:12, Peter Humphrey wrote:
> I can't find any of those. Any clues for the uninitiated?

I am running Docker fine on 5.6.12 and I am missing a lot:

- CONFIG_NF_NAT_IPV4: missing
- CONFIG_IP_NF_FILTER: missing
- CONFIG_IP_NF_TARGET_MASQUERADE: missing
- CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: missing
- CONFIG_NETFILTER_XT_MATCH_CONNTRACK: missing
- CONFIG_NETFILTER_XT_MATCH_IPVS: missing
- CONFIG_IP_NF_NAT: missing
- CONFIG_NF_NAT_NEEDED: missing
- CONFIG_IOSCHED_CFQ: missing
- CONFIG_CFQ_GROUP_IOSCHED: missing
- CONFIG_IP_NF_TARGET_REDIRECT: missing
- CONFIG_EXT3_FS_XATTR: missing
      - CONFIG_INET_XFRM_MODE_TRANSPORT: missing
    - CONFIG_NF_NAT_FTP: missing
    - CONFIG_NF_CONNTRACK_FTP: missing
    - CONFIG_NF_NAT_TFTP: missing
    - CONFIG_NF_CONNTRACK_TFTP: missing
    - CONFIG_AUFS_FS: missing
    - CONFIG_BTRFS_FS: missing
    - CONFIG_BTRFS_FS_POSIX_ACL: missing
    - /dev/zfs: missing
    - zfs command: missing
    - zpool command: missing

In regards to NF options, I use nftables and I manage the firewall
manually for Docker (I set {"iptables": false} in
/etc/docker/daemon.json). Docker has been extremely slow at adopting
nftables.

You definitely do not need zfs installed to use Docker. This machine
doesn't have it.

As Victor stated, CFQ is deprecated and gone and BFQ will work fine. And
the script is basically for Red Hat (or corporate) users who still use
iptables, CFQ, ext3 apparently, and a much older kernel.

On my slightly stabler server (running 5.4.38), this is the output and
Docker still works fine. Again on that server I use nftables and manage
the firewall manually. The system has ext4 for its root and the rest of
it is ZFS.

- CONFIG_BRIDGE_NETFILTER: missing
- CONFIG_NF_NAT_IPV4: missing
- CONFIG_IP_NF_FILTER: missing
- CONFIG_IP_NF_TARGET_MASQUERADE: missing
- CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: missing
- CONFIG_NETFILTER_XT_MATCH_CONNTRACK: missing
- CONFIG_NETFILTER_XT_MATCH_IPVS: missing
- CONFIG_IP_NF_NAT: missing
- CONFIG_NF_NAT_NEEDED: missing
- CONFIG_IOSCHED_CFQ: missing
- CONFIG_CFQ_GROUP_IOSCHED: missing
- CONFIG_IP_NF_TARGET_REDIRECT: missing
- CONFIG_EXT3_FS: missing
- CONFIG_EXT3_FS_XATTR: missing
- CONFIG_EXT3_FS_POSIX_ACL: missing
- CONFIG_EXT3_FS_SECURITY: missing
      - CONFIG_INET_XFRM_MODE_TRANSPORT: missing
    - CONFIG_NF_NAT_FTP: missing
    - CONFIG_NF_CONNTRACK_FTP: missing
    - CONFIG_NF_NAT_TFTP: missing
    - CONFIG_NF_CONNTRACK_TFTP: missing
    - CONFIG_AUFS_FS: missing
    - CONFIG_BTRFS_FS: missing
    - CONFIG_BTRFS_FS_POSIX_ACL: missing

Andrew


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Kernel config for Docker

[Peter Humphrey]

On Sunday, 17 May 2020 00:58:54 BST Andrew Udvare wrote:
> On 16/05/2020 13:12, Peter Humphrey wrote:
> > I can't find any of those. Any clues for the uninitiated?
> 
> I am running Docker fine on 5.6.12 and I am missing a lot:

--->8

> In regards to NF options, I use nftables and I manage the firewall
> manually for Docker (I set {"iptables": false} in
> /etc/docker/daemon.json). Docker has been extremely slow at adopting
> nftables.

I'm still pretty much in the dark about setting up nftables and iptables in 
the kernel config. Not to worry, though; I dare say it'll become clearer in 
time.

> You definitely do not need zfs installed to use Docker. This machine
> doesn't have it.
> 
> As Victor stated, CFQ is deprecated and gone and BFQ will work fine. And
> the script is basically for Red Hat (or corporate) users who still use
> iptables, CFQ, ext3 apparently, and a much older kernel.
> 
> On my slightly stabler server (running 5.4.38), this is the output and
> Docker still works fine. Again on that server I use nftables and manage
> the firewall manually. The system has ext4 for its root and the rest of
> it is ZFS.

--->8

Thanks gents. I'll just try it and see what happens.

-- 
Regards,
Peter.

Re: [gentoo-user] Kernel config for Docker

[Victor Ivanov]

[-- Attachment #1.1: Type: text/plain, Size: 2791 bytes --]

Andrew makes a good point that, of course, not all options will be
relevant to a particular image or use case. The script is aimed to check
for "full" compatibility. Having some reported as missing is by no means
a deal breaker.

Re nftables it's a very valid point as well. I too use nftables instead
of iptables and, in general, anything that dares touch my rules I will
either disable the option for it to do so or, if that's not possible,
swiftly eradicate it off my system with vengeance. I'm not a big fan of
how Docker manages netfilter rules so I too tend to disable that from
the config and, as Andrew said, it has been slow at adopting nftables.
It seems Docker is being developed with primary consideration for stable
(read archaic) distributions that have long release cycles.

If you use nftables at all - even via other software such as firewalld,
etc - Docker may or may not like that. Previously, though admitedly
quite a while ago, Docker just loved adding iptables rules in addition
to my nftables rules. Needless to say, that quickly became a mess.

nftables is _a lot_ easier to manage, even writing rules manually feels
a lot more intuitive. So I think the learning curve (at least in terms
of syntax) tends to be less steep IMO if you decide to go down that road
at some point.

Anyway, this probably wasn't a post of high contribution value haha

Keep us updated in case you encounter any issues!

Cheers,
Victor

On 17/05/2020 09:31, Peter Humphrey wrote:
> On Sunday, 17 May 2020 00:58:54 BST Andrew Udvare wrote:
>> On 16/05/2020 13:12, Peter Humphrey wrote:
>>> I can't find any of those. Any clues for the uninitiated?
>>
>> I am running Docker fine on 5.6.12 and I am missing a lot:
> 
> --->8
> 
>> In regards to NF options, I use nftables and I manage the firewall
>> manually for Docker (I set {"iptables": false} in
>> /etc/docker/daemon.json). Docker has been extremely slow at adopting
>> nftables.
> 
> I'm still pretty much in the dark about setting up nftables and iptables in 
> the kernel config. Not to worry, though; I dare say it'll become clearer in 
> time.
> 
>> You definitely do not need zfs installed to use Docker. This machine
>> doesn't have it.
>>
>> As Victor stated, CFQ is deprecated and gone and BFQ will work fine. And
>> the script is basically for Red Hat (or corporate) users who still use
>> iptables, CFQ, ext3 apparently, and a much older kernel.
>>
>> On my slightly stabler server (running 5.4.38), this is the output and
>> Docker still works fine. Again on that server I use nftables and manage
>> the firewall manually. The system has ext4 for its root and the rest of
>> it is ZFS.
> 
> --->8
> 
> Thanks gents. I'll just try it and see what happens.
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Kernel config for Docker

[Peter Humphrey]

On Sunday, 17 May 2020 12:26:02 BST Victor Ivanov wrote:
> Andrew makes a good point that, of course, not all options will be
> relevant to a particular image or use case. The script is aimed to check
> for "full" compatibility. Having some reported as missing is by no means
> a deal breaker.
> 
> Re nftables it's a very valid point as well. I too use nftables instead
> of iptables and, in general, anything that dares touch my rules I will
> either disable the option for it to do so or, if that's not possible,
> swiftly eradicate it off my system with vengeance. I'm not a big fan of
> how Docker manages netfilter rules so I too tend to disable that from
> the config and, as Andrew said, it has been slow at adopting nftables.
> It seems Docker is being developed with primary consideration for stable
> (read archaic) distributions that have long release cycles.

Ah. I scent Debian.

> If you use nftables at all - even via other software such as firewalld,
> etc - Docker may or may not like that. Previously, though admitedly
> quite a while ago, Docker just loved adding iptables rules in addition
> to my nftables rules. Needless to say, that quickly became a mess.

I've been using shorewall for many years.

> nftables is _a lot_ easier to manage, even writing rules manually feels
> a lot more intuitive. So I think the learning curve (at least in terms
> of syntax) tends to be less steep IMO if you decide to go down that road
> at some point.
> 
> Anyway, this probably wasn't a post of high contribution value haha

All grist to the mill - thanks.

-- 
Regards,
Peter.