[gentoo-user] Do I need NUMA set up in my kernel?

[gentoo-user] Do I need NUMA set up in my kernel?

[Peter Humphrey]

Hello list,

I see "[    0.003162] No NUMA configuration found" in dmesg. Does that mean I 
should, or can, remove the NUMA settings from the kernel? This is a Ryzen M9 
5900X machine.

I have this at the moment:

$ grep NUMA /usr/src/linux/.config
CONFIG_ARCH_SUPPORTS_NUMA_BALANCING=y
# CONFIG_NUMA_BALANCING is not set
CONFIG_NUMA=y
CONFIG_AMD_NUMA=y
CONFIG_X86_64_ACPI_NUMA=y
# CONFIG_NUMA_EMU is not set
CONFIG_USE_PERCPU_NUMA_NODE_ID=y
CONFIG_ACPI_NUMA=y
CONFIG_NUMA_KEEP_MEMINFO=y

-- 
Regards,
Peter.

Re: [gentoo-user] Do I need NUMA set up in my kernel?

[Charlotte Delenk]

[-- Attachment #1.1.1: Type: text/plain, Size: 349 bytes --]

Hi Peter,

On 9/23/21 10:59, Peter Humphrey wrote:
> Hello list,
>
> I see "[    0.003162] No NUMA configuration found" in dmesg. Does that mean I
> should, or can, remove the NUMA settings from the kernel? This is a Ryzen M9
> 5900X machine.

I have CONFIG_NUMA unset on both of my AMD Ryzen machines (Zen+ and 
Zen2) with no issues



[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 4319 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: [gentoo-user] Do I need NUMA set up in my kernel?

[Miles Malone]

You'd need NUMA if you had a NUMA machine.  In current context, that
would be either a) a dual socket system, b) an amd threadripper, or c)
some of the really high core xeons.  If your motherboard doesnt have
certain memory banks allocated to certain processors or cores, you're
probably not running a NUMA machine.

NUMA stands for non-uniform memory access, it means that certain
processor cores have more direct access to certain parts of memory
than others do (e.g. to access the other memory they need the other
cpu core to pass it through)

On Thu, 23 Sept 2021 at 19:39, Charlotte Delenk <darkkirb@darkkirb.de> wrote:
>
> Hi Peter,
>
> On 9/23/21 10:59, Peter Humphrey wrote:
> > Hello list,
> >
> > I see "[    0.003162] No NUMA configuration found" in dmesg. Does that mean I
> > should, or can, remove the NUMA settings from the kernel? This is a Ryzen M9
> > 5900X machine.
>
> I have CONFIG_NUMA unset on both of my AMD Ryzen machines (Zen+ and
> Zen2) with no issues
>
>

Re: [gentoo-user] Do I need NUMA set up in my kernel?

[Peter Humphrey]

On Thursday, 23 September 2021 11:39:39 BST Miles Malone wrote:
> You'd need NUMA if you had a NUMA machine.  In current context, that
> would be either a) a dual socket system, b) an amd threadripper, or c)
> some of the really high core xeons.  If your motherboard doesnt have
> certain memory banks allocated to certain processors or cores, you're
> probably not running a NUMA machine.
> 
> NUMA stands for non-uniform memory access, it means that certain
> processor cores have more direct access to certain parts of memory
> than others do (e.g. to access the other memory they need the other
> cpu core to pass it through)
> 
> On Thu, 23 Sept 2021 at 19:39, Charlotte Delenk <darkkirb@darkkirb.de> 
wrote:
> > Hi Peter,
> > 
> > On 9/23/21 10:59, Peter Humphrey wrote:
> > > Hello list,
> > > 
> > > I see "[    0.003162] No NUMA configuration found" in dmesg. Does that
> > > mean I should, or can, remove the NUMA settings from the kernel? This
> > > is a Ryzen M9 5900X machine.
> > 
> > I have CONFIG_NUMA unset on both of my AMD Ryzen machines (Zen+ and
> > Zen2) with no issues

Thank you both. I'll try removing it and see what happens.

-- 
Regards,
Peter.

Re: [gentoo-user] Do I need NUMA set up in my kernel?

[Grant Taylor]

On 9/23/21 4:39 AM, Miles Malone wrote:
> You'd need NUMA if you had a NUMA machine.  In current context, that 
> would be either a) a dual socket system, b) an amd threadripper, or 
> c) some of the really high core xeons.  If your motherboard doesnt 
> have certain memory banks allocated to certain processors or cores, 
> you're probably not running a NUMA machine.

Will a kernel without NUMA support boot and run on a system that has a 
NUMA architecture?

If it will boot and run, does it simply do so in a sub-optimal way?

Flipping the coin on the other side, is there any negative effect (other 
than kernel size / lines of code / attack surface) for having NUMA 
support enabled on a non-NUMA system?



-- 
Grant. . . .
unix || die

Re: [gentoo-user] Do I need NUMA set up in my kernel?

[Charlotte Delenk]

[-- Attachment #1.1.1: Type: text/plain, Size: 1116 bytes --]

On 9/23/21 18:30, Grant Taylor wrote:
> On 9/23/21 4:39 AM, Miles Malone wrote:
>> You'd need NUMA if you had a NUMA machine.  In current context, that 
>> would be either a) a dual socket system, b) an amd threadripper, or 
>> c) some of the really high core xeons.  If your motherboard doesnt 
>> have certain memory banks allocated to certain processors or cores, 
>> you're probably not running a NUMA machine.
>
> Will a kernel without NUMA support boot and run on a system that has a 
> NUMA architecture?
>
> If it will boot and run, does it simply do so in a sub-optimal way?
Pure speculation for this one but if it works it's probably only going 
to detect part of the memory and some of the devices and one cpu, since 
the other resources are physically connected to the other cpus.
>
> Flipping the coin on the other side, is there any negative effect 
> (other than kernel size / lines of code / attack surface) for having 
> NUMA support enabled on a non-NUMA system?
There is no meaningful downside to leaving it enabled, it's enabled in 
many distribution kernels for a reason.

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 4319 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: [gentoo-user] Do I need NUMA set up in my kernel?

[Peter Humphrey]

On Thursday, 23 September 2021 17:32:46 BST Charlotte Delenk wrote:
> On 9/23/21 18:30, Grant Taylor wrote:
> 
> > On 9/23/21 4:39 AM, Miles Malone wrote:
> > 
> >> You'd need NUMA if you had a NUMA machine.  In current context, that 
> >> would be either a) a dual socket system, b) an amd threadripper, or 
> >> c) some of the really high core xeons.  If your motherboard doesnt 
> >> have certain memory banks allocated to certain processors or cores, 
> >> you're probably not running a NUMA machine.
> >
> >
> >
> > Will a kernel without NUMA support boot and run on a system that has a 
> > NUMA architecture?
> >
> >
> >
> > If it will boot and run, does it simply do so in a sub-optimal way?
> 
> Pure speculation for this one but if it works it's probably only going 
> to detect part of the memory and some of the devices and one cpu, since 
> the other resources are physically connected to the other cpus.
> 
> >
> >
> > Flipping the coin on the other side, is there any negative effect 
> > (other than kernel size / lines of code / attack surface) for having 
> > NUMA support enabled on a non-NUMA system?
> 
> There is no meaningful downside to leaving it enabled, it's enabled in 
> many distribution kernels for a reason.

I've booted a kernel with no NUMA config, and it seems to run fine on this 
single-socket Ryzen motherboard. I just get the one entry in dmesg:

$ dmesg | grep -i numa
[    0.297998] pci_bus 0000:00: on NUMA node 0

That's it. I'm running five BOINC projects, some of which run on vbox, so on 
this motherboard it seems clear that I don't need NUMA.

-- 
Regards,
Peter.

Re: [gentoo-user] Do I need NUMA set up in my kernel?

[Michael]

[-- Attachment #1: Type: text/plain, Size: 808 bytes --]

On Thursday, 23 September 2021 17:38:15 BST Peter Humphrey wrote:

> I've booted a kernel with no NUMA config, and it seems to run fine on this
> single-socket Ryzen motherboard. I just get the one entry in dmesg:
> 
> $ dmesg | grep -i numa
> [    0.297998] pci_bus 0000:00: on NUMA node 0
> 
> That's it. I'm running five BOINC projects, some of which run on vbox, so on
> this motherboard it seems clear that I don't need NUMA.

Out of interest, have you tried booting a NUMA enabled kernel to see what 
dmesg reports?  On an old laptop, which definitely has only a single AMD APU, 
I get:

$ dmesg | grep -i NUMA -A2
[    0.002078] No NUMA configuration found
[    0.002080] Faking a node at [mem 0x0000000000000000-0x000000042effffff]
[    0.002085] NODE_DATA(0) allocated [mem 0x42effc000-0x42effffff]

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Do I need NUMA set up in my kernel?

[Peter Humphrey]

On Thursday, 23 September 2021 19:20:52 BST Michael wrote:

> Out of interest, have you tried booting a NUMA enabled kernel to see what
> dmesg reports?

Yes, it's been enabled ever since I had a dual-socket motherboard, years ago. 
I didn't understand why I did or didn't need it until I read Miles's post 
yesterday (thanks, Miles). I don't know why it hadn't been made clear in any 
websites I've visited.

> On an old laptop, which definitely has only a single AMD
> APU, I get:
> 
> $ dmesg | grep -i NUMA -A2
> [    0.002078] No NUMA configuration found
> [    0.002080] Faking a node at [mem 
0x0000000000000000-0x000000042effffff]
> [    0.002085] NODE_DATA(0) allocated [mem 0x42effc000-0x42effffff]

I had something similar. Oddly, with NUMA configured I get "not found" and 
without it I get "pci_bus 0000:00: on NUMA node 0". The system seems to run 
happily either way.

-- 
Regards,
Peter.

Re: [gentoo-user] Do I need NUMA set up in my kernel?

[Michael]

[-- Attachment #1: Type: text/plain, Size: 1605 bytes --]

On Friday, 24 September 2021 10:06:49 BST Peter Humphrey wrote:
> On Thursday, 23 September 2021 19:20:52 BST Michael wrote:
> > Out of interest, have you tried booting a NUMA enabled kernel to see what
> > dmesg reports?
> 
> Yes, it's been enabled ever since I had a dual-socket motherboard, years
> ago. I didn't understand why I did or didn't need it until I read Miles's
> post yesterday (thanks, Miles). I don't know why it hadn't been made clear
> in any websites I've visited.
> 
> > On an old laptop, which definitely has only a single AMD
> > APU, I get:
> > 
> > $ dmesg | grep -i NUMA -A2
> > [    0.002078] No NUMA configuration found
> > [    0.002080] Faking a node at [mem
> 
> 0x0000000000000000-0x000000042effffff]
> 
> > [    0.002085] NODE_DATA(0) allocated [mem 0x42effc000-0x42effffff]
> 
> I had something similar. Oddly, with NUMA configured I get "not found" and
> without it I get "pci_bus 0000:00: on NUMA node 0". The system seems to run
> happily either way.

Sorry I should have made it clear - the above "No NUMA configuration found" 
message was obtained *with* NUMA enabled in my kernel.

I suppose "NUMA on node 0" is the default first socket, which the kernel sets 
up.  If the kernel can't find a second CPU it will be 'faking' a multi-CPU 
memory allocation setup, when it comes to allocate memory to the only CPU 
available.  If the kernel does not have NUMA enabled then it doesn't need to 
fake anything.  It will treat the hardware as a single socket MoBo and no 
further tests would be undertaken.  All suppositions of course, I haven't 
looked at the code.  ;-)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Do I need NUMA set up in my kernel?

[William Kenworthy]

On 24/9/21 5:38 pm, Michael wrote:
> On Friday, 24 September 2021 10:06:49 BST Peter Humphrey wrote:
>> On Thursday, 23 September 2021 19:20:52 BST Michael wrote:
>>> Out of interest, have you tried booting a NUMA enabled kernel to see what
>>> dmesg reports?
>> Yes, it's been enabled ever since I had a dual-socket motherboard, years
>> ago. I didn't understand why I did or didn't need it until I read Miles's
>> post yesterday (thanks, Miles). I don't know why it hadn't been made clear
>> in any websites I've visited.
>>
>>> On an old laptop, which definitely has only a single AMD
>>> APU, I get:
>>>
>>> $ dmesg | grep -i NUMA -A2
>>> [    0.002078] No NUMA configuration found
>>> [    0.002080] Faking a node at [mem
>> 0x0000000000000000-0x000000042effffff]
>>
>>> [    0.002085] NODE_DATA(0) allocated [mem 0x42effc000-0x42effffff]
>> I had something similar. Oddly, with NUMA configured I get "not found" and
>> without it I get "pci_bus 0000:00: on NUMA node 0". The system seems to run
>> happily either way.
> Sorry I should have made it clear - the above "No NUMA configuration found" 
> message was obtained *with* NUMA enabled in my kernel.
>
> I suppose "NUMA on node 0" is the default first socket, which the kernel sets 
> up.  If the kernel can't find a second CPU it will be 'faking' a multi-CPU 
> memory allocation setup, when it comes to allocate memory to the only CPU 
> available.  If the kernel does not have NUMA enabled then it doesn't need to 
> fake anything.  It will treat the hardware as a single socket MoBo and no 
> further tests would be undertaken.  All suppositions of course, I haven't 
> looked at the code.  ;-)

Try "numactl --hardware" (from the numactl package)

rattus ~ # numactl --hardware
available: 1 nodes (0)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11
node 0 size: 31942 MB
node 0 free: 7210 MB
node distances:
node   0
  0:  10
rattus ~ #

(Intel 6 core - NUMA emulation in the kernel.)

I can only find testing NUMA code and hardware as a reason to have
emulation enabled on a non-NUMA system?

BillK

Re: [gentoo-user] Do I need NUMA set up in my kernel?

[William Kenworthy]

On 24/9/21 7:25 pm, William Kenworthy wrote:
> On 24/9/21 5:38 pm, Michael wrote:
>> On Friday, 24 September 2021 10:06:49 BST Peter Humphrey wrote:
>>> On Thursday, 23 September 2021 19:20:52 BST Michael wrote:
>>>> Out of interest, have you tried booting a NUMA enabled kernel to see what
>>>> dmesg reports?
>>> Yes, it's been enabled ever since I had a dual-socket motherboard, years
>>> ago. I didn't understand why I did or didn't need it until I read Miles's
>>> post yesterday (thanks, Miles). I don't know why it hadn't been made clear
>>> in any websites I've visited.
>>>
>>>> On an old laptop, which definitely has only a single AMD
>>>> APU, I get:
>>>>
>>>> $ dmesg | grep -i NUMA -A2
>>>> [    0.002078] No NUMA configuration found
>>>> [    0.002080] Faking a node at [mem
>>> 0x0000000000000000-0x000000042effffff]
>>>
>>>> [    0.002085] NODE_DATA(0) allocated [mem 0x42effc000-0x42effffff]
>>> I had something similar. Oddly, with NUMA configured I get "not found" and
>>> without it I get "pci_bus 0000:00: on NUMA node 0". The system seems to run
>>> happily either way.
>> Sorry I should have made it clear - the above "No NUMA configuration found" 
>> message was obtained *with* NUMA enabled in my kernel.
>>
>> I suppose "NUMA on node 0" is the default first socket, which the kernel sets 
>> up.  If the kernel can't find a second CPU it will be 'faking' a multi-CPU 
>> memory allocation setup, when it comes to allocate memory to the only CPU 
>> available.  If the kernel does not have NUMA enabled then it doesn't need to 
>> fake anything.  It will treat the hardware as a single socket MoBo and no 
>> further tests would be undertaken.  All suppositions of course, I haven't 
>> looked at the code.  ;-)
> Try "numactl --hardware" (from the numactl package)
>
> rattus ~ # numactl --hardware
> available: 1 nodes (0)
> node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11
> node 0 size: 31942 MB
> node 0 free: 7210 MB
> node distances:
> node   0
>   0:  10
> rattus ~ #
>
> (Intel 6 core - NUMA emulation in the kernel.)
>
> I can only find testing NUMA code and hardware as a reason to have
> emulation enabled on a non-NUMA system?
>
> BillK
>
>
Actually Iam using "numactl -C 4,5 /etc/init.d/amavisd start" to lock
processes to particular cpu's (on an arm big.LITTLE architecture.

I will need to compile a new kernel without NUMA emulation to see if it
still works.

BillK