# questions about tuning a NUMA server

**URL:** <https://forum.kx.com/t/questions-about-tuning-a-numa-server/12337>\
**Category:** Community Support\
**Tags:** add-ons\
**Created:** [December 29, 2021, 3:19am UTC](https://forum.kx.com/t/questions-about-tuning-a-numa-server/12337 "2021-12-29T03:19:00Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![leguan1](https://avatars.discourse-cdn.com/v4/letter/l/6f9a4e/32.png) [@leguan1](https://forum.kx.com/u/leguan1)\
**Post date:** [December 29, 2021, 3:19am UTC](https://forum.kx.com/t/questions-about-tuning-a-numa-server/12337/1 "2021-12-29T03:19:00Z")

</div>

Hello everyone,&nbsp;

i have some questions about tuning q processes on NUMA server (linux kernel version is above 3.x).&nbsp;

1. According to&nbsp;https://code.kx.com/q/kb/linux-production/&nbsp; [1]&nbsp; it is suggested to disable interleave and enable zone\_reclaim, however&nbsp;https://code.kx.com/q/kb/cpu-affinity/#activating-numa [2]&nbsp; suggests&nbsp;

a) “numactl --interleave= **all** --physcpubind=0,1,2 q”&nbsp;

**Q1.** Why allows the process to access all node’s memory instead of bind the process to local memory only?&nbsp;

b) “echo 0 \> /proc/sys/vm/zone\_reclaim\_mode”&nbsp; which lets the process closes zone-reclaim and takes other numa nodes’s memory.&nbsp;

**Q2.** &nbsp;Is zone\_reclaim\_mode=1&nbsp; the correct answer according to [1] ?&nbsp;

The attached shows one of our q process,&nbsp; on a numa-enabled server, binds to mem 0-3,&nbsp;

**Q3.** Is&nbsp;it identical to “interleave=all” ?&nbsp;

**Q4**. If i want this process access local mem only,&nbsp; shall i execute&nbsp; “numactl --interleave=0” ?

**Q5. following up Q4,&nbsp;** How to start a q process and put numactl command’s to work in kx-platform?&nbsp; which Reversed Parameter shall be edited? or, should i “run”/start a process first, then&nbsp; execute system "numactl --interleave=0 --physcpubind=0 " inside process’s console ?

thank you and the best,

leguan,&nbsp;

&nbsp;

---

<div class="post-metadata">

**Author:** ![leades2](https://avatars.discourse-cdn.com/v4/letter/l/a87d85/32.png) [@leades2](https://forum.kx.com/u/leades2)\
**Post date:** [December 29, 2021, 12:56pm UTC](https://forum.kx.com/t/questions-about-tuning-a-numa-server/12337/2 "2021-12-29T12:56:00Z")

</div>

Hello,

Great questions for the Community! I will share them internally and come back with any feedback.

Many thanks

Luke

---

<div class="post-metadata">

**Author:** ![leguan1](https://avatars.discourse-cdn.com/v4/letter/l/6f9a4e/32.png) [@leguan1](https://forum.kx.com/u/leguan1)\
**Post date:** [January 11, 2022, 8:10am UTC](https://forum.kx.com/t/questions-about-tuning-a-numa-server/12337/3 "2022-01-11T08:10:00Z")

</div>

Hello Luke,

May i know if there is any update from tech-team?&nbsp;

best,

leguan

---

<div class="post-metadata">

**Author:** ![leades2](https://avatars.discourse-cdn.com/v4/letter/l/a87d85/32.png) [@leades2](https://forum.kx.com/u/leades2)\
**Post date:** [January 17, 2022, 1:00pm UTC](https://forum.kx.com/t/questions-about-tuning-a-numa-server/12337/4 "2022-01-17T13:00:00Z")

</div>

Hello,

Thank you for your patience.

We have escalated it with the appropriate teams internally and will hopefully get back with an answer soon.

Many Thanks

Luke

---

<div class="post-metadata">

**Author:** ![leades2](https://avatars.discourse-cdn.com/v4/letter/l/a87d85/32.png) [@leades2](https://forum.kx.com/u/leades2)\
**Post date:** [February 18, 2022, 12:21pm UTC](https://forum.kx.com/t/questions-about-tuning-a-numa-server/12337/5 "2022-02-18T12:21:00Z")

</div>

Hello,

Reverting back with feedback for some of the questions:

Q1) Keeping the process within one node is the best option for performance. However, if the working set exceeds one node’s memory capacity, it then has to spill in to other nodes.

Q2) No, this rarely helps with recent kernels. This discussion will be removed altogether from [code.kx.com](http://code.kx.com). The default for modern systems is 0, which should be fine.&nbsp;(Regardless, this setting has no effect if confining the process to a single node)

Q3) No, the default NUMA policy attempts to allocate from the current node, and from others if no free memory is available. See&nbsp;https://man7.org/linux/man-pages/man2/set\_mempolicy.2.html&nbsp;for more details.

Q4) You would usually express that as “numactl -N 0 -m 0” (allocate from node 0, and also run on the cores that belong to node 0)

I hope this information helps.

Many Thanks

Luke

---

<div class="post-metadata">

**Author:** ![leguan1](https://avatars.discourse-cdn.com/v4/letter/l/6f9a4e/32.png) [@leguan1](https://forum.kx.com/u/leguan1)\
**Post date:** [February 22, 2022, 11:33am UTC](https://forum.kx.com/t/questions-about-tuning-a-numa-server/12337/6 "2022-02-22T11:33:00Z")

</div>

Hello Luke ,&nbsp;

Thank you very much for the above helpful answers.&nbsp;

I would like to ask how to start a q instance/task in kx-platform with specified numactrl command?&nbsp;

best,

&nbsp;

leguan

---

<div class="post-metadata">

**Author:** ![leades2](https://avatars.discourse-cdn.com/v4/letter/l/a87d85/32.png) [@leades2](https://forum.kx.com/u/leades2)\
**Post date:** [February 24, 2022, 9:26am UTC](https://forum.kx.com/t/questions-about-tuning-a-numa-server/12337/7 "2022-02-24T09:26:00Z")

</div>

Hello,

You could try “numactl --cpubind=0 --membind=0,1 \<command to start the q process\>”. Let me know if this works.

Many Thanks

Luke

---

<div class="post-metadata">

**Author:** ![leguan1](https://avatars.discourse-cdn.com/v4/letter/l/6f9a4e/32.png) [@leguan1](https://forum.kx.com/u/leguan1)\
**Post date:** [February 25, 2022, 10:56am UTC](https://forum.kx.com/t/questions-about-tuning-a-numa-server/12337/8 "2022-02-25T10:56:00Z")

</div>

Hello Luck,

Thank you for the answer.

“numactl --cpubind=0 --membind=0,1 q blablabla…” This is a good practice to start a raw-kdb, to my understanding.

Well, what i am trying to ask is :&nbsp;

when we wanna start a task of an instance in kx-platform(w r using v4.5.0), we can&nbsp; right-click on the task and select “RUN”.&nbsp; &nbsp;Therefore which instance’s RESERVED PARAM(e.g host/port/taskSet/enableIPSharding/memLimit …) i can edit so that when we “RUN” the task, it works just as start a raw-kdb process in the way as you suggested.

the best and wish you a great weekend!

leguan

---

<div class="post-metadata">

**Author:** ![leguan1](https://avatars.discourse-cdn.com/v4/letter/l/6f9a4e/32.png) [@leguan1](https://forum.kx.com/u/leguan1)\
**Post date:** [February 25, 2022, 10:58am UTC](https://forum.kx.com/t/questions-about-tuning-a-numa-server/12337/9 "2022-02-25T10:58:00Z")

</div>

Following up, i started a q instance as suggested, but it could not be recognized and managed by kx-platform, i guess it fails to register with the DC.&nbsp;

best,&nbsp;

---

<div class="post-metadata">

**Author:** ![leguan1](https://avatars.discourse-cdn.com/v4/letter/l/6f9a4e/32.png) [@leguan1](https://forum.kx.com/u/leguan1)\
**Post date:** [March 15, 2022, 10:28am UTC](https://forum.kx.com/t/questions-about-tuning-a-numa-server/12337/10 "2022-03-15T10:28:00Z")

</div>

Hello Luke and everyone,

I tried again&nbsp; this morning but it still does not work. &nbsp;I executed the following line:

“numactl -membind=0,1 /mykxPlatform\_home/delta-bin/softeware/KDBPlus\_myKdbVersion/l64/q q/prcl.q\_ -process myInstanceName -prhost myControlFQDN -prhost portOfControl&nbsp; -name CentralConfig-PidOfControl -logfile /pathOfLogdir/myProcessName.out -o 8”.

The result is: the process is seemingly UP, i can see its status is “running” in “Process Status Library” page, however after i right-click the process and select “connect”, i got a busy process which is unable to interact with, and there is no log file of it.

I would like to trouble you all to let me know how to start a process/task and making system command like “numactl” works on that process.  
This is significant to us, given i re-installed whole platform last week, finding there is one server has high numa\_miss, albeit the server has exactly the same hardware,even firmware version, with the other 3.  
I would like to manage processes on-demand to see if we can reduce numa\_miss and boost up rdb’s speed.  
Thank you all!

best,

leguan
