Production memory usage

Community Support
1

Given the following 2 scenarios:

  1. A real time inmemory tick database generally not queried to frequently
  2. A database of mostly static tables to which users can place unconstrained queries

what kind of max memory consumption does kx recommend for production usage? Should scenario 1 max out at ~50% physical memory before maybe requiring another database instance on another machine? Since scenario 2 could potentially result in extra memory being allocated from unconstrained queries would somewhere around 25-30% of physical memory be more reasonable? Or is there some general rule of thumb?

Thank you.

2

Memory consumption and raw data capacity are different. Kdb 32 bit can report up to 4 GB memory usage. Windows task manager reports up to 2GB ram usage. My guess is half of the signed int/pointer range goes to negative numbers. Another half goes to garbage collection. That leaves 1 GB for raw data.

http://kx.com/q/d/a/kdb+.htm#Limits

22 Limits

Each database runs in memory and/or disk map-on-demand – possibly partitioned. There is no limit on the size of a partitioned database but on 32-bit systems the main memory OLTP portion of a database is limited to about 1GB of raw data, i.e. 1/4 of the address space. The raw data of a main memory 64bit process should be limited to about 1/2 of available RAM.

/ memoryConsumptionTest.q

value each {“t”, (string x), “: til 1024*1024”} each til 231;

show .Q.w%(1024*1024);

$ q memoryConsumptionTest.q

KDB+ 3.2 2015.02.10 Copyright (C) 1993-2015 Kx Systems

w32/ 4()core 4095MB yan yanicolaptop 192.168.30.219 NONEXPIRE


Welcome to kdb+ 32bit edition

For support please see http://groups.google.com/d/forum/personal-kdbplus

Tutorials can be found at http://code.kx.com/wiki/Tutorials

To exit, type \

To remove this startup msg, edit q.q

used| 3696.113

heap| 3712

peak| 3712

wmax| 0

mmap| 0

mphy| 4096

syms| 0.0007648468

symw| 0.03010845

q)