PyKX 2.4.0 Released!

Community Support
,
1

https://learninghub.kx.com/forums/topic/pykx-2-4-0-released

Hi all,

PyKX 2.4.0 has been released. Full details on the release can be found here.

Additions

  • Support for q/kdb+ 4.1 documentation here added as an opt-in capability, this functionality is enabled through setting PYKX_4_1_ENABLED environment variable.
>>> import os 
>>> os.environ['PYKX_4_1_ENABLED'] = 'True' 
>>> import pykx as kx 
>>> kx.q.z.K 
pykx.FloatAtom(pykx.q('4.1'))
  • Added support for Python 3.12
    • Support for PyArrow in this python version is currently in Beta.
  • Added conversion of NumPy arrays of type datetime64[s], datetime64[ms], datetime64[us] to kx.TimestampVector
  • Added Table.sort_values(), Table.nsmallest() and Table.nlargest() to the Pandas like API for sorting tables.
  • Table.rename() now supports non-numerical index columns and improved the quality of errors thrown.
  • Added the reconnection_attempts key word argument to SyncQConnection, SecureQConnection, and AsyncQConnection 0IPC classes. This argument allows IPC connection to be automatically re-established when it is lost and a server has reinitialized.
>>> import pykx as kx 
>>> conn = kx.SyncQConnection(port = 5050, reconnection_attempts=4) 
>>> conn('1+1')    # Following this call the server on port 5050 was closed for 2 seconds 
pykx.LongVector(pykx.q('2')) 
>>> conn('1+2') 
WARNING: Connection lost attempting to reconnect. Failed to reconnect, trying again in 0.5 seconds. 
Failed to reconnect, trying again in 1.0 seconds. 
Connection successfully reestablished. 
pykx.LongAtom(pykx.q('3'))
  • Added --reconnection_attempts option to Jupyter %%q magic making use of the above IPC logic changes.
  • Addition of environment variable/configuration value PYKX_QDEBUG which allows debugging backtrace to be displayed for all calls into q instead of requiring a user to specify debugging is enabled per-call. This additionally works for remote IPC calls and utilisation of Jupyter magic commands.
>>> import os 
>>> os.environ['PYKX_QDEBUG'] = 'True' 
>>> import pykx as kx 
>>> kx.q('{x+1}', 'e') 
backtrace:
   [2]  {x+1}
          ^
   [1]  (.Q.trp)
   [0]  {[pykxquery] .Q.trp[value; pykxquery; {if[y~();:(::)];2@"backtrace:
                     ^ 
",.Q.sbt y;'x}]} 
Traceback (most recent call last):
   File "", line 1, in 
   File "/usr/local/anaconda3/lib/python3.8/site-packages/pykx/embedded_q.py", line 230, in __call__
     return factory(result, False)
   File "pykx/_wrappers.pyx", line 493, in pykx._wrappers._factory
   File "pykx/_wrappers.pyx", line 486, in pykx._wrappers.factory
 pykx.exceptions.QError: type

Fixes and Improvements

  • Resolved segfaults on Windows when PyKX calls Python functions under q.
>>> import pykx as kx 
>>> kx.q('{[f;x] f  x}', sum, kx.q('4 4#til 16')) 
pykx.LongVector(pykx.q('24 28 32 36'))
  • Updated kdb Insights Core libraries to 4.0.8, see here for more information.
  • Updated libq 4.0 version to 2024.03.04 for all supported OS’s.
  • Fix issue where use of valid C backed q code APIs could result in segmentation faults when called.
>>> import pykx as kx 
>>> isf = kx.q('.pykx.util.isf') 
>>> isf 
pykx.Foreign(pykx.q('code')) 
>>> isf(True) 
pykx.BooleanAtom(pykx.q('0b'))
  • Each call to the PyKX query API interned 3 new unique symbols. This has now been removed.

Beta Features

  • Addition of Compress and Encrypt classes to allow users to set global configuration and for usage within Database partition persistence.

Standalone

>>> import pykx as kx 
>>> compress = kx.Compress(algo=kx.CompressionAlgorithm.gzip, level=8) 
>>> kx.q.z.zd 
pykx.Identity(pykx.q('::')) 
>>> compress.global_init() 
pykx.LongVector(pykx.q('17 2 8')) 
>>> encrypt = kx.Encrypt(path='/path/to/the.key', password='PassWord') 
>>> encrypt.load_key()

Database

>>> import pykx as kx 
>>> compress = kx.Compress(algo=kx.CompressionAlgorithm.lz4hc, level=10) 
>>> db = kx.DB(path='/tmp/db') 
>>> db.create(kx.q('([]10?1f;10?1f)', 'tab', kx.q('2020.03m'), compress=compress) 
>>> kx.q('-21!`:/tmp/db/2020.03/tab/x') 
pykx.Dictionary(pykx.q('
compressedLength  | 140
uncompressedLength| 96
algorithm         | 4i
logicalBlockSize  | 17i
zipLevel          | 10i 
'))
2

Thanks for sharing @cmccarthy1 !?