https://learninghub.kx.com/forums/topic/pykx-3-0-is-released
We are thrilled to announce a huge milestone in the development of PyKX - version 3.0. The PyKX team has been working hard to deliver a new and improved user experience that’s easier and faster to get started with PyKX than ever before.
?Major Features/Changes
- Major overhaul in the documentation on code.kx.com/pykx
- Addition of functionality to allow for development of end-to-end streaming workflows consisting of data-ingestion, persistence and query. This functionality is outlined in-depth here.
- Update to the PyKX Query API to support a significantly more Python first approach to querying kdb+ in-memory and on-disk databases.
>>> table = kx.Table(data={
... 'sym': kx.random.random(100, ['AAPL', 'GOOG', 'MSFT']),
... 'date': kx.random.random(100, kx.q('2022.01.01') + [0,1,2]),
... 'price': kx.random.random(100, 1000.0),
... 'size': kx.random.random(100, 100)
... })
>>> table.select(columns=kx.Column('price').max(), where=kx.Column('size') > 5)
>>> table.update(column=kx.Column('price').wavg(kx.Column('size')).rename('vwap'), by=kx.Column('sym'))
>>> table.delete(column=kx.Column('sym'))
>>> table.update(column=(kx.Column('price') * kx.Column('size')).rename('total'))Beta features available in the 2.* versions of PyKX have now been migrated to full support.
- The full list of these features are as follows:
- Database Creation and Management
- Compression and Encryption Module
- Remote Function Execution
- Streamlit Integration
- Multi-threaded use of PyKX
❓What else?
- Extension to our integration with Jupyter Notebooks by adding a q-first mode of operation which allows users working between the two languages to more easily automate workflows depending on both
- Addition of a new utility function kx.util.detect_bad_columns to validate if the columns of a table object conform to the naming conventions supported by kdb+ and highlighting if the table contains duplicate column names raising a warning indicating potential issues and returning True if the table contains invalid columns.
- When generating IPC connections with reconnection_attempts users can now configure the initial delay between first and second attempts and the function which updates the delay on successive attempts using the reconnection_delay and reconnection_function keywords. See here for a worked example.
- Two new options added on first initialisation of PyKX to allow users to:
- Use the path to their already downloaded kc.lic/k4.lic licenses without going through the “Do you want to install a license” workflow
- Allow users to persist for future use that they wish to use the IPC only unlicensed mode of PyKX, this will persist a file ~/.pykx-config which sets configuration denoting unlicensed mode is to be used.
- Addition of function kx.util.install_q to allow users who do not have access to a q executable at the time of installing PyKX. See here for instructions regarding its use.
- Addition of function kx.util.start_q_subprocess to allow a q process to be started on a specified port with supplied initialisation arguments
?Fixes & Improvements As with any release PyKX 3.0 provides a significant number of bug fixes and improvements, the following are a subset:
- Addition of support for help when interacting with q keywords and operators via PyKX
- Previously loading pykx.q during q startup using QINIT or QHOME/q.q resulted in a segfault or a corruption.
- The function kx.util.debug_environment now returns the applied configuration values at startup instead of customised values
- Operations on kx.GroupbyTable objects which have been indexed previously would raise an error indicating invalid key access
- Attempts to load a database using the kx.DB module previously would raise an nyi error if the path to the database contained a space
?Additional Resources
Blog for PyKX 3.0: https://kx.com/blog/introducing-pykx-3-0/ First look at PyKX 3.0 video: https://www.youtube.com/watch?v=8pqpQdCpJMg&t=772s&pp=ygUIcHlreCAzLjA%3D