embedPy installation error

Community Support
,
1

https://learninghub.kx.com/forums/topic/embedpy-installation-error

Hi everyone, I'm starting out installing embedPy, and I'm doing so via downloading and installing the latest release (as of now it's 1.5.0) to run on my local machine.


I unzipped the 1.5.0 release and am currently trying to run the tests (q test.q), when I get the following error. I'm not sure what it means and I'm wondering if anyone can add some input?


M:\M\embedPy_windows-1.5.0>q test.q
KDB+ 4.0 2022.01.14 Copyright (C) 1993-2022 Kx Systems
++++++++++++++++++++++++++++++++++++++++++++++++++++
Loading custom startup.  Customize in C:/q/q.q
Start timestamp (local): 2024.03.26D10:48:22.584873000
Full display precisions: \P 0
C:/q/help.q
C:/q/c/odbc.k
C:/q/ps.k
'libpython
  [5]  M:\M\embedPy_windows-1.5.0\p.q:14:
 .P.env:not H~P;
 .p:(:./p 2:(init;3))[L;H;B]]
    ^
q.p))
2

Related thread: embedPy installation: os error with kdb and python - Stack Overflow


I would try:

python -m pip install libpython


If still failing see output from https://pypi.org/project/find-libpython/ 

pip install find_libpython 


>>> from find_libpython import find_libpython
>>> find_libpython()


3

It looks like I didn't install pip correctly, so after making sure I updated my environment variables and running 

C:\Users\a623>python -m ensurepip

I was able to run the following:

C:\Users\a623>python -m pip install libpython
Collecting libpython
  Downloading libpython-0.2.tar.gz (15 kB)
Collecting requests
  Downloading requests-2.31.0-py3-none-any.whl (62 kB)
     |████████████████████████████████| 62 kB 408 kB/s
Collecting idna<4,>=2.5
  Downloading idna-3.6-py3-none-any.whl (61 kB)
     |████████████████████████████████| 61 kB 55 kB/s
Collecting urllib3<3,>=1.21.1
  Downloading urllib3-2.2.1-py3-none-any.whl (121 kB)
     |████████████████████████████████| 121 kB 3.3 MB/s
Collecting certifi>=2017.4.17
  Downloading certifi-2024.2.2-py3-none-any.whl (163 kB)
     |████████████████████████████████| 163 kB ...
Collecting charset-normalizer<4,>=2
  Downloading charset_normalizer-3.3.2-cp39-cp39-win_amd64.whl (100 kB)
     |████████████████████████████████| 100 kB 5.5 MB/s
Using legacy 'setup.py install' for libpython, since package 'wheel' is not installed.
Installing collected packages: urllib3, idna, charset-normalizer, certifi, requests, libpython
    Running setup.py install for libpython ... done
Successfully installed certifi-2024.2.2 charset-normalizer-3.3.2 idna-3.6 libpython-0.2 requests-2.31.0 urllib3-2.2.1
WARNING: You are using pip version 21.1.3; however, version 24.0 is available.
You should consider upgrading via the 'C:\Users\a623\AppData\Local\Programs\Python\Python39\python.exe -m pip install --upgrade pip' command.

Then, I ran the test.q script, which weirdly gives me the same error as before:

M:\M\embedPy_windows-1.5.0>q test.q
KDB+ 4.0 2022.01.14 Copyright (C) 1993-2022 Kx Systems
++++++++++++++++++++++++++++++++++++++++++++++++++++
Loading custom startup.  Customize in C:/q/q.q
Start timestamp (local): 2024.03.27D13:50:41.738296000
Full display precisions: \P 0
C:/q/help.q
C:/q/c/odbc.k
C:/q/ps.k
'libpython
  [5]  M:\M\embedPy_windows-1.5.0\p.q:14:
 .P.env:not H~P;
 .p:(:./p 2:(init;3))[L;H;B]]
    ^

I'm not sure why I'm getting the same error, because it looks like I have pip set up correctly and installed libpython, as you can see above.


When I try to reinstall libpython I get the error below:

M:\M\embedPy_windows-1.5.0>python -m pip install libpython
Requirement already satisfied: libpython in c:\users\a623\appdata\local\programs\python\python39\lib\site-packages (0.2)
Requirement already satisfied: requests in c:\users\a623\appdata\local\programs\python\python39\lib\site-packages (from libpython) (2.31.0)
Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\a623\appdata\local\programs\python\python39\lib\site-packages (from requests->libpython) (3.3.2)
Requirement already satisfied: urllib3<3,>=1.21.1 in c:\users\a623\appdata\local\programs\python\python39\lib\site-packages (from requests->libpython) (2.2.1)
Requirement already satisfied: idna<4,>=2.5 in c:\users\a623\appdata\local\programs\python\python39\lib\site-packages (from requests->libpython) (3.6)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\a623\appdata\local\programs\python\python39\lib\site-packages (from requests->libpython) (2024.2.2)
WARNING: You are using pip version 21.1.3; however, version 24.0 is available.
You should consider upgrading via the 'C:\Users\a623\AppData\Local\Programs\Python\Python39\python.exe -m pip install --upgrade pip' command.

Any idea what's going on here?

4

Can you try https://pypi.org/project/find-libpython/

pip install find_libpython 
>>> from find_libpython import find_libpython
>>> find_libpython()

What is the output?

5

Hey
I have a similar issue.
Most likely caused by how our UNIX team force users specific environments when working with python.
I have the same issue and the find_libpython() returns an empty string for me.
Any idea how to fix this ?

6

Hello, I still haven't been able to figure it out. Not sure if my team wants to continue using embedPy but if you figure it out let me know. I reached out to someone who works at kdb and they recommended taking a look at the FAQs link in the github: https://github.com/KxSystems/embedPy/blob/master/docs/faq.md. The first question/answer is related to the libpython error I've been getting and may help you. It looks like we need to enable python to allow shared libraries, which is not enabled if python is installed using the recommended method.

7

Going forward PyKX under q is the recommended replacement for embedPy.

*It is not operational yet on Windows

8

Hey

for me it was the UNIX team which messed up.
They installed a new version of python and forgot to configure the shared library folder.

9

PyKX under q requires the pykx license feature flag right, so there is a license fee to use it. This is the part where you are using python within the q interpreter (loading pykx.q). You can use PyKx free when sending data from python to kdb, when you set export PYKX_UNLICENSED="true". Whereas embedPy is free to use (apache 2.0 license). This is my understanding, please correct me if I am wrong

10

There is a docs page on PyKX modes of operation.


The 3 modes are:

  1. Unlicensed mode - used for IPC query and publish. No license needed.
  2. Licensed mode - import pykx in Python session.
  3. PyKX under q - \l pykx.q in a q session.

Both 2 & 3 require a KX license which includes insights.lib.pykx and insights.lib.embedq licensing flags.


11

Thanks for providing clarification on the modes Rian. So the replacement for the embedPy part is a licensed model. Noting that Pykx has more advanced functionality than embedPy.


Still, I hope they continue to maintain embedPy for users who only require what it offers, while keeping costs limited to the kdb license.