I have managed to load Python into the Q process using PyQ (http://code.kx.com/wiki/Contrib/PyQ).
Does anybody know how to do the same with IPython (Notebook)? I imagine that you could set the interpreter of IPython to
q python.q
Many thanks!
Ok, so it’s possible to load IPython by typing
from IPython import embedembed()
To open IPython Notebook you can do something like
import sysimport impipython_path = '/usr/local/bin/ipython'site_pkgs = '/usr/local/lib/python2.7/site-packages'sys.path.append(site_pkgs)sys.argv = [ipython_path, 'notebook']_ipython = imp.load_source(' __main__', ipython_path)
although I get some errors from tornado (the webserver).
If anybody has come up with a stable solution, please let me know.
You can also launch an ipython notebook within q by doing this:
q python.q -m@ IPython notebook
But the problem is - when ipython notebook tries to launch a backend kernel (to run python commands and return their results over websocket) it tries to spawn the executable w/ path ‘sys.executable’. In a normal python process that would be something like ‘/usr/bin/python’, but in PyQ (maybe due to python being embedded), sys.executable = ‘’, and this leads to:
OSError: [Errno 13] Permission denied
(just try subprocess.Popen([‘’]) to reproduce).
Luckily, the kernel spawn command can be overridden in the ipython notebook config. Create an ipython notebook profile, e.g:
ipython profile create james
then edit its config, e.g:
vim /Users/james/.ipython/profile_james/ipython_notebook_config.py
and set:
c.KernelManager.kernel_cmd = [‘q’, ‘python.q’, ‘-c@’, ‘from IPython.kernel.zmq.kernelapp import main; main()’, ‘-f@’, ‘{connection_file}’]
(you need q to be in $PATH for the above to work).
and then launch the ipython notebook w/
ipython notebook --profile=james
Note that you don’t need to wrap this with the q process anymore, because ipython notebook will spawn the “PyQ kernel” for you.
Although you can do this:
because the q process is not sending its output on the ipython notebook websocket, you will of course not see this in the browser (you’ll see it in your console stdout instead). But you can do something like this instead:
The next step would be to use the .z.ws websocket handler to seamlessly send q output to the notebook UI.
Hope this helps.
James
Thanks James, you’re the man!
I’m still struggling a bit with PyQ though. Would you happen to know how to for example create a pandas DataFrame from PyQ?
In case anybody is wondering how to do convert a q table to a pandas dataframe, one way to do it is:
from q import qfrom pandas.core.api import DataFramefrom itertools import izipdef to_data_frame(table, index='Date'): cols = [col for col in q.cols(table) if not getattr(table, col).inspect(b't') == 20] data = izip(*(getattr(table, col) for col in cols)) return DataFrame.from_records(data, columns=cols, index=index)
For some reason, type 20’s (sym) aren’t iterable. If anybody know a workaround, please let me know.
A hack to convert a table with a enumerated column (such as sym for my tickers in this case):
def pyq_to_data_frame(table, index=['Date', 'Sym']): cols = [col for col in q.cols(table)] data = izip(*(getattr(table, col) if getattr(table, col).inspect(b't') < 20 else q("{sym x%s}" % col, table) for col in cols)) return DataFrame.from_records(data, columns=cols, index=index)`