Web Function and Data Parsing Question

Community Support
1

I am trying to get this q code to work

yahoo: {[offset;stocks]
tbl:();i:0;zs:(ze:.z.d)-offset;
parms:“&d=”,(string -1+mm$ze),"&e=",(stringdd$ze),“&f=”,
(stringyear$ze),"&g=d&a=",(string -1+mm$zs),“&b=”,(stringdd $zs),"&c=",(stringyear$zs),“&ignore=.csv”;
do[count stocks:distinct stocks,();
txt:`:http://ichart.finance.yahoo.com “GET /table.csv?s=”,
(string stock:stocks[i]),parms," http/1.1\r
\nhost:ichart.finance.yahoo.com\r\n\r\n";
tbl,:update Sym:stock from select from (“DEEEEI “;enlist”,”)0:(txt
ss"Date,Open")_ txt;i+:1];
(lower cols tbl)xcol`Date`Sym xasc select from tbl where not null
Volume}

from

https://code.kx.com/svn/cookbook\_code/yahoo\_compact.q

I cannot seem to get it to work. Any suggestions?

2

Make sure you do not have connectivity problems to Yahoo. Try this
link: http:. You should
get a CSV file. If you do, then this should work:

q)yahoo[7;GOOGAAPL]
date open high low close volume sym
----------------------------------------------------
2008.07.28 162.34 162.47 154.02 154.4 27882600 AAPL
2008.07.28 492.09 492.09 475.13 477.12 3160000 GOOG
2008.07.29 155.41 159.45 153.65 157.08 24431100 AAPL
2008.07.29 479.3 487.26 478 483.11 2802800 GOOG
2008.07.30 157.78 160.49 156.08 159.88 25899400 AAPL
2008.07.30 485.5 486.02 472.81 482.7 3490700 GOOG
2008.07.31 157.54 162.2 156.98 158.95 22744600 AAPL
2008.07.31 474.56 480.89 471.44 473.75 2865100 GOOG
2008.08.01 159.9 159.99 155.75 156.66 19451400 AAPL
2008.08.01 472.51 473.22 462.5 467.86 3007900 GOOG
2008.08.04 156.6 157.9 152.91 153.23 21161700 AAPL
2008.08.04 468.12 473.01 461.9 463 2487000 GOOG

Swee Heng
</http:>

3

I am having same issue. Yahoo connectivity is fine. Have saved code athttps://code.kx.com/svn/cookbook_code/yahoo_compact.q to a file, andloaded using the \l command. Then tried yahoo[7;GOOGAAPL], output isbelowq)\l c:/yahoo.qq)yahoo[7;GOOGAAPL]{[offset;stocks] tbl:();i:0;zs:(ze:.z.d)-offset; parms:“&d=”,(string -1+mm$ze),"&e=",(stringdd$ze),“&f=”,(stringyear$ze),"&g=d&a=",(string -1+mm$zs),“&b=”,(stringdd$zs),"&c=",(stringyear$zs),“&ignore=.csv”; do[count stocks:distinct stocks,(); txt::http://ichart.finance.yahoo.com "GET /table.csv?s=",(string stock:stocks[i]),parms," http/1.1\r\nhost:ichart.finance.yahoo.com\r\n\r\n"; tbl,:update Sym:stock from select from ("DEEEEI ";enlist",")0:(txt ss"Date,Open")_ txt;i+:1]; (lower cols tbl)xcolDateSym xasc select from tbl where not nullVolume}'length0:("DEEEEI ";,",")()q))Suggestions?On Aug 4, 7:56?pm, Swee Heng <thesweeh...> wrote:&gt; Make sure you do not have connectivity problems to Yahoo. Try this&gt; link: <http:>. You should&gt; get a CSV file. If you do, then this should work:&gt;&gt; q)yahoo[7;GOOGAAPL]&gt; date ? ? ? open ? high ? low ? ?close ?volume ? sym&gt; ----------------------------------------------------&gt; 2008.07.28 162.34 162.47 154.02 154.4 ?27882600 AAPL&gt; 2008.07.28 492.09 492.09 475.13 477.12 3160000 ?GOOG&gt; 2008.07.29 155.41 159.45 153.65 157.08 24431100 AAPL&gt; 2008.07.29 479.3 ?487.26 478 ? ?483.11 2802800 ?GOOG&gt; 2008.07.30 157.78 160.49 156.08 159.88 25899400 AAPL&gt; 2008.07.30 485.5 ?486.02 472.81 482.7 ?3490700 ?GOOG&gt; 2008.07.31 157.54 162.2 ?156.98 158.95 22744600 AAPL&gt; 2008.07.31 474.56 480.89 471.44 473.75 2865100 ?GOOG&gt; 2008.08.01 159.9 ?159.99 155.75 156.66 19451400 AAPL&gt; 2008.08.01 472.51 473.22 462.5 ?467.86 3007900 ?GOOG&gt; 2008.08.04 156.6 ?157.9 ?152.91 153.23 21161700 AAPL&gt; 2008.08.04 468.12 473.01 461.9 ?463 ? ?2487000 ?GOOG&gt;&gt; Swee Heng&gt;&gt; On Aug 4, 1:48 pm, DuoCentillion <duocentill...> wrote:&gt;&gt; &gt; I am trying to get this q code to work&gt;&gt; &gt; yahoo: {[offset;stocks]&gt; &gt; ? ? ? ? tbl:();i:0;zs:(ze:.z.d)-offset;&gt; &gt; ? ? parms:"&amp;d=",(string -1+mm$ze),“&e=”,(stringdd$ze),"&amp;f=",&gt; &gt; (stringyear$ze),“&g=d&a=”,(string -1+mm$zs),"&amp;b=",(stringdd> > $zs),“&c=”,(stringyear$zs),"&amp;ignore=.csv";&gt; &gt; ? ? do[count stocks:distinct stocks,();&gt; &gt; ? ? ? ? txt::http://ichart.finance.yahoo.com"GET /table.csv?s=“,> > (string stock:stocks[i]),parms,” http/1.1\r> > \nhost:ichart.finance.yahoo.com\r\n\r\n";> > ? ? ? ? ? ? ? ? tbl,:update Sym:stock from select from (“DEEEEI “;enlist”,”)0:(txt> > ss"Date,Open")_ txt;i+:1];> > ? ? (lower cols tbl)xcolDateSym xasc select from tbl where not null> > Volume}>> > from>> >https://code.kx.com/svn/cookbook_code/yahoo_compact.q&gt;&gt; > I cannot seem to get it to work. ?Any suggestions?</duocentill…></http:></thesweeh…>

4

I get your error if I use an offset of 0. That is:
q)yahoo[0;GOOGAAPL]
… (output snipped)…
'length
0:
(“DEEEEI “;,”,”)
()

What I’d suggest is that you modify yahoo to show “txt” and where
the string “Date,Open” appears in it. E.g.
ya: {[offset;stocks]
tbl:();i:0;zs:(ze:.z.d)-offset;
parms:“&d=”,(string -1+mm$ze),"&e=",(stringdd$ze),
“&f=”,(stringyear$ze),"&g=d&a=",(string -1+mm$zs),
“&b=”,(stringdd$zs),"&c=",(stringyear$zs),“&ignore=.csv”;
do[count stocks:distinct stocks,();
txt:`:http://ichart.finance.yahoo.com “GET /table.csv?s=”,
(string stock:stocks[i]),parms,
" http/1.1\r\nhost:ichart.finance.yahoo.com\r\n\r\n";
show txt; show txt ss “Date,Open”;i+:1]}

Then try:
q)ya[0;GOOGAAPL]
"HTTP/1.1 404 Not Found\r\nProxy-Connection: Keep-Alive\r\nConnection:
Keep-A..
int$() "HTTP/1.1 404 Not Found\r\nDate: Tue, 05 Aug 2008 17:54:59 GMT\r\nP3P: policy.. int$()
q)ya[7;GOOGAAPL]
"HTTP/1.1 200 OK\r\nDate: Tue, 05 Aug 2008 17:55:03 GMT\r\nP3P:
policyref="h..
,291
"HTTP/1.1 200 OK\r\nProxy-Connection: Keep-Alive\r\nConnection: Keep-
Alive\r..
,382

In the first case, Yahoo returned a HTTP 404 Error and so the content
does not contain the expected “Date,Open” string. That seems to be the
error in yahoo function when 0: parsed the records. You can also use
a sniffer like Wireshark to inspect the traffic Yahoo returns you if
the problem persists.

Swee Heng