calculating correlations in functional form

Community Support
1

I have a table with columns a, b, c. Can I calculate the correlation matrix of cor[a;a], cor[a;b],cor[a;c] using functional form somehow?

?[table; (); 0b; (`aa`ab`ac)!((cor; `a; `a); (cor; `a; `b);(cor; `a; `b));

How can i generate the list of the last argument? (cor; a;b)

2

q)?[t; (); 0b; (`aa`ab`ac)!((cor; `a; `a); (cor; `a; `b);(cor; `a; `b))]

aa ab ? ? ? ? ac

1 ?-0.4255875 -0.4255875

Are you looking for this

You missed “]” in given code?

3

You can use functional form for sure:<o:p></o:p>

<o:p> </o:p>

t:flipab`c!“f”$3 cut 9?1.0<o:p></o:p>

a:( `$(“_” sv) each string a)!(cor),/:a:raze a,/::a:cols t<o:p></o:p>

?[t; (); 0b;a]<o:p></o:p>

<o:p> </o:p>

But I won’t suggest you to do this, since this will result in a table with many columns.<o:p></o:p>

<o:p> </o:p>

It is much better to use dictionary instead:<o:p></o:p>

<o:p> </o:p>

ft cor/::ft:flip t<o:p></o:p>

<o:p> </o:p>

But using  /:: is also not optimal since you are calculating twice the same numbers. So you should try to optimize it.<o:p></o:p>

<o:p> </o:p>

But even this is not optimal. When you calculate the correlation matrix you are more interested in numbers which will exceed a given threshold, say 0.85 .<o:p></o:p>

<o:p> </o:p>

You can calculate for each series the fourier transformation. Afterwards you can use  first few coefficients to index a grid structure. This will help you to  find similar series. Details can be found here:<o:p></o:p>

http://cs.nyu.edu/shasha/papers/statstream.html<o:p></o:p>

<o:p> </o:p>

HTH<o:p></o:p>

<o:p> </o:p>

Kim<o:p></o:p>

<o:p> </o:p>

<o:p> </o:p>

<o:p> </o:p>

Von: personal-kdbplus@googlegroups.com [mailto:personal-kdbplus@googlegroups.com] Im Auftrag von sharma.naveen888@gmail.com
Gesendet: Mittwoch, 13. November 2013 17:52
An: personal-kdbplus@googlegroups.com
Betreff: [personal kdb+] calculating correlations in functional form<o:p></o:p>

<o:p> </o:p>

I have a table with columns a, b, c. Can I calculate the correlation matrix of cor[a;a], cor[a;b],cor[a;c] using functional form somehow?<o:p></o:p>

?[table; (); 0b; (`aa`ab`ac)!((cor; `a; `a); (cor; `a; `b);(cor; `a; `b));

How can i generate the list of the last argument? (cor; a;b)<o:p></o:p>

–
Submitted via Google Groups

4

Thanks Kim! two beautiful solutions and the added bonus of the link.

regards,