how to avoid each?

Community Support
1

Hi Q Gods,

Is there any trick to avoid using each?

I found out this by implementing simple normal inverse function that each is fairly slow to use. It seems that one should avoid each as much as possible to take leverage of the vectorisation in Q.

However, for an example below, taking a Poisson distribution

(exp neg lambda) * prd (x#lambda) % 1 + til x

til itself is not vectorisable… i.e. If i put a vector after til, it will not work.

Any advice?

Thanks

2

yes, this is an example where eaching is necessary.

you can hide this fact from the end user by allowing them to pass a vector of parameters and handling it within the function.

for example:

poisson:{[l;k]
 if[0<max type each (l;k);:.z.s’[l;k]];
 p:exp[neg l] * l xexp k;
 p%:prd 1+til k;
 p}

q)poisson[2.5] 1
0.2052125
q)poisson[2.5] til 5
0.082085 0.2052125 0.2565156 0.213763 0.1336019
q)poisson[til 5] 1
0 0.3678794 0.2706706 0.1493612 0.07326256

3

This would hide the interface but in terms of the computational intensity it is the really the same isn’t it?

For normal and its inverse, i used rational approximation so the code can be easily vectorised.

It seems there is no other way in this case. (unless there is a rational approximation for discrete pdf…)

Thanks anyway.

4

Hi,

For discrete distribution like Poisson, think you can vectorize the power/factorial bit, that should be much quicker.

p:{exp[neg x]*(prds[“f”$i#x]j)%prds[“f”$1+til i:max y]j:-1+y}

p[1.5;1 2 3 4 5]

For large lambda (>50) you can use normal pdf with correct to approximate it anyway.

Regards,

Xi

5

For large lambda (>50) you can use normal pdf with correct to approximate it anyway. // i like this approach. brilliant. Thanks!

6

actually I tried to implement it today. This augmented solution is a beast. The only concern I have is that essentially we are trading off between speed and memory.

But I guess this is the most Q way of doing it.