Dear all,
i have a question: if I have 8 csv file that togheter occupy 1GB on Hard Drive, how much memory RAM do I need to work whit these?
It’ll depend on what’s the data inside of those. Usually it’ll be less memory, but I can think of an example when it’ll be more ;) (0,0,0… Read as ints or longs for instance)
Why don’t you try loading one of them and looking at memory load?
Cheers,
Oleg
I have tried to load them and kdb get out of memory :(
Il giorno lunedì 17 dicembre 2012 12:08:03 UTC+1, _Oz_ ha scritto:
It’ll depend on what’s the data inside of those. Usually it’ll be less memory, but I can think of an example when it’ll be more ;) (0,0,0… Read as ints or longs for instance)
Why don’t you try loading one of them and looking at memory load?
Cheers,
Oleg
have you tried loading them into a splayed table?
http://code.kx.com/wiki/Cookbook/LoadingFromLargeFilesAndSplaying
If low on memory, use -g 1 command line option and also perhaps .Q.gc after each chunk of loading.