# Too many open files **URL:** \ **Category:** Community Support\ **Tags:** kdb-and-q\ **Created:** [June 10, 2016, 7:59am UTC](https://forum.kx.com/t/too-many-open-files/11066 "2016-06-10T07:59:00Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![carfield11](https://avatars.discourse-cdn.com/v4/letter/c/ec9cab/32.png) [@carfield11](https://forum.kx.com/u/carfield11)\ **Post date:** [June 10, 2016, 7:59am UTC](https://forum.kx.com/t/too-many-open-files/11066/1 "2016-06-10T07:59:00Z")
Hi all, just found out that it is possible to get “Too many open files” error by running a select statement on a HDB across a lot splay tables It look like current KDB will close the file handler only after the select statement return, I guess it would be better to close the file handler after KDB load the data file by file? ---
**Author:** ![charlie1](https://avatars.discourse-cdn.com/v4/letter/c/ec9cab/32.png) [@charlie1](https://forum.kx.com/u/charlie1)\ **Post date:** [June 10, 2016, 9:12am UTC](https://forum.kx.com/t/too-many-open-files/11066/2 "2016-06-10T09:12:00Z")
I expect you are encountering this when using compressed files, and the select has no row constraint? We can easily remove this error, however it would be at the cost of using much more address space - which would then make compressed files practically unusable in the 32bit version. There’s no limit within kdb+ itself which forces this error - it’s due to restrictions from the environment only, and you should be able to increase that with e.g. ulimit -n 4096 hth, Charlie ---
**Author:** ![ikorkhov1](https://avatars.discourse-cdn.com/v4/letter/i/d2c977/32.png) [@ikorkhov1](https://forum.kx.com/u/ikorkhov1)\ **Post date:** [June 10, 2016, 10:47am UTC](https://forum.kx.com/t/too-many-open-files/11066/3 "2016-06-10T10:47:00Z")
Hello Charles, Do you mean that setting ulimit -n to a higher value can increase a number of _compressed_ files that can be open simultaneously? I thought 4096 was hardcoded: [http://code.kx.com/wiki/Cookbook/FileCompression:](http://code.kx.com/wiki/Cookbook/FileCompression:) **\> Q) Is there a limit on the number of compressed files that can be open simultaneously?** \> A) Yes, currently the limit is 4096 files. There is no practical internal limit on the number of uncompressed files. I was never able to open more that 4096 compressed files at once no matter how high ulimit was. Or were you talking about a situation where low ulimit prevented kdb from opening _uncompressed_ files? Thank you, Igor ---
**Author:** ![charlie1](https://avatars.discourse-cdn.com/v4/letter/c/ec9cab/32.png) [@charlie1](https://forum.kx.com/u/charlie1)\ **Post date:** [June 10, 2016, 11:17am UTC](https://forum.kx.com/t/too-many-open-files/11066/4 "2016-06-10T11:17:00Z")
Hi Igor, I’m referring to compressed files only, and this scenario should arise very rarely assuming your schema is lots of rows per partition rather than lots of partitions with small number of rows per partition. It looks like we need to update that documentation. :-) In v3.1, release 2013.02.21, we removed the limit of 4096 open compressed files; number of open files is now just the limit from the OS. $ rlwrap q KDB+ 3.4 q).z.zd:17 2 6;{(hsym `$string x)set 1000#x}each til 10000 `:0`:1`:2`:3`:4`:5`:6`:7`:8`:9`:10`:11`:12`:13`:14`:15`:16`:17`:18`:19`:20`:2.. q)v:get each hsym key`:. q)count v 10000 q)system"ulimit -n" “32768” q)-21!`:0 compressedLength  | 96 uncompressedLength| 8016 algorithm         | 2i logicalBlockSize  | 17i zipLevel          | 6i upping the ulimit may require additional changes, e.g. see https://access.redhat.com/documentation/en-US/Red\_Hat\_Enterprise\_Linux/5/html/Tuning\_and\_Optimizing\_Red\_Hat\_Enterprise\_Linux\_for\_Oracle\_9i\_and\_10g\_Databases/chap-Oracle\_9i\_and\_10g\_Tuning\_Guide-Setting\_Shell\_Limits\_for\_the\_Oracle\_User.html thanks, Charlie ---
**Author:** ![ikorkhov1](https://avatars.discourse-cdn.com/v4/letter/i/d2c977/32.png) [@ikorkhov1](https://forum.kx.com/u/ikorkhov1)\ **Post date:** [June 10, 2016, 11:32am UTC](https://forum.kx.com/t/too-many-open-files/11066/5 "2016-06-10T11:32:00Z")
Thank you Charles, As far as I remember I got “Too many files” when I tried to run .Q.chk on a compressed database with _very_ wide tables, one of them had 2500 columns or so. But that was kdb 2.8 which explains why upping ulimit didn’t help. I had to create a slightly modified version of .Q.chk by  to solve the problem. Best regards, Igor ---
**Author:** ![carfield1](https://avatars.discourse-cdn.com/v4/letter/c/ba8739/32.png) [@carfield1](https://forum.kx.com/u/carfield1)\ **Post date:** [June 10, 2016, 3:57pm UTC](https://forum.kx.com/t/too-many-open-files/11066/6 "2016-06-10T15:57:00Z")
I see, thanks