# Compression Erorr

**URL:** <https://forum.kx.com/t/compression-erorr/9375>\
**Category:** Community Support\
**Tags:** kdb-and-q\
**Created:** [January 13, 2015, 10:40am UTC](https://forum.kx.com/t/compression-erorr/9375 "2015-01-13T10:40:00Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![vikas\_agarwal1](https://avatars.discourse-cdn.com/v4/letter/v/22d042/32.png) [@vikas\_agarwal1](https://forum.kx.com/u/vikas_agarwal1)\
**Post date:** [January 13, 2015, 10:40am UTC](https://forum.kx.com/t/compression-erorr/9375/1 "2015-01-13T10:40:00Z")

</div>

Hi,

I am using q on windows. I have to compress tables at the time of splaying itself.

For simplicity i have taken following example:

.z.zd:(16;2;1)

`:db/t/ set ( ti:09:31:00 09:30:01 09:30:02 09:30:03; p:33 33.5 33.2 33.3)

When i run above command i get following error:

Error: db/t/ti: This operation completed successfully.

.d file is created in db/t. running get `:db/t/.d give me list of columns.

Do i need to include some libraries to make compression work.

---

<div class="post-metadata">

**Author:** ![charlie1](https://avatars.discourse-cdn.com/v4/letter/c/ec9cab/32.png) [@charlie1](https://forum.kx.com/u/charlie1)\
**Post date:** [January 13, 2015, 10:42am UTC](https://forum.kx.com/t/compression-erorr/9375/2 "2015-01-13T10:42:00Z")

</div>

http://code.kx.com/wiki/Cookbook/FileCompression

**Q) I want to use algorithm #2 (gzip). Do I need additional libraries??**  
A) Yes, but these may already be installed on your system. It binds dynamically to zlib (more info at&nbsp;http://zlib.net). For windows we chose to be compatible with the dlls from&nbsp;http://www.winimage.com/zLibDll/zlib125dll.zip. For linux and solaris you may find it convenient to install zlib using your package manager software, or consult your system administrator for assistance. Note that your will require the matching 32 or 64bit libs for kdb+, i.e. 32bit libs for 32bit kdb+, 64bit libs for 64bit kdb+.

---

<div class="post-metadata">

**Author:** ![vikas\_agarwal1](https://avatars.discourse-cdn.com/v4/letter/v/22d042/32.png) [@vikas\_agarwal1](https://forum.kx.com/u/vikas_agarwal1)\
**Post date:** [January 13, 2015, 1:33pm UTC](https://forum.kx.com/t/compression-erorr/9375/3 "2015-01-13T13:33:00Z")

</div>

Thanks Charles. I should have seen it first place.

Its working fine with no errors. The issue i am facing is the slow compression rate, As i am using .Q.fs for 1 gb file with upsert command

.Q.fs[{`:newfile/ upsert .Q.en[`:.;] &nbsp;flip colnames!(“PSFFFcI”;“,”)0:x}]`:examplecsv/trades2014\_01.csv

it takes around 4 minutes. I think the reason may be upsert for the 131000j buffer at a time, it decompress the existing compress file and merge with compression.May be i can use .Q.fsn

Is these any workaround?

&nbsp;

---

<div class="post-metadata">

**Author:** ![charlie1](https://avatars.discourse-cdn.com/v4/letter/c/ec9cab/32.png) [@charlie1](https://forum.kx.com/u/charlie1)\
**Post date:** [January 13, 2015, 1:38pm UTC](https://forum.kx.com/t/compression-erorr/9375/4 "2015-01-13T13:38:00Z")

</div>

appending to a compressed file is expensive (lots of file ops plus decompress/recompress of last block),  
so try to minimize how many times it calls to do that.  
e.g. as you say by using .Q.fsn

or by using a hardware accelerator card (such as an AHA card)

or initial write out uncompressed, and then compress afterwards.

---

<div class="post-metadata">

**Author:** ![charlie1](https://avatars.discourse-cdn.com/v4/letter/c/ec9cab/32.png) [@charlie1](https://forum.kx.com/u/charlie1)\
**Post date:** [January 13, 2015, 1:49pm UTC](https://forum.kx.com/t/compression-erorr/9375/5 "2015-01-13T13:49:00Z")

</div>

and/or multithread the compression part in peach. Left as an exercise for the reader ;-)