# Saving with auto partition

**URL:** <https://forum.kx.com/t/saving-with-auto-partition/8919>\
**Category:** Community Support\
**Tags:** kdb-and-q\
**Created:** [July 7, 2014, 2:46pm UTC](https://forum.kx.com/t/saving-with-auto-partition/8919 "2014-07-07T14:46:00Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![skuvvv1](https://avatars.discourse-cdn.com/v4/letter/s/7ba0ec/32.png) [@skuvvv1](https://forum.kx.com/u/skuvvv1)\
**Post date:** [July 7, 2014, 2:46pm UTC](https://forum.kx.com/t/saving-with-auto-partition/8919/1 "2014-07-07T14:46:00Z")

</div>

Hi.  
At current moment I’m saving all incoming data directly to partitions(Date) which selected from my c# application(every record checking Date)…

Can it possible to save automatically all data?

Simple example what I mean, look at table:

2014.07.06 23:50:00.000 somedata

2014.07.06 23:55:00.000 somedata

2014.07.07 00:05:00.000 somedata

2014.07.06 23:57:00.000 somedata

2014.07.06 00:07:00.000 somedata

Save it to 2 partition 2014.07.06 and 2014.07.07 by generic ability of q?

If not, can it possible to do with premade func(example please)?

---

<div class="post-metadata">

**Author:** ![sohagan](https://avatars.discourse-cdn.com/v4/letter/s/6a8cbe/32.png) [@sohagan](https://forum.kx.com/u/sohagan)\
**Post date:** [July 7, 2014, 3:14pm UTC](https://forum.kx.com/t/saving-with-auto-partition/8919/2 "2014-07-07T15:14:00Z")

</div>

yes…have a look at .Q.dcfgnt…[http://code.kx.com/wiki/Cookbook/LoadingFromLargeFiles.](http://code.kx.com/wiki/Cookbook/LoadingFromLargeFiles.)

More explanation here: [https://groups.google.com/forum/#!searchin/personal-kdbplus/save$20all$20partitions/personal-kdbplus/bb\_AosPGMHk/71JDykKS4HUJ](https://groups.google.com/forum/#!searchin/personal-kdbplus/save$20all$20partitions/personal-kdbplus/bb_AosPGMHk/71JDykKS4HUJ)

Thanks,

Sean

---

<div class="post-metadata">

**Author:** ![skuvvv1](https://avatars.discourse-cdn.com/v4/letter/s/7ba0ec/32.png) [@skuvvv1](https://forum.kx.com/u/skuvvv1)\
**Post date:** [July 9, 2014, 9:45am UTC](https://forum.kx.com/t/saving-with-auto-partition/8919/3 "2014-07-09T09:45:00Z")

</div>

Thank, it what I wanted.

One thing I saw that after saving on disk, partition column saved too.

Better will be if it will be deleted because partition already contains that information.

But I have not skills to update this function -)

---

<div class="post-metadata">

**Author:** ![sohagan](https://avatars.discourse-cdn.com/v4/letter/s/6a8cbe/32.png) [@sohagan](https://forum.kx.com/u/sohagan)\
**Post date:** [July 10, 2014, 12:04pm UTC](https://forum.kx.com/t/saving-with-auto-partition/8919/4 "2014-07-10T12:04:00Z")

</div>

Vadim…this is untested, but works on my test db.

tab:(date:10?2014.07.06 2014.07.07;time:10?.z.t;vol:10?100f);

\d .Q

k)dpfnt:{[d;p;f;n;t];if[~&/qm’r:+en[d]t;'`unmappable];{[d;t;i;x]@[d;x;:;t[x]]}[d:par[d;p;n];r;\<r f]'!r;@[;f;`p#]@[d;`.d;:;r@&~f=r:!r];n}

k)dcft:{[d;c;f;t]\*(dpfnt[d;;f;t]).‘{[t;c;p](p;?[`. t;,(=;c;p);0b;a!a:.q.except[.q.cols `. t;c]])}[t;c]’?[;();();c]?[`. t;();1b;(,c)!,c]}

\d .

.Q.dcft[`:.;`date;`;`tab]

`tab

Slight mods to Aarons…removed the reference to the partition column in the .d file, and the writing process. If you have time can you test and get back to me.

Cheers,

Sean

---

<div class="post-metadata">

**Author:** ![skuvvv1](https://avatars.discourse-cdn.com/v4/letter/s/7ba0ec/32.png) [@skuvvv1](https://forum.kx.com/u/skuvvv1)\
**Post date:** [July 10, 2014, 6:31pm UTC](https://forum.kx.com/t/saving-with-auto-partition/8919/5 "2014-07-10T18:31:00Z")

</div>

Not works at my pc. Got error:

![]()

---

<div class="post-metadata">

**Author:** ![nat40023362](https://avatars.discourse-cdn.com/v4/letter/n/47e85d/32.png) [@nat40023362](https://forum.kx.com/u/nat40023362)\
**Post date:** [July 11, 2014, 10:56pm UTC](https://forum.kx.com/t/saving-with-auto-partition/8919/6 "2014-07-11T22:56:00Z")

</div>

Hi Vadim

You could use the attached script to save down data directly to directory. The first step you should perform in your q session is to load in  
your raw data with the correct data types. We can use your data layout in the initial post as an example:

e.g. rawData: flip (`date`time`sym`price)!0:[(“DTSF”;“,”);\<rawDataFileLocation\>]

Then once the raw data has been loaded into q, this must be enumerated like below where the result is saved to a directory path:

e.g. loadedData: .Q.en[dirPath;rawData]

dirPath could be in the form `:hdb/testdata/ for example.

You can now load the attached script by performing the below command into your q session:

\l dataSave.q

This script will save the data down to directory but without having the partition (date) column listed within your table. The data will by sectioned by date when saved under the directory path. The savefunc function uses five parameters which you should input when running the function (one of these being the loadedData enumerated table above).  
Some features are that the function can sort and apply attributes to the data within your table. The function will append data to the generated datapath directory in the correct format. If there is a problem with saving the data down, the error will be caught and displayed.

Once the script is loaded, you can run the function in the following format:

savefunc[loadedData;loadedDate;hdbdir;tableNamePath;sortCol]

e.g. to save down data for one particular date, run:

savefunc[loadedData;2014.04.01;`:hdb/testdata;`trade;`sym]

e.g. to save down data for all dates within your table, run:

savefunc[loadedData;;`:hdb/testdata;`trade;`sym] each exec distinct `date$date from loadedData

Expected output can be seen below

Saving data to directory :hdb/testdata/2014.04.01/trade/  
Saving data to directory :hdb/testdata/2014.04.02/trade/  
`:hdb/testdata/2014.04.01/trade/`:hdb/testdata/2014.04.02/trade/

Hope this helps!  
(Please see script attached)

Kind Regards

Natalie Inkpin

Financial Software Developer

AquaQAnalytics

---

<div class="post-metadata">

**Author:** ![sohagan](https://avatars.discourse-cdn.com/v4/letter/s/6a8cbe/32.png) [@sohagan](https://forum.kx.com/u/sohagan)\
**Post date:** [July 13, 2014, 8:54am UTC](https://forum.kx.com/t/saving-with-auto-partition/8919/7 "2014-07-13T08:54:00Z")

</div>

copy and paste the code I wrote exactly as it is and it will work. The funcs are k funcs, and you have missed the k) prefix.

Thanks,

Sean

---

<div class="post-metadata">

**Author:** ![skuvvv1](https://avatars.discourse-cdn.com/v4/letter/s/7ba0ec/32.png) [@skuvvv1](https://forum.kx.com/u/skuvvv1)\
**Post date:** [July 13, 2014, 12:24pm UTC](https://forum.kx.com/t/saving-with-auto-partition/8919/8 "2014-07-13T12:24:00Z")

</div>

Hello Natalie  
Thank you for your script, it helped me and I can modify it further.&nbsp;