corrupt .d file

Community Support
1

Situation:
Lets say in a partitioned hdb, the .d file for some of the dates is corrupt i.e (missing cols and/or the cols are not in the same order) ?

Question:

how does q catch it ?

2

Wouldn’t call it corrupt per se, I would just say not up to date/not modified

When loading a hdb, q reads the the .d file of any table located in the latest partition and any splayed tables living in the root. This is to get the columns for those tables. The meta of a table also respects the latest partition, so when running meta you are actually just getting the types and attributes of the latest partition.

When selecting any column that is existent in this .d file, q will default to thinking that that column also lives in every other partition. Thus it will attempt to read that file on a query involving that column. If the file is missing you simply will get an error from the os saying it does not exist.

example:

//splay
q):./testdir/2017.01.01/t/ set ([]a:10?10;b:10?10) :./testdir/2017.01.01/t/
q):./testdir/2017.01.02/t/ set ([]a:10?10;b:10?10;c:10?10) :./testdir/2017.01.02/t/

//load
q)\l ./testdir

//tables and meta data, notice the meta data is taken from latest partition
q)\a
,`t
q)meta t

c   t f a
date d   
a   j   
b   j   
c   j   
q)cols t
dateabc
q)get`:./2017.01.01/t/.d
ab
q)get`:./2017.01.02/t/.d
ab`c

//query b works
q)count select b from t
20

//query c doesn’t work
q)count select c from t
k){0!(?).@[x;0;p1[;y;z]]}
'./2017.01.01/t/c. OS reports: No such file or directory
.
?
(+abc!:./2017.01.01/t;();0b;(,c)!,c)
q.Q))\

//add the c column to the old partition manually and select works
q):./2017.01.01/t/c set 10?10 :./2017.01.01/t/c
q)count select c from t
20

I would keep the .d files up to date though as a good dba practice

Also - the order of the columns in your result is according to the order in the latest .d file too.

Fwiw - some functions in this tool may be useful to you
http://code.kx.com/wiki/Contrib/dbmaintdotq

HTH,
Sean