.Q.dpft is adding another date

Community Support
1

I have an hdb patitioned by date and I am trying to overwrite a table on disk.  .Q.dpft seems to work in my test environment but in production its adding a duplicate partition.  Anyone know why this may be happening?

I am just using it like

.Q.dpft[]:c:/hdb;2015.02.05;sym;table]`

There is already a 2015.02.05 partition so I thought the `table would get overwritten?

2

I see the problem, .Q.par[…] is pointing to the wrong location on disk.  I had moved some date folders to anther segmented disk to give more space.  Is there a way to reload and find all the correct date locations on disk?

3

yeh that’s a common problem. I tend to use http://code.kx.com/wiki/DotQ#Partitioned\_Database\_State

e.g.

f:{.Q.PV! sv/: (.Q.PD,'$string .Q.PV),:x,`}

ftable will give you all partitions for table sensitive to drives etc

f[`table]2015.02.05 will give you the physical location/s of the table of that partition. It will also give you all drives that the partition exists on. 

HTH,

Sean

4

Thanks very much Sean

 
On Wednesday, May 27, 2015 at 1:54:22 AM UTC-4, Sean O’Hagan wrote:

yeh that’s a common problem. I tend to use http://code.kx.com/wiki/DotQ#Partitioned_Database_State

e.g.

f:{.Q.PV! sv/: (.Q.PD,'$string .Q.PV),:x,`}

ftable will give you all partitions for table sensitive to drives etc

f[`table]2015.02.05 will give you the physical location/s of the table of that partition. It will also give you all drives that the partition exists on. 

HTH,

Sean

5

Roni,

Are you using par.txt file? You can specify how your hdb is segmented there (just list all the sub-directories).

Once you have that, you can use .Q.par to get the correct location. Without it, .Q.par is not very useful.

You can read more about par.txt here:
http://code.kx.com/wiki/Reference/pardottxt

6

Hi Himanshu, yes I am using par.txt, but if the partition is moved .Q.par will still point to the wrong location.  I got a great answer from someone at First Derivatives

.Q.dpft looks at how many segments there is in par.txtand decides which segment to write a particular date to based on <date>mod <# segments>. Naturally the outcome of modulo changes after you add asegment for a given date. Randomly distributing partitions to a new segmentisn’t an issue per se - but you will get duplicate partitions if you attempt towrite down data using .Q,dpft for a daily partition that already exists if itwas not originally mapped according to modulo. <o:p></o:p>

I can’t think of a way to change this behaviour whilestill using .Q.dpft. You could instead use .Q.PV!.Q.PD which will give you adictionary with dates as values and segments as keys, then lookup the segmentfor the date you want to save data down to disk, and use the set command insteadof .Q.dpft to overwrite the existing partition in that segment. However, itmight be tidier to just move partitions to the correct segment based on<date> mod <no segments>…

<o:p></o:p>

On Thursday, May 28, 2015 at 9:37:22 AM UTC-4, Himanshu Gupta wrote:

Roni,

Are you using par.txt file? You can specify how your hdb is segmented there (just list all the sub-directories).

Once you have that, you can use .Q.par to get the correct location. Without it, .Q.par is not very useful.

You can read more about par.txt here:
http://code.kx.com/wiki/Reference/pardottxt