partitioned and splayed table

Community Support
1

I created a table that is partitioned (by date) and splayed and saved to disk using .Q.dpft . The documentation says the following: “The table cannot be keyed at present”. Does this mean you can’t add a the grouped attribute (`g#) to one or more of the columns? Is there another way of doing the partitioning and splaying such that this can be accomplished?

Thanks,
Bob

2

Bob,keying and setting attributes are two different thingsyou can’t key the splayed tablebut you can set p, g or s attributes wherever the data allows itnote that a p attribute takes -far- less space than a gand kdb+ is not like many other databases where you have to indexevery which way to get decent performanceyou may get acceptable speed without any attributesSimonOn Aug 11, 3:37?pm, "Bob Pisani" <bob.pis...> wrote:&gt; I created a table that is partitioned (by date) and splayed and saved to&gt; disk using .Q.dpft . The documentation says the following: "The table cannot&gt; be keyed at present". Does this mean you can't add a the grouped attribute&gt; (g#) to one or more of the columns? Is there another way of doing the> partitioning and splaying such that this can be accomplished?>> Thanks,> Bob</bob.pis…>

3

Simon,

I’m trying to understand how to most effectively warehouse and data mine using kdb+. I’ve loaded several fact tables from one of our other databases. These tables range in size from 50m records to 400m. I partitioned and splayed all of them. As you stated, you can’t key a splayed table. In reading the q-sql and joins documentation, it seems like keys are very important and crucial to doing most sql like queries. I did load some smaller dimension tables that are keyed. But something we do quite often is to join fact tables together across common dimensions. Without keys, how would I do this?

I’m also not clear on when splaying and partitioning should be used based on size of table, i.e. < 1m, < 100m, < 1b, >1b etc… I ask because the data I loaded is only for 1 year. So one of those fact tables would go from 400m to 1.5b+ very easily as an example.

Thanks,
Bob

4


<11a2a48c0809021139w71b82ccfmba70d0c6ef9dc3be@mail.gmail.com>

eW91J2xsIGZpbmQgc29tZSBiYWNrZ3JvdW5kIGFib3V0IGpvaW5zIGhlcmU6CgpodHRwczovL2Nv
ZGUua3guY29tL3RyYWMvd2lraS9SZWZlcmVuY2Uvam9pbnMKCnRhYmxlIHNpemUgLSBhbnN3ZXIg
aXMgb2YgY291cnNlICJpdCBkZXBlbmRz4oCmIgoKaG93IG1hbnkgY29sdW1ucz8gaG93IG1hbnkg
Y29sdW1ucyB0eXBpY2FsbHkgaW4gdXNlPwp3aGF0IHNvcnQgb2YgcXVlcmllcyAoaWUgd2hvbGUg
eWVhcnMgZGF0YSBhdCBvbmNlLCBvciBmb3Igc2VsZWN0ZWQgbW9udGhzKT8KCm1lbW9yeSBwZXJt
aXR0aW5nIG9uIGEgNjRiaXQgT1MgeW91IGNvdWxkIHByb2JhYmx5IGp1c3Qgc3BsYXkgdGhlCjQw
ME0tMS41QiB0YWJsZSBhbmQgbm90IGJvdGhlciB0byBwYXJ0aXRpb24sIGJ1dCBJJ2QgZ3Vlc3Mg
cGVyIG1vbnRoCndvdWxkIGJlIG1vcmUgbm9ybWFsCgoyMDA4LzkvMiBCb2IgUGlzYW5pIDxib2Iu
cGlzYW5pQGdtYWlsLmNvbT46Cj4gU2ltb24sCj4KPiBJJ20gdHJ5aW5nIHRvIHVuZGVyc3RhbmQg
aG93IHRvIG1vc3QgZWZmZWN0aXZlbHkgd2FyZWhvdXNlIGFuZCBkYXRhIG1pbmUKPiB1c2luZyBr
ZGIrLiBJJ3ZlIGxvYWRlZCBzZXZlcmFsIGZhY3QgdGFibGVzIGZyb20gb25lIG9mIG91ciBvdGhl
ciBkYXRhYmFzZXMuCj4gVGhlc2UgdGFibGVzIHJhbmdlIGluIHNpemUgZnJvbSA1MG0gcmVjb3Jk
cyB0byA0MDBtLiBJIHBhcnRpdGlvbmVkIGFuZAo+IHNwbGF5ZWQgYWxsIG9mIHRoZW0uIEFzIHlv
dSBzdGF0ZWQsIHlvdSBjYW4ndCBrZXkgYSBzcGxheWVkIHRhYmxlLiBJbgo+IHJlYWRpbmcgdGhl
IHEtc3FsIGFuZCBqb2lucyBkb2N1bWVudGF0aW9uLCBpdCBzZWVtcyBsaWtlIGtleXMgYXJlIHZl
cnkKPiBpbXBvcnRhbnQgYW5kIGNydWNpYWwgdG8gZG9pbmcgbW9zdCBzcWwgbGlrZSBxdWVyaWVz
LiBJIGRpZCBsb2FkIHNvbWUKPiBzbWFsbGVyIGRpbWVuc2lvbiB0YWJsZXMgdGhhdCBhcmUga2V5
ZWQuIEJ1dCBzb21ldGhpbmcgd2UgZG8gcXVpdGUgb2Z0ZW4gaXMKPiB0byBqb2luIGZhY3QgdGFi
bGVzIHRvZ2V0aGVyIGFjcm9zcyBjb21tb24gZGltZW5zaW9ucy4gV2l0aG91dCBrZXlzLCBob3cK
PiB3b3VsZCBJIGRvIHRoaXM/Cj4KPiBJJ20gYWxzbyBub3QgY2xlYXIgb24gd2hlbiBzcGxheWlu
ZyBhbmQgcGFydGl0aW9uaW5nIHNob3VsZCBiZSB1c2VkIGJhc2VkIG9uCj4gc2l6ZSBvZiB0YWJs
ZSwgaS5lLiA8IDFtLCA8IDEwMG0sIDwgMWIsID4xYiBldGMuLi4gSSBhc2sgYmVjYXVzZSB0aGUg
ZGF0YSBJCj4gbG9hZGVkIGlzIG9ubHkgZm9yIDEgeWVhci4gU28gb25lIG9mIHRob3NlIGZhY3Qg
dGFibGVzIHdvdWxkIGdvIGZyb20gNDAwbSB0bwo+IDEuNWIrIHZlcnkgZWFzaWx5IGFzIGFuIGV4
YW1wbGUuCj4KPiBUaGFua3MsCj4gQm9iCj4KPgo+Cj4gT24gVHVlLCBBdWcgMTIsIDIwMDggYXQg
Mzo0NCBBTSwgc2ltb24gPHNpbW9uLmdhcmxhbmRAZ21haWwuY29tPiB3cm90ZToKPj4KPj4gQm9i
LAo+Pgo+PiBrZXlpbmcgYW5kIHNldHRpbmcgYXR0cmlidXRlcyBhcmUgdHdvIGRpZmZlcmVudCB0
aGluZ3MKPj4geW91IGNhbid0IGtleSB0aGUgc3BsYXllZCB0YWJsZQo+PiBidXQgeW91IGNhbiBz
ZXQgYHAsIGBnIG9yIGBzIGF0dHJpYnV0ZXMgd2hlcmV2ZXIgdGhlIGRhdGEgYWxsb3dzIGl0Cj4+
Cj4+IG5vdGUgdGhhdCBhIGBwIGF0dHJpYnV0ZSB0YWtlcyAtZmFyLSBsZXNzIHNwYWNlIHRoYW4g
YSBgZwo+PiBhbmQga2RiKyBpcyBub3QgbGlrZSBtYW55IG90aGVyIGRhdGFiYXNlcyB3aGVyZSB5
b3UgaGF2ZSB0byBpbmRleAo+PiBldmVyeSB3aGljaCB3YXkgdG8gZ2V0IGRlY2VudCBwZXJmb3Jt
YW5jZQo+PiB5b3UgbWF5IGdldCBhY2NlcHRhYmxlIHNwZWVkIHdpdGhvdXQgYW55IGF0dHJpYnV0
ZXMKPj4KPj4gU2ltb24KPj4KPj4gT24gQXVnIDExLCAzOjM3IHBtLCAiQm9iIFBpc2FuaSIgPGJv
Yi5waXMuLi5AZ21haWwuY29tPiB3cm90ZToKPj4gPiBJIGNyZWF0ZWQgYSB0YWJsZSB0aGF0IGlz
IHBhcnRpdGlvbmVkIChieSBkYXRlKSBhbmQgc3BsYXllZCBhbmQgc2F2ZWQgdG8KPj4gPiBkaXNr
IHVzaW5nIC5RLmRwZnQgLiBUaGUgZG9jdW1lbnRhdGlvbiBzYXlzIHRoZSBmb2xsb3dpbmc6ICJU
aGUgdGFibGUKPj4gPiBjYW5ub3QKPj4gPiBiZSBrZXllZCBhdCBwcmVzZW50Ii4gRG9lcyB0aGlz
IG1lYW4geW91IGNhbid0IGFkZCBhIHRoZSBncm91cGVkCj4+ID4gYXR0cmlidXRlCj4+ID4gKGBn
IykgdG8gb25lIG9yIG1vcmUgb2YgdGhlIGNvbHVtbnM/IElzIHRoZXJlIGFub3RoZXIgd2F5IG9m
IGRvaW5nIHRoZQo+PiA+IHBhcnRpdGlvbmluZyBhbmQgc3BsYXlpbmcgc3VjaCB0aGF0IHRoaXMg
Y2FuIGJlIGFjY29tcGxpc2hlZD8KPj4gPgo+PiA+IFRoYW5rcywKPj4gPiBCb2IKPj4KPgo+Cj4g
Pgo+Cg==

5

Thanks for the response. But what about my question regarding no keys on splayed tables. How do you effectively join splayed tables, i.e. 2 fact tables such as trade order data with market data?