read /write csv with line break in fields

Community Support
1

Hello:
   Is there a way to read/write csv with line breaks in fields? 

The standard 0: will not work.

Thanks in advance. 

2

Can you give a snippet of your csv file?<o:p></o:p>

<o:p> </o:p>

Thanks,<o:p></o:p>

Kim<o:p></o:p>

<o:p> </o:p>

Von: personal-kdbplus@googlegroups.com [mailto:personal-kdbplus@googlegroups.com] Im Auftrag von Hao Deng
Gesendet: Donnerstag, 25. Dezember 2014 02:37
An: personal-kdbplus@googlegroups.com
Betreff: [personal kdb+] read /write csv with line break in fields<o:p></o:p>

<o:p> </o:p>

Hello:<o:p></o:p>

   Is there a way to read/write csv with line breaks in fields? <o:p></o:p>

The standard 0: will not work.<o:p></o:p>

<o:p> </o:p>

Thanks in advance. <o:p></o:p>

<o:p> </o:p>

–
Submitted via Google Groups

3

How about simply representing and storing a new line with a semi colon instead of a \n character

q)  t:( sym:$(); price:float$(); text:())

q) t insert (A;10.0;“Yes its a lovely day;Better than yesterday”)

q) t

sym price text

A   10    “Yes its a lovely day;Better than yesterday”

This you can easily write and read in csv format

write: q) save `:/home/t.csv

read:  q) m:update text:string[text] from (“SFS”; enlist (“,”))  0: `:/home/t.csv

q) m~t

1b

If you wish to output the lines to a user in stdout

as they were originally intended to be you could do something like this..

q) {show “;” vs x} each m`text

“Yes its a lovely day”

“Better than yesterday”

4

Well no, the CSV is not generated by me , and there might already semicolon exists

5

can you give us example data?

6

I don’t have access to a computer now, but you can create one with one line, two fields, and one of the field like “ABC \n def”

7

Ah ok, I assumed you ran into the issue below when you tried using

the \n within a string

 q) t:( sym:AB; text:(“Hello \n World”; “Back \n Tomorrow”))

 q)  save `:/home/t.csv

t.csv looks like this, which is hard to read back in

sym,text

A,Hello

 World

B,Back

 Tomorrow

Was this the problem originally?

8

/ write text containing \n to file.txt

q)hopen[`:file.txt] 0N!“\n” sv ","0:5#enlist til 5;

“0,0,0,0,0\n1,1,1,1,1\n2,2,2,2,2\n3,3,3,3,3\n4,4,4,4,4”


/ lets take a look at whats written - ‘cat -E’ displays new lines as $

q)\cat -E file.txt

“0,0,0,0,0$”

“1,1,1,1,1$”

“2,2,2,2,2$”

“3,3,3,3,3$”

“4,4,4,4,4”


/ read in the file as bytes with (1:)

q)(1:)(`:file.txt;0;0Wj)

0x302c302c302c302c300a312c312c312c312c310a322c322c322c322c320a332c332c332c332..


/ can cast to chars to display \n

q)“c”$(1:)(`:file.txt;0;0Wj)

“0,0,0,0,0\n1,1,1,1,1\n2,2,2,2,2\n3,3,3,3,3\n4,4,4,4,4”

You can replace 1: with 0: in the above, but 0: is built to work with text/ascii – so will do the char conversion + break a string into a list of strings at each \n it encounters. If you wanted to, you could get the \n back with sv when using 0:

q)“\n” sv (0:)(`:file.txt;0;0Wj)

“0,0,0,0,0\n1,1,1,1,1\n2,2,2,2,2\n3,3,3,3,3\n4,4,4,4,4”

Further reading:

You can also take a look at implementation of .Q.fsn/.Q.fs to see how it utilises 1: to read csv data in chunks.

Kind Regards,

Mohammad Noor