How could q read Unicode csv file correctly?

Hello all. I use the code below to read a csv file which contains Chinese characters, but the output is not shown correctly.

CstInfo:("sdsssbs"; 1# ",")0:CustomerInfo1.csv`

The content of csv file is:

CstID,CstACDate,CstName,CstProp,CstCatag,IsFinanceInv,CstContact
GFSM00001,???OTC?,??,1,??,

 When I look into the variable  CstInfo  in q terminal, it shows a totally different string:

How could I solve this problem?

Thanks!

Tongwei,

Your file was encoding in UTF-8, and you command has already successfully loaded the file into q workspace.

The reason you are not seeing the proper characters on screen is due to the encoding of your console, which defaults to GB18030 on Chinese versions of Windows. You can run this command to switch your console encoding to UTF-8 as well:

\chcp 65000

However, please note that Windows’ implementation of UTF-8 is somewhat buggy. So do expect to see some partly corrupted displays when you are showing information on a UTF-8 console sometimes. One possible work-around is to use an IDE instead, or work through the web interface of kdb+.

Hi, Flying,

Thank you so much for your reply. I can use the web interface and the method really helps me out.

? 2017?9?23??? UTC+8??8:48:21?Flying???

Tongwei,

Your file was encoding in UTF-8, and you command has already successfully loaded the file into q workspace.

The reason you are not seeing the proper characters on screen is due to the encoding of your console, which defaults to GB18030 on Chinese versions of Windows. You can run this command to switch your console encoding to UTF-8 as well:

\chcp 65000

However, please note that Windows’ implementation of UTF-8 is somewhat buggy. So do expect to see some partly corrupted displays when you are showing information on a UTF-8 console sometimes. One possible work-around is to use an IDE instead, or work through the web interface of kdb+.