Processing non-structured text data

Hi all,I have huge logs files that need to be processed. Each log file isabout 500MB.These are non-structured mixtures of strings and numbers.To give an example, the first row might contain trader id, trade_id,quantity, limit_price, etc.And then the next row contains the acknowledgement returned back fromthe exchange after the trade is sent to the exchange.The next a few rows may contain info related to earlier trades.And then the next row may contain the fills info for that trade sentto the exchange. Of course, we use the “trade_id” to keep track of theinfo.And also, in between rows, the log file contains the status of theorder book (order depth, etc.)Is there a way to read the whole text file into KDB and then I can useMatlab to interface with KDB and read one line at a time to parse thetext data?I am familiar with the regular expressions in Matlab that’s the why Iprefer to do the data extraction that way.Any thoughts on how to do the KDB part?Thanks a lot!

Check out the read0 function:

https://code.kx.com/trac/wiki/Reference/read0