To load large files, we need to use .Q.fs and the following recipe:https://code.kx.com/trac/wiki/Cookbook/LoadingFromLargeFilesAndSplayingHowever, the recipe assumes that the csv file does not give the columntitles. What if the first row are the column symbols? I know the’enlist’ in the syntax:(“SDFIS”;enlist “,”)will read the first line as a the column names, but when this iscalled repeatedly in .Q.fs, the program seems to interpret subsequentdatarows as header symbols.Does anyone know of a nice way of handling this?
if you’re using a *nix machine, you could use sed to easily delete thefirst line before q gets to process it.If that’s not an option, you can write your own .Q.fs that takes abyte offset which denotes where to start processing the .csv file.this way you can ignore the first line where the headers are, anduse (“SDFIS”; “,”)On Feb 8, 1:25?pm, Michael <princetonbac…> wrote:> To load large files, we need to use .Q.fs and the following recipe:https://code.kx.com/trac/wiki/Cookbook/LoadingFromLargeFilesAndSplaying>> However, the recipe assumes that the csv file does not give the column> titles. ?What if the first row are the column symbols? ?I know the> ‘enlist’ in the syntax:> (“SDFIS”;enlist “,”)> will read the first line as a the column names, but when this is> called repeatedly in .Q.fs, the program seems to interpret subsequent> datarows as header symbols.>> Does anyone know of a nice way of handling this?</princetonbac…>
csvguess.q or csvutil.q can simplify loading+saving csvs
https://code.kx.com/trac/browser/contrib/simon/csvguess/csvguess.q