sql - Selecting different number of columns in a CSV file -
the task extract data multiple csv files according criteria. file contains sampleid (this criteria) , other columns. @ end of file there measurement values under 0...100 named columns (the numbers actual names of columns). make bit more interesting there can variations in different csv files, depending on customer needs. means measurement data count can 15, 25, 50 etc. no more 100 , no variations within 1 file. data placed in end of line, there set of columns before numbers.
i'd have sql statement can accept parameters:
select {0} {1} sampleid = {2}
0 numbers, 1 csv file name , 2 sampleid looking for. other solution came mind columns after last fix column. don't know possible or not, thinking out loud.
please descriptive, sql knowledge basic. appreciated.
so managed solve it. code in vb.net, logic quite clear.
private function getdatafromcsv(sampleids integer()) list(of keyvaluepair(of string, list(of integer))) dim datafiles() string = system.io.directory.getfiles(outputfolder(), "*.csv") dim results list(of keyvaluepair(of string, list(of integer))) = new list(of keyvaluepair(of string, list(of integer))) if datafiles.length > 0 , sampleids.length > 0 index integer = 0 sampleids.length - 1 if sampleids(index) > 0 each file in datafiles if system.io.file.exists(file) dim currentid string = sampleids(index).tostring() dim filename string = path.getfilename(file) dim strpath string = path.getdirectoryname(file) dim conn oledb.oledbconnection = new oledb.oledbconnection("provider=microsoft.jet.oledb.4.0; data source=" & strpath & "; extended properties='text; hdr=yes; fmt=delimited'") dim command oledb.oledbcommand = conn.createcommand() command.commandtext = "select * [" & filename & "] 'where sample id = " & currentid conn.open() dim reader oledb.oledbdatareader = command.executereader() dim numberoffields = reader.fieldcount while reader.read() if reader("sample id").tostring() = currentid 'if found write particle data output file dim particles list(of integer) = new list(of integer) field integer = 0 numberoffields - 1 particles.add(cint(reader(field.tostring()))) next field results.add(new keyvaluepair(of string, list(of integer))(currentid, particles)) end if end while conn.close() end if next file end if next index return results else messagebox.show("missing csv files or invalid sample id(s)", "internal error", messageboxbuttons.ok, messageboxicon.exclamation) end if end function
Comments
Post a Comment