Author |
Topic: LB 4.5.0 INPUTCSV 'broken' (Read 835 times) |
|
Richard Russell
Administrator
member is offline
 
Posts: 1348
|
 |
LB 4.5.0 INPUTCSV 'broken'
« Thread started on: Aug 16th, 2015, 1:42pm » |
|
I tried the new INPUTCSV statement in LB 4.5.0 with the test data from the Wikipedia page (listed below) and it doesn't work! Specifically, it seems to get confused when a record contains one or more embedded quote characters (which in a CSV file are doubled-up from " to "").
The whole point of 'quoted' CSV records is that they can contain any characters, including commas, quotes and newlines. It looks to me as though INPUTCSV copes only with the case when commas are included, but that doesn't allow for arbitrary strings.
I am tempted to release a version 3.01 of LBB with the INPUTCSV statement correctly implemented. What do you think?
Richard.
CSV Code:Year,Make,Model,Description,Price
1997,Ford,E350,"ac, abs, moon",3000.00
1999,Chevy,"Venture ""Extended Edition""","",4900.00
1999,Chevy,"Venture ""Extended Edition, Very Large""",,5000.00
1996,Jeep,Grand Cherokee,"MUST SELL!
air, moon roof, loaded",4799.00 |
|
|
|
Richard Russell
Administrator
member is offline
Posts: 1348
|
|
Re: LB 4.5.0 INPUTCSV 'broken'
« Reply #1 on: Aug 17th, 2015, 08:33am » |
|
on Aug 16th, 2015, 1:42pm, Richard Russell wrote:| I am tempted to release a version 3.01 of LBB with the INPUTCSV statement correctly implemented. What do you think? |
|
INPUTCSV is the only new statement in LB 4.5.0 (the other keyword additions are all functions and can easily be supported in LBB by means of a library). It's potentially a useful feature, or at least it would be if it wasn't seriously broken. 
So in the interests of compatibility, and to give LB users an implementation of INPUTCSV that actually works, I could quite easily add it to LBB. But it would be helpful to know whether anybody else thinks it's worthwhile. 
Richard.
|
|
 Logged
|
|
|
|
tsh73
Full Member
member is offline
Gender: 
Posts: 210
|
|
Re: LB 4.5.0 INPUTCSV 'broken'
« Reply #2 on: Aug 17th, 2015, 11:07am » |
|
Sure working tool is better then no tool.
CSV is pretty much standard exchange format. Last time I mucked around it it was phone book transfer from old phone to new one.
Is where a possibility to specify a delimiter? Here in Russia Excel reads/writes CSV delimited with semicolon ( ; ).
|
| « Last Edit: Aug 17th, 2015, 11:08am by tsh73 » |
Logged
|
|
|
|
Richard Russell
Administrator
member is offline
Posts: 1348
|
|
Re: LB 4.5.0 INPUTCSV 'broken'
« Reply #3 on: Aug 17th, 2015, 1:13pm » |
|
on Aug 17th, 2015, 11:07am, tsh73 wrote:| Is there a possibility to specify a delimiter? |
|
Yes, I would like to do that. The use of an alternative delimiter (usually a semicolon) is common in countries which use the 'decimal comma' (Germany for example) and it is what Excel does automatically when running on a PC which is so configured.
I will need to devise a suitable syntax. I was wondering about:
Code: inputcsv #file, a, b, c, d, ... ' Comma delimiter
inputcsv ";", #file, a, b, c, d, ... ' Specified delimiter
Sadly Liberty BASIC provides very poor support for 'internationalisation'. This used to be a common problem with American software, and in the case of LB I'm afraid it still is.
I've tried my best to rectify this in LBB.
Richard.
|
|
Logged
|
|
|
|
net2014
New Member
member is offline
Posts: 37
|
|
Re: LB 4.5.0 INPUTCSV 'broken'
« Reply #4 on: Aug 17th, 2015, 2:13pm » |
|
I agree, I use csv files for transfer between different machines, but my functions seem to be much to slow so a faster fully functioning INPUTCSV would be much appreciated. Any additions/improvements are always most welcome.
Thanks for your time and effort in maintaining LBBooster. 
|
|
Logged
|
|
|
|
Richard Russell
Administrator
member is offline
Posts: 1348
|
|
Re: LB 4.5.0 INPUTCSV 'broken'
« Reply #5 on: Aug 19th, 2015, 12:46pm » |
|
on Aug 17th, 2015, 1:13pm, Richard Russell wrote:I was wondering about:
Code: inputcsv #file, a, b, c, d, ... ' Comma delimiter
inputcsv ";", #file, a, b, c, d, ... ' Specified delimiter |
|
I don't think anybody commented on this proposal. If I receive no objections I will assume it is acceptable; it could of course be used to read a Tab Separated Value (TSV) file:
Code: inputcsv chr$(9), #file, a, b, c, d, ... ' TSV file
I also propose to make INPUTCSV accept files with 'Unix-style' line terminations (i.e. LF rather than CRLF). This is a little non-standard for a Windows application but many CSV samples I have found are of that type, and it could significantly simplify using LBB to read CSV files generated on different platforms.
Here are some sources of 'reference' CSV files I have found so far, which I will use to validate INPUTCSV:
https://en.wikipedia.org/wiki/Comma-separated_values
http://www.creativyst.com/Doc/Articles/CSV/CSV01.htm
https://github.com/maxogden/csv-spectrum
Richard.
|
|
Logged
|
|
|
|
tsh73
Full Member
member is offline
Gender:
Posts: 210
|
|
Re: LB 4.5.0 INPUTCSV 'broken'
« Reply #6 on: Aug 19th, 2015, 6:01pm » |
|
Quote:| If I receive no objections I will assume it is acceptable |
|
it looks good so far.
Quote:| I also propose to make INPUTCSV accept files with 'Unix-style' line terminations (i.e. LF rather than CRLF). |
|
I see no problem if it will work on both.
|
|
Logged
|
|
|
|
pierscintilla
New Member
member is offline
Gender:
Posts: 30
|
|
Re: LB 4.5.0 INPUTCSV 'broken'
« Reply #7 on: Aug 31st, 2015, 08:45am » |
|
Every your observation is like a lighthouse!
It would be a very useful function.
Thank you for existing!!
Hi Pier
|
|
Logged
|
|
|
|
Richard Russell
Administrator
member is offline
Posts: 1348
|
|
Re: LB 4.5.0 INPUTCSV 'broken'
« Reply #8 on: Aug 31st, 2015, 11:17am » |
|
on Aug 31st, 2015, 08:45am, pierscintilla wrote:| It would be a very useful function. |
|
LBB v3.01, with the INPUTCSV statement, was released ten days ago! 
Does anybody know whether Carl has reacted to my report that INPUTCSV in LB 4.5.0 is broken (I posted it on the LB Bug Tracker wiki)? It seems very odd to me that he didn't check it with even the test data in the Wikipedia article, let alone the more comprehensive CSV example data sets that can be readily found on the web.
Richard.
|
|
Logged
|
|
|
|
Mystic
Junior Member
member is offline
Gender:
Posts: 53
|
|
Re: LB 4.5.0 INPUTCSV 'broken'
« Reply #9 on: Sep 10th, 2015, 9:45pm » |
|
This was a great add Richard! I only wish I would have saw it before I spent yesterday going through a CSV file manually pulling comma's from the data.
Was only 1250 lines and search worked great for it. I had to pull the weeds before my program would properly read in all of the lines so I could manipulate them.
Live and learn. I look forward to using that feature in the future. Looks like I may be converting another file soon.
Many thanks for your continued awesomeness!
|
|
Logged
|
- Rick
|
|
|
|
