Author |
Topic: Maximum size of Random Access File (Read 688 times) |
|
Richard Russell
Administrator
member is offline
 
Posts: 1348
|
 |
Maximum size of Random Access File
« Thread started on: Apr 14th, 2016, 4:24pm » |
|
There's a comment at the LB Community Forum which refers to a 4 Gigabytes "Windows imposed limit" for a Random Access File. I don't know whether LB 4 imposes such a limit, but Windows certainly doesn't (at least, so long as the storage device is formatted NTFS rather than FAT).
In LBB not only can the file be of any length, up to the capacity of the storage device, but even the record number isn't restricted to a 4G limit!
Try this, if you have enough free space on your drive. It first writes and then reads the 5-billionth record in a file defined as having records which are 1 byte long (it will take a few seconds to run):
Code: open "hugeraf.dat" for random as #f len = 1
field #f, 1 as v$
v$ = "x"
put #f, 5000000000
close #f
open "hugeraf.dat" for random as #g len = 1
field #g, 1 as w$
print "File size is "; lof(#g)
get #g, 5000000000
close #g
print "Data read = "; w$
kill "hugeraf.dat"
Richard.
|
|
 Logged
|
|
|
|
RobM
Junior Member
member is offline
Posts: 91
|
|
Re: Maximum size of Random Access File
« Reply #1 on: Apr 14th, 2016, 5:16pm » |
|
As I was typing that I wasn't sure if it was correct for the modern file standard 
I would doubt anyone working with LB or LBB would want to work with a database that large. As your code proves, it is possible but exceedingly slow.
|
| « Last Edit: Apr 14th, 2016, 5:16pm by RobM » |
Logged
|
|
|
|
Richard Russell
Administrator
member is offline
Posts: 1348
|
|
Re: Maximum size of Random Access File
« Reply #2 on: Apr 14th, 2016, 5:36pm » |
|
on Apr 14th, 2016, 5:16pm, RobM wrote:| As your code proves, it is possible but exceedingly slow. |
|
Creating a 5 Gbyte file from scratch is slow, but that's not what typically happens with a database. Once the file is created you should find that seeking and reading/writing is in fact exceedingly fast, and having a large, sparse, file may be easier than deriving the record number with a hash and having to deal with collisions.
With modern storage devices having typically hundreds of Gigabytes, or even Terabytes, of capacity I certainly wouldn't rule out using a RAF considerably bigger than 4 Gbytes. LBB is entirely suitable for that.
The real point I was trying to make is that LB 4 is very limiting. Having a 4 Gbyte file-size limit or a 32-bit record-number limit is not really acceptable in 2016. LBB has neither of those limits.
Richard.
|
|
|
|
Richard Russell
Administrator
member is offline
Posts: 1348
|
|
Re: Maximum size of Random Access File
« Reply #3 on: Apr 14th, 2016, 9:42pm » |
|
on Apr 14th, 2016, 5:36pm, Richard Russell wrote:| Having a 4 Gbyte file-size limit or a 32-bit record-number limit is not really acceptable in 2016. |
|
Chris Iverson writes at the Community Forum that "The file size parameter in many of the Win32 file access APIs is still 32-bit for compatibility reasons" but what APIs are those? GetFileSize, SetFilePointer and the WIN32_FIND_DATA structure have all been 64-bit-size compatible since at least Windows 95 - that's over twenty years!
Whatever the reason for LB 4 having a 4 Gbyte file-size limit might be, it's not that the Win32 API ever enforced it.
Richard.
|
|
Logged
|
|
|
|
RobM
Junior Member
member is offline
Posts: 91
|
|
Re: Maximum size of Random Access File
« Reply #4 on: Apr 14th, 2016, 10:29pm » |
|
Chris recently mentioned that much of what he writes in regards to how LB works in the background is just speculation 
|
|
Logged
|
|
|
|
Richard Russell
Administrator
member is offline
Posts: 1348
|
|
Re: Maximum size of Random Access File
« Reply #5 on: Apr 15th, 2016, 10:03am » |
|
on Apr 14th, 2016, 10:29pm, RobM wrote:| Chris recently mentioned that much of what he writes in regards to how LB works in the background is just speculation |
|
Well indeed it must be, since only Carl knows what actually happens (and even he may not always know if it's buried within SmallTalk). Similarly any comments I make on the internal workings of LB are just intelligent guesses based on the symptoms and experience gained from writing LBB.
But in this case Chris was referring not to the internals of LB but to the Windows API. Claiming that many Win32 APIs support only a 32-bit file-size is, I'm afraid, plain wrong. Yes it can be easier to use a 32-bit size value, and I've done that many times myself (indeed it was a limitation of LBB pre v3.00), but it's not actually enforced by the API.
Richard.
|
|
Logged
|
|
|
|
|
