Author |
Topic: Re: Help with Multi-user application (Read 1396 times) |
|
Richard Russell
Administrator
member is offline
 
Posts: 1348
|
 |
Re: Help with Multi-user application
« Thread started on: Sep 5th, 2016, 3:34pm » |
|
There's a thread at the Liberty BASIC Yahoo! Group about sharing files between multiple users. The underlying problem is that in LB 4.04 and LB 4.5.0 a random access file is always opened for both reading and writing - even if you only need to do one or the other!
This is a serious design flaw, because it makes it impossible for one LB program to be writing a random file whilst another LB program concurrently reads that same file, even though there's no fundamental reason why that should not work.
Fortunately LBB provides a solution. An undocumented feature (I really should add it to the help file) is that you can open a random file in a read-only mode, and in so doing allow concurrent access from multiple programs (a maximum of one of which is writing to the file while the others read from it).
Here's a simplistic example; in this primitive case you must start the 'writing' program before you start the 'reading' program, but it demonstrates the principle:
Writing program Code: open "SharedFile.dat" for output as #f len = 30
field #f, 20 as string$, 10 as number
do
i += 1
string$ = "My string ";i
number = i
put #f,i
call delay 100
loop until 0
end
sub delay d
timer d, [pause]
wait
[pause]
timer 0
end sub
Reading program Code: open "SharedFile.dat" for input as #f len = 30
field #f, 20 as string$, 10 as number
do
i += 1
gettrim #f,i
print string$; " "; number
call delay 110
loop until 0
end
sub delay d
timer d, [pause]
wait
[pause]
timer 0
end sub
Perhaps somebody would be kind enough to post something to the Yahoo! group about this capability of LBB.
Richard.
|
|
|
|
Alincon
Full Member
member is offline
Posts: 147
|
|
Re: Help with Multi-user application
« Reply #1 on: Sep 5th, 2016, 8:09pm » |
|
I think I understand the point of your 'primitive' example is to demonstrate the 'for output' and 'for input' phrases for random files.
But, please explain some things that I don't recall seeing before.
I assume that 'i += 1' is equivalent to 'i=i+1'
What is it that becomes 0 and ends the do loop? A binary condition?
How many records are written to the file?
What does the delay sub do that a timer statement does not do?
r.m.
|
|
 Logged
|
|
|
|
Richard Russell
Administrator
member is offline
Posts: 1348
|
|
Re: Help with Multi-user application
« Reply #2 on: Sep 5th, 2016, 8:51pm » |
|
on Sep 5th, 2016, 8:09pm, Alincon wrote:| I assume that 'i += 1' is equivalent to 'i=i+1' |
|
Correct.
Quote:| What is it that becomes 0 and ends the do loop? |
|
Nothing - that's the point: it's an infinite loop. Some people like to emphasise this by setting a variable to zero, such as:
Code: the.cows.come.home = 0
do
' an infinite loop
loop until the.cows.come.home
Actually in LBB there's a shorthand way of creating an infinite loop but it's not standard Liberty BASIC code:
Code: do
' an infinite loop
loop
Quote:| How many records are written to the file? |
|
It writes ten per second, so it depends on how long the program is left running. As I said, it was a primitive example to emphasise the principle.
Quote:| What does the delay sub do that a timer statement does not do? |
|
It's just an example of making the program more modular. If you want different delays in different places, calling a common subroutine is an elegant way to achieve it. However it's a technique that cannot safely be used in a program that uses branch labels as event handlers (because whilst the delay subroutine is executing the label will be out-of-scope). So it's almost always better to use SUB event handlers to avoid this problem.
Richard.
|
|
Logged
|
|
|
|
Rod
Full Member
member is offline
Gender: 
Posts: 110
|
|
Re: Help with Multi-user application
« Reply #3 on: Sep 6th, 2016, 08:03am » |
|
Richard, is it not the writing conflict that needs a solution. In a multiuser environment there will be multiple write attempts that need managed. A solution would be a welcome addition.
Perhaps I am misunderstanding the example.
|
| « Last Edit: Sep 6th, 2016, 08:05am by Rod » |
Logged
|
|
|
|
Richard Russell
Administrator
member is offline
Posts: 1348
|
|
Re: Help with Multi-user application
« Reply #4 on: Sep 6th, 2016, 11:41am » |
|
on Sep 6th, 2016, 08:03am, Rod wrote:| Richard, is it not the writing conflict that needs a solution. In a multiuser environment there will be multiple write attempts that need managed. |
|
A common usage case is that multiple users won't be simultaneously attempting to write; rather only one will be responsible for writing and the rest will only need to read. The point is that this case should be simple to implement, because it doesn't violate the usual file sharing provisions (a file may be concurrently open for writing once and for reading multiple times) but it isn't because LB 4 doesn't allow you to open a random file for input only.
That's the scenario that LBB has a simple solution for. If multiple users need to write to the file then that will require either a file locking protocol (such that only one user at a time has acquired the right to write) or opening the file in a shared-write mode. Contrary to popular perception, Windows does allow a file to be opened for shared writes (pass the FILE_SHARE_WRITE flag to the CreateFile API) but it can be tricky to guarantee data integrity.
Obviously both those scenarios are more complex to implement than the simple 'single writer' case. File locking is likely to require error trapping and retries (LBB's Structured Exception Handling will make this nicer to code) and the shared-write case will require opening the file using the Windows API. Unlike LB 4, LBB will allow you to use the native file-handling statements (like FIELD and PUT) with a file opened using the API, although I'd need to explain how it's done.
But there comes a point when it would be better to use a proper database manager designed to provide shared access (ODBC, MySQL, SQLite etc.) rather than to reinvent the wheel.
Richard.
|
|
|
|
Richard Russell
Administrator
member is offline
Posts: 1348
|
|
Re: Help with Multi-user application
« Reply #5 on: Sep 6th, 2016, 2:51pm » |
|
Here's one way to acquire write access using SEH:
Code: do
try
open "SharedFile.dat" for output as #f len = 30
success = 1
catch
call delay rnd(1) * 100
success = 0
end try
loop until success
Much nicer than ON ERROR GOTO!
To ensure that the file is locked for as short a time as possible it should be closed again as soon as the record(s) have been written. The file can be opened for input on a different handle, and kept open continuously, for reads to take place.
Something to bear in mind when implementing any kind of file sharing is that there may be various levels of data-buffering, with the result that a PUT statement will not necessarily cause the data to reach the file immediately! If this is an issue, following it with a second, dummy, PUT (perhaps you can reserve record 1 for this purpose) will usually be sufficient to flush the previously-written data to the file.
Richard.
|
|
Logged
|
|
|
|
Mystic
Junior Member
member is offline
Gender:
Posts: 53
|
|
Re: Help with Multi-user application
« Reply #6 on: Sep 6th, 2016, 3:23pm » |
|
Love the SEH concept. I will now attempt to implement this in a few of my programs. Elegant.
Unfortunately, where I work I cannot spin up any form of database server (long story), so I am stuck with writing my own databases, and do have a few I'm currently working on that may encounter the multiple user write issue.
I was going to try to do a crude form of checking if the file is being written to, such as create a lock-check file, if it's "1", try again in a few seconds sort of thing.
Or, in the case of the SEH, this might work better because if a user attempt to write to the file it SHOULD error, and the program can just retry until successful.
Because my programs are multiple user across a network, I always open and close my files only when actually required to be reading or writing to them. I never leave one hanging open like in a single user usage.
|
|
Logged
|
- Rick
|
|
|
Richard Russell
Administrator
member is offline
Posts: 1348
|
|
Re: Help with Multi-user application
« Reply #7 on: Sep 6th, 2016, 4:22pm » |
|
on Sep 6th, 2016, 3:23pm, Mystic wrote:| I was going to try to do a crude form of checking if the file is being written to, such as create a lock-check file, if it's "1", try again in a few seconds sort of thing. |
|
To avoid race hazards, you really need to test for write access being available and acquire that write access atomically (otherwise somebody else might have acquired the lock in the short period between the two). So a separate lock file or record won't really do the job, rather the only safe way is to try to open the file for output and retry if you can't.
It's the same issue as trying to protect against an error occurring when you attempt to delete or rename a file, by testing for the existence of the relevant file beforehand. That will succeed most of the time, but one day you will be unlucky and another process will jump in between the test and the delete/rename and it will fail.
The only safe way is to attempt the operation and accept that an error might occur. I've seen some people argue that trapping errors is a sign of bad coding, and that there is always a better way. But in fact in this case the converse is true: not doing it by error trapping is a sign of sloppy coding!
That's one reason why having a structured means of trapping and recovering from errors (such as SEH) is desirable, because it may be unavoidable.
Richard.
|
|
Logged
|
|
|
|
michael
New Member
member is offline
Posts: 28
|
|
Re: Help with Multi-user application
« Reply #8 on: Sep 6th, 2016, 5:36pm » |
|
Interesting concept.
|
|
Logged
|
I make program generators and some utilities. Its my hobby
|
|
|
roxyryan
New Member
member is offline
Posts: 4
|
|
Re: Help with Multi-user application
« Reply #9 on: Sep 7th, 2016, 4:57pm » |
|
Richard good news it seems as though I finally have a multi-user program without having to use any third party database software, and without any API calls!
The final clincher was the try/catch clause. I could not it working with 'on error goto' which does not seem to like jumping back into subroutines. But as soon as I used the try/catch clause it started working! It is an amazing enhancement!
I have been running my Pub software from 2 programs, one with my left hand and one with my right, deliberately trying to break it, but so far it has has got through without crashing.
I want to do some more testing and then will send my code, which is really quite simple at the end of the day.
Dermot
|
|
Logged
|
|
|
|
Richard Russell
Administrator
member is offline
Posts: 1348
|
|
Re: Help with Multi-user application
« Reply #10 on: Sep 7th, 2016, 9:02pm » |
|
on Sep 7th, 2016, 4:57pm, roxyryan wrote:| 'on error goto' which does not seem to like jumping back into subroutines. |
|
Once you've exited the scope of a subroutine you can't jump back in. That is, the GOTO may seem to work but something bad is sure to happen before too long! So if, when an error occurs, you need to jump to a label within a subroutine, put the ON ERROR statement itself within the scope of that subroutine.
Of course I'd rather that you use SEH (try...catch...end try) instead. There's no GOTO involved so no danger that you might, deliberately or by accident, try to jump somewhere you shouldn't.
Richard.
|
|
Logged
|
|
|
|
roxyryan
New Member
member is offline
Posts: 4
|
|
Re: Help with Multi-user application
« Reply #11 on: Sep 8th, 2016, 09:01am » |
|
Richard
I cannot break the program with the code below, it really works. I am sure there is prettier code to achieve the same, but this is simple and it works.
While testing I had displays to show when it was "caught" in a conflict, and I only had that happen in the second "catch" shown in the routine. Maybe I dont need the first catch?
' the example below is the code that has to be done for each file, in this case the orders file, #10
Code: ' -------------------------------------------------------
' If a file is to be accessed for read-only then:
gosub [openorders]
' do processing
close #10
' -------------------------------------------------------
' If a file is to be accessed for writing then:
gosub [set.orders.lock]
' do processing
gosub [rel.orders.lock]
' -------------------------------------------------------
[openorders] ' opens the file in the appropriate mode
if file.mode$ = "OUT" then
OPEN ordersname$ FOR random AS #10 len=215
else
OPEN ordersname$ FOR input AS #10 len=215
end if
file.mode$ = "" ' set flag back to default, ie input
FIELD #10,_
1 AS ORlock$,_ ' only used on record 1 (no data held on record 1)
1 AS ORuser$,_ ' only used on record 1
etc other fields
return
' -------------------------------------------------------
[set.orders.lock] ' This subroutine called to open the file and set the lock indicator
waited = 0
gosub [openorders] ' file.mode$ not set, so it defaults to open input mode
[set.orders.lock.repeat]
get #10, 1
if ORlock$ = "Y" then ' Indicator on record 1 of random file
close #10
waited += 1
if waited < 11 then
call delaysub, 500 ' keep trying 10 times
gosub [openorders] ' open in input mode
goto [set.orders.lock.repeat]
end if
for x = 1 to max.users
if ORuser$ = userscodes$(x) then
other.user$ = usersnames$(x)
end if
next x
noticetext1$ = "Orders File is locked by " + other.user$ ' This message should never normally appear
notice.x = 20: notice.y = 60: gosub [notice.box] ' (my own notice routine)
waited = 0
gosub [openorders] ' open in input mode
goto [set.orders.lock.repeat]
else
close #10
try
file.mode$ = "OUT": gosub [openorders] ' open in read/write mode
catch
call delaysub, 100
gosub [openorders]
goto [set.orders.lock.repeat]
end try
try
ORlock$ = "Y" ' set the lock now to prohibit other users
ORuser$ = user.code$ ' the user code for this user
put #10, 1
catch
close #10
call delaysub, 100
gosub [openorders] ' open in input mode
goto [set.orders.lock.repeat]
end try
end if
return
' -------------------------------------------------------
[rel.orders.lock] ' Release the lock and close the file
get #10, 1
ORlock$ = "N" ' Indicates no longer locked
ORuser$ = ""
put #10, 1
close #10
return
' -------------------------------------------------------
sub delaysub delaytime
timer delaytime, [pause]
wait
[pause]
timer 0
end sub
' -------------------------------------------------------
|
|
|
|
Richard Russell
Administrator
member is offline
Posts: 1348
|
|
Re: Help with Multi-user application
« Reply #12 on: Sep 8th, 2016, 1:29pm » |
|
on Sep 8th, 2016, 09:01am, roxyryan wrote:| I only had that happen in the second "catch" shown in the routine. Maybe I dont need the first catch? |
|
So, that means the OPEN never failed but the PUT did? I expect the explanation is that you opened the file for RANDOM (in order to preserve the existing contents) which probably succeeds even if the file is currently being written by another user. Then when you try to write to the file the sharing violation is triggered.
Personally I'd be inclined to leave the first TRY clause in place. It doesn't do any harm and you might find that the behaviour is different if (for example) the file is stored on a shared network server running a different OS.
I notice that you haven't randomized the delays. You'll probably get away with it, but it does increase the probability of a deadlock if two users keep retrying simultaneously.
Incidentally I don't know if this limitation of LBB is going to hit you, but there is a maximum of eight simultaneously-open random files.
Richard.
|
|
|
|
roxyryan
New Member
member is offline
Posts: 4
|
|
Re: Help with Multi-user application
« Reply #13 on: Sep 8th, 2016, 2:36pm » |
|
Richard
Thanks for the reminder, yes I will randomise the delays.
Also now that I am only opening when needed, I wont be hitting that 8 file limit, I have a maximum of 3 or 4 open at a time
Dermot
|
|
Logged
|
|
|
|
Mystic
Junior Member
member is offline
Gender:
Posts: 53
|
|
Re: Help with Multi-user application
« Reply #14 on: Sep 8th, 2016, 2:39pm » |
|
on Sep 8th, 2016, 1:29pm, Richard Russell wrote:I notice that you haven't randomized the delays. You'll probably get away with it, but it does increase the probability of a deadlock if two users keep retrying simultaneously.
|
|
Ah ha! Was wondering about the randomization of the delay. Now it makes sense! I didn't even think of the potential user-standoff occurring.
I love learning new things.
Thanks Richard, and everyone for a great discussion.
|
|
Logged
|
- Rick
|
|
|
|
