Welcome Guest. Please Login or Register. Apr 1st, 2018, 03:28am
ATTENTION MEMBERS: Conforums will be closing it doors and discontinuing its service on April 15, 2018. We apologize Conforums does not have any export functions to migrate data. Ad-Free has been deactivated. Outstanding Ad-Free credits will be reimbursed to respective payment methods.
Thank you Conforums members.
Speed up Liberty BASIC programs by up to ten times!
Compile Liberty BASIC programs to compact, standalone executables!
Overcome many of Liberty BASIC's bugs and limitations!
ITEACH and IQPDF
« Thread started on: Mar 15th, 2014, 11:11pm »
This could be a long post, so get a coffee and settle down ....
Conversely you could watch these short (8-9 Min) videos that introduce the concepts and results.
The ITEACH overview:
The Q&A Database Editor:
However from the viewpoint of this forum, none of this would have been as positive a result without LBB. Using LBB got the application(s) over a critical speed bump and also allowed packaging into a single EXE file. The latter is important for the environment into which they work, a large corporate network. I mainly run them off a data stick as portable (non installed) apps and having no DLLs or extra files helps keep my IT people at bay
The Primary reason though is that LBB gives such a significant speed boost, more so on the slower, older PCs where the CPU is the limiting factor rather than the file system (as on my home PC room heater).
Why is speed so important? Well the ITEACH apps are all about teaching and over the past year when these apps have been live teaching B777 technical to line pilots, I have been taking very close notice on whether and how they work.
The problem has been to work out just why it works.
Interestingly for us in the aviation industry, the move to PDFs for our tech books was seen originally as a huge problem. There was something comforting about your dog eared multi-tagged manuals. However they were heavy and tedious to amend (a frequent occurrence). The bean-counters didn't like the expense of 20 lbs of paper either.
However the emergence of the PDF did open up possibilities for a very old idea of mine - IQUIZ. I could possibly take my Q&A database package and instead of just using it to generate test papers, use it to teach as well. All good ideas have problems, and mine was the update cycle. Those PDFs were changing every couple of weeks so linking a PDF page number to a question was pointless.
IQPDF was born - a dynamic real time text search into the mangled interior of complex PDFs. I then put that idea into IQUIZ and ITEACH came along.
But why this speed problem? We humans have a finite perception of time and even our spoken and written language is riddled with references, not to scientific time, but to perceptual, body clock referenced time.
For example, 'a blink of an eye' and 'a heartbeat'. The former is approximately a 25th of a second. We use this to generate the illusion of moving pictures (think frame-rate) and is effectively the limiting refresh rate of the optical nerves, rods and cones etc. The latter is around a second, the average resting pulse being 60 beats per minute. So why is this relevant?
An aside - On board the B777 we have a useful, but actually flawed piece of hardware called the Electronic Flight Bag (EFB). For reasons that are not relevant here, it has a very slow processor and memory architecture that makes it very slow indeed. The control is by touch screen and an array of hardware buttons around the screen perimeter (here is a link to a picture to avoid straining your imagination: http://www.boeing.com/Features/2010/04/img/bca_EFB_K62824-02_300.jpg
Anyway we (and other airlines) were having many hardware failures centered on those buttons. A real puzzle - and an expensive one. However it only took a few minutes of reflection about how they were being used, to solve it. It was too slow, so when a button was pressed there was no response in a 'heartbeat', so the button was pressed again - harder. This started the interrupt cycle again and successive repeats of this process saw the button being pushed harder and harder each time. End result - failure. Solution - education in the sort term, faster hardware, better software (multiple button press traps) in the long term.
ITEACH teaches because it is accurate and fast. It wouldn't work as well (or indeed at all) without either of those attributes. Without the speed of LBB generated code I could not do the multiple 3D matrix calculations that enable the search function to work accurately in anything like the time available (1 second max display time for the correct PDF page).
I wrote ITEACH / IQPDF originally in LB and it worked, but not that well, for the reasons above. I tried other languages, nothing in it. LBB proved the charm really and made it possible. There are still areas I can improve on, only this morning I was playing with the new PROFILER function and found a couple of areas where even LBB can be improved dramatically. Even, to be fair, an area where LBB is actually slower than LB - but that was being masked overall by all the rest of speed improvements.
The problem is now that the whole application(s) is so fast that I have real problems with any form of subjective timing at all. I have to write dedicated test sequences to compare changes in techniques. Vagaries in OS responses are now more significant for individual searches.
Which is a nice place to be. The IQPDF search app is just 1360 lines of code (including a debug function - which I have never taken out). The Search engine is now a stand alone EXE that is called by the ITEACH / IEDIT modules, which makes them easier to modify as I am continuously 'playing' with the GUI to (mostly) improve user interaction. The separate search EXE makes life very easy.
But the screen is saying I am running out of words - so enough already. You will probably find this long and boring, apologies if you do - watch the movies above.
Later perhaps I will share some code. It ain't that clever, now that I know how to do it
Ken
This is the beginning of the 'blurb' that goes with the full documentation. Mainly of interest is the back story, which really shows how improving / changing technology and Moore's Law have changed the game continuously.
IQUIZ and its’ associated modules has a genesis that goes back 20+ years to a requirement to digitize an aircraft technical quiz. In the technology of the day these were normally produced by typewriter stencils and a Gestetner duplicator. Tests were problematic and time consuming to produce apart from constructing the questions and answers themselves. Once used, tests were rapidly compromised - often to the point of uselessness - by word of mouth sharing or blatant copying.
Today, we have digital solutions that are often so complex, awkward and time consuming to use that the questions are rarely changed, have reduced relevance over time, and no prime document linkage. So despite the use of modern technology, little functional or pedagogical improvement has been achieved.
IQUIZ was originally conceived, designed and written 24 years ago with a L1011 database on a TRS-80 and CP/M. Although the core idea was sound and the concepts of randomized question delivery and answer order presentation (with a serialized marking sheet) were very successful, there was always the next important logical step that just could not be taken.
A good Question & Answer database is an incredibly valuable piece of IP and teaching resource, and if you can integrate that database into a teaching mechanism alongside the quiz itself, then you have a very powerful learning tool. The single, insurmountable, problem in the past was that to take that step; each question has to be linked to a section / page of a core document. In subjects such as Bible studies, Basic Sciences etc, the core document does not change (much) over time. Technical subjects and particularly aviation related subjects are, by their very nature, often quite the opposite. Documents change. Content, placement and page numbers migrate unpredictably. Revision cycles are variable. Linking a question to a single page of a PDF is a straightforward technique and this relationship can be set at time of composition. However when that core document mutates, then the question database must be suitably modified. This is so time consuming and difficult as to be impracticable in a normal world.
Time passes and the core document referencing problem remained an itch that could not be scratched. Paradoxically the solution came when working on some software for a very old PC running, of all things, CP/M and attempting to mine a depreciated database. To cut an otherwise long story short(er), I found that quite complex binary matrix structures could be represented and reproduced reliably in a hexadecimal text format using a little math and some rather mind twisting matrix manipulations.
The modern document package of choice is the PDF and although this is an excellent format, the internal structure makes any searching function slow and inflexible to the point of tedium. The simple answer was to post-process the PDF, extract the text and page number details, then convert that rather untidy data into a compressed 3D matrix in hexadecimal form.
That was the charm. Using this concept, building a 'words found on a page' and 'pages associated with words' relationship was relatively straightforward. More important it was robust and repeatable. The method was spun off into IQPDF as a PDF search tool.
Logged