::SHE+ILA:: Multi-Page Segments [$94] (updated)

More about ::SHE+ILA:: Multi-Page Segments [$94] (updated)

If a segment needs to be bigger than 32K bytes, then it will need more than one page of storage. The first page contains a list of LBA's of the other pages forming the segment (like a PRODOS type 3 file) as part of the page header, and each subsequent page contains a pointer back to the first page. We also need to make the dope vector pointers larger, as they may need to span several megabytes. (For example, the final element of a buf situated on the first page may reside on the final page.) We can't split the dope vector into multiple chunks, one per page, because that would mean having to read every page of the segment from disk if we wanted to access an element near the end. The garbage collector (::CKPT::) needs to follow all the dope vectors (not just those situated on the first page of the segment), and the table of LBA's (which point to the subsequent pages). Note that ::CKPT:: may need to swap-in those pages from disk while it compacts the data and reallocates free space. This can be queued in a "fork block" to run at a lower priority. So far this is not yet implemented, as the test datastore has been confined to single-page segments. (Except for external data, such as MP3 or graphics images, which are currently "raw binary" blocks in their own dataspace.)
Normally, each entry in the dope vector takes 16 bits for the offset (allowing access anywhere on the same 32K page) and 16 bits for the element length; but where the segment is multi-page then each entry in the dope vector takes 32 bits for the offset (allowing access up to 4 gigabytes away) and 16 bits for the length. Only the SHE kernel is allowed to read/write the Buf descriptor fields (i.e. the flag, dope vector length, and dope vector entries) so it is okay for them to be in binary rather than BCD (as here there is no danger that binary patterns could be misinterpreted as tags).