Training ::SHE+ILA::

More about Training ::SHE+ILA::

In order to test the effectiveness of the Markov FSM, we built a small LLM (should we call it a SLM -- Small Language Model) with only 4096 nodes, that we could run in Real Mode (640K byte limit) under DOS 5.0 . We used the public domain version of the Bhagavad Gita as the training text. In practice we would use a much larger sample of plaintext and more like 500,000 nodes. Each node occupies 128 bytes (32*2 for the branches, 32*2 for the weightings). The following Assembler code was used to set up the data :
MOV ES, 1000h ; Point to the root node
MORE: .READ AL ; Read the next prompt or training data character
JC ENDFILE ; Out of data
AND AL, 1Fh ; Make uppercase
SHL AL,1
SHL AL,1 ; Keep only the lower 5 bits of the character (A_Z, and 5 spares)
XOR AH,AH ; Set AX to 0000 0000 0dat ta00
MOV BX,AX ; Put it in BX to select the field in the current node for this character
MOV AX,ES:[BX] ; Get the appropriate branch out of this node
OR AX,AX ; Is there already a node for this exit?
JNZ ALREADY ; Yes there is
MOV AX,[NEWNODE] ; No, we need a new node
ADD AX,8 ; Increment by 8 paragraphs (128 bytes)
CMP AX,9000h ; Have we run out of free nodes?
JA ENDFILE ; Yep so save and quit
MOV [NEWNODE],AX
MOV ES:[BX],AX ; Make the current node point to it
ALREADY: INC word ptr ES:[BX+2] ; Increment the Markov weighting for this branch
MOV ES,AX ; And go to the next node
JMP MORE ; Go round for the next prompt (or training) character
ENDFILE: .SAVE ; Checkpoint it all to disk
INT 20h ; And quit