Search Order: Wordlists, Vocabularies and Modules

The definitions within the Forth dictionary are divided into groups called WORDLISTs. A wordlist is identified by a unique number called a WID (Wordlist IDentifier), which is returned when a wordlist is created by the word WORDLIST.

At any given time the system has a "search-order", which is an array of WID values representing the wordlists which are searched. This is the CONTEXT array. The system also uses one WID to contain any new definitions. This is called the CURRENT wid.

Vocabularies are named wordlists. When a vocabulary is created by VOCABULARY <name> a word is built which has a new wordlist. When <name> executes the wordlist replaces the first entry in the search order

Modules are special wordlists for hiding implementation details that should not be modified by application programmers.

Wordlists and Vocabularies

Creation

: WORDLIST      \ -- wid                              16.6.1.2460
Create a new wordlist and return a unique identifier for it.

: VOCABULARY    \ -- ; VOCABULARY <name>
Create a VOCABULARY called <name>. When <name> executes, its wordlist replaces the first entry in the search order

: voc>wid       \ xt(voc) -- wid
Return the WID from a vocabulary with the XT supplied.

Searching

1 value CheckSynonym?   \ -- flag
If true, words with the synonym bit set in the header will return the original word's xt, otherwise the xt of the child of SYNONYM will be returned. The setting affects SEARCH-WORDLIST and any words that use it, e.g. ', ['] and FIND.

: SEARCH-WORDLIST       \ c-addr u wid -- 0 | xt 1 | xt -1    16.6.1.2192
Search the given wordlist for a definition. If the definition is not found then 0 is returned, otherwise the XT of the definition is returned along with a non-zero code. A -ve code indicates a "normal" definition and a +ve code indicates an IMMEDIATE word.

: Search-Context        \ c-addr len -- 0 | xt 1 | xt -1
Perform the SEARCH-WORDLIST operation on all wordlists within the search order. Returns -1 for a "normal" word and +1 for an IMMEDIATE word.

: FIND          \ c-addr -- c-addr 0 | xt 1 | xt -1           6.1.1550
Perform the SEARCH-WORDLIST operation on all wordlists within the current search order. This definition takes a counted string rather than a c-addr/u pair. The counted string is returned as well as the 0 on failure. On success, -1 is returned for normal words, or 1 is returned for immediate words.

: FORTH         \ --                                  16.6.2.1590
Install Forth Wordlist into search-order.

: FORTH-WORDLIST        \ -- wid                              16.6.1.1595
Return the WID of the FORTH wordlist.

: ResetMinSearchOrder   \ --
Reset the minimum search-order. The minimum search-order reflects a minimal set of WIDs which make up the search order when ONLY is executed.

: >MIN-ORDER    \ wid --
Add a given WID to the minimum search-order.

: GET-CURRENT   \ -- wid                                      16.6.1.1643
Return the WID for the wordlist which holds any definitions made at this point.

: SET-CURRENT   \ wid --                                      16.6.1.2195
Change the wordlist which will hold future definitions.

: GET-ORDER     \ -- widn...wid1 n                            16.6.1.1647
Return the list of WIDs which make up the current search-order. The last value returned on top-of-stack is the number of WIDs returned.

: SET-ORDER     \ widn...wid1 n -- ; unless n = -1            16.6.1.2197
Set the new search-order. The top-of-stack is the number of WIDs to place in the search-order. If N is -1 then the minimum search order is inserted.

: ONLY          \ --                                          16.6.2.1965
Set the minimum search order as the current search-order.

: ALSO          \ --                                          16.6.2.0715
Duplicate the first WID in the search order.

: PREVIOUS      \ --                                          16.6.2.2037
Drop the current top of the search order.

: DEFINITIONS   \ --                                          16.6.1.1180
Set the current top of the search order as the current definitions wordlist.

: VOC?          \ wid -- flag
Return TRUE if 'wid' is actually a vocabulary

: .VOC          \ wid --
Display the name of a vocabulary if the WID is a valid wordlist identifier associated with a vocabulary.

: ORDER         \ --
Display the current search-order. WIDs created with VOCABULARY are displayed by name, others are displayed as numeric representations of the WID.

: VOCS          \ --
Display all vocabularies by name.

: WIDS          \ -- ; display wordlists by address or VOC name
Display all created wordlists by address or vocabulary name.

: -ORDER        \ wid --
Remove all instances of the given wordlist from the CONTEXT search order.

: +ORDER        \ wid --
The given wordlist becomes the top of the search order. Duplicate entries are removed.

: ?ORDER        \ wid -- flag
Return true if the given wordlist is in the search order.

Removing words

: trim-dictionary       \ start end --
Unlink all definitions in the memory region from start to end. See MARKER for more details of removing words from the dictionary. The dictionary pointer DP and HERE remain unchanged.

: cut-dictionary        \ start --
Unlink all definitions in the memory region from start to HERE. Reset the dictionary pointer to start.

: prune:        \ --
Starts a nameless definition that is added to the prune chain and is later executed by children of MARKER. See MARKER for more details. Pruning words are passed the start and end addresses of the region being pruned. The stack action of the definition must be:

  start end -- start end

: prunes        \ xt -- ; add xt to prune chain
Adds the xt to the prune chain that is executed by children of MARKER. See MARKER for more details. Pruning words are passed the start and end addresses of the region being pruned. The stack action of xt must be:

  start end -- start end

: remember:     \ --
Starts a nameless definition that is added to the remember chain that is executed by MARKER. See MARKER for more details. The stack action of the new definition must be:

  --

: remembers     \ xt -- ; add xt to remember chain
Adds xt to the remember chain that is executed by MARKER. See MARKER for more details. The stack action of xt must be:

  --

: marker        \ "<spaces>name"                        6.2.1850
MARKER <name> creates a word that when executed removes itself and all following definitions from the dictionary. MARKER is the ANS replacement for FORGET. MARKER automatically trims all vocabulary and wordlist vocabulary-based chains. If you need to clean up your data structures, you can add code to do this using the words PRUNE:, PRUNES, REMEMBER: and REMEMBERS. When MARKER runs, the 'remember' chain words are executed to construct preservation data. When the child of MARKER, <name>, is run, all the words in the 'prune' chain are executed to remove/restore the data to its previous state.

: anew          \ " name" --
A variant of MARKER that executes a previous child of MARKER of the same name if it exists, and then creates the marker. This allows you to place ANEW FOO at the start of a source file being debugged so that previous versions of the code are always replaced.

: Empty         \ --
Remove all words added since the system was loaded or SAVEd.

: forget        \ "<spaces>name"                        15.6.2.1580
Used in the form FORGET <name>, <name> and all following words are removed from the dictionary. This word is marked as obsolescent in the ANS specification, and is replaced by the extensible and more powerful word MARKER.

: $forget       \ $ --
FORGET the word whose name is the given counted string. See FORGET and MARKER.

Processing words in a wordlist

: (MAX-DEF)     \ wid-copy -- addr c-addr
Returns addr and top definition pointer from a copy of a wordlist. The thread is then truncated by one ready for the next call.

: WalkWordList  \ xt wid --
Walk through a wordlist calling the definition XT for each word. The definitions are walked in reverse chronological order. The definition at XT will be passed the THREAD# and NFA. This provides a future-proof method of parsing through a wordlist. It will be supported by future versions of the compiler. The XT definition has the stack form:


: MyDef    \ thread# nfa -- flag ; Return TRUE to continue

: WalkAllWordLists      \ xt-to-call --
Call the given XT once for each WORDLIST. The callback is given the WID and a flag and will return TRUE to continue the walk or false to abandon it. The FLAG supplied will be TRUE if the WID represents a VOCABULARY and FALSE if the WID represents a child of WORDLIST.


: MyDef    \ wid flag -- t/f ; return TRUE to continue

: WalkAllWords  \ xt --
Walk through all wordlists calling the given XT for each word. The definitions are walked in reverse chronological order of wordlists and then by reverse chronological order within each wordlist. When run, the XT will be passed the THREAD# and NFA. This provides a future-proof method of parsing through all wordlists. The XT definition has the stack form:


: MyDef    \ thread# nfa -- flag ; return TRUE to continue

: traverse-wordlist  \ xt wid -- ; Forth2012
Walk through the wordlist identified by wid calling the definition xt for each word. The words in the wordlist are walked in reverse chronological order. The word defined by xt is passed an nt, which in VFX Forth is an NFA. The XT definition has the stack form:


: MyDef    \ nt -- flag ; Return TRUE to continue

: name>string   \ nt/nfa -- caddr len ; Forth2012
Given an NT/NFA return the name string.

: name>interpret  \ nt/nfa -- xt ; Forth2012
Convert an NT/NFA to the corresponding xt.

: name>compile  \ nt/nfa -- xt1 xt2 ; Forth2012
Given an NT/NFA return xt1, the xt of the word, and xt2, the word used to compile it. If xt1 is immediate, xt2 is of EXECUTE, otherwise it is of COMPILE-WORD.

: CheckDict     \ --
Check the dictionary for corruption and if corrupt perform a #-418 THROW.

: Xt>Wid        \ xt -- wid|0
Attempts to locate the wordlist which contains the given XT. This word is designed for interpreter extensions and tools - it is not thread safe or re-entrant! Xt>Wid searches all wordlists and vocabularies - it can be slow.

: MoveNameToWid \ nfa new-wid -- okay?
Detach the the word whose nfa is given from its wordlist and attach it to the wordlist specified by new-wid. The word is attached to the new WID at the correct place in a thread to match its original chronological origin. Okay? is returned true if the operation was successful.

: changeNameWid { nfa oldwid newwid -- }
Move the word whose nfa is given from the wordlist oldwid to the wordlist newwid. No error checking is performed. ChangeNameWid is much faster than MoveNameToWid.

Source Code Modules

Apart from wordlists and vocabularies, VFX Forth provides 'source modules'. A MODULE is a section of source code which handles a given task. Rather than having all the factored 'sub-words' built into the public dictionary, a module exists in its own private wordlist and only provides visible access to those words which have been deliberately EXPORTED by the author. This method helps to improve the maintainability of large source projects both for single programmers and for group efforts.

When using this system, the implication is that a function exported by the author will be maintained and not change its meaning or implementation in an 'invisible' manner. Unexported words may change at any time.

For example, a module written by one person for use by another may require a sub-word to lay a string in the dictionary. If initially this word takes a counted string and builds a 0 terminated one in the dictionary it is possible that other sources will use this function for their own use. If at a later date the author of the original module needs to store strings in unicode format due to a change in the overall architecture of the module all other unauthorised uses of the sub-word will break through no fault of the original author. By hiding the mechanics of an API in a module this breakage cannot happen.

Module definition

: Module        \ <"name"> -- old-current
Begin the definition of a new source module. Modules can be nested and the EXPORTs from any module are placed in the current user definitions vocabulary.

: End-Module    \ old-current --
Mark the end of the current module under definition.

: EXPORT        \ old-current -- old-current ; EXPORT <name>
Export a module definition into the user's definition wordlist. The dictionary header for the word is relinked from the wordlist in which it was defined to the user's definition wordlist.

  EXPORT <name>

: Set-Init-Module       \ xt --
St the initialisation action of a module, which can be triggered by INIT-MODULE <name>. Must be executed within a module definition, and the xt must have no stack effect ( -- ).

  ' <action> SET-INIT-MODULE

: Set-Term-Module       \ xt --
Set the termination action of a module, which can be triggered by TERM-MODULE <name>. Must be executed within a module definition, and the xt must have no stack effect ( -- ).

  ' <action> SET-TERM-MODULE

Module management

: INIT-MODULE   \ "<name>" -- ; INIT-MODULE <name>
Calls the initialiser of the module whose name follows.

: TERM-MODULE   \ "<name>" -- ; TERM-MODULE <name>
Calls the terminator of the module whose name follows.

: REQUIRES      \ "<name>" -- ; REQUIRES <name>
Specifies by name a module which is required in order to compile the current source code. If the required module is not present compilation is ABORTed.

 requires MyModule

: EXPOSE-MODULE         \ -- ; EXPOSE-MODULE <name>
This word will add the private word-list of the module <name> to the search order. It is a debugging aid and should only be used as such. Using this method to get at a module's internal definitions defeats the purpose of the module mechanism.

 expose-module MyModule

An Example Module

The code below defines a module with one public word. The module itself doesn't actually do anything of consequence but it does show the definition syntax.

After compilation the only publically available words will be the two exported at the bottom of the module. All other definitions will be hidden and can only be accessed after an EXPOSE-MODULE is executed. In this way the actual implementation of the API can be isolated, only the author needs to worry about it.


   MODULE  counter

   variable counter

   : incr-counter          \ --
     1 counter +!
   ;

   : get-counter           \ -- val
     counter @
   ;

   : set-counter           \ val --
     counter !
   ;

   : CounterInitialise     \ --
     0 set-counter
   ;

   : Counter@++            \ -- value
     get-counter incr-counter
   ;

   ' CounterInitialise SET-INIT-MODULE
   ' CounterInitialise SET-TERM-MODULE

   EXPORT  CounterInitialise     \ Public word to init
   EXPORT  Counter@++            \ Fetch value and incr.

   END-MODULE

: WIDInfo       \ wid --
Display loads of information about a given wordlist