Functions in DLLs and shared libraries

Introduction

VFX Forth supports calling external API calls in dynamic link libraries (DLLs) for Windows and shared libraries in Linux and other Unix-derived operating systems. Various API libraries export functions in a variety of methods mostly transparent to programmers in languages such as C, Pascal and Fortran. Floating point data is supported for use with Lib\x86\Ndp387.fth.

Before a library function can be used, the library itself must be declared, e.g.


LIBRARY: Kernel32.dll

Access to functions in a library is provided by the EXTERN: syntax which is similar to a C style function prototype, e.g.


EXTERN: int PASCAL SendMessage(
  HWND hwnd, DWORD mesg, WPARAM wparam, LPARAM lparam
);

This can be used to prototype the function SendMessage from the Microsoft Windows API, and produces a Forth word SendMessage.

  SendMessage  \ hwnd mesg wparam lparam -- int

For Linux and other Unices, the same notation is used. The default calling convention is nearly always applicable. The following example shows that definitions can occupy more than one line. It also indicates that some token separation may be necessary for pointers:


Library: libc.so.6

Extern: int execve(
  const char * path,
  char * const argv[],
  char * const envp[]
);

This produces a Forth word execve.

  execve       \ path argv envp -- int

The parser used to separate the tokens is not ideal. If you have problems with a definition, make sure that * tokens are white-space separated. Formal parameter names, e.g. argv above are ignored. Array indicators, [] above, are also ignored when part of the names.

The input types may be followed by a dummy name which is discarded. Everything on the source line after the closing ')' is discarded.

From VFX Forth v4.3 onwards, PASCAL is the default calling convention in the Windows version. The default for the Linux and OS X versions is "C". The default is always used unless overridden in the declaration.

Format


EXTERN: <return> [ <callconv> ] <name> '(' <arglist> ')' ';'

<return>    := { <type> [ '*' ] |  void }
<arg>       := { <type> [ '*' ] [ <name> ] }
<args>      := { [ <arg>, ]* <arg> }
<arglist>   := { <args> | void }        Note: "void, void" etc. is illegal.
<callconv>  := { PASCAL | WINAPI | STDCALL | "PASCAL" | "C" }
<name>      := <any Forth acceptable namestring>
<type>      := ... (see below, "void" is a valid type)

Note that during searches <name> is passed to the operating system exactly as it is written, i.e. case sensitive. The Forth name is case-insensitive.

As a standard Forth's string length for dictionary names is only guaranteed up to 31 characters for portable source code, very long API names can cause problems. Therefore the word AliasedExtern: allows separate specification of API and Forth names (see below). AliasedExtern: also solves problems when API functions only differ in case or their names conflict with existing Forth word names.

Calling Conventions

In the discussion caller refers to the Forth system (below the application layer and callee refers to a a function in a DLL or shared library. The EXTERN: mechanism supports three calling conventions.

Unless otherwise specified, the Forth system's default convention is used. Under Windows this is WINAPI and under Linux and other Unices it is "C".

Promotion and Demotion

The system generates code to either promote or demote non-CELL sized arguments and return results which can be either signed or unsigned. Although Forth is an un-typed language it must deal with libraries which do have typed calling conventions. In general the use of non-CELL arguments should be avoided but return results should be declared in Forth with the same size as the C or PASCAL convention documented.

Argument Reversal

The default calling convention for the host operating system is used. The right-most argument/parameter in the C-style prototype is on the top the Forth data stack. When calling an external function the parameters are reordered if required by the operating system; this is to enable the argument list to read left to right in Forth source as well as in the C-style operating system documentation.

Under certain conditions, the order can be reversed. See the words "C" and "PASCAL" which define the order for the operating system. See L>R and R>L which define the Forth stack order with respect to the arguments in the prototype.

C comments in declarations

Very rudimentary support for C comments in declarations is provided, but is good enough for the vast majority of declarations.

The example below is taken from a SQLite interface.


Extern: "C" int sqlite3_open16(
  const void * filename,  /* Database filename [UTF-16] */
  sqlite3 ** ppDb         /* OUT: SQLite db handle */
);

Controlling external references

1 value ExternWarnings? \ -- n
Set this true to get warning messages when an external reference is redefined.

0 value ExternRedefs?   \ -- n
If non-zero, redefinitions of existing imports are permitted. Zero is the default for VFX Forth so that redefinitions of existing imports are ignored.

1 value LibRedefs?      \ -- n
If non-zero, redefinitions of existing libraries are permitted. Non-zero is the default for VFX Forth so that redefinitions of existing libraries and OS X frameworks are permitted. When set to zero, redefinitions are silently ignored.

1 value InExternals?    \ -- n
Set this true if following import definitions are to be in the EXTERNALS vocabulary, false if they are to go into the wordlist specified in CURRENT. Non-Zero is the default for VFX Forth.

: InExternals   \ --
External imports are created in the EXTERNALS vocabulary.

: InCurrent     \ --
External imports are created in the wordlist specified by CURRENT.

Library Imports

In VFX Forth, libraries are held in the EXTERNALS vocabulary, which is part of the minimum search order. Other Forth systems may use the CURRENT wordlist.

For turnkey applications, initialisation, release and reload of required libraries is handled at start up.

variable lib-link       \ -- addr
Anchors the chain of dynamic/shared libraries.

variable lib-mask       \ -- addr
If non-zero, this value is used as the mode for dlopen() calls in Linux and OS X.

struct /libstr  \ -- size
The structure used by a Library: definition.

  int >liblink          \ link to previous library
  int >libaddr          \ library Id/handle/address, depends on O/S
  int >libmask          \ mask for dlopen()
  0 field >libname      \ zero terminated string of library name
end-struct

struct /funcstr \ -- size
The structure used by an imported function.

: init-lib      \ libstr --
Given the address of a library structure, load the library.

: clear-lib     \ libstr --
Unload the given library and zero its load address.

: clear-libs    \ --
Clear all library addresses.

: init-libs     \ --
Release and reload the required libraries.

: find-libfunction      \ z-addr -- address|0
Given a zero terminated function name, attempt to find the function somewhere within the already active libraries.

: .Libs         \ --
Display the list of declared libraries.

: #BadLibs      \ -- u
Return the number of declared libraries that have not yet been loaded.

: .BadLibs         \ --
Display a list of declared libraries that have not yet been loaded.

: Library:      \ "<name>" -- ; -- loadaddr|0
Register a new library by name. If LibRedefs? is set to zero, redefinitions are silently ignored. Use in the form:

  LIBRARY: <name>

Executing <name> later will return its load address. This is useful when checking for libraries that may not be present. After definition, the library is the first one searched by import declarations.

: topLib        \ libstr --
Make the library structure the top/first in the library search order.

: firstLib      \ "<name>" --
Make the library first in the library search order. Use during interpretation in the form:

  FirstLib <name>

to make the library first in the search order. This is useful when you know that there may be several functions of the same name in different libraries.

: [firstLib]    \ "<name>" --
Make the library first in the library search order. Use during compilation in the form:

  [firstLib] <name>

to make the library first in the search order. This is useful when you know that there may be several functions of the same name in different libraries.

Mac OS X extensions

The phrase Framework <name.framework> creates two Forth words, one for the library access, the other to make that library top in the search order. For example:

  framework Cocoa.framework

produces two words

  Cocoa.framework/Cocoa
  Cocoa.framework

The first word is the library definition itself, which behaves in the normal VFX Forth way, returning its load address or zero if not loaded. The second word forces the library to be top/first in the library search order. Thanks to Roelf Toxopeus.

As of OSX 10.7, FRAMEWORK (actually dlopen()) will search for frameworks in all the default Frameworks directories:

: framework     \ --
Build the two framework words. See above for more details. If LibRedefs? is set to zero, redefinitions are silently ignored.

Function Imports

Function declarations in shared libraries are compiled into the EXTERNALS vocabulary. They form a single linked list. When a new function is declared, the list of previously declared libraries is scanned to find the function. If the function has already been declared, the new definition is ignored if ExternRedefs? is set to zero. Otherwise, the new definition overrides the old one as is usual in Forth.

In VFX Forth, ExternRedefs? is zero by default.

variable import-func-link       \ -- addr
Anchors the chain of imported functions in shared libraries.

: ExternLinked  \ c-addr u -- address|0
Given a string, attempt to find the named function in the already active libraries. Returns zero when the function is not found.

: init-imports  \ --
Initialise Import libraries. INIT-IMPORTS is called by the system cold chain.

defer preExtCall        \ --
Windows only. A hook provided for debugging and extending external calls without floating point parameters or return items. It is executed at the start of the external call before any parameter processing.

defer postExtCall       \ --
Windows only. A hook provided for debugging and extending external calls without floating point parameters or return items. It is executed at the end of the external call after return data processing.

defer preFPExtCall      \ --
Windows only. A hook provided for debugging and extending external calls with floating point parameters or return items. . It is executed at the start of the external call before any parameter processing.

defer postFPExtCall     \ --
Windows only. A hook provided for debugging and extending external calls with floating point parameters or return items. It is executed at the end of the external call after return data processing.

: InExternals   \ --
External imports are created in the EXTERNALS vocabulary.

: InCurrent     \ --
External imports are created in the wordlist specified by CURRENT.

: Extern:       \ "text" --
Declare an external API reference. See the syntax above. The Forth word has the same name as the function in the library, but the Forth word name is not case-sensitive. The length of the function's name may not be longer than a Forth word name. For example:

  Extern: DWORD Pascal GetLastError( void );

: AliasedExtern:        \ "forthname" "text" --
Like EXTERN: but the declared external API reference is called by the explicitly specified forthname. The Forth word name follows and then the API name. Used to avoid name conflicts, e.g.

AliasedExtern: saccept int accept( HANDLE, void *, unsigned int *);

which references the Winsock accept function but gives it the Forth name SACCEPT. Note that here we use the fact that formal parameter names are optional.

: LocalExtern:  \ "forthname" "text" --
As AliasedExtern:, but the import is always built into the CURRENT wordlist.

: extern        \ "text" --
An alias for EXTERN:.

: ExternVar     \ "<name>" -- ; ExternVar <name>
Used in the form

  ExternVar <name>

to find a variable in a DLL or shared library. When executed, <name> returns its address.

: AliasedExternVar      \ "<forthname>" "<dllname>" --
Used in the form

  AliasedExternnVar <forthname> <varname>

to find a variable in a DLL or shared library. When executed, <forthname> returns its address.

: .Externs      \ -- ; display EXTERNs
Display a list of the external API calls.

: #BadExterns   \ -- u
Silently return the number of unresolved external API calls.

: .BadExterns   \ --
Display a list of any external API calls that have not been resolved.

: func-pointer  \ xt -- addr
Given the XT of a word defined by EXTERN: or friends, returns the address that contains the run-time address.

: func-loaded?  \ xt -- addr|0
Given the XT of a word defined by EXTERN: or friends, returns the address of the DLL function in the DLL, or 0 if the function has not been loaded/imported yet.

Pre-Defined parameter types

The types known by the system are all found in the vocabulary TYPES. You can add new ones at will. Each TYPE definition modifies one or more of the following VALUEs. )

argSIZE

Size in bytes of data type.

argDEFSIGN

Default sign of data type if no override is supplied.

argREQSIGN

Sign OverRide. This and the previous use 0 = unsigned and 1 = signed.

argISPOINTER

1 if type is a pointer, 0 otherwise

Each TYPES definition can either set these flags directly or can be made up of existing types.

Note that you should explicitly specify a calling convention for every function defined.

Calling conventions

: "C"           \ --
Set Calling convention to "C" standard. Arguments are reversed, and the caller cleans up the stack.

: "PASCAL"      \ --
Set the calling convention to the "PASCAL" standard as used by Pascal compilers. Arguments are not reversed, and the called routine cleans up the stack. This is not the same as PASCAL below.

: PASCAL        \ --
Set the calling convention to the Windows PASCAL standard. Arguments are reversed in C style, but the called routine cleans up the stack. This is the standard Win32 API calling convention. N.B. There are exceptions! This convention is also called "stdcall" and "winapi" by Microsoft, and is commonly used by Fortran programs. This is not the same as "PASCAL" above.

: WinApi        \ --
A synonym for PASCAL.

: StdCall       \ --
A synonym for PASCAL.

: VC++          \ --
Defines the calling convention as being for a C++ member function which requires "this" in the ECX register. The function must be defined with an explicit this pointer (void * this). Because exported VC++ member functions can have either "C" or "PASCAL" styles, the this pointer must be positioned so that it is leftmost when reversed (C/WINAPI/StdCall style) or is rightmost when not reversed ("PASCAL" style). See also the later section on interfacing to C++ DLLs.

: R>L   \ --
By default, arguments are assumed to be on the Forth stack with the top item matching the rightmost argument in the declaration so that the Forth parameter order matches that in the C-style declaration. R>L reverses this.

: L>R   \ --
By default, arguments are assumed to be on the Forth stack with the top item matching the rightmost argument in the declaration so that the Forth parameter order matches that in the C-style declaration. L>R confirms this.

Basic Types

: unsigned      \ --
Request current parameter as being unsigned.

: signed        \ --
Request current parameter as being signed.

: int           \ --
Declare parameter as integer. This is a signed 32 bit quantity unless preceeded by unsigned.

: char          \ --
Declare parameter as character. This is a signed 8 bit quantity unless preceeded by unsigned.

: void          \ --
Declare parameter as void. A VOID parameter has no size. It is used to declare an empty parameter list, a null return type or is combined with * to indicate a generic pointer.

: *             \ --
Mark current parameter as a pointer.

: **            \ --
Mark current parameter as a pointer.

: ***           \ --
Mark current parameter as a pointer.

: const ;       \ --
Marks next item as constant in C terminology. Ignored by VFX Forth.

: int32         \ --
A 32bit signed quantity.

: int16         \ --
A 16 bit signed quantity.

: int8          \ --
An 8 bit signed quantity.

: uint32        \ --
32bit unsigned quantity.

: uint16        \ --
16bit unsigned quantity.

: uint8         \ --
8bit unsigned quantity.

: LongLong      \ --
A 64 bit signed or unsigned integer. At run-time, the argument is taken from the Forth data stack as a normal Forth double with the top item on the top of the data stack.

: LONG          int  ;
A 32 bit signed quantity.

: SHORT         \ --
For most compilers a short is a 16 bit signed item, unless preceded by unsigned.

: BYTE          \ --
An 8 bit unsigned quantity.

: float         \ --
32 bit float.

: double        \ --
64 bit float.

: bool1         \ --
One byte boolean.

: bool4         \ --
Four byte boolean.

: ...           \ --
The parameter list is of unknown size. This is an indicator for a C varargs call. Run-time support for this varies between operating system implementations of VFX Forth. Test, test, test.

Windows Types

The following parameter types are non "C" standard and are used by Windows in function declarations. They are all defined in terms of existing types.

: OSCALL        PASCAL  ;
Used for portable code to avoid three sets of declarations. For Windows, this is a synonym for PASCAL and under Linux and other Unices this is a synonym for "C".

: DWORD         unsigned int  ;
32 bit unsigned quantity.

: WORD          unsigned int  2 to argSIZE  ;
16 bit unsigned quantity.

: HANDLE        void *  ;
HANDLEs under Windows are effectively pointers.

: HMENU         handle  ;
A Menu HANDLE.

: HDWP          handle  ;
A DEFERWINDOWPOS structure Handle.

: HWND          handle  ;
A Window Handle.

: HDC           handle  ;
A Device Context Handle.

: HPEN          handle  ;
A Pen Handle.

: HINSTANCE     handle  ;
An Instance Handle.

: HBITMAP       handle  ;
A Bitmap Handle.

: HACCEL        handle  ;
An Accelerator Table Handle.

: HBRUSH        handle  ;
A Brush Handle.

: HMODULE       handle  ;
A module handle.

: HENHMETAFILE  handle  ;
A Meta File Handle.

: HFONT         handle  ;
A Font Handle.

: HRESULT       DWORD   ;
A 32bit Error/Warning code as returned by various COM/OLE calls.

: LPPOINT       void *  ;
Pointer to a POINT structure.

: LPACCEL       void *  ;
Pointer to an ACCEL structure.

: LPPAINTSTRUCT void *  ;
Pointer to a PAINTSTRUCT structure.

: LPSTR         void *  ;
Pointer to a zero terminated string buffer which may be modified.

: LPCTSTR       void *  ;
Pointer to a zero terminated string constant.

: LPCSTR        void *  ;
Another string pointer.

: LPTSTR        void *  ;
Another string pointer.

: LPDWORD       void *  ;
Pointer to a 32 bit DWORD.

: LPRECT        void *  ;
Pointer to a RECT structure.

: LPWNDPROC     void *  ;
Pointer to a WindowProc function.

: PLONG         long *  ;
Pointer to a long (signed 32 bit).

: ATOM          word  ;
An identifier used to represent an atomic string in the OS table. See RegisterClass() in the Windows API for details.

: WPARAM        dword  ;
A parameter type which used to be 16 bit but under Win32 is an alias for DWORD.

: LPARAM        dword  ;
Used to mean LONG-PARAMETER (i.e. 32 bits, not 16 as under Win311) and is now effectively a DWORD.

: UINT          dword  ;
Windows type for unsigned INT.

: BOOL          int  ;
Windows Boolean type. 0 is false and non-zero is true.

: LRESULT       int  ;
Long-Result, under Win32 this is basically an integer.

: colorref      DWORD  ;
A packed encoding of a color made up of 8 bits RED, 8 bits GREEN, 8 bits BLUE and 8 bits ALPHA.

: SOCKET        dword  ;
Winsock socket reference.

: CURRENCYFMT   void * ;
Contains information that defines the format of a currency string.

: ENUMRESNAMEPROC void * ;
An application-defined callback function used with the EnumResourceNames and EnumResourceNamesEx functions.

: FILETIME      void * ;
Contains a 64-bit value representing the number of 100-nanosecond intervals since January 1, 1601 (UTC).

: HGLOBAL       void * ;
A handle to the global memory object.

: HRSRC         void * ;
A handle to a resource.

: LANGID        void * ;
A language identifier.

: LCID          void * ;
A locale identifier.

: LCTYPE        void * ;
A locale information type.

: LONG_PTR      void * ;
A signed long type for pointer precision. Use when casting a pointer to a long to perform pointer arithmetic.

: LP            void * ;
A long pointer.

: LPBOOL        void * ;
A pointer to a BOOL.

: LPCWSTR       void * ;
A pointer to a constant null-terminated string of 16-bit Unicode characters.

: LPFILETIME    void * ;
A pointer to a FILETIME structure.

: LPMEMORYSTATUS  void * ;
A pointer to a MEMORYSTATUS structure.

: LPMODULEENTRY32 void * ;
A pointer to a MODULEENTRY32 structure.

: LPOSVERSIONINFO void * ;
A pointer to a OSVERSIONINFO structure.

: LPOVERLAPPED  void * ;
A pointer to a OVERLAPPED structure.

: LPWSTR        void * ;
A pointer to a null-terminated string of 16-bit Unicode characters.

: LPVOID        void * ;
A pointer to any type.

: LPCVOID       void * ;
A pointer to a constant of any type.

: MSG           void * ;
Contains message information from a thread's message queue.

: NORM_FORM     void * ;
Specifies the supported normalization forms.

: NUMBERFMT     void * ;
Contains information that defines the format of a number string.

: PACTCTX       void * ;
Pointer to an ACTCTX structure that contains information about the activation context to be created.

: PBOOL         void * ;
A pointer to a BOOL.

: PDWORD        void * ;
A pointer to a DWORD.

: PHANDLE       void * ;
A pointer to a HANDLE.

: PVOID         void * ;
A pointer to any type.

: PULARGE_INTEGER void * ;
A pointer to a ULARGE_INTEGER structure.

: SIZE_T        Dword  ;
The maximum number of bytes to which a pointer can point. Use for a count that must span the full range of a pointer.

: SYSTEMTIME    void * ;
Specifies a date and time, using individual members for the month, day, year, weekday, hour, minute, second, and millisecond. The time is either in coordinated universal time (UTC) or local time, depending on the function that is being called.

: ULONG_PTR     void * ;
An unsigned LONG_PTR.

: VA_LIST       void * ;
A variable argument list.

: LPWIN32_FIND_DATA     void * ;
A pointer to a WIN32_FIND_DATA structure.

: LPTPMPARAMS           void * ;
A pointer to a TPMPARAMS structure.

: CODEPAGE_ENUMPROC     void * ;
An application-defined callback function that processes enumerated code page information provided by the EnumSystemCodePages function. The CODEPAGE_ENUMPROC type defines a pointer to this callback function.

: LPPROCESSENTRY32      void * ;
A pointer to a PROCESSENTRY32 structure.

: LPPROGRESS_ROUTINE    void * ;
The LPPROGRESS_ROUTINE type defines a pointer to this callback function. CopyProgressRoutine is a placeholder for the application-defined function name.

: LPSECURITY_ATTRIBUTES void * ;
A pointer to a SECURITY_ATTRIBUTES structure.

: LPSYSTEMTIME  void * ;
A pointer to a SYSTEMTIME structure.

: LPTCH         void * ;
A pointer to the environment block.

: LPTIME_ZONE_INFORMATION       void * ;
A pointer to a TIME_ZONE_INFORMATION structure.

: PMEMORY_BASIC_INFORMATION     void * ;
A pointer to a MEMORY_BASIC_INFORMATION structure.

: LPBY_HANDLE_FILE_INFORMATION  void * ;
A pointer to a BY_HANDLE_FILE_INFORMATION structure.

: DEVMODE       void * ;
The DEVMODE data structure contains information about the initialization and environment of a printer or a display device.

: FONTENUMPROC  void * ;
A pointer to the application defined callback function.

: HGDIOBJ       void * ;
A handle to the graphics object.

: HPALETTE      void * ;
A handle to a logical palette.

: HRGN          void * ;
Handle to a region.

: LINEDDAPROC   void * ;
The LineDDAProc function is an application-defined callback function.

: LOGBRUSH      void * ;
The LOGBRUSH structure defines the style, color, and pattern of a physical brush.

: LOGFONT       void * ;
The LOGFONT structure defines the attributes of a font.

: LOGPALETTE    void * ;
The LOGPALETTE structure defines a logical palette.

: LOGPEN        void * ;
The LOGPEN structure defines the style, width, and color of a pen.

: LPENHMETAHEADER       void * ;
A pointer to an ENHMETAHEADER structure that receives the header record.

: LPFONTSIGNATURE       void * ;
Pointer to a FONTSIGNATURE data structure.

: LPINT         void * ;
A pointer to an INT.

: LPLOGFONT     void * ;
A pointer to a LOGFONT structure.

: LPPALETTEENTRY        void * ;
A pointer to a PALETTEENTRY structure.

: LPSIZE        void * ;
A pointer to a SIZE structure.

: LPTEXTMETRIC  void * ;
A pointer to a TEXTMETRIC structure.

: POINT         void * ;
The POINT structure defines the x- and y- coordinates of a point.

: RECT          void * ;
The RECT structure defines the coordinates of the upper-left and lower-right corners of a rectangle.

: DLGPROC       void * ;
Application-defined callback function used with the CreateDialog and DialogBox families of functions.

: DRAWSTATEPROC void * ;
The DrawStateProc function is an application-defined callback function that renders a complex image for the DrawState function.

: DWORD_PTR     void * ;
A DWORD_PTR is an unsigned long type used for pointer precision.

: GRAYSTRINGPROC        void * ;
A pointer to the application-defined function.

: HCONV         void * ;
A conversation handle.

: HCONVLIST     void * ;
A handle to the conversation list.

: HCURSOR       void * ;
A cursor handle.

: HDDEDATA      void * ;
A handle to a DDE object.

: HICON         void * ;
A handle to a Icon.

: HKL           void * ;
A handle to a keyboard layout.

: HMONITOR      void * ;
A handle to the display monitor.

: HOOKPROC      void * ;
HookProc is a placeholder for an application-defined name.

: HSZ           void * ;
A handle to the string that specifies the service name of the server application with which a conversation is to be established.

: INT_PTR       void * ;
A signed integer type for pointer precision. Use when casting a pointer to an integer to perform pointer arithmetic.

: LPBYTE        void * ;
A pointer to a BYTE.

: LPCDLGTEMPLATE        void * ;
A pointer to a DLGTEMPLATE structure.

: LPCMENUINFO   void * ;
A pointer to a MENUINFO structure.

: LPCRECT       void * ;
A pointer to a RECT structure.

: LPCSCROLLINFO void * ;
A pointer to a SCROLLINFO structure.

: LPDRAWTEXTPARAMS      void * ;
A pointer to a DRAWTEXTPARAMS structure.

: LPINPUT       void * ;
An array of INPUT structures.

: LPMENUITEMINFO        void * ;
A pointer to a MENUITEMINFO structure.

: LPMONITORINFO void * ;
A pointer to a MONITORINFO or MONITORINFOEX structure that receives information about the specified display monitor.

: LPMSG         void * ;
A pointer to a MSG structure.

: LPMSGBOXPARAMS        void * ;
A pointer to a MSGBOXPARAMS structure.

: LPSCROLLINFO  void * ;
Pointer to a SCROLLINFO structure.

: LPTRACKMOUSEEVENT     void * ;
A pointer to a TRACKMOUSEEVENT structure

: LPWNDCLASSEX  void * ;
A pointer to a WNDCLASSEX structure.

: MONITORENUMPROC       void * ;
A MonitorEnumProc function is an application-defined callback function.

: PAINTSTRUCT   void * ;
The PAINTSTRUCT structure contains information for an application. This information can be used to paint the client area of a window owned by that application.

: PCOMBOBOXINFO void * ;
A pointer to a COMBOBOXINFO structure.

: PCONVCONTEXT  void * ;
A pointer to the CONVCONTEXT structure.

: PCONVINFO     void * ;
A pointer to the CONVINFO structure.

: PFLASHWINFO   void * ;
A pointer to a FLASHWINFO structure.

: PFNCALLBACK   void * ;
A pointer to the application-defined DDE callback function.

: PICONINFO     void * ;
A pointer to an ICONINFO structure.

: PROCESS_DPI_AWARENESS void * ;
PROCESS_DPI_AWARENESS enumeration.

: PSECURITY_QUALITY_OF_SERVICE  void * ;
A pointer to a SECURITY_QUALITY_OF_SERVICE data structure.

: SECURITY_QUALITY_OF_SERVICE   void * ;
The SECURITY_QUALITY_OF_SERVICE data structure contains information used to support client impersonation.

: TCHAR         void * ;
A Win32 character string that can be used to describe ANSI, DBCS, or Unicode strings.

: WNDCLASSEX    void * ;
The WNDCLASSEX structure is similar to the WNDCLASS structure. There are two differences. WNDCLASSEX includes the cbSize member, which specifies the size of the structure, and the hIconSm member, which contains a handle to a small icon associated with the window class.

: WNDENUMPROC   void * ;
A pointer to an application-defined callback function.

: LPNETRESOURCE void * ;
A pointer to the NETRESOURCE structure.

: LPHANDLE      void * ;
A pointer to a handle.

: LPSHFILEOPSTRUCT      void * ;
A pointer to an SHFILEOPSTRUCT structure.

: LPBROWSEINFO  void * ;
A pointer to a BROWSEINFO structure.

: SHELLEXECUTEINFO      void * ;
A structure that contains information used by ShellExecuteEx.

: REFKNOWNFOLDERID      void * ;
A reference to the KNOWNFOLDERID.

: PIDLIST_ABSOLUTE      void * ;
The ITEMIDLIST is absolute and has been allocated, as indicated by its being non-constant.

: PCIDLIST_ABSOLUTE     void * ;
The ITEMIDLIST is absolute and constant.

: PWSTR         void * ;
A pointer to a null-terminated string of 16-bit Unicode characters.

: LPPRINTER_DEFAULTS    void * ;
A pointer to a PRINTER_DEFAULTS structure.

: PDEVMODE      void * ;
A pointer to a DEVMODE data structure.

: __in  ;
Microsoft header annotation.

: __inout  ;
Microsoft header annotation.

: __out  ;
Microsoft header annotation.

: __in_opt  ;
Microsoft header annotation.

: __inout_opt  ;
Microsoft header annotation.

: __out_opt  ;
Microsoft header annotation.

: _in_  ;
Microsoft header annotation.

: _inout_  ;
Microsoft header annotation.

: _out_  ;
Microsoft header annotation.

: _in_opt_  ;
Microsoft header annotation.

: _out_opt_  ;
Microsoft header annotation.

: _Reserved_   ;
Microsoft header annotation.

: [in]  ;
Microsoft header annotation.

Linux Types

: OSCALL        "C"  ;
Used for portable code to avoid three sets of declarations. For Windows, this is a synonym for PASCAL and under Linux this is a synonym for "C".

: FILE          uint32  ;
Always use as FILE * stream.

: DIR           uint32  ;
Always use as DIR * stream.

: size_t        uint32  ;
Linux type for unsigned INT.

: off_t uint32  ;
Linux type for unsigned INT.

: int32_t       int32  ;
Synonym for int32.

: int16_t       int16  ;
Synonym for int16.

: int8_t        int8  ;
Synonym for int8.

: uint32_t      uint32  ;
Synonym for uint32.

: uint16_t      uint16  ;
Synonym for uint16.

: uint8_t       uint8  ;
Synonym for uint8.

: time_t        uint32  ;
Number of seconds since midnight UTC of January 1, 1970.

: clock_t       uint32  ;
Processor time in terms of CLOCKS_PER_SEC.

: pid_t         int32  ;
Process ID.

: uid_t         uint32  ;
User ID.

: mode_t        uint32  ;
File mode.

Mac OS X Types

: OSCALL        "C"  ;
Used for portable code to avoid three sets of declarations. For Windows, this is a synonym for PASCAL and under OS X this is a synonym for "C".

: FILE          uint32  ;
Always use as FILE * stream.

: DIR           uint32  ;
Always use as DIR * stream.

: size_t        uint32  ;
Unix type for unsigned INT.

: off_t uint32  ;
Unix type for unsigned INT.

: int32_t       int32  ;
Synonym for int32.

: int16_t       int16  ;
Synonym for int16.

: int8_t        int8  ;
Synonym for int8.

: uint32_t      uint32  ;
Synonym for uint32.

: uint16_t      uint16  ;
Synonym for uint16.

: uint8_t       uint8  ;
Synonym for uint8.

: time_t        uint32  ;
Number of seconds since midnight UTC of January 1, 1970.

: clock_t       uint32  ;
Processor time in terms of CLOCKS_PER_SEC.

: pid_t         int32  ;
Process ID.

: uid_t         uint32  ;
User ID.

: mode_t        uint32  ;
File mode.

Compatibility words

These words are mainly for users converting code from other Forth systems.

This section provides shared library imports in the form:

  function: foo  ( a b c d -- x )

where the brackets must be space delimited. Imports use the default calling convention for the operating system.

: FUNCTION:     \ "<name>" "<parameter list>" --
Generate a reference to an external function. The Forth name is the same as the name of the external function. Use in the form:

 function: foo1 ( a b c d -- )
 function: foo2 ( a b c d -- e )
 function: foo3 ( a b c d -- el eh )

The returned value may be 0, 1 or 2 items corresponding to void, int/long and long long on most 32 bit systems.

: ASCALL:       \ "<synonym-name>" "<name>" "<parameter list>" --
Generate a reference to an external function. The Forth name is not the same as the name of the external function. Use in the form:

 ascall: forthname funcname ( a b c d -- e )

: GLOBAL:       \ "<name>" --
Generate a reference to an external variable. Use in the form:

 global: varname

Using the Windows hooks

The hooks preExtCall and postExtCall are DEFERred words into which you can plug actions that will be run before and after any external call. They are principally used:

The hooks preFPExtCall and postFPExtCall are compiled into calls with floating point parameters or return values. They do not affect the NDP state.

The examples below illustrate both actions.

Deferred words and variables

defer preExtCall        \ --
Windows only. A hook provided for debugging and extending external calls. It is executed at the start of the external call before any parameter processing.

defer postExtCall       \ --
Windows only. A hook provided for debugging and extending external calls. It is executed at the end of the external call after return data processing.

defer preFPExtCall      \ --
Windows only. A hook provided for debugging and extending external calls with floating point parameters or return items. It is executed at the start of the external call before any parameter processing.

defer postFPExtCall     \ --
Windows only. A hook provided for debugging and extending external calls with floating point parameters or return items. It is executed at the end of the external call after return data processing.

variable XcallSaveNDP?  \ -- addr
Set true when imports must save and restore the NDP state. Windows only. From build 2069 onwards, the default behaviour for Windows includes saving and restoring the FPU state. This can be inhibited by clearing XcallSaveNDP? before execution.

variable abort-code     \ -- addr
Holds error code for higher level routines, especially RECOVERY below. Windows versions only.

variable aborting?      \ -- addr
Holds a flag to indicate whether error recovery should be performed by a calling routine.

defer xcall-fault       \ -- ; handles errors in winprocs
Used by application code in the DEFERred words preExtCall and postExtCall above to install user-defined actions.

Default versions

code PreExtern  \ -- ; R: -- sys
\ Clears the abort code and saves the NDP state if XcallSaveNDP?
\ is set.
  mov   dword ptr abort-code , # 0      \ no previous abort code
  cmp     [] XcallSaveNDP? , # 0        \ Win: required
  nz, if,
    pop     eax
    lea     esp, -/fsave [esp]
    fsave   0 [esp]
    push    eax
  endif,
  ret
end-code
assign preExtern to-do preExtCall

code PostExtern \ -- ; R: sys --
\ Restore the NDP state if XcallSaveNDP? is set and test the
\ abort code.
  cmp   [] XcallSaveNDP? , # 0          \ required
  nz, if,
    pop     eax
    frstor  0 [esp]
    lea     esp, /fsave [esp]
    push eax
  endif,
  \ Detecting faults in nested callbacks.
  cmp   dword ptr abort-code , # 0      \ test previous aborting code
  nz, if,
    call  [] ' xcall-fault 5 +          \ execute xcall-fault if set
  endif,
  ret
end-code
assign postExtern to-do postExtCall

code PreFPExtern        \ -- ; R: -- sys ; SFP006
\ Clears the abort code.
  mov   dword ptr abort-code , # 0      \ no previous abort code
  ret
end-code
assign preFPExtern to-do preFPExtCall

code PostFPExtern       \ -- ; R: sys -- ; SFP006
\ Test the abort code.
  cmp   dword ptr abort-code , # 0      \ test previous aborting code
  nz, if,
    call  [] ' xcall-fault 5 +          \ execute xcall-fault if set
  endif,
  ret
end-code
assign postFPExtern to-do postFPExtCall

: DefaultExterns        \ --
Set the default PRE and POST EXTERN handlers.

Protected EXTERNs

Protected EXTERNs allow VFX Forth to recover when a crash occurs inside a Windows call and the Forth registers have been corrupted. For example

  255 0 GetCurrentDirectory

will crash because an address of zero is invalid. Protected EXTERNs save the Forth registers befor making the call so that exception handlers can restore VFX Forth to a known state.

code PreProtExtern      \ -- ; R: -- sys
\ Clears the abort code and saves the NDP state if XcallSaveNDP?
\ is set.
  mov    edx, # XcallBuffer             \ where the saved data goes
  mov    eax, 0 [esp]                   \ return address
  sub    eax, # 6                       \ xt of EXTERN (call [] prexx)
  mov    0 scb.xt [edx], eax            \ save it
  lea    eax, 4 [esp]                   \ RSP on entry
  mov    0 scb.esp [edx], eax
  mov    0 scb.ebp [edx], ebp
  mov    0 scb.esi [edx], esi
  mov    0 scb.edi [edx], edi

  mov   dword ptr abort-code , # 0      \ no previous abort code
  cmp     [] XcallSaveNDP? , # 0        \ Win: required
  nz, if,
    pop     eax                         \ return address
    lea     esp, -/fsave [esp]
    fsave   0 [esp]
    push    eax
  endif,
  ret
end-code

code PostProtExtern     \ -- ; R: sys --
\ Restore the NDP state if XcallSaveNDP? is set and test the
\ abort code.
  mov   dword ptr XcallBuffer scb.xt , # 0  \ reset Extern in progress
  cmp   [] XcallSaveNDP? , # 0          \ required
  nz, if,
    pop     eax                         \ return address
    frstor  0 [esp]
    lea     esp, /fsave [esp]
    push eax
  endif,
  \ Detecting faults in nested callbacks.
  cmp   dword ptr abort-code , # 0      \ test previous aborting code
  nz, if,
    call  [] ' xcall-fault 5 +          \ execute xcall-fault if set
  endif,
  ret
end-code

code PreProtFPExtern    \ -- ; R: -- sys ; SFP006
\ Clears the abort code.
  mov    edx, # XcallBuffer             \ where the saved data goes
  mov    eax, 0 [esp]                   \ return address
  sub    eax, # 6                       \ xt of EXTERN (call [] prexx)
  mov    0 scb.xt [edx], eax            \ save it
  lea    eax, 4 [esp]                   \ RSP on entry
  mov    0 scb.esp [edx], eax
  mov    0 scb.ebp [edx], ebp
  mov    0 scb.esi [edx], esi
  mov    0 scb.edi [edx], edi

  mov   dword ptr abort-code , # 0      \ no previous abort code
  ret
end-code

code PostProtFPExtern   \ -- ; R: sys -- ; SFP006
\ Test the abort code.
  mov   dword ptr XcallBuffer scb.xt , # 0      \ reset Extern in progress
  cmp   dword ptr abort-code , # 0      \ test previous aborting code
  nz, if,
    call  [] ' xcall-fault 5 +          \ execute xcall-fault if set
  endif,
  ret
end-code

: ProtectedExterns      \ --
Set the protected PRE and POST EXTERN handlers.

Interfacing to C++ DLLs

Caveats

These notes were written after testing on Visual C++ v6.0. Don't blame us if the rules change!

Example code

The example code may be found in the directory EXAMPLES\VC++. Because of the inordinate amount of time we spent wandering around inside debuggers to get this far, we recommend that you adopt a cooperative and investigative attitude when requesting technical support on this topic.

Accessing constructors and destructors

Example code for accessing the constructor of class is provided in TRYCPP.FTH which accesses the class DllTest in DLLTEST.CPP.

Since C++ is supposed to provide a higher level of abstraction, apparently simple operations may generate reams of code. So it is with the equivalent of

  pClass = new SomeClass;

The actual code generated may/will be a call to a function new to generate an object structure (not a single cell) followed by passing the return value from new to the class constructor.

The class constructor (in C++ CDllTest::CDllTest()) is not normally exported from C++ without some extra characters being added to the name. For example, the reference to it in the example code is:

  extern: PASCAL void * ??0CDLLTest@@QAE@XZ( void );

This function is not directly callable because it has to be passed the result of the new operator. To solve this problem DLLTest.dll contains a helper function CallNew which is passed the address of the constructor for the class. This is redefined as NEW for normal use.


\ C++ Helpers
extern: PASCAL void * CallNew( void * );
extern: PASCAL void CallDelete( void * this);

\ CDLLTest class specific
extern: PASCAL void * ??0CDLLTest@@QAE@XZ( void );

0 value CDLLTest        \ -- class|0

: InitCDLLTest  \ -- ; initialise the CPP interface
  ['] ??0CDLLTest@@QAE@XZ func-loaded? -> CDLLTest
;

: New   \ class -- *obj|0
  CallNew
;

: Delete        \ *obj --
  CallDelete
;

The word INITCDLLTEST gets the address of the constructor for the class, and NEW then runs the CallNew function which executes the C++ new operator and calls the constructor. Unfortunately, you will have to do this for each class that use in the DLL. What is returned by CallNew is an object pointer. This is not the object itself, but the address of another (undocumented) data structure. It can be used as the this pointer for all following member function calls.

Once you have finished with the onject, you must relase its resources using the delete method (the destructor). This is implemented in VC++ by passing the object pointer to the delete function. This is performed by the CallDelete function exported from the DLL. Again, the Forth word DELETE provides syntactic sugar by just calling CallDelete.

Accessing member functions

A Visual C++ member function exported from a DLL requires the "this" pointer in the ECX register. This can be achieved using the following form:


extern: VC++ PASCAL BOOL TestWindow1( void * this, char * ch, int n, int nbyte );

The function must be defined with an explicit this pointer (void * this). Because exported VC++ member functions can have either C or "PASCAL" styles, the this pointer must be positioned so that it is leftmost when reversed (C/PASCAL/WINAPI/StdCall/APIENTRY style) or is rightmost when not reversed ("PASCAL" style).


extern: PASCAL VC++ BOOL GetHello( void * this, char * buff, int len );
extern: PASCAL VC++ BOOL TestWindow1( void * this, char * ch, int n, int nbyte );
extern: PASCAL VC++ BOOL TestWindow2( void * this, void * pvoid, int ndword, int nlong );

0 value CDLLTest        \ -- constructor/class

: InitCDLLTest  \ --;  Initialise the CPP interface
  ['] ??0CDLLTest@@QAE@XZ func-loaded? to CDLLTest
;

create Magic#  $AAAA5555 ,      \ -- addr

#64 buffer: StringBuff  \ -- ; buffer for GetHello

: TestCDLLTest  \ -- ; test CDLLTest interface
  InitCDLLTest  CDLLTest if
    cr ." Initialisation succeeded"
    CDLLTest new ?dup if
      cr ." new succeeded"
      dup StringBuff #64 GetHello drop
      cr ." GetHello returns: " StringBuff .z$
      dup Magic# 4 5 TestWindow1 drop
      dup Magic# #20 #30 TestWindow2 drop
      delete
      cr ." delete done"
    else
      cr ." new failed"
    endif
  else
    cr ." Initialisation failed"
  endif
;

Please note that the actual code in TRYCPP.FTH may/will be different as we extend the facilities. See the source code itself!

Accessing third party C++ DLLs

Most third party C++ DLLs are provided with C header files which define the interfaces. Study of these will provide the information you need to determine how to access them.

For simple C++ classes, the DllTest.dll file can be used to provide constructor and destructor access. Note that classes with multiple constructors will export these as functions with the same basic name differentiated by the name mangling.

The DLL Fth2VC60.dll contains new and delete access for use with other DLLs. Note that the third party DLLs must be compatible with VC++ v6.0. The example file EXAMPLES\VC++\USECPP.FTH demonstrates using Fth2VC60.dll.


library: Fth2VC60.dll

extern: PASCAL void * FTH2CPPNew( void * constructor);
extern: PASCAL void FTH2CPPDelete( void * this);

: New           \ *class -- *obj|0
  FTH2CPPNew
;

: Delete        \ *obj --
  FTH2CPPDelete
;

If you are using an incompatible compiler or DLL, create a similar support DLL for that compiler. You can use the source code for Fth2VC60.dll as an example.

Changes at v4.3

The guts of the EXTERN: mechanism have been rewritten to provide more features and to support more operating systems.

Additional C types

The following C data types are now supported:

Floating point numbers are taken from the NDP floating point unit. This is directly compatible with the the Forth floating point pack in Lib\x86\Ndp387.fth.

More Operating Systems

The requirements of newer operating systems, especially those for 64-bit operation, are more stringent for things like data alignment. Consequently the underlying mechanism has changed.

Miscellaneous

These notes are probably only relevant for code that has carnal knowledge of the VFX Forth internals.

The word NXCALL is provided for constructing your own import mechanisms, but it only deals with single-cell arguments and provides no type safety at all. It is used internally by VFX Forth in the first stage build of a console-mode kernel.

code NXCALL     \ i*x addr i -- res
Calls the operating system function at addr with i arguments, returning the result res. As far as the operating systems is concerned, i*x appear on the CPU return stack pointed to by ESP, and the return value is taken from EAX. After executing NXCALL the return value res is the contents of the EAX register.