Thanks for adding F~ to VfxForthOsx64.  I'm running this:
 VFX Forth 64 for Mac OS X x64
 © MicroProcessor Engineering Ltd, 1998-2020 
 Version: 5.11 Beta 2 [build 0310]
 Build date: 27 September 2020
Is this the appropriate place for a bug report?
My tests reveal a bug in F~ when called as 0e F~, which turns out to be
due to a failure to take into account that FSIGN does not consume its
argument.  The F~ doc comment also has a typo in the f3 positive case,
is missing the case f3 = zero, and needs to replace abs[f3*abs[f1+f2]]
by abs[f3*[abs[f1]+abs[f2]]] in the f3 negative case.
The following fix in FPSSE64S.fth passes all of my fprox tests.
: f~            \ F: f1 f2 f3 -- ; -- flag
\ *G Approximation function. If f3 is positive, flag is true if abs[f1-f2]
\ ** less than f3. IF f3 is zero, flag is true if f1 and f2 encodings are
\ ** the same. If f3 is negative, flag is true if abs[f1-f2] less than
\ ** abs[f3*[abs[f1]+abs[f2]]].
  fdup f0> if                           \ if f3 > 0
    frot frot  f- fabs \ -- f3 |f1-f2|
    f>  exit                            \ -- t/f ; SFP007
  then
  fdup f0= if \ if f3 = 0
\   fdrop  fdup fsign >r
\   fover fsign >r
    fdrop fsign >r
    fswap fsign >r
    f= r> r> xor 0= and  exit
  then
  frot frot  fover fover \ -- f3 f1 f2 f1 f2
  f- fabs \ -- f3 f1 f2 |f1-f2|
  frot frot \ -- f3 |f1-f2| f1 f2
  fabs fswap fabs \ -- f3 |f1-f2| |f1| |f2|
  f+ frot fabs f* \ -- |f1-f2| |f3|*(|f1|+|f2|)
  f< \ -- t/f
;