POST handlers and HTTP updates

The file <Pnet>/Examples/WebPost.fth provides example handlers for HTTP POST requests. According to the RFCs, GET requests are made when the server is left unchanged by the request. POST requests are made when the state of the server is (or may be) changed by the request.

The example here is of uploading a new application binary image to the server. DEFERred words are used to handle the target specific actions. This example can be used as the basis of other POST handlers, especially those using multipart forms. For details of form encoding, see RFCs 2045 and 2046. These can be obtained from http://www.rfc-editor.org.

To use the system, you need PowerNet version 4.62 or later. Compile the file <Pnet>/Examples/WebPost.fth after the PowerNet build file. When the system is running, point your browser at the page Reflash.htm. When you have selected the required file, press the Submit button. This returns the form results and new binary to a page called NewApp.asp, which is actually a Forth word that performs the process. When it is complete, the page /naresp.asp is served to display the results to the user.

To provide flexibility in how the new binary is handled, a simple interface is provided that can be used to save the new binary code to Flash or a file system, e.g. on an SD card.

Discussion

The requirement is to be able to point any browser at the PowerNet web server, and to be able to upload a new binary image to it using a form for data entry. What is done with this file is application dependent, but it may replace the existing application, or it may be saved to an SD card.

Form

In order to send binary data to a server (file upload), a POST request must be made by the form. Here is the HTML for an example form. Following sections indicate the response, in our case using Firefox.


<html>
<head><title>Select new application</title></head>
<body>
<h4>Select application</h4>
<p>Select the application using the Browse button,<br>
then press the "Submit" button to send it to me.<br>
Once you have pressed "Submit" you are committed.<br>
To avoid sending a file, return to the previous page.
</p>
<form enctype="multipart/form-data" method=POST action="newapp.asp">
  <table>
  <tr>
    <td>New application file</td>
    <td colspan=4><input type=file name=appfile></td>
  </tr>
  <tr>
    <td>Upload to remote:</td>
    <td colspan=2><input type=submit name=send value=Submit></td>
  </tr>
  </table>
</form>
</body>
</html>

Headers

The following is a response by Firefox to the form above. Note that whenever you press a button of the submit type, you get the complete response, including any selected file.


POST /newapp.asp HTTP/1.1
Host: 192.168.0.227
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://192.168.0.227/Reflash.htm
Content-Type: multipart/form-data; boundary=---------------------------805274455224
Content-Length: 9386

-----------------------------805274455224

The Content-Type and Content-Length headers are essential. The first tells us that this is a form response in the style we need and the second tells us the overal size of the response. If you need security, you can use the Referer field to indentify which host and file contained the form.

In particular the Content-Type field contains a boundary string. This is the only data that identifies where one part of the form starts and ends.

Form boundaries

The boundary string is defined by the browser. The PowerNet server has no control over it. However, it is mandatory that the string is not contained in the data.

The example below shows three versions of the boundary string.


---------------------------805274455224
-----------------------------805274455224
-----------------------------805274455224--

The first is the separator defined by the boundary portion of the Content-Type header. Under some circumstances, this may be delimited by '"' characters which are not part of the boundary string.

The second is applied to all part separators except the last one. The second form is the same as the first but with two leading '-' characters. It is always preceded by a CR/LF pair and terminated by a CR/LF pair.

The third marks the end of the form. It is identical to the second type but has two additional '-' characters at the end of the line.

Form data

Using the form above, we receive two blocks of form data between boundary markers.


Content-Disposition: form-data; name="appfile"; filename="memcopy.s"
Content-Type: application/octet-stream

/******************** (C) COPYRIGHT
...

Note that in our form, the submit button is after the file browser, so the file is sent first! The order in which the elements of the form are sent is the order in which they appear in the HTML of the form.

The Content-Disposition header tells us which element of the form is being returned, and the Content-Type header tells us the file is being transferred as 8 bit binary. There is blank line (CR/LF pair) between the last header and the start of the binary data.

Note that there is nothing to tell us the size of the data! We must rely on detecting the boundary separator unless there is some magic data at the start of the binary data section.


Content-Disposition: form-data; name="send"

Submit

This represents the Submit button. You can have several buttons of type submit. They will each cause the form response. You can use them to decide where to place the data, e.g. primary or secondary application.

If you want a "Cancel" button, your page will need a second form pointing to a different page.

After the form

After the last item of form data will come the residue, which is usually null. Note that more than one form may be contained in a web page.

Restrictions

The following assumptions and restrictions should be noted.

Parsing multipart boundaries

There are two conditions we have to deal with, parsing text lines and detecting the marker at the end of a binary file. The second situation is coded within the binary file handler.

: Not--         \ caddr -- flag ; true for not '--'
Return true if the two characters at caddr are not '--'.

: Bdry$?        \ caddr len -- 0|-1
Returns true if the string matches the boundary string plus two leading dashes.

: Boundary?     \ caddr len -- -1|0|1
Returns -1 if the string is a normal boundary marker, 0 if it is not a boundary marker, and 1 if it is the last boundary marker.

: NextBoundary  \ -- flag
Read text up to the next boundary marker. Flag is returned true if the boundary was the last one.

Flash update application

System interface

The flash interface with the underlying system is handled by three DEFERred words.

#1024 equ /FlashBlock   \ -- len
Unit size passed to the application.

Defer InitUpd   \ caddr len --
Initialise the app's binary update system. The input string is the file name to be received. This is passed for validation purposes and in case the file is saved in a local file system.

Defer AddUpd    \ caddr len --
Add the given memory block to the update. All blocks except the last contain /FlashBlock bytes of data.

Defer TermUpd   \ --
Terminate the app's binary update system.

Receiving a file

#256 buffer: Filename$  \ -- addr
Buffer holding file name as a counted string.

#80 equ /BndryBlock     \ -- len
Size of additional buffer for boundary detection. The maximum size of a boundary string is 70 characters (RFC2046). These are prefixed by CR/LF/-/- at the start and will be followed by -/- in the last boundary line.

/FlashBlock buffer: BinBuff     \ -- addr
Buffer for storing binary data.

/BndryBlock buffer: BndryBuff$  \ -- addr
Buffer for storing potential boundary string data. Holds a counted string starting with a CR.

variable NextByte       \ -- addr
Holds the address (in BinBuff) of the next character to be received.

variable #FileSize      \ -- addr
Holds the number of bytes stored as binary data.

variable #AppSize       \ -- addr
Holds the number of bytes transferred to the application.

variable BinDone        \ -- addr
Set if the file has been transferred.

variable BinStatus      \ -- addr
Holds the file receive status (0=good).

: ResetBlock    \ --
Reset the next byte pointer.

: StartRxApp    \ --
Initialise variables and buffers for file reception.

: Block>App     \ --
Send the data block to the application. All blocks except the last contain /FlashBlock bytes of data.

: StopRxApp     \ --
Clean up after receiving a file.

: addData       \ byte --
Add the byte to the binary buffer.

: NotBoundary   \ --
Add the partial boundary string to the data and clear the buffer.

create CrLf--   \ -- caddr
Holds the four characters CR/LF/-/-.

: FullBndry?    \ -- flag
Check the contents of BndryBuff$ against the start of Boundary$ which holds the received boundary separator. If the data does not match copy the data to the binary buffer. If the data matches the complete boundary string, return true, otherwise return false.

By definition the marker string does not appear in the the binary. The code reads the binary into a buffer that is 80 characters longer than the required Flash block size to accommodate the boundary string (70 chars maximum). We check for the sequence CR/LF/-/-, and if found check for the following boundary string.

: ReceiveBinary \ --
The separator line between the part headers and the binary file has been read. Read the binary up to the concluding part boundary. Later code will check if this is the final boundary. If the boundary tail matches the last boundary condition, return true, otherwise return false.

: ReadBinFile   \ -- flag
The separator line between the part headers and the binary file has been read. Read the binary up to the concluding part boundary. If the boundary string is the last one, return true, otherwise return false.

Part scanning

In this example, the only form part we need to process is the one containing a binary file. We detect this by parsing the headers for the pair:


Content-Disposition: form-data; name="appfile"; filename="foo"
Content-Type: application/octet-stream

We assume that a valid file is present when

variable OctetStream    \ -- addr
Set true when Content-Type: application/octet-stream has been received.

variable FormData       \ -- addr
Set true when the "form-data" marker has been received.

: ?formData     \ caddr len --
Check for the "form-data" marker.

: ?appfile      \ caddr len --
Check for 'name="appfile"'.

: doPdisp       \ caddr len --
Check the given content disposition line.

: doPtype       \ caddr len --
Check the given content type line.

create PartHdrs \ -- addr
Table holding the headers to be processed by part header handlers.

: CheckPartHdrs \ -- x
Check the data received in the part headers, and return x, indicating what to do with the part. If x is zero, just skip the part.

: ProcessPart   \ -- flag
After receiving a boundary, process a part up to and including the next boundary, returning true if it is the last one.

: ProcessParts  \ --
Read the parts up to the last boundary

File update handler

: doNewApp      \ caddr len --
This is the handler for the update request.

' doNewApp xtPage: /NewApp.asp
Defines the page which performs the upload.

: .UploadDone   \ --
Used in the body of an ASP page to indicate the response to the upload.

MemPage: /Reflash.htm %IPSTACK%\TestPages\Reflash.htm
A form used to request a new application.

MemPage: /nareply.asp %IPSTACK%\TestPages\nareply.asp
The response delivered after an upload.

Example for pages stored in files

This example can be used to upload binary files, e.g. a new application image, to the file system. In order to compile this code the FAT file system must be present, and the equate FilePages? (used in Pages.fth} must be set non-zero.

Defer InitUpd   \ caddr len --
Initialise the app's binary update system. The input string is the file name to be received. This is passed for validation purposes and in case the file is saved in a local file system.

Defer AddUpd    \ caddr len --
Add the given memory block to the update. All blocks except the last contain /FlashBlock bytes of data.

Defer TermUpd   \ --
Terminate the app's binary update system.

create FileDir$ \ -- addr
The base directory for files. This will be prepended to the page name given to InitUpd. The string is held as a counted string in the buffer. Make sure that your application sets FileDir$ to point to its own pages. The directory name must not end in a separator. The directory must exist before use.

-1 value hPostFile      \ -- handle
Handle of the file used to save the download data.

: InitFileUpd   \ caddr len --
Initialise the file reception.

: AddFileUpd    \ caddr len --
Add the given memory block to the update. All blocks except the last contain /FlashBlock bytes of data.

: TermFileUpd   \ --
Terminate the app's binary file update system.