Monday, August 20, 2018

C64Net WiFi Modem Filter State Machine

So, believe it or not, work continues on the C64Net WiFi Modem firmware.  Especially since the same firmware is being used in a traditional RS-232 version of the modem based on the ESP32 module.




Some of the newer features include:
  1. New AT+CONFIG configuration menu
  2. Ability to set the hostname
  3. X-Modem and Z-Modem downloads
  4. AT+SHELL to access an SD-card interface.
  5. NTP client with configurable timezone.
  6. New configurable socket filtering state machine.
That last feature is what I wanted to document here.

The use case was the ability to have the modem filter or transform bytes coming either from a socket connection, or from a web page via the AT&G command.  There were already existing commands to mask out specific bytes, but users needed something more complex.  I searched the web the best I could to find an existing language definition for doing such a thing, and couldn't come up with anything.  I therefore chose to invent a really simple filtering code/language.  

My requirements were that it had to be completely definable in ascii, using only characters available for the AT command set in quotes.  It needed to be as compact as possible for memory constraints, and needed to handle cases like filtering out everything inside html comments <!-- -->, or possibly filtering out everything NOT inside html comments.

Here is what I came up with:

State Machine entry format:
MMcCCNN
MM - byte value to match, in hex. The value 00 matches ALL.
c - Command character: e)at char, p)ush to que, d)isplay char, r)eplace char, q)ue display and empty, x)empty que
CC - if c == 'r', then hex value of replacement byte
C - if c != 'r', then same as 'c', or '-' to do nothing further.
NN - next state, in hex, starting with state 00.

The machine starts with state 00, and, for each character byte, increments the state until a match is made, at which point is executes commands and proceeds to state NN.

Example:
Suppose you wanted to filter out everything in a web page EXCEPT the contents of the comments.

Important chars and their hex values:
< 3c
! 21
- 2d
> 3e

So, to grab only the stuff from <!-- -->, your state machine would look like this:
## -CODE--  COMMENT
00 3Ce--02  <-- if a '<' go to state 02
01 00e--00  <-- anything else, ignore it, go back to state 00
02 21e--04  <-- if '<!', go to state 04
03 00x--00  <-- anything else, ignore it, go back to state 00
04 2de--06  <-- if '<!-', go to state 06
05 00x--00  <-- anything else, ignore it, go back to state 00
06 2de--08  <-- if '<!--', go to state 08
07 00x--00  <-- anything else, ignore it, go back to state 00
08 2dp--0a  <-- now inside the <!--.  If '-', then state 0A
09 00qd-08  <-- anything else, display que & char, go to state 08
0a 2dp--0c  <-- if '--', then que the char, go to state 0C
0b 00qd-08  <-- anything else, display que & char, go to state 08
0c 3ex--00  <-- if '-->', dump the que, ignore char, go to state 00
0d 2dqd-0a  <-- anything else, display que & char, go to state 08


So, to do the AT&Y command, we just combine the codes in order:
AT&Y"3Ce--0200e--0021e--0400x--002de--0600x--002de--0800x--002dp--0a00qd-082dp--0c00qd-083ex--002dqd-0a"

Then any subsequent packets received from an open socket, or from the AT&G command (which dumps a web page to the modem) will use the above filter.

A few extra utility arguments were added for convenience:
AT&Y     with no arguments clears the state machine definition entirely
AT&Yn   where n is a decimal number, will set the state machine state.

All of this will be in 3.4 of Zimodem.

Amiga Recaps, Pt. 1

Lately I've been recapping Amigas with a fellow from the CTCUG group.  That's been going pretty well.  I've been practicing on spare Amiga 600 boards, of which I had 3 non-booting boards.  All 3 began booting after the recap, which is amazing happiness.  Once I've recapped all my spare boards, I'll move on to the computers in my collection.


Some lessons I've learned:
1. You can use hot air to remove surface mount caps with tin-foil to isolate the cap and the damage, but be super-careful not to disturb nearby components for several minutes after removing the hot air.  Easier is to use a wire cutter to cut into the cap from the top, being sure not to pinch along the same line as the solder contacts, so as not to pull a trace.  When the cap is cut away, it is easy and safe to de-solder normally.

2. Remove those tiny-legged through-hold electrolytics with a soldering iron on max heat by applying heat to the bottom pin while carefully pulling that pin up through the board.  Once you've exposed the top-side of the pin, resume pulling it up from the top of the board to make sure you aren't pulling up any traces on the top.  Then repeat for the second pin.

3. Clean the area with alcohol and q-tips afterwards, and carefully clean the through-holes with solder-wick.

4. When applying the surface mounted caps, do one leg at a time after tinning the pads.

5. Check the smd caps for a short afterwards.