Monday, August 20, 2018

C64Net WiFi Modem Filter State Machine

So, believe it or not, work continues on the C64Net WiFi Modem firmware.  Especially since the same firmware is being used in a traditional RS-232 version of the modem based on the ESP32 module.

Some of the newer features include:
  1. New AT+CONFIG configuration menu
  2. Ability to set the hostname
  3. X-Modem and Z-Modem downloads
  4. AT+SHELL to access an SD-card interface.
  5. NTP client with configurable timezone.
  6. New configurable socket filtering state machine.
That last feature is what I wanted to document here.

The use case was the ability to have the modem filter or transform bytes coming either from a socket connection, or from a web page via the AT&G command.  There were already existing commands to mask out specific bytes, but users needed something more complex.  I searched the web the best I could to find an existing language definition for doing such a thing, and couldn't come up with anything.  I therefore chose to invent a really simple filtering code/language.  

My requirements were that it had to be completely definable in ascii, using only characters available for the AT command set in quotes.  It needed to be as compact as possible for memory constraints, and needed to handle cases like filtering out everything inside html comments <!-- -->, or possibly filtering out everything NOT inside html comments.

Here is what I came up with:

State Machine entry format:
MMcCCNN
MM - byte value to match, in hex. The value 00 matches ALL.
c - Command character: e)at char, p)ush to que, d)isplay char, r)eplace char, q)ue display and empty, x)empty que
CC - if c == 'r', then hex value of replacement byte
C - if c != 'r', then same as 'c', or '-' to do nothing further.
NN - next state, in hex, starting with state 00.

The machine starts with state 00, and, for each character byte, increments the state until a match is made, at which point is executes commands and proceeds to state NN.

Example:
Suppose you wanted to filter out everything in a web page EXCEPT the contents of the comments.

Important chars and their hex values:
< 3c
! 21
- 2d
> 3e

So, to grab only the stuff from <!-- -->, your state machine would look like this:
## -CODE--  COMMENT
00 3Ce--02  <-- if a '<' go to state 02
01 00e--00  <-- anything else, ignore it, go back to state 00
02 21e--04  <-- if '<!', go to state 04
03 00x--00  <-- anything else, ignore it, go back to state 00
04 2de--06  <-- if '<!-', go to state 06
05 00x--00  <-- anything else, ignore it, go back to state 00
06 2de--08  <-- if '<!--', go to state 08
07 00x--00  <-- anything else, ignore it, go back to state 00
08 2dp--0a  <-- now inside the <!--.  If '-', then state 0A
09 00qd-08  <-- anything else, display que & char, go to state 08
0a 2dp--0c  <-- if '--', then que the char, go to state 0C
0b 00qd-08  <-- anything else, display que & char, go to state 08
0c 3ex--00  <-- if '-->', dump the que, ignore char, go to state 00
0d 2dqd-0a  <-- anything else, display que & char, go to state 08


So, to do the AT&Y command, we just combine the codes in order:
AT&Y"3Ce--0200e--0021e--0400x--002de--0600x--002de--0800x--002dp--0a00qd-082dp--0c00qd-083ex--002dqd-0a"

Then any subsequent packets received from an open socket, or from the AT&G command (which dumps a web page to the modem) will use the above filter.

A few extra utility arguments were added for convenience:
AT&Y     with no arguments clears the state machine definition entirely
AT&Yn   where n is a decimal number, will set the state machine state.

All of this will be in 3.4 of Zimodem.


Amiga Recaps, Pt. 1

Lately I've been recapping Amigas with a fellow from the CTCUG group.  That's been going pretty well.  I've been practicing on spare Amiga 600 boards, of which I had 3 non-booting boards.  All 3 began booting after the recap, which is amazing happiness.  Once I've recapped all my spare boards, I'll move on to the computers in my collection.

Some lessons I've learned:
1. Use hot air to remove surface mount caps with tin-foil to isolate the cap and the damage.  Be super-careful not to disturb nearby components for several minutes after removing the hot air.

2. Remove those tiny-legged through-hold electrolytics with a soldering iron on max heat by applying heat to the bottom pin while carefully pulling that pin up through the board.  Once you've exposed the top-side of the pin, resume pulling it up from the top of the board to make sure you aren't pulling up any traces on the top.  Then repeat for the second pin.

3. Clean the area with alcohol and q-tips afterwards, and carefully clean the through-holes with solder-wick.

4. When applying the surface mounted caps, do one leg at a time after tinning the pads.

5. Check the smd caps for a short afterwards.


Thursday, May 17, 2018

My new friend KIM.


I've had a pair of Commodore KIM-1 computers for at least 15 years now, but never really did anything with them.  That all changed over the last few weeks, and now I feel sorry for how much fun I've been missing out on.



The KIM-1 is a single board, 1mhz, 1 kilobyte computer (the green thing in the picture above) with a 23 key hexadecimal keypad for input, and a 6 digit LED display for output.  It also has two edge connectors on the left-hand-side.  One is called the Expansion port, and the lower one is the Application port.  The computer is powered by applying 5V to a pin on the Application port.  This port also includes a Current-Loop RS232 interface and cassette interface for saving/loading programs.

It may be a very minimal micro-computer, and it is definitely not a Home/Personal Computer, but it is certainly a computer!




The screen is divided into two segments.  The left four hex-digits are the current working Address, and the right two hex-digits are the contents of that address.  The address can represent the program counter when stepping through a program, or just the address currently being read or written to when using the KIM-1s built-in monitor.



The keyboard has the sixteen hex digits for entering addresses or data.  The AD button selects "Address" mode, where any entry on the keyboard will change the current working address.  The DA button selects "Data" mode, where any entry will change the contents of the address shown on the LED screen.  When in either Data or Address mode, the + key will advance to the next address byte.

The switch in the upper left selects between SST mode (on), which allows you to step through an executing program one instruction at a time, and normal mode (off), which allows a program to run freely to completion.  In both modes, the GO button will begin executing the program from the current address shown.  In SST mode, the GO button is then used to execute the next instruction, and then immediately return to the monitor program.

The RS button will reset the CPU and cause the KIM to re-enter the built-in monitor.  It is the first button you must press after powering-on the KIM-1.  Lastly, the PC button does nothing except replace the current address with the program counter address.



The KIM-1 is powered by three principle chips.  One, of course, is the MOS 6502 cpu/processor.  The other two are a pair of MOS 6530 RRIOT chips.  Each has 64 bytes (!) of ram, a pair of 8-bit bi-directional I/O controllers, timer registers, and 1k of ROM.  On one of the 6530s is the ROM code for the ml-monitor and cassette interface, while the other has the paper-tape interface.



I have a couple of Corsham expansion boards hooked up to my KIM, which are awfully convenient, and part of the reason why I finally dug the KIM-1 out for play.  Pictured above is their I/O board, which provides a standard 9-pin RS-232 port, which converts to the KIM's strange current-loop port.  It also has convenient interface for regulated 5V, 12V, and ground (the colored wires you see above), as well as an audio input and output jack for plugging in a standard cassette deck.  The little switch on the bottom lets me pick between TTY input, and using the KIM's keypad.


On the Expansion port is a Corsham 60k ram expansion card, gives the computer ram from $2000 all the way through $FFF7.



Once I got it working, I hooked the KIM-1 up to my VIC-20 through the VIC-1011A TTL interface and used the VIC's keyboard and monitor to program the computer.

One of my first programs was a memory-page address tester to make sure that the Corsham 60K ram expander was working correctly.  The program simply puts the page number into every byte in every page above $2000, and I then use the monitor to spot-check the results.

The first program I wrote on it was done by writing the assembly on paper for address $0200, and then translating the mnemonics into the machine language op-code numbers, and the branch addresses into one byte 6502 offsets.

Later, I realized that I could simply write the code in assembly, assemble it on the c64, and then key in the generated bytes onto the KIM's keyboard.

I am seriously interested in translating Microsoft BASIC for the KIM, and have discovered numerous resources on the web for doing that.  Check back here in the future for progress on that project.













Thursday, November 2, 2017

PETs are my Business



For a long time, I've been trying to get my hands on one of the U.S. PET 2001-B 'Business' series computers.  And boy are they rare!  I've seen at least two go through eBay, but over-confidence and under-estimations caused me to lose them. :(

But finally, I struck gold .. well, sorta.  A 2001B-16 turned up on eBay, but was advertised as non-functional.  It was still an opportunity worth taking, so I took it:



So, let's talk about this computer and what makes it unique!

At first glance, it appears to be just another PET with a 9" screen, just like it's Home series brother, the 2001N.  However, look closer and you'll see that the keyboard keycaps are not only different, but the entire arrangement of the keyboard has changed! There are no PETSCII graphic symbols on the front of the keys, the punctuation symbols and number keys are arranged together at the top row, just like a typewriter, and the numeric keypad is smaller.   This keyboard would go on to be used in the PET 8032 also.



Under the hood is another important difference.  The standard Editor ROM, which handles screen editor control and keyboard mapping, is replaced with a 'Business series' editor ROM, whose model number is 901474-01 (second rom from the left in the above picture).  This rom not only handles the new keyboard arrangement, but forces the computer to boot into Uppercase/Lowercase character set mode, instead of the Uppercase/Graphics mode the other PETs booted into.  The computer otherwise has the standard BASIC-2 Kernal and BASIC.



And lastly, of course, if the above was not convincing that this is not a 2001N series, the model number is "PET 2001-16B".


When I first got the machine, the computer did not boot, which the seller did warn me about.  Luckily, I had the Tynemouth 6502 PET diagnostics chip, v1.1!



This chip replaces the 6502 on the motherboard, and performs several tests on main ram, video ram, and provides a hash and identification of the roms.


The first run showed the video ram and ROMs in good shape (the -UNKNOWN- rom is the business editor, which is pretty rare, and the BASIC-4 was just a silly add-in I removed later).  However, it also shows ALL of the main ram as DOA.  One or two bad chips, and I might have started replacing them, but when they are all gone like that, something more fundamental was wrong.

And I quickly determined that the 12V rail was grounded.  The 4116 ram chips require +5VDC, -5VDC, and +12VDC.  So, starting at the regulator, I began removing and testing every damn component in the 12V circuit.



First up was this regulator, and the two caps and diodes you see here around C24 and C25.   The electrolytic was out of spec (it's a 47uF, but reported 68uF), so I replaced it with a high quality long-lasting modern one that was closer to spec.  Still, none of the components were shorted, so I followed the 12V rail down to the ram.



The 4116 ram chips I desoldered, socketed, and then re-inserted, after checking each one for a short at the 12V pin.  I broke one of the 4116s during this process, but had plenty of replacements.  Otherwise, I kept the original chips.

Next, on the advice of a friend, I moved to the tantalum capacitors (the larger blue caps on the LHS and RHS -- not the tiny blue ones next to the ram).  Turned out that was it!  TWO of the three tantalum capacitors I removed were shorted.  I therefore replaced all three.  The Tan colored capacitors you see intermingled with the blue ones were the ones I replaced.


And Walah! Now I have working RAM! Woohoo! You can see from the test screen that I also removed the silly BASIC-4 chip I had added for no reason other than the socket looked lonely.


You can see the char rom test is also good.

However, when I put the working 6502 processor back in, the computer still would not boot!

For giggles, I next tried the "PET tester ROM", which replaces the standard PET kernal instead of the processor, and runs a few simple memory tests.



As near as I can tell, those also came back with good results.

So, well, I thought about it.  When the computer starts, the CPU is reset, and it reads a location from the Kernal ROM at the bottom of memory and starts executing code.  So the CPU needs to work, and the Kernal needs to be readable.  Since the Pet Tester ROM test worked, I know it is doing those things.  After that, it will starting writing to main ram, and eventually screen ram.  I know all the RAM is good also, and that the CPU can read from and write to it.  What else does the PET do on startup?  I wasn't sure, but I imagined it also had to initialize the I/O chips so it can read from the keyboard.

So, I swapped out the 6522 VIA, and the two 6520 PIAs.  After swapping the PIA closest to the keyboard, BOOM:




And they lived happily ever after!







Monday, September 11, 2017

A Post! Now with even more WiFi Modem!



Well, I hope you aren't tired of news about the CTCUG "C64Net WiFi Modem", because it has remained my development focus.  Even as I type this I'm busy adding SSL support to the firmware, and wondering if the feature is really worth the 50k of program space and 5k of global ram it appears to consume.  The answer? Probably Yes.



My last posting talked about the Telnetd server for the Commodore 64.  That program turned out to be my last new application for the standard C64 kernal.  Since then, I've moved onto the Commodore VIC-20.  Some quick tinkering allowed me to discover that this little 22 column wonder can easily do 1200 baud with our new modem.  That was all I needed to know.

I proceeded to port the machine language code for the WiFi modem applications, which I call the "PML" for "Packet Mode Library", to the VIC-20 memory map, which I was surprised to find so closely resembled the C64 memory map as to need almost no changes at all!

After that, it was a simple matter to adapt most of the standard BASIC applications to run on it, provided you had at least a 16K ram expansion for the VIC.

So that means our modem comes with a nice suite of internet programs for the Commodore 64, 128, and the VIC!  I also tried the Plus/4, but we found that the DCD line on our modem was causing problems, and that a Plus/4 version of the modem would have to wait.


The next thing that happened was that our modems went on sale on eBay! You can find it by searching for "C64NET WiFi Card"



Since then, I've been working on porting my internet applications to the GEOS graphical operating system.  Back in the late 90s, I was quite infatuated with coding for this environment, and have found it quite easy to slip back into its elegant arms.

My first step, of course, was figuring out how to do modem I/O from inside an operating system that ignored the existence of the user port.  I quickly decided I would not try to replicate the C64 Kernal routines, which have some notorious speed issues.  Instead I decided to port Ilker Fi├žicilar's "commLib2" library, which claims to be able to do much higher speeds.  You can download it and find other information here.

Porting commLib2 to GEOS was done in several painful steps.  The easiest part was first: I took his binary and ran it through the WFDis interactive disassembler and generated something approaching the original assembly source.  I then stripped out the terminal, which I would not use, and filled in as many labels as I could.

The next step was hardest: actually getting it working in GEOS.  Like the C64 Kernal, commLib2 relies on timer interrupts from the CIA to handle serial.  However, GEOS does not use NMI interrupts, and, by default, uses a memory map that "hides" the CIA addresses from the CPU.  After working through these problems, I uncovered a few minor bugs in the library, and discovered that the timing was optimized for a European PAL C64, which runs slightly slower than my NTSC C64.  Lastly, I replaced his more complex support for large receive buffers with a faster and simpler small-buffer design that used hardware flow control. Getting through all this took the better part of a month, but eventually I had a working driver!


Once the driver was done, I started working on an 80 column ANSI terminal program, which would be my first GEOS application for the modem.  I discovered that the driver does up to 4800 baud without any troubles.  The author says I should be able to get 7200, but I'll probably need to tweek my NTSC timing numbers at some point to actually see this.


Adding the ANSI color support was tricky, since the font is 4 pixels wide, but the C64 hi-resolution screen supports background and foreground colors in pixel blocks of a minimum 8x8.  I went ahead and allowed this fudge to go through, but I don't think it looks too bad at all.

I still have quite a ways to go before it's finished.  It will need a phonebook/dialer, and support for X-Modem file transfers, but after that, it will be ready to release.

It will eventually be followed by an IRC client, CBM graphics "BBS" client, an FTP client, and probably a combination of the WGET/D64WGET program I wrote in BASIC.  At first, all of these programs will be for the C64 version of GEOS (including Wheels, MP3, and gateWay).  For GEOS 128, I would basically need to start from scratch with the driver all the way up, so I'll leave that for the distant future.

Until next time!





































































Sunday, July 2, 2017

C64 Telnet Server







Of late, I have not been able to get enough of my Commodore user-port wireless Ethernet modems. Yes, the same modem I described before >here<.

Since that last blog post, we've added hardware flow control, and gone through two more revisions of the hardware.  Changes to the firmware have slowed down at this point, as I focus more of my energy on different applications of my new internet toy.

In addition to all the programs mentioned in the previous post, I've since added a version of WGET specifically for downloading .D64 (or D81, etc..) disk image and writing the sectors directly to a blank floppy.  This is perfect for users who want to enjoy games on the internet in .D64 format, but only have a single 1541 disk drive and our modem.  The program is called D64WGET on our disk.



I also ported one of my favorite old Commodore PET games, "weather", to the C64, and then made it playable over the internet using our modems.  I got my brother to help me play test it several times during testing, and I was soundly defeated. :(



I even managed to hook my Commodore SuperPET's RS232 port to the FTDI pins on an old rev 2 version of our modem (after going through a RS232/TTL converter), and playing around with it online.  This little project did require adding some more features to the firmware, since the SuperPET needed 7 bit characters and Even parity.  The fact that it uses 7-bit characters instead of the standard 8-bit was especially hard to discern, since neither the computer manual nor the uart setup menu mentioned it.  It was only after hooking up a logic analyzer to the TX/RX pins that this was figured out.



But my latest project has been a telnetd server for the C64.  From the internet you can connect to my C64 using a telnet-client of some sort, or even another ethernet modem like I'm doing here, and "take over" the C64's READY prompt.  You can write, run, load, and save programs, and so long as they limit themselves to text and keyboard input, everything works great.

In the picture above, the breadbox C64 on the left is running the server using a prototype of the rev 4 modem, while the C64C on the right is running a little four-line BASIC terminal program at 1200 baud on a rev 3.

The telnetd server resides in C64 memory at $C000 and is hard coded to initialize the modem and listen on port 6400.  The server injects itself into all the the Kernal vectors and thereby redirects modem input to the keyboard buffer, and screen output to the modem.  I've been learning quite a bit about the way the C64 Kernal RS232 routines and Kernal IEC (disk drive/printer) routines share CIA timers and thereby step all over each other.  At one point running the telnetd server all but ensured that programs LOADed would be corrupted, while also disabling remote user input.  It was a mess.  But since then I've gotten it to the point where you can remotely do disk access commands and maintain your control, which I debugged by simple disabling all modem output during LOAD, SAVE, and OPEN kernal calls.

My next mission was security.  One wrong POKE or SYS and the server is toast.  This was fixed by injecting code into the 'IGONE' BASIC vector that converted the token values for POKE and SYS into PRINT. :)



An idle timer, and an overall time limit was then put in, so that tinkerers can't hog the system forever, or just sit around doing nothing.  I chose an overall limit of 1 hour, and idle time of 5 minutes, though I'm sure that will be configurable in the future.  I tried using the ability of the TOD clock to generate interrupts to handle my time limits, but then discovered that i/o and the c64 kernal's default NMI step all over those as well.  I mean, the first thing the C64 Kernal NMI interrupt does is turn OFF all interrupts on CIA#2.  What the hell?!  In the end, I just had my normal interrupt code watch the two TOD clocks for timeouts.

Lastly, I needed a welcome message for when users connected, and, to enforce my time limits, the ability to send the delayed "+++" (pause) "ath" commands to the modem.  Since I was detecting users coming online and time limit timeouts in the standard periodic interrupt, that was where I handled this 'forced disconnect' feature as well by simply adding a state machine and letting the TOD clock used for the idle timer do double duty as a delay-pause timer as well.

When completed, this program will join its brothers over in my Zimodem/C64Net WiFi project, which you can follow at https://github.com/bozimmerman/Zimodem.

I'm sure there will be more on this forthcoming.. unless some other shiny C= bauble catches my eye of course.









Wednesday, January 11, 2017

So, What's New? A WiFi Modem, That's What!


I haven't updated the blog in awhile, but this is not because I've been idle.  In fact, I've been so constantly busy on Commodore things that I just never took a breath long enough to talk about it.  As a result, you can expect an endless barrage of posts as I try to get you, and my future self, caught up.



The Commodore WiFi Modem project has been an effort by our local Commodore users group, and has been progressing nicely, both at the hardware and software level.  A WiFi modem is a wireless ethernet device usable by Commodore computers.  Anyway, as I've been handling the software side, I'll talk a bit about the hardware side first.

Shown above is the REV1 fab.  We (CTCUG) are just days away from Carlos having REV2 in his hands.    REV1 featured a fully powered ESP8266 board with proper RS232 levels for the Commodore 64 and 128, allowing basic KERNAL I/O between the two.  REV2 will feature DCD detection for BBS support, and support for UP9600 in C64 mode.  It will also have an optimized and cheaper design, and be only half the length of REV1.

I've done almost all of my software development using the REV1 board, as well as my own hand-made version.



I have several features I desire in a Commodore 8-bit internet device, which this WiFi modem is capable of fulfilling, with the right firmware.  These desires are:

  1. Uses the User Port (No cartridge port/bus hogging)
  2. Supports KERNAL RS232 (easy simple BASIC/ML programming)
  3. Does not hog system CPU/Memory (the TCP stack is in the device, not the Commodore)
  4. Supports socket listening and multiple client connections.
  5. Supports both streaming and controllable packet-based data.
  6. Supports CRC for error detection and correction of packet data.
  7. Can look enough like a modem to the Commodore to support old BBS programs.
  8. Supports boot-options (again, for BBS support)
With those goals in mind, I wrote firmware for the ESP8266 that would run in this device, which I called Zimodem.   The project is mirrored here on GitHub. 



After some initial frustrations, development went pretty well.  If you check out the build instructions, you'll see that I had all kinds of problems with the ESP8266 Arduino libraries.  And I probably should never have bothered with trying to program for it using Arduino IDE, but, well, it seemed like the best idea at the time.

To make a long story short, all of the features listed above, and many more, are implemented, with periodic bug fixes as the capabilities are tested and stretched.

While the initial landmark was being able to use a standard Commodore terminal program, such as Novaterm in the picture above, to connect to MUDs and BBS programs, the second landmark of a suite of internet applications that run natively in the C64 and C128 is also coming along.


Here is my C128D testing an early version of the IRC chat client.  Like all the clients, the same BASIC program works in a C64, in a C128 in C64 mode, in a C128 in C128 mode, in 40 or 80 columns.  The biggest difference is which machine language support file it loads.  All the clients will be made available on my ftp site, and in a separate GitHub project when available.

In addition to the IRC client, a TELNET client has also been completed, which does full ASCII/PETSCII translation, supports color as far as it can, supports TELNET codes, at least to the extent of eating them, and supports local keystroke echo, all in super-fast machine language core wrapped by the BASIC handler.  It also has a little phonebook that gets saved to your disk.

A WGET client has ALSO been completed, which allows any text or binary file to be downloaded from a non-SSL Url off the internet.  So, if the URL starts with HTTP://, it can probably download it to your commodore.

Lastly, an FTP client is currently in the works.  It uses PASSIVE modem FTP to get around firewall issues, although early tests with non-PASSIVE socket listeners were also promising.  This will be super cool.

Beyond these, a special MUD client, similar to TELNET except for some extra features unique to MUD clients, will be done.  A special C= BBS client will also be done.  It's also similar to TELNET except that it skips ascii and telnet translation.

So, that's it.  Keep your eyes posted here for more updates!