SPMP_SendSignal() Research (and ScummVM)

I did some rote work and added a new header file (mcatch_cmd.h) to the library that defines all the commands SPMP_SendSignal() accepts, 128 in total. I tried my best to guess reasonable names, but the only ones we can be sure about are those that are mentioned in debug strings. There is some really interesting stuff in there: power management, TV out, system update(!), SD card detection, various audio settings (speaker switching, volume, equalizer, Dolby), FM radio control, LCD panel control (suspend/resume, brightness), camera control, media playing, and much more.

SPMP_SendSignal() most often gets a pointer to a parameter structure that is used as input and/or output. So far I have only figured out the one used to read the real-time clock, in order to give ScummVM save games the proper timestamp.

And with that we come to our main attraction: ScummVM for SPMP8000 devices. This build only contains the SCUMM v1-v6 engine (memory is tight on this platform), but supports most features already, including sound and save games. The major feature that is not supported yet is compression, both for save games and game data. This is not difficult to fix, I simply haven't gotten around to it yet. Another feature I absolutely want to add is touchscreen support, but that will have to wait a bit; my A3300 is still in flight from China. :)

I have tested this build with Fate of Atlantis (VGA/Floppy) and Maniac Mansion (Enhanced), and they both work fine. It is quite conceivable that bigger games might not run due to a lack of memory, but that remains to be tested. Source code is at the usual place. Have fun!

[UPDATE: Here's the key mapping.

• F5 -> START (brings up the in-game menu)
• ESC -> SELECT (skip cutscenes)
• Mouse cursor -> d-pad
• Left mouse button -> O
• Right mouse button -> X

]

Win32 Toolchain (and a teaser)

I painstakingly built a Win32 (MinGW) arm-eabi toolchain, maybe that'll be of use to some people. Extract it in the root of your MinGW system. (Or, better yet, get yourself a real operating system, even if it's just Mac OS or something like that.)

Apart from the previously mentioned adbg and better support for building on Windows, the library has received a few small touchups:

• The startup code, header files, and linker script have been made C++-ready.
• The newlib/eCos glue translates stat()'s st_mode correctly now, so you can distinguish between files and directories.
• The utility function libgame_chdir_game() changes to a well-defined directory (the "GAME" folder), making it easier to find data files.

And these unassuming little fixes allow us to do this:

(Yes, that's ScummVM. The photos have been taken with the JXDs, my apologies. :)

Stay tuned...

Debugging SPMP8000 Programs Without Soldering

As everybody knows, the best way to debug code is to print lots of crap to the console, not the least because that requires very little platform support: All you need is a place where you can dump data and where it can be retrieved later. On the SPMP8000, I have so far used the file system for that purpose: Just redirect stderr to a file and write everything there. There is, however, a big problem with that approach: On a system without memory protection, programming errors usually result in a system crash, which in turn leads to total loss of the logged output. A better solution would be to have a channel of communication from which the debug output can be read as it is written, such as a serial port.

Luckily, all SPMP8000 microcontrollers come with an on-chip serial port, and breaking it out seems simple enough: Most devices appear to have soldering pads where you can attach wires. So I set out to perform this operation on my JXD 100 and promptly ripped the soldering pad off the board when I tried to verify whether the solder joint is good. Determined to make this work, I then attempted to solder the TX wire directly to the leads on the microcontroller, with a soldering iron that, at its narrowest, is about as wide as two pins. Predictably, I ended up shorting the UART TX and RX lines, and I suppose the JXD 100 is now happily talking to itself when printing debug output. I briefly tried to aggravate the situation by using a knife to scratch out the solder between the pins, but luckily came to my senses before causing more serious damage.

So what now? After some contemplation I came up with an alternative solution that does not require any hardware shenanigans and thus allows me to keep my white gloves on and my hardware alive: using the line out connector as a serial port.

The audio debugging interface (adbg)

The latest feature in libspmp8k is the adbg debug interface. It takes your debug output, serializes it and sends it to the headphone jack. When you connect that to the line input on your PC and run adbg_console.py there, it will decode the signal and print the debug output to your console. No soldering, no need to employ serial-to-USB converters, just plug in a cable and you're done.

The Catch(es)

Of course nothing is ever that simple, and there are a few things that you have to keep in mind:

• If you get no output at all, make sure to you have turned up the volume up to the maximum on both the transmitting and receiving ends. We want as much clipping as possible to get a nice square wave. You may also have to tell adbg_console.py which input it should listen to.
• If you usually get correct output, but there are bursts of garbled data, make sure you have good sound hardware with good drivers on the PC side. My computer at work (HP desktop with Intel HDA sound, Linux kernel 3.4.x) inserts random bursts of noise into the input signal every once in a while and is therefore largely unusable for this application. The machine at home (EVGA 780i SLI FTW board, also Intel HDA, kernel 3.1.x), on the other hand, works without a hitch. YMMV.
• Your computer may be oddly wired. If you receive output, but it's all garbled, you may have to invert the signal (adbg_console.py option "-i").
• There is no error detection or correction, nor is there a back channel, so this is not something you can or should use to transmit data that has to be 100% error-free.
• The devices I have seen so far (JXD 100 and A1000) seem to detect the presence of headphones by impedance. A line input has a much higher impedance that headphones and is therefore not detected. This is not a problem for signal transmission because the output on the headphone jack seems to be always on anyway, but it will be a problem for your ears because the internal speaker will not be switched off. You can, however, switch the speaker off manually in the "Settings" app.
• Finally, you have to disable sound in your program when debugging because it would interfere with the debug signal.

For an example of how to use audio debugging, check the adbg demo in the demos directory. The console program adbg_console.py can be found in tools.

The console application is written in Python, supports ALSA and PortAudio, and requires pyalsaaudio (for ALSA) and/or PyAudio (for PortAudio) to be installed. ALSA is of course Linux-only, but using PortAudio it should work on Windows and Mac OS X as well, although I have not tested that.

Have fun!

xrick for SPMP8000

Here's a port of xrick for the SPMP8000 platform. I always wanted to be the first to port the most ported game of all times to a "new" platform. :) Source code is at the usual place.
Please note that I have only tested this build on the JXD 100 because my wife brought the A1000 on a trip, so I can't use it for development ATM.

There are a few new features in the library as well:

• A couple of new test firmwares (JXD A16, 2000, and 300); all passed the tests without any changes.
• I have added two NativeGE hooks I found in the A16 firmware, NativeGE_SPUCommand() and NativeGE_getTPEvent(). I have no idea what the former does, but the latter obviously allows you to poll for touchscreen events. I do not have a touchscreen SPMP8000 device (yet), so I cannot test this right now.
• Followed a suggestion by Sea S to fall back to the HZX16 font if the Sunplus fonts are not available. This is a phenomenon I have never encountered in any official firmware, but there seem to be custom firmwares with those fonts removed.
• I have exposed _putc (and named it _diag_putc to avoid clashing with newlib). It's the character output function used by diag_printf(), which is used for all operating system debug output. By redirecting it to your own implementation you can, for instance, redirect system debug output to a file.

Have fun!

Better SPMP8000 Homebrewing (and PC Engine emulator!)

I recently got myself a JXD A1000 and found it quite an enjoyable device to develop for. The "official" API ("native game" interface) isn't any fun, though: Fixed screen size of 320x240, no access to all but three buttons, no support for generated audio, no access to the ROM file system, not even a way to list directories.

The firmware is, of course, able to do much better than that, but while all the functionality is there, we don't know how to access it: function and memory addresses are different from device to device and from firmware revision to firmware revision. While it was easy to adapt, for instance, Triple Oxygen's libemu for the Cybergame to my A1000 to get sound support, the result would again only run on a single machine. I didn't really want to spend a lot of time developing software that won't run on other people's systems, so I did some thinking and came up with a solution: On-the-fly firmware reverse-engineering.

Whenever I found something interesting in the firmware, I devised a simple heuristic that weasels its way through the running operating system, usually starting from a function the address of which we know because it's part of the "official" interface, and taking advantage of typical subroutine calls, address loads, debug strings, and function table offsets to get to the sweet stuff in a way that works on all SPMP8000 firmwares. To check that it really does work everywhere I availed myself of eleven different firmwares (and an additional JXD 100) and wrote a test harness that verifies that the heuristic actually arrives at the right spot.

The result is a library that provides you with a greatly extended API in a portable fashion, so your programs will (hopefully) run on any SPMP8k device. Here's the stuff you couldn't get before:

• Emulator interface: This is the API used by the built-in emulators. It provides hardware scaling from arbitrary resolutions, access to all keys on all controllers (both raw scan codes and standardized mappings), and streaming audio.
• Frame buffer access: Direct access to the currently active display device's frame buffers, with support for single and double buffering.
• eCos POSIX-like file API: Access to all files without restrictions or filtering. Full set of directory functions. (Seriously, with the official API you can't even read a directory!) I have glued this interface to newlib, so you can use stdio functions (fopen() etc.).
• High-resolution timer: The official API gives you mere 100 Hz; with the new libgame_utime(), you can get microsecond resolution. Indispensable for accurate speed and profiling.
• CPU scaling: Saves battery power, you can go down as low as 5 MHz. (Sorry, no overclocking.)
• Built-in fonts: A few simple routines I wrote to use the bitmap fonts in ROM. Not as black magic as the other stuff, but with full Unicode support. :)

There's more stuff in there, like the mysterious SPMP_SendSignal() function that takes dozens of commands, or the eCos threading interface, that I will unearth as needed. I also more or less worked out the signatures of all the as-yet undocumented functions in both the "native game" and the emulator API, if you want to experiment a little.

Using this souped-up libspmp8k, I have ported PC Engine emulator TGEmu, and it runs like hell (after some optimization) on both the JXD A1000 and 100, and probably on your SPMP8000 device, too. Check it out! (You need PC Engine ROMs, of course. Uncompressed, zipped, and gzipped files are supported.) The source code should also give you a good idea of how to use the library. Have fun!

snes-sdk update

After about a year I finally got around to do something with snes-sdk again. I fixed it to build (ahem), and added some of the optimizations from a tool called optimore to the peephole optimizer. (I do not recommend to use optimore, BTW. I have found at least some of the optimizations to cause test suite failures...)

Linux and the built-in flash

I don't know if this is generally known, but I've been digging into the original firmware, and it appears that the flash translation layer used for the internal memory is NFTL. At least some left-over debug strings strongly point in that direction. NFTL is readily supported by the Linux kernel, but there's a problem: When I enable the jz4740 NAND flash support in the opendingux kernel, it freezes at some point during bootup, so I cannot test my assumption.

(Also tried the legacy kernel, but it wouldn't even compile with NFTL enabled.)