Design notes on Educube

The inner workings and state changes in the Educubes are (duh!) getting more complex.  I can barely keep them straight during one coding session, and by the next day I have only a clue about how they work.  This post is an attempt to keep some non-volatile notes about how it all works.  I’ll just edit this post rather than posting separate updates.  It’s probably useful for others to jump into the code.

Overview

The goal of the Educube project is to produce a set of “cubes” which present problems to the user.  By observing the screen on each cube and physically arranging the cubes into a line in the right order , the user solves the problem.  The cube system provides feedback about the success of the solution.

We (Workshop 88) were inspired to attempt the Educubes by the Element 14 Great Global Hackerspace Challenge.  Our blog on Element 14 has more details.

The cubes communicate wirelessly (IR) with their neighbors to the left and right.  A 3″ diagonal 240×320 color touch screen provides user interaction beyond the physical rearranging of the cubes.  Each cube also has a small speaker to produce tones.  An Arduino clone provides the brains; its “shield” mechanism provides electrical and physical mounting for the display and communications boards; a battery pack provides power.  A production version could be very much smaller than these clunky prototypes.

User interface

The current screen display consists of several elements.  Each cube has an unchanging unique color – here it’s purple.  The border and 3 buttons are displayed in that color.   A vertical band appears at each edge when a neighbor is detected, painted in the neighbor’s color.  When a neighbor goes away, the edge band goes away after a short timeout.

The central block contains this cube’s problem data.  Here it’s just the letter ‘D’.  The background color changes for each new problem presented.  All cubes stay in sync and display the same background color.  Touching anywhere in a large area in the center of the display goes to the next problem.

A second text area, below the main 3 characters provides up to 10 characters of additional information.  (The picture at the right has no neighbors present.)  This could be used, for example, to tell the user to treat the data characters as an anagram rather than sorting them in alpha order.

When the user presses the lower screen button, marked “GO” on the rightmost cube, it tells that cube to evaluate the order and data on all the cubes.  If the solution is correct, a short happy tune plays, and a success message is displayed.  That’s currently just text “YES”, but we hope to make it a simple bitmap image.  After a few seconds, the displays revert to their previous contents.

Depending on the application/problem type, the “+” and “-” buttons may allow the user to change the displayed data. This might be appropriate on the rightmost cube for a math problem. Each button press is acknowledged with a short beep.

A small button on the upper right corner, near the battery pack, provides a hardware hook that could be used for some kind of debugging or back door access.

Low level datagram structure

A simple datagram protocol is implemented with the serial IR communication hardware at each side of each cube.  One IR transceiver can be seen, out of focus, above and to the right of the small button.  The protocol looks like this:

<sync byte><length byte><variable length payload><2 byte checksum>

The sync byte (0x88) helps identify the start of a datagram.  If the checksum does not match the data, the datagram is silently discarded.  This is completely consistent with an environment where cubes may come or go at any time.  Datagrams are sent continuously out both sides of each cube, whether they see neighbors or not.  Communication is successful if some packets get through when cubes are close together.

High level (app) messages

The main loop polls the left and right interfaces continuously to see if a message is available.  If it is, it is parsed expecting the following structure:

<opcode ><sender's color/ID ><current problem number><zero or more bytes of data>

Normal data goes from left to right.  Each cube takes what it hears on the left, appends its own data, and passes that all on to the right.  Only the rightmost cube knows the data from all the cubes, so it is the one that does the evaluation when GO is pressed.

Opcode OPDATA is used on such messages as they’re sent out the right interface.

Opcode OPNOP is sent out the left interface with message containing only the sender’s color and current problem number.  No data is sent to the left.

Opcode OPRIGHT is sent to the left only when a problem is solved successfully.  This is propagated from right to left by all cubes.  This way all cubes can celebrate success together.

Opcode OPWRONG is also sent to the left (only) when a solution is incorrect.

Messaging may be expanded with additional opcodes.  One early additional opcode would allow a cube creating a new problem to send distinct data to each other cube to display.

Neighbors

Each cube is constantly aware of whether it has a neighbor on each side based on whether it is receiving messages on that side. The visual acknowledgment to the user of the vertical bars that appear in the neighbor’s color when a neighbor is detected provide a little interesting “magic” to the user as well as providing reassurance that things are working well.

Whether a neighbor exists to the right is critical information for processing the GO button:  Only the rightmost cube can possibly successfully evaluate the order and data of all the cubes, and so the GO button is dynamically only displayed on that cube.

The data structure for each (L, R) neighbor contains a boolean exists flag, his color, what he thinks the current problem number is (probNum), and a timeout value.  The timeout is reloaded every time a message is received.  The timeout is decremented every time through the main loop.  When it reaches zero, the exists flag is cleared and the vertical bar on the display is also cleared.

In order to keep the “color” sent in each message header and stored in the neighbor’s data structure to one byte (“colors” as sent to the display are 2 bytes), the value actually sent is the cube’s unique one byte Id.  The 5 cubes have Id’s 1-5 burned into EEPROM.  Value zero is used to paint the vertical band black (“remove” it).  Value 6 is reserved for a development Arduino.  The actual 16 bit color values are fetched when needed by using the Id as an index into the cubeColors array.

The main loop

The real loop is implemented with a goto within the main Arduino loop() routine.  It implements a very crude multi-tasking system.

We seem to go around the loop about every 20 ms.  Here’s what we do:

• See if any message is available from left, then right channels.  Update neighbor info and timeout, update current problem number, set WIN or LOSE state if we get those opcodes.  If it’s data (from the left), update what we have to send on to the right.  If problem number is greater than ours, update ours and generate new problem data.
• Decrement neighbor timeout values, dropping the neighbor if timeout hits zero, and reevaluating who displays the GO button.
• Decrement WIN/LOSE state timeouts, going back to state NORMAL if timeout hits zero.
• Check the small button.  If pressed, increment current problem number and generate our new problem data.
• Send message out right, left sides if it’s time to do so.  We currently do it every 60 times through the main loop.  If we’re in WIN or LOSE state, send immediately, changing opcodes from the normal OPDATA and OPNOP.
• Check the touchscreen.  If any presses, deal with them.  This is where the GO button triggers solution evaluation.
• Call the IR read lib routine.  I’d prefer this to be done by a timer interrupt, but I couldn’t get it to work, and putting it in the main loop works.
• Goto top.

Timeouts

We time neighbors out if we haven’t heard from them in NTIME times through the main loop.  Their timers are reloaded with NTIME (currently 100) each time we get a message.

Much like with neighbors, we load stateTimeout with STIME as soon as we hear an OPRIGHT or OPWRONG.  There should be nobody reloading it, so those states just time out after STIME loops and we revert to state NORMAL.  To avoid a feedback loop, we send OPRIGHT/OPWRONG out ONLY on the left interface.  This lets it propagate all the way to the leftmost cube without fear of making a cube to our right inappropriately enter a WIN/LOSE state.

Win/lose state

We want to stay in WIN/LOSE state for a little while so users can enjoy the feedback.  When the rightmost cube gets a GO screenpress, it evaluates the data from all the other cubes, determines right/wrong and sets the cube’s state to WIN or LOSE.  In addition to putting the win/lose displays up briefly, it plays tune HAPPY or SAD.  The propagation delay to the leftmost cube is so long that if we had them all play tunes, it would just sound chaotic and be laughable.  So only one guy plays.  The propagation delay is long enough that we may drop trying to have cubes other than the rightmost one display a win/lose message.

Problem types

There are two dimensions to the multiple problems the system will present to the user.  First, we have the concept of the “current problem number”.  Each time a problem is solved and the user asks for a new problem, we increment this monotonically increasing current problem number.  The two things this number allows us to do are:  Pass the word to all the other cubes that we have a new problem and they need new data, and potentially change to a different type or class of problem.

We have evaluator code for 3 games/problem types: alpha sort, numeric sort, and the addme math problems to run on the rightmost cube when GO is pressed.  Of course all cubes contain identical code, and which one is rightmost depends on how the user orders them.

We’d like to be able to have not all the same problems all the time, if only to keep users from getting bored.  So we have multiple different “problem types” all loaded at the same time.  We’ve referred to these types as “applications” in the past.

We need two bits of coding to implement each problem “type”.  Obviously the evaluate-the-answer code triggered by the GO button will be quite distinct for each kind of problem.  But especially to the extent that we have randomly generated content for each problem, we need to know what kind of data to generate for each cube for each new problem of a given type.  Obviously these must be kept in sync.  We keep the current problem number in sync across the cubes by broadcasting it and always updating our current number if any neighbor advertises a higher number.  But each cube must know what the problem type is before it can generate its random data for that problem.

While global CurrentProblem has been there for some time, support for multiple problem types is just starting to be included.  I suspect for the near future (including “showtime” for the final video we’ll submit) we’ll end up with a fixed common array of problem types, indexed by problem number.  Such an array could be hard coded or computed at run time using the random() function starting with the same seed.  The global ProbType array provides access to the current type.  It is populated in  genNewProblem() by a lookup using CurrentProblem.  That genNewProblem function knows how to generate different data based on ProbType.

Screen management

Since screen painting is painfully slow, screen refresh is broken out into a couple of functions.  The various areas – borders and buttons – are pretty logically #defined and can almost be changed just by making the obvious changes.

refrScrn() repaints the whole screen with current colors and information.  It paints or omits the +/- buttons depending on global doUpDn.  The GO button is always painted.  There are various helpers to repaint various bits without the visual insult of refrScrn().

refreshNeighborBorders() is used to paint the vertical bars when neighbors are detected or go away.

refreshDispString() just paints the 3 chars of data in the middle of the screen from char array DisplayString.

clearDispString() just repaints the 3 chars of data in the middle of the screen in the current background color.  This is a much faster way to “erase” the using the fillRect() function.

refreshDispInfo() and clearDispInfo() provide the same functionality as the two above, but for the 10 character “info” space below the 3 main display characters.

clearDisp() repaints the central block with the current color to erase the current display data.  It does not repaint enough for a color change from a new problem number.  That requires a full refrScrn().

Touch screen

I pretty much just copied Bill’s initial work making the touch screen work, including a resistance calibration, and it seems OK.  The boundaries of the buttons are all #defined.  There are comments showing “process + button here”.  Lots of lines of code and defines, but many have been moved to Educube.h.

Sound

The first version will do nothing but support the Arduino tone() function.  D11 is connected to a small speaker through 100Ω.

Variable length “Tunes” are implemented in EducubeNoises.h as a bunch of #defined names and 4 arrays of data. Each tune consists of a series of notes, each with a note duration and a silence duration to follow that note.  Those allow for rests as well as control of whether the notes are staccato.  The arrays containing those 3 quantities are all managed as one line of values per tune in the header file.  The makeNoise() function that plays the tunes knows how to index into the 3 arrays based on the tune number and the lengths of that tune and preceeding ones.  Ugly, but it works.  To play a tune call makeNoise(tune name).  To add a tune, add a #defined name at the end of the #defines, put the length at the end of noiseLen[], and add a line at the end of noiseTone[], noiseOn[], and noiseOff[].

Because this routine plays a whole tune – possibly a second or more – atomically in the foreground, it’s not very friendly to other things going on, like reading from the IR ports or sending messages out.  It should be made just another task in the main loop, playing one note or rest and relinquishing control to the main loop.  Someday.

Code

The code is pretty messy.  Capitalization of variables and functions is not consistent.  Organization of everything needs cleanup.  There are a fair number of comments, but those, too, could be better.  There are lots of commented out leftovers from debugging various bits, though many more have been removed.  There is terrible inconsistency in using multiple 8 bit variable types for no good reason, resulting in lots of casts.

Hardware

The brains of the Educube is an Amtel ATMega328 (from Element 14!) on an Arduino clone board called the Diavolino from Evil Mad Science.  Using the Arduino shield interconnect standard and some stackable headers (male + female), we built a 3 layer sandwich of a custom IR comms board, the Diavolino, and a plug-on display shield.

The display shield hosts an Adafruit 240×320 LCD touchscreen.  That shield also provides mounting and a mini-infinite baffle for a 0.7″ speaker.  The speaker is driven in the simplest possible way – with 100 ohms to one ATMega I/O pin (D11).  After a board hack, we routed D12 to the display’s reset line to be able to start it reliably in software.

To get short range directional wireless communication between cubes, we chose IR.  Since the ATMega doesn’t have enough serial I/O to run two serial channels (plus the console), we added an NXP SC16SI752 dual UART.  This chip provides the pulse shaping needed to work with the IrDA IR transceivers we chose.  That UART chip talks I2C to the ATMega.

While the Wire library does provide basic I2C primitives, there is nothing to talk to the 40+ registers in this particular UART chip.  We had the good fortune to find Strich Labs’ MultiSerial shield in pre-production, which not only uses the ‘752, but which has an open source library to talk to it!  That gave us a great starting point.  We added some very crude support for multi-byte reads and writes to the library, gaining two orders of magnitude in performance.

Our shield for the UART also contains a few resistors and caps as recommended by the IR transceiver manufacturer, a 7.3728 MHz crystal for baud rate generation, and a 3.3V low dropout regulator to power the chip.  Since the UART also provide 8 general purpose I/O pins, we put an LED (and limiting resistor) on one pin as a first “hello world” test that we could talk to the chip, later used as a poll routine status indicator.  A tiny surface mount push button is also mounted on a corner of that board grounding D10 for a general purpose input.  It was used before we had good touch screen routines to cycle between Educube applications.  Since the IR shield is between the Diavolino and the battery pack (and provides some pin sockets to connect the battery pack), it was also the logical place to mount a small slide switch for power.  And since the 4 AA cell battery with fresh cells at 1.6V each would put out more than the absolute maximum input voltage for the ATMega, the shield puts a silicon diode in series with it to drop the voltage a bit.

Since at this writing the cube has no housing, we needed a way to prevent IR from other cubes down the line from being seen by the receiver.   Some black duct tape provided just the IR-proof barrier we needed.

Eagle files for both boards (including the display reset connection to D12 now on the board) are here.

Next steps

The biggest usability issue and annoyance is the not infrequent lockups, which require power cycling the cube.  In second place is slightly flaky neighbor communication.  The neighbor acknowledgment bands show brief disruptions in communications that shouldn’t happen.  This means you have to look carefully to make sure everybody is in sync before you hit the GO button to test the solution.  Ugh.

The lockups are almost certainly caused by the very crude implementation of multi-byte reads in the UART driver library.  No return values, no error checks – the functions work on a sunny day with a tailwind, but really should be updated to better protect themselves from some buffer overflow conditions.  That will require some digging into the implementation of the I2C functions in the Wire and twi libraries.

Neighbor communications could probably be improved with a different approach to the send/receive timing.  Since the transceivers are half duplex and echo transmitted characters to the receive channel, we have to completely flush the receive buffers immediately after each transmit, potentially dropping a just-received transmission from a neighbor.  That shouldn’t be a problem, but if we miss a few transmissions in a row, we time the neighbor out.  We currently send once every 60 times through the main loop.  If two cubes happen to get in bad sync, one could flush just at the wrong times and miss all of the other’s transmissions.  There’s a small randomizing feature that drifts the timing slightly based on the cube ID in EEPROM, but it doesn’t seem to be enough.  I’d like to try really randomizing the send times by replacing the fixed number of loop before a transmission with a random number.  Sending more frequently should help as well.  While the timeout – 2 seconds or so – is reasonable, it should have to miss several transmissions be timed out.  That means increasing the number of transmissions per second.  Should be an interesting afternoon’s play.

After those, enhancing the math app to multiple digits and other operators is on the list.  And a “story” app where you get pieces of text and must order them to make a sensible story should be interesting.

…end…