Jim's Projects

After digging into I²C protocol some more I felt armed to hack the MultiSerial library to do multi-byte reads and writes.  I added 2 user level functions – writeMulti() and readBuf().  I had to add 2 more internal functions – the ones that talk to the Wire lib.  But as I tried to write the internal msWriteRegisterMulti() and msReadRegisterBuf(), while Wire did seem to support multi-byte operations, the bits I needed weren’t exposed as public.

OK – if I have to I’ll make a private Wire.h with more things exposed.  But that wasn’t enough:  the code used by Wire isn’t in wire.cpp – it’s in the underlying twi.cpp!  I really didn’t want to make a private version of that, too!

Google and I could find little or nothing about heavy use of Wire (and twi) for making other libs like the one I needed.  But I finally figured out how it actually works – and by working with the system, everything I needed was there.  I put crude versions of all the routines in.  No sanity checking, no error returns.  Use it perfectly and it works.  Else – no promises.

Wrapping my application level routines – writeMsg() and rxpoll() in millis() timers, I got some numbers.  All are for sending/receiving a 27 byte message.

byte-at-a-time send: 330 ms       multi-byte send:  4 ms

byte-at-a-time read: 800 ms       multi-byte read:  7 ms

So how fast will it go end to end?  I got up to about 1300 B/sec, but if I went faster the receive routine locked up.  I suppose I should start handling some error conditions more gracefully 🙂

That’s plenty fast for the target applications, but not so much for bulk downloading of applications/problem sets/whatever via IR.

I sent a little update note to the original author of the lib and learned she was having performance problems with a VGA shield she was making and using I²C to talk to.  So my multi-byte efforts may be the starting point for a performance boost for Strich Labs as well!

I was frustrated because message transport between cube bases was slow/erratic.  At 115200 b/sec, the best case is ~11KB/sec.  For a 22 byte message, that’s 2 ms best case.  That’s a pretty short in timelines measured in 10s of ms.  But I’d see a successful message every few seconds.  Huh?

I put a hook in to use the new button to toggle between the current send+receive and receive-only behaviors.  It was still slow, but more reliable when transmit was turned off on the receiving box.  Maybe that had to do with not throwing away messages in the flush() call.  The flush() call also flushes both transmit and receive fifos.  Since we flush immediately after sending, there was the possibility that it would truncate the send buffer, so I tried clearing only the receive buffer.  It didn’t seem to make any difference.  That makes sense in the light of later observations.

Let’s measure something

A quick check with a one char print every time a message was transmitted showed maybe one message going out per second!  The example app does 10 reads with 10ms delay between and then one write (to both sides).  Assuming the programmed 10 ms delay is the main delay, it should transmit about every 100ms.  Where’s the time going?  About the only thing in the loop 9 times out of 10 is a call to rxpoll().  Bracketing that call with calls to millis(), it turns out rxpoll() takes ~72 ms to read both channels with no data coming in!

When receiving actual messages, rxpoll() took 4 reads for a total of 1220 ms to receive a 55 B message, and 523 ms in 2 reads for a 13 B message.  That’s 20-40 ms/byte in receive time.

How about the send side?  Bracketing just the new writeMsg() call, that took 186 ms to write a 13 B message (including sync, len, cksum) and 610 ms for 55 B.  That’s ~12 ms/byte.  Hmm – that’s still noticeably less than the time it takes to receive, so the analysis isn’t very complete.

What can be done?

Since the I²C lib calls are obviously expensive, how can we reduce that time?  I noticed when I was looking into the lib that the transmit and receive functions transfer a single byte at a time.  The I²C bus supports multi-byte reads and writes.  If we could use those, it could help a lot.  I don’t know how directly that’s supported by the Wire lib, but it looks like the primitives Wire exposes are extremely low level – like “send a START condition”.  With a little digging into the I²C protocol, we might be able to build up appropriate commands to do multi-byte sends and receives.

The Hackstrich MultiSerial lib only provides one-byte reads and writes, so we’d have to add new lib calls for the efficient stuff.  That shouldn’t be very difficult (if we knew how to do it).

The good news is that if slow, the current implementation essentially works.  We can tweak with different transmit/receive ratios for R and L channels based on how we need apps to behave and improve performance some.  And if/when we add more efficient multi-byte reads/writes to the lib it will be just a few-line change to the app and perhaps an order of magnitude performance improvement.

Testing the last board

I’m almost done with what I’d hoped to do!  I just finished soldering up the last of the five boards.  I only have four Diavolinos, so I tested the last one by hooking it up to a regular Arduino.  All it takes is two wires for clock and data for the I2C plus ground and 5V.  OK – the button is another digital pin to the Arduino.  I didn’t bother to hook that one up.

You can see the others in the background.

The other thing I wanted to contribute to the platform was a working library to talk to the UARTs and some simple example code showing how to use the library.  I also wanted at least a basic UDP-like datagram protocol with checksum.  I got most of that.

There were really only two routines I had to write.  There’s a writeMsg() function that takes a buffer of payload bytes and packages up a message – sync byte, length, payload, checksum and sends that to the UART.  And a poll received data function.  I hoped to have that one run by a timer interrupt, but couldn’t get that to work.  The application now has to call that routine frequently.  The example program does it every 10ms.  That routine reads bytes coming in and assembles and check messages.  When it’s got a good one, it sets a flag so the main app can grab the message.

I also put a simple app together that loops reading both channels and periodically writing a random message (one of 7 canned strings) out both sides, and loaded on all the Diavolino/IR board combos.  If you point either end of one at another which is connected serial to the IDE, it displays the message,  who sent it, which side they sent it from, and which side it was received on.  And if you push the button on the IR board it lights the Arduino standard LED.  But its main claim to fame is showing how to use the send and receive routines.  It’s far from perfect, and needs tuning (Tx vs Rx), but it’s a good start.

The biggest disappointment (besides not being able to get the time interrupt to work with the receive poll routine) is that I couldn’t get the two routines into the library and have them still work.  It was just dumb C++ problems, and I wasn’t good enough to over come them.  The example program has both routine in the main source file.  Ugh.  I hope somebody else can figure out how to put them in the lib.

I even wrote up a readme with most of the relevant info – see it below.

It’s great to be about done.  I’ve been off for two weeks, but have done almost nothing but work on the boards and the software.  Done is good.

W88MultiSerial library README  ver 0.9  4/15/11 jw
---------------------------------------------------


BACKGROUND
----------
This code is to allow an Arduino (or clone) to talk to the custom IR comms
board used in the Workshop 88 Educube project.

The comms board uses an NXP SC16IS752 chip with the following relevant
features:
- 2 independent UARTs
- talks I2C to the host Arduino
- UARTs have appropriate pulse shaping hardware to be used with IrDA IR
transceivers such as the Vishay TFDU4101 on the board
- UARTs each have a 64-byte receive and a 64 byte transmit FIFO
- chip has 8 bits of General Purpose IO
- the chip is capable of generating Tx and Rx interrupts, but we don't have
interrupt lines available

Hackstrich Labs is in the process of producing an Arduino shield which
implements 2 serial ports using the SC16IS752.  They have graciously made
their code open-source, and that code has provided an excellent starting
point for the W88 code.  This includes initialization, register access,
single-byte read and write functions, and a non-blocking "is anything
available on receive" function.


OVERVIEW
--------
The example program IrBoard3.pde works with the W88 IR comms boards attached
to Diavolino Arduino clone hosts.  It's a suitable starting point for
developing real applications, providing:
- initialization of the UARTs for IrDA
- a higher level function to write a message
- a poll function that reads from both channels, assembles and checks
messages, and provides notification when a valid message is available
- access to a GPIO pin with an LED on the IR comms board
- an application level read, write, and message handling loop structure

The initial message structure provided is as follows:
sync byte | length byte | variable number of data bytes | 2 byte checksum

The sync byte is 0x88 (ooow!).  Length is limited by a #defined buffer size
of 70 bytes.  The current checksum is the most basic
sum-of-all-bytes-truncated-to-16-bits.  Checksum includes the length byte,
but not the sync byte.  Zero length messages may not work.


Timing is critical, particularly in light of the facts that the IR channels
are necessarily half-duplex, and transmitted characters reliably appear in
the receive buffer.  This means the receive buffer on each UART MUST be
flushed immediately after transmitting a message.  If this is not done, the
transmitted characters will be comingled with the received characters,
invalidating the message checksum.

The example program transmits once for every 10 reads.  This timing tuning
may not be optimal.

There's also a hardware button on the IR board, connected to D10 on the
Diavolino.  Very basic, but works.

I also wrote a unique identifier into EEPROM of each Diavolino.  They somehow
have to know who they are, and this one-byte value, 0-6 does that.  It may
also be useful for randomizing delays so the cubes don't end in a
pathological lockstep mode not able to communicate.


FUNCTIONS
---------
The init is just begin() call for each channel, and a call to set one GPIO
pin as output (for the LED).

void writeMsg(byte chan, char len, unsigned char* buf)
	chan is 0 or 1
	len is number of data bytes to send
	buf is a pointer to a buffer containing the data payload

The function prepends the sync and length bytes, and computes and appends the
2 checksum bytes.  It writes the message to the specified UART's transmit
FIFO.


void rxpoll()

Call this often - the example program does it every 10 ms.  It takes care of
reading to exhaustion, resyncing using the sync byte, doing sanity check on
length, and verifying the checksum.  When a valid message has been received,
it copies the data payload to msgbuf[channel].  To tell the main application
when a message is available, it puts the length in msglen[channel].

To allow the main program to deal with the message before it gets overwritten
by the next message, rxpoll will not construct a new message until it sees
that the main program has set msglen[channel] back to zero.  It is the
obligation of the main program to do this.


SAMPLE OUTPUT
-------------
Output of example program
   - R|L msg means message received on right|left channel
   - B|b= means sent from the right|left side of device 2 (B)
   - . means chars were available
   - x means bad checksum
   - digit at end of message is a sequence number (0-9) to check for lost
msgs

I took a device on the battery pack and pointed each end of it at each end of
the receiver.  Output is from receiver to IDE serial monitor.

IR Board Test
I am device 4
...R msg of len 14 b=Hello mom!0
...R msg of len 27 b=Arduino clone Diavolino2
...R msg of len 54 b=The quick brown fox jumps over the lazy dog's back3
...R msg of len 14 B=Hello mom!7
.....L msg of len 15 b=hackerspace0
....x.....L msg of len 13 B=Educubes!8
...R msg of len 27 B=Arduino clone Diavolino9
....L msg of len 15 hackerspapace0


STATE OF THE CODE
-----------------
It all works, but isn't optimal yet.

- both rxpoll() and writeMsg() should be in the library, but I couldn't get
them to work when I put them there.  Some programmer more facile with C++
should figure out how to put them in the lib.  For now, you must include
those functions in the file with loop().  Sorry about that.

- It would be very nice if rxpoll could be hooked to a timer interrupt so the
main line didn't have to mess with it.  Using MsTimer2 I could call rxpoll,
but it didn't work when called that way.  I hope somebody smarter than me can
fix that.

- The reported length on received messages isn't always right, although
there's never garbage at the end or anything.

- rxpoll() could reassemble the next message in its private buffer and just
not write to the public buffer until msglen was reset.  This might throw away
fewer messages.

- There are functions like word length that are stubbed out in the lib that
should be populated.  But I hacked the begin() function so it does exactly
the inits we need for IrDA, so that's a lower priority.

- There should be some versioning of the lib .cpp and .h.

- It should be put in github or some such.

- Fixes for known problems should be reported to the original author.

- The original code from Hackstrich Labs is in the "original" directory.

- All remain bugs should be removed :)

		...end...

IR board

I’m well into the prototype stage of 2-channel IrDA IR comms Arduino shield for the Workshop 88 Educubes GGHC project.

Very efficiently, IrDA uses very short (~10μs) IR pulses, which must be delivered by the UART driving the IR transmitter.  This really requires a special pulse generator on the send side, and a pulse stretcher on the receive side – a special hardware driver.

The first chip I found with IrDA drivers, the MCP2120, took TTL levels in and did the pulse shaping to talk to an IrDA transceiver like the Vishay TFDU4101s we have.  Unfortunately, the ATMega 328 only has one UART, and usually uses it to talk to the console in its IDE.  It does, however support I²C and SPI, so I next looked for a dual UART chip I could talk to with I²C/SPI and had built-in IrDA drivers.  I settled on the NXP SC16IS752.

We were obviously going to need a board to host the UART and the IR transceivers.  Since the main brain of the Educube is a Diavolino (Arduino clone), a shield was appropriate.  Since the LCD display is hosted on another shield, there were shield-stacking considerations, too.  Fortunately, the crystals I’d already bought for the unused MCP2120s would also work with the ‘752.

After a round of stupidity about which side the foil was on for the shield pins to line up, I had a board layout with the ‘752, its crystal and caps, Arduino shield pins, an LED on one of the ‘752’s GPIO lines for ‘hello world’ or whatever, the IR transceivers, easily replaceable transmit current limiting resistors and a local cap for nice sharp rising edges on the IR pulses, a button on one of the unused Arduino pins (D10) possibly for a ‘debug’ mode, pads for receptacles for the pins on the battery packs we had, a diode in series with the battery (since the 4×1.6V from new cells is easily beyond the absolute max voltage of 5.5V on some of the chips), an on-off switch, and a layout for a 3.3V low-dropout regulator (which is ordered but not in).

New "green" etchant

Since I was working on a part that is in the critical path for a project with a short deadline, it seemed like a good time to try out a new PCB etching technology I had never used and was completely unfamiliar with (rather than my always-works ammonium persulfate).  But I read about the cupric chloride etchant someplace on the Internet, so it must work 🙂  I don’t know why they never laugh when I say things like that at work.

The board came out just about perfect, and the new etchant is a pretty green.  I don’t know if it has enough Cu++ in it yet to work next time I use it, when the H2O2 initial oxidizer is gone.  That’s for another time.

IR board on Diavolino

I didn’t have much trouble soldering the fine-pitch 28-pin chip (bless you, solder wick!), and the rest of the board was very straight forward.  Since I didn’t have the stackable headers yet, I just soldered the shield to a few pins on the still-headerless Diavolino.  I needed 7 pins and added 2 more for more mechanical strength.  The extras were overkill – it’s mechanically very solid.  An interesting part was the dire warnings

IR Transceiver packaging

on the outside of the package about the IR transceivers being moisture sensitive.  Huh?  But a read of the Jedec standard explained it.  Who knew?  I was very careful to seal the package up as soon as I’d removed the two transceivers I needed.

After I figured out the ATMega on the Diavolino didn’t have a bootloader, I followed a cookbook page and used my Duemilanove as an ISP to burn a bootloader into the Diavolino.  Worked fine.  The picture below still has the connections I used to burn it.

Cheap USB-TTL adapter

The one missing piece was 3.3V for the UART.  Fortunately, I’d bought a couple of $6 USB-TTL adapters (from China via Ebay) which used a CP2102 rather than the standard FTDI chip.  (And cheaper than the $15 FTDI cables!)  But it’s just a serial port, so I figured it would work.  The adapter has a 6-pin 0.1″ male header, with a different pinout than the standard FTDI cables (and the header on non-USB Arduinos, like our Diavolinos).  I made up a dual female cable that got the right signals to the right places.  The bad part was that they didn’t bring DTR out on the cheapie adapter.  I think the Arduino IDE bounces DTR to reset the processor before (and after?) a download, so I have to watch and push the Diavolino’s reset button after the compile is complete each time I download code to it.  (The 2102 does have DTR, and a hack with some very fine soldering to bring it out may be possible.)  But the good news is that the CP2102 has a 3.3V regulator on the chip – and it’s brought out to the header!  So I left one flying lead from the cable carrying 3.3V and soldered it on to where the missing 3.3V regulator on the shield would go.

OK – board is populated, I can talk to the Diavolino, and I have 3.3V.  How do I talk to it?

A Google search for things like “arduino sc16si752” got 2 hits.  One was a woman who was searching, precisely as I was, for an Arduino library for this chip.  (It claims to be register-compatible with the good old 16C450 UART, so that was what we were actually looking for.)  She didn’t find anything.  The other was from Sarah Nordstrom, apparently a founder of StrichLabs, which is about to produce a MultiSerial Arduino shield – using a ‘752!  She pointed me at her github repo with code for her almost-complete board.  I was in luck!

So after studying some I²C tutorials so I could figure out how she would be talking to the chip, and studying the ‘752 datasheet and the 40+ registers it contains (and which ones must contain what magic bit patterns to allow writing to other registers – like the Modem Control Register, which contains the bit to enable the IrDA driver) so I could figure out what registers she was talking to and why, and studying her code so I’d know what objects and methods were available, I started with one of her test programs and hacked it down to a ‘hello world’ to blink the LED on the GPIO pin.

It worked!  (Well, after I fixed a couple of basic syntax errors so her lib would compile (!?), and made public the private register read/write methods so I could talk to registers, along with a couple more stumbles.)  The I²C connection was up, my stolen 3.3V worked, I had the right chip address, and I could write to a couple of registers!

Could I talk to the IrDA transceivers?  I added a one-byte serial write to my LED blink loop and looked at the transceivers with a digital camera.  Blinking!  Can’t tell if it’s intelligible, but the UART is putting something out when told to, and the transceivers are working well enough to light their LEDs!

But the two transceivers behave differently.  When sent two bytes 200ms apart, one channel blinks as expected.  But the other blinks a lot more randomly.  There is one difference in the board:  the transceiver has separate Vcc inputs for the electronics and current for the LED.  The datasheet suggests a series resistor to current-limit the LED, at least for short range and for power saving.  I’d put large pads for those resistors on the foil side of the shield – which faces away from the Diavolino, so it’s accessible.  I didn’t know what values to use, so I put 100Ω on one transceiver and 330 on the other.  The one with the 330Ω blinked more sensibly.  Maybe the 100Ω was too small and the chip was doing some protective thing.  I tried a 470 and the blinks looked a little different, but still not right.  Hmm.

How about receiving?  I added a while loop for received chars on each channel, printing whatever it got.  I tried holding a mirror up to the transceivers, hoping to hear their “L” and “R” data bytes coming back.  Nothing.  But the ‘good blinking’ channel reliably reported receiving an “R” for each blink – even with no mirror.  The IR transceiver data sheet says something about the receiver working in ‘echo mode’ while transmitting.  I wonder if that’s what’s happening.  Also – I think it’s half duplex at best, so a transceiver can’t listen to its own IR.  Maybe I can set up three mirrors and have them hear each other.  Ugh.

Lots of debug prints later… The chip has duplicates of the first 10 or so registers – one for each channel.  Part of the I²C protocol is a sub-address mechanism which the ‘752 uses to specify which channel is being addressed with those duplicate registers.  I tried to read back values from some of the registers, but despite identical channel initializations, got different values for the 2 channels.  Only the second channel I initted (using MultiSerial::begin()) had the right values.  Some digging later, I found that the lib used a UART reset flag to ensure it was starting from a known condition.  That’s fine, but unfortunately there’s only one of those – not one per UART – so the second init would unset whatever we’d set with the first begin().  This is a real bug in the library.  I put a static state flag in so it would only do the reset the first time begin() was called.

After that worked, I told each side to ignore its own transmissions (easy, because one side only transmitted ‘R’, the other only ‘L’) and print anything else it heard.  Then I arranged three mirrors so each transceiver should see the other – and they saw each other’s transmissions!

OK, but that won’t work well in the real world.  I need to flush received data very immediately after sending.  There is a flush() method, and it seems to work.  My main loop is now:

read & print R chan
send R chan
flush R recv
delay 2 ms
read & print L chan
send L chan
flush L chan
delay 2 ms
goto top

I think I’m now confident enough that the board works that I can make another!

I really also need to pack up the hacks and fixes to Sarah’s lib and send them to her.  That’s what open source is all about.