Interfacing to my Neptune water meter

My water meter (with a Neptune T-10 encoder) is connected to an external RF transmitter (a Neptune R900) to allow drive-by meter readings.  Great!  I’d like that data for my home monitoring system.  (The bottom line is here.)

Can I listen in?

There’s a cool 900 MHz utility meter monitor tool mentioned in this Hackaday post.  Unfortunately, while it can receive the transmissions from an R900, it can’t decode the data (yet).  I sent a note to Greg at gridinsight.com and asked if I could help with development of the R900 code, but what’s left would involve crypto stuff I know nothing about.  Nice try.  I guess I’ll have to try to eavesdrop on the data going from the meter to the transmitter.

The 3-wire connection between the two seems to consist of common, data (from meter to transmitter), and clock/power (from transmitter to meter).  The transmitter apparently polls the meter periodically by sending a string of clock pulses to it.  The meter responds by putting (usage) data on the data line, clearly clocked out by the clock pulses from the transmitter.  I can see both the clocks and the data; now I just need to decode the data so my home monitoring system can use it.

The encoder

The water meter seems to be cleverly designed with a separate “encoder” unit magnetically coupled to whatever it is that actually moves with the water flow inside the meter body.  That encoder, which can be replaced on the fly, contains the clear plastic head with 6-digit odometer style counter, a big hand with a calibrated dial for the least significant reading, and an uncalibrated double ended red “spinner” that shows flows too small to be seen easily on the big hand.  The 3-wire interface is part of that encoder.  While it might be possible to harvest a little energy from the water flow, I think the encoder has no internal power.

Apparently (my inference) the device at the other end of the cable (here, the R900 battery-powered transmitter) polls the encoder by sending a stream of clock pulses to it.  The encoder extracts enough power from the 5V logic level clock stream to allow it to reply on the data line with the current values of the odometer counters.  An interesting feature of this mechanism is that the polling device always gets the complete total usage reading – not a delta.  Thus the reading is essentially guaranteed to be accurate.

A little more detail

From the product information on the R900, while it sends a 900 MHz spread-spectrum RF burst with the “latest” usage data every 14 seconds, it only polls the meter once an hour.  (Thus the quotes.)  While I’d like to just be a non-invasive listener, I’d really like more granular data.  I suspect, but haven’t tried yet, that if I inject a suitable series of clock pulses at times of my choosing (maybe every 10 or 15 minutes?) the encoder will respond to me just like it does to the R900.

Of course it would be very inappropriate to impose my clock pulses on to the R900’s clock driver output.  One possibility would be to put a (probably Schottky) diode in series with both the R900’s clock output and mine, hoping the 0.3V lower voltage level on the clock line would still allow the encoder to respond reliably.

And then there’s the interesting question of how the unsolicited data transitions caused by my clock pulses might affect the R900.  I suppose if I wanted to be very polite about that I could interrupt the data line to the R900 with a relay so it would only hear the data it expected.  But that’s a way out yet.

Decoding the data

Back when I hoped to be really non-invasive (never disconnecting the original wires), I made up something with spring contacts to connect to the screw heads of the 3-wire interface.  That gave me a nice wire coming out with all 3 connections I could play with.

I connected the wire to my USBee SX logic analyzer (the first time I’ve used it!) and set it for 1 M samples at 1 μs/sample (lowest number and lowest rate offered).  I set a trigger for lo-hi edge on one of the clock/data lines (I didn’t know which) and left it for a while.  It triggered, and I looked at the data, but it didn’t make any sense at all.  After some head-scratching, it looked like the wire I’d used for common (black – duh) was in fact one of the clock or data lines.  Trying again with green as common, I got (what I think to be) a good capture.  A nice stream of clock pulses on one line (black) and some kind of data on the other (red).  Maybe 0.26 sec of data out of the one second capture, but I can ignore the silent times.  I’m home free!

Well, no.  While the data transitions always occur right on a clock edge, some are on a rising clock edge, some on falling.

Looking at a larger scale and starting right after the very first rising clock edge, the same pattern appears on the data line 6 times in a row.  (Picture on right is just the first instance of that pattern.)  Counting a clock pulse as one hi time and one low time, that pattern takes just 16 clock pulses.  That’s a nice number.  Unfortunately, looking past that initial (sync?) pattern, there are “blips” (I don’t know any appropriate word) of data – between a rising and a falling edge, but not necessarily in that order – that are only 1/2 clock pulse wide.  (Is that really a bit time?)  There are both low blips (falling edge followed by rising edge) and high blips (rising -> falling).  POSSIBLE CLUE: (it looks like) all high blips occur during a high clock half-cycle and all lo blips occur during a lo clock half-cycle.  Here’s more:

Hmm – all the one-half-clock-pulse wide blips I could find start on a clock falling edge and end on a rising clock edge.  Also, all the longer N-and-a-half clock width blips are adjacent to a half-clock blip.  I don’t understand the implications of all that – it’s just some random observations.

I could really use some help figuring out the protocol – at least step zero to read bytes from it.  If anybody wants to play, the USBee binary capture file is here: water2ulb.txt.  You’ll need to rename the file to water2.ulb.  The USBee SX logic analyzer code – which will run in demo mode without the USBee module plugged in – can open that .ulb file so you can root around in it.  You can freely download the (Windows 32 bit) logic analyzer code here: http://www.usbee.com/sxdemo.zip (more versions here: http://www.usbee.com/download.htm).  Fifteen second GUI lesson: left/right mouse clicks zoom in/out; click & drag to pan; left/right clicking in “Cursors” bar sets “X”/”O” marks; click something in “Cursor Insta-measure” and mouse over the data for some tools.

Update 9/4/12: After staring at it for a long time I noticed a pattern:  There’s always a 1/2 clock cycle lo “blip” on the low half of every 16th clock cycle, always followed by a high blip of either 1/2 or 3 1/2 clock cycles.  And those are the ONLY instances of 1/2 cycle nonsense.  Otherwise all data line transitions are reassuringly on hi->lo clock edges.  Those statements apply to both sections of the communication capture:  The first ~500 clock cycles of 50% duty cycle and 0.3ms width, and the last ~250 clock cycles of 33% duty cycle and 0.42ms width.  (What’s that about??)

So the half-cycle stuff is clearly some kind of frame delimiter.  But with 15 clock cycles of usable data time per frame, the bit encoding is still unclear.  More head scratching ahead.

To avoid conflicts with my someday higher frequency polls and the polls from the R900, I needed to know how often the R900 actually polls.  I set up an Arduino to look for lo-hi transitions on the clock line and timestamp them.  (It sleeps 1000 ms after it logs a transition to ignore the ~0.25 seconds of actual data.)  The R900 is quite uniform in polling about every 3030 seconds.

Even more interesting is that the polls are sometimes repeated after a second or few, up to 6 times in the day or two I’ve been logging.  Most polls are not repeated.  A first guess is that a repeat poll is sent when some kind of checksum fails.

Update 9/6/12: It occurred to me that my simple assumption that the reader provides a series of clock pulses and the encoder drove the data line might not be completely right.  (Especially in light of the change in clock duty cycle after the first 500 or so clock pulses.)  To see who was really driving the bus (?), I thought I’d isolate the two sides with Schottky diodes oriented so the R900 power/clock could get thru and the data line high states from the encoder could get thru.  (Two diodes pointing in opposite directions.)  I’ll spend 2 more channels of the USBee SX and monitor on both sides of both diodes.  I’m hoping the logic thresholds will let me see if there is any difference on the two sides of the diodes, indicating a more interesting story than I originally assumed.

The only Schottkys I had were SMT, so I sliced two insulating slots in a bit of scrap copper clad, soldered the diodes on, and set it up so it would be easy to insert inline in the black (clock/power) and red (data) lines.  (Green is common and will remain connected.)

I didn’t want to interrupt an R900 poll (though I don’t think that would cause any real trouble), so I needed to know when to expect the next poll.  My Arduino sketch logging time between polls was still running, but it didn’t tell me how long ’til the next poll.  I added a Serial.available() check so any keypress will print time since last event (and, assuming 3030 secs between, estimated time ’til next).  Now I know how much time I have to remove my spring contacts, rewire the cable to the R900, reposition my spring contacts, and set up the USBee SX.

Grumpf.  The PC software had somehow lost contact with the USBee module and wouldn’t run and capture.  I had to shut it down, unplug module, replug module, restart software, and set up for the triggered capture again.  In doing that, I missed the poll event.  Grumpf.

And then several more misses.  Can’t find the USBee help files – bad link on their web site!

OK – got a capture.  (USBee setup: set CLK input so  you can change Capture on TRIG to hi; set CLK back to output; set 1st 2 cols on clock sig to lo, hi; click Acquire.)  Clock is identical on both sides of the diode.  Good – no funny business from the encoder (like providing the clock!).  So I have to believe if I injected a clock pulse stream, I’d elicit data.  But the data line is not identical.  Data is clearly driven on the encoder side.  Unfortunately the signal on the R900 side goes high and stays high.  Very high input impedance?  Anyway, I don’t think the R900 gets the data.  Its clock stream extends way longer than normal.  Jumpered around the data line diode.  New capture much better, but not quite like first one.  That capture is water4.ulb.

The Arduino monitoring polls also saw re-polls until I jumpered the data line diode, then R900 got data first poll.  The bad news is that now I still can’t tell if the R900 ever writes on the data line.

Encoding details

Update 9/9/12:  OK – I think I’ve won.  The apparent re-polls from the R900 when the (weak-drive) data line was connected directly to an Arduino input caused me to add a simple buffer to reduce the loading on the encoder’s data line.  That seemed to help.  After I fixed dumb bugs in my code, the Arduino started getting data. Then I replaced the function that waited for a clock edge with code that toggled a bit so the Arduino provided the clocks.  Once I could generate my own clocks, I made a lot faster progress than back when I had to wait 50 minutes for the R900’s next poll!

And that let me figure it out.  Here’s how the data is encoded:

The (brown) clock is regular, with a period of ~300 μsec.  The basic bit time is from clock falling edge to the next falling edge.  Five bits of data (red line) are shown – pretty obvious.

There’s a frame delimiter (Frm) consisting of a half-cycle of low (starting on clock falling edge) followed by a half cycle of high (starting on clock rising edge).  This frame mark (from the register encoder) occurs regularly every 16th full clock pulse.  That gives us 15 bits of data per frame.  When data bits are being clocked out, a data line transition is never allowed on a rising edge, making the frame mark completely unique.

Looking at the “sync” frame, here’s how the data is decoded.  After the frame mark, we take blocks of 4 bit times and express them in binary.  With LSB first, the bits in each nibble are reversed, and all the nibbles are reversed.  There are only 15 data bits, so the most significant bit (X) is missing; we assume it’s a zero.

Very interestingly, the only nibble values that I’ve ever seen are 4 and 7.  Now, for reasons I don’t understand, we assign a nibble value of 4 to be logical “1” and nibble value of 7 to be logical “0”.  The sync frame above then has the logical value 1110 – or 0xE.

The actual usage data is expressed in BCD, one digit per frame.  That 0xE sync value is reassuringly not a valid BCD value.  Along with and bounded by a LOT of sync frames, the most recent capture has four consecutive frames with values of 7477, 4774, 4774, 4777.  Converting to logical and expressing that BCD as decimal, that’s 4998.  This meter shot, taken a week or two ago shows a close, but lower value in the high 4 dials.  Right after I took that capture, the meter read 4998.  (Update 9/11/12:  The meter just rolled over to 5000 (in the hi 4 digits) and a poll from the Arduino produced the same value.  I declare the decoding described here to be correct.)

The bad news is that it provides terrible resolution for my needs (like verifying that the watering system is working), although perfectly adequate for the city’s billing purposes.

One shred of hope was in the two phases of the R900’s poll, with different duty cycles.  Might that second, 33% duty cycle poll elicit data from those lower two wheels and maybe even the big red hand?  I found an early R900-clocked capture and manually decoded the data.  Nope.  Same patterns and same data as the 50% duty cycle poll.  Bummer.

A few more observations about the whole data sequence.  Starting from the clock line going from 0V to 5V (rising edge), we have the second half of a frame marker, followed by 6 frames of the 0xE sync pattern.  Then we have 4 frames containing the (most significant) meter usage wheel data, 5 more frames of 0xE, then one frame of 0xF.  Then the whole pattern repeats.  The R900 poll sequence provided 300 μsec 50% duty cycle clocks for: 6 0xE, 4 data, 5 0xE, 1 0xF,  6 0xE, 4 data, 5 0xE; then 424 μsec 33% duty cycle (1 lo, 2 hi) clocks for 1 0xF, 6 0xE, 4 data, 5 0xE.

I tried 100 frames’ worth of clocks and the pattern of 6 sync, 4 data, 5 sync, 1 0xF just repeated and repeated.  And no checksums – My earlier guess about a checksum fail was wrong.  I guess the data integrity check is just to watch the data for more than one cycle and make sure it’s the same.

The bottom line is that while this was an interesting reverse engineering project, the results are not useful at all to me.  I’ll just go back to my optical telescope.  <sigh>

Final Arduino code to generate clocks and read data is here.

Earlier attempts

I’ve wanted water usage data for a long time.  I had a temperature sensor on the incoming water line that could very crudely sense when water was flowing (with better luck in winter when incoming temp was farther from basement temp).  I considered taping a mic to the pipe and looking for noise.  But the water meter was the obvious source of the data.  While I seem to have gotten over it, at the time I didn’t want to hook up to it electrically.  But maybe I could get the data optically.

I built an optical telescope looking at the spinner, and even got it to work and calibrated it in gallons used.  It ran for a while, failed for a while, and is now back online.  Details are here.

This entry was posted in Water meter reader and tagged , , , , , , . Bookmark the permalink.

19 Responses to Interfacing to my Neptune water meter

  1. Jim says:

    I have considered a web cam pointed at the meter, and some sort of OCR software.. Sounds like a lot of trouble though.

  2. Pingback: Optical interface to water meter | Jim's Projects

  3. Eric says:

    Very cool; good job! I wish I’d had them put an outside transmitter on when they retrofitted mine (mine has the transmitter indoors – I’m not all that edgy about RF, but having it outside would be nice, with the added benefit of having wires to tap into!)

    • Jim says:

      Is your meter and transmitter one monolithic blob or is the transmitter wired on, but just located indoors?

      The place the 3 interface wires come out of mine is covered with a curved, fitted black plastic cap. There’s a wire with a seal to keep users from removing/altering the register/encoder part (or at least make it obvious if they have). The cap covering the interface wires is threaded thru that security wire, but I was able to finesse the cap off without breaking the seal or cutting the wire. If your transmitter is wired, might that help?

      • Eric says:

        It’s monolithic sadly. Like the one in this image.

      • Eric says:

        What’s really frustrating is that this thing has a lot of smarts; I just can’t get to it:

        The E-Coder)R900i stores consumption in hourly intervals for a rolling total of 96 days. This is equal to 2,304 hourly intervals of consumption. A special data logging activation flashlight is required for this process. This flashlight is used to activate the E-Coder)R900i for data transmission for collection by the CE handheld unit (HHU). The flashlight must be held within 2-3 inches from the photo sensor (recessed hole on dial plate) for 3-4 seconds or until the LCD changes to read “RF Log”. After activation and the data is being received by the HHU, it takes approximately five minutes to download all 96 days worth of data
        logging information.

  4. Peter says:

    http://www.edcheung.com/automa/water.htm

    Just thought you might be interested in the above website (not mine).
    It describes a method using a hall effect sensor to bypass the meter mechanism and simply measure the rotating magnetic fields from inside the Neptune meter.
    This provides a much higher resolution than the default interface.
    Plus it has the benefit of not interfering with, or even connecting to, the wireless transmitter.

    • Jim says:

      Hi Peter,

      Nice job by Edward. Thanks for the link!

      I was aware that there was magnetic coupling in the meter; sensing that was in fact the first thing I tried. Unfortunately, there’s about 3 amps of AC going thru my water meter (!?), and the 60 cycle magnetic field from that so thoroughly drowns out what I could pick up from the moving magnet in the meter that I gave up. I made some serious attempts to null it out, but failed. Filtering wouldn’t work either because at high flow the fluctuations from the magnet could be around 60 Hz. (There was a little comment about it in this post.)

      It was an interesting moment when I first put an Amprobe around that pipe! Of course I tried to figure out why that current was there, but failed at that, too. I did get far enough to be fairly convinced it wasn’t a wiring problem in my house – but rather a problem in a neighboring house sharing the same power transformer. I suppose I should revisit that some day.

      • Anony says:

        IF you have 3 amps of AC (or even DC) going through an underground pipe, methinks you ARE doomed to that pipe failing through electrolysis.

        • Jim says:

          Unfortunately you’re probably right. It worried me a lot when I first found it, and I did about as much troubleshooting as I could from here and couldn’t find any reason for it. I think a possibility that came up was a fault in the ground/neutral connection in the panel of a neighbor who shares a transformer, but I wasn’t willing to try to extend my troubleshooting to his house. :(

  5. Sam Teague says:

    Jim,
    Your hack may be incomplete. From my understanding, the account number (meter serial number(?)) should also be included in the read. This to tie the reading to a specific meter. I wonder if there is a command sequence from the uP that is necessary to fully implement the protocol? Incidentally, I understand the Sensus protocol is also known as UI 1203. This is some form of (unpublished) standard.

    • Jim says:

      Interesting comments, Sam – thanks! In all the captures I’ve looked at, I’ve never seen anything but data polls. Of course that certainly doesn’t mean that serial number polls don’t happen. I guess it doesn’t argue that there couldn’t be a serial number poll command/mechanism, but the fact that there’s no command for the data poll – just start sending clocks – raises the thought that the protocol looks too dumb to have any other commands.

      Of course maybe the 5 or 6 bytes of 0xE I’ve attributed to being sync bytes are really the serial number, and my meter happens to have an unusual serial number that happens to encode to 5 or 6 bytes of 0xE. Seems kind of unlikely, but without more information (protocol details or observations from other meters) it’s hard to know.

      If there is such a serial number poll command/mechanism, it’s quite plausible that when the R900 first starts up – maybe after getting a new battery? – it polls the meter encoder for a serial number and then just remembers it. No need to ask again. But I guess I’m not quite bold (or curious) enough to open the R900 and pull the battery (or look for a reset switch) while logging the data from the encoder. :)

      Also plausible would be that the commercial account mapping is between the account and the R900. That device has a 10-digit serial number, and could very easily include it in every transmission. And I saw reference in some Neptune doc to a 10-digit number in every transmission from an MIU (Meter Interface Unit). Maybe the same number?

      My water bill shows an 8 digit number as “meter number” which matches a number in the cast bronze housing of the meter. I doubt that there’s any way for the “encoder” part to be able to know that number (unless it was hard coded in by switches/jumpers/links when the encoder was installed on the meter housing).

      I ran across mention of the 1203 protocol in my research, too. While I thought finding the details of that protocol might help a lot – and I tried to find it (but failed) – I seem to recall only reading that is was very similar, but not necessarily identical to the Neptune protocol. If anybody reading this happens to have a copy of that protocol, I’d love to see it!

      Jim

  6. Sam says:

    Jim,
    You’re probably right that Neptune protocol not identical to Sensus. As I understand it, the meter number must be hard coded into the meter electronics. Otherwise, possible to move interface electronics between meters possibly resulting in chaos. I too have had no success in acquiring UI 1203 protocol. Will keep trying, though!

  7. PEC-Memphis says:

    Jim,

    If monitoring and/or recording water flow is your end-goal, you can do anything you want on your (the customer) side of the meter at most water utilities. You have indicated that you have a basement with the incoming water line, this would be an easy access point to the water line. There are a variety of flowmeter devices and sensors here: http://www.omega.com/prodinfo/flowmeters.html. You could then get resolution suitable for your purposes.

    • Jim says:

      Interesting comment. I’m very quick to mark commercial comments as spam to help the filters identify inappropriate comments. But yours was very much on topic and relevant, and your web site had some interesting info on flowmeters, so I’m happy to keep the comment.

      Thanks!

  8. Bob says:

    Hi Jim,
    I have implemented data capture with an ethernet Arduino for both the ProRead and the E-Coder versions of Neptune water meters. Your information was invaluable for the ProRead.
    I would be happy to send you the design data and setup if you would host the files.

    Bob

  9. Steve G. says:

    Jim: I have a Neptune T10 with a R900 V2 transmitter. The steplight group (website above) out of Australia has a monitor that would fit my needs yet it is not compatible with the Neptune. They told me I would need to put another water meter in series with the T10. There are a variety of e-monitors on the Chinese merchant websites yet the Aquamonitor has the best functionality for me. I am not a tech savvy as most of your followers. Has your research discovered any other devices similar to the AMRUSB-1 or can match the Aquamonitor ?

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>