« Phidgets projects | Main | Pure software »

Wednesday, March 13, 2013

New toy - logic analyzer

Kicking myself that I didn't buy this logic analyzer weeks ago. Got bogged down in debugging my Propeller assembly language (PASM) code and my first attempt at making debugging easier was to write yet more code which ran in a separate cog and which flashed LED's when the code would run past a certain point. Fine for slow code, but I needed something which could deal with the Propeller running at full speed.

First thing I bought was a DSO Nano single channel oscilloscope; very nice little toy an small enough that I can slip it into my pocket but not fast enough for debugging 20 MIPS Propeller code. Then it struck me that a logic analyzer would be the thing to get. A bit reticent given the $150 price tag of the Saleae 8 channel 24 MHz logic analyzer, but ordered it from Sparkfun a couple of days ago and picked it up at my office today. Rarely have I been as impressed with a new toy. It's a tiny little thing, the square metal box with all the colored wires coming out of it on the bottom left of the photo.

The Propeller board is connected to a bank of LED's which was my first debugging tool. What amazes me about microprocessor development nowadays is how tiny all the pieces are and I can easily use my already overcrowded desk for development instead of needing a large bench area for my test equipment. 

One of the most frustrating bits of debugging I was doing was trying to get code working to produce two offset 1 msec pulses which will be used to drive the red and IR LED's of the pulse oximeter portion of my ambulatory physiologic monitor (APM). This is a trivial bit of code but, when coding in PASM, often stupid errors are hard to find unless one is thinking like an assembler which, when I start to think that way I can't seem to do medicine because everything is so illogical and fuzzy. 5 minutes after I had the item out of the box, I hooked it up to Propeller pins 0 to 7 while the software was downloaded from the Saleae site. Then, once I captured my first signal, I had the problem solved. A quick rewrite of the code and the satisfying picture below appeared:

The clock pulse displayed is a half-speed system clock which still needs some tweaking as the period should be 2.000 msec, not 2.006 msec. One can clearly see the LED drive waveforms on Channel 6 and Pin7. That's the way they're supposed to look and, because they came out inverted when I first made the changes, found yet another bug in my code.

Below is a magnified view of the clock pulse and a precise measurement of how long it takes the clock cog to go from generating the debug clock pulse on pin0 to get to the PulseOx driver code.


All of the in-between code from the primary clock loop, the RTC and Polar HR monitor pulse detection takes 1.8333 microseconds to execute.

Needless to say I'm thrilled with my new toy and would highly recommend it to anyone who needs a logic analyzer that deals with moderate speed digital circuits. People on the Sparkfun site are grumbling that it doesn't do 50 MHz SPI connection debugging, but I bit bang all of my SPI protocols on the the Propeller and so will be easily able to decode them with this logic analyzer. Conclusion: well worth the money and should have invested in some diagnostic tools earlier than trying to emulate PASM in my wetware -- works for small bits of code but this logic analyzer rocks.

Friday, March 08, 2013

Windoze sucks bigtime

I promise this is my last entry on my Geiger counter (GC) explorations for the indefinite future. The most important point in this post is that windoze sucks bigtime. Now I don't know if the problems I've unearthed belong to the windoze serial driver or MSComm VB6 control, but they raise some very serious issues with using windoze for any form of data acquisition.

M$ doesn't advertise windoze as a real-time OS and the 10 msec quantum used by WinXP certainly relegates it to the non-realtime mode. What I naively assumed was that in a WinXP system with a 10 msec quantum, and most of the threads in idle mode, that I'd be capable of getting 20 msec temporal precision. The windoze scheduling algorithms are fairly efficient and they run through the list of runnable threads in order of priority and, if a thread has nothing to do, it immediately relinquishes its quantum. Thus one would expect a process which has been assigned real-time priority to be able to time events with a precision of 1-2 quanta.

There are near realtime aspects of windoze as when one samples the sound card or video card. If there weren't, then one would get distorted sound or video. It may be that this is a driver issue and that M$ made the assumption that all that one needs to deal with serial data is to allocate a large enough buffer and then who cares about the temporal spacing of serial events. I don't know if this is the case since windoze is a closed source system and the serial driver is a black box. In such a setting I don't know if the driver black box is defective or the OS black box is the culprit. It may be that the driver is fine and the problem lies in the MSComm control black box. That's what happens when one deals with a closed OS -- the source of the problems is not at all obvious and, one can't simply peruse the source code to find and fix bugs like this. That's why I'm switching to Linux.

In the Propeller pulse timing routine that I discussed in a previous blog posting, I had deterministic timing of all GC events as well as measureing the duration of the GC output pulse. The resolution of this time was 2 microseconds; if windoze allowed one the amount of freedom to write interface code that I have on the Propeller chip, that temporal resolution would be 100 nsec or less because my laptop is a 1.6 GHz Pentium processor. As it turns out, my estimate of 20 msec temporal precision for windows was wildly optimistic.

The Propeller program that I wrote grabbed the 2 microsecond clock time and width of the GC pulse and sent it via a serial line to the Teraterm program which logged it to a disk file. As expected, when I plotted a histogram of the GC inter-event intervals at a 50 msec bin width, the scatter around the best fit exponential decreased steadily with sqrt(# of intervals). When one samples the same GC under windoze, however, something totally different happens. The histogram contains regularly spaced peaks which don't decrease as one gets more and more intervals. These peaks are quite periodic and have periods ranging from 200 to 400 msec.

The output of the Sparkfun GC is a single character for each event; either a 0 or a 1 depending on whether the current inter-event interval was less than or greater than the previous interval. I don't know the exact relationship but there is a very short latency between a sufficiently energetic photon hitting the GM tube and the Sparkfun GC outputting a serial byte. This latency can be easily computed by perusing the GC MCU source code which Sparkfun freely supplies. My VB6 program to time GC events sets the MSComm buffer size to 1 which means that an event is generated as soon as a single character is recieved on the serial input. This event is used to grab the time on the windoze msec timer and the event data is then written to a file. The VB6 portion of the code executes in < 1 microsecond.

I don't know how windoze prioritizes drivers, but it seems that the USB serial driver is of very little importance to M$. The only way in which these 200-400 msec spaced peaks can be produced is if the serial driver is ignored for such long periods of time. Presumably the wizards of Redmond assumed that a 4096 byte buffer meant that ignoring the serial driver for 400 msec while data accumulated in the buffer was a perfectly acceptable thing to do. They likely never envisioned the possibility that one would ever think of precisely measuring the exact time at which a particular byte was recieved. So, in this particular application, windoze is completely defective. The peaks in the inter-event histogram suggest bunching of bytes in the buffer which are then released as a group when windoze thinks it's time to deal with obsolete serial connections. I haven't run into anything about the serial driver in Mark Russinovich's very detailed reverse engineering of windoze, but W7 has the same problem as WinXP. Only Win 3.1 would be able to precisely deliver bytes as they arrive to the serial driver.

So, the only conclusion one can derive is that, if one wants to do precise timing, stay as far away from windoze as possible (there's good reason to do so for ideologic reasons as well). Most of my data acquisition applications involve precise timing of events to sub-millisecond precision. My Commodore 64 was more than capable of doing so as was the PDP-11. However, a 3 GHz superscalar processor, when infected with windoze, behaves more like an ancient mechanical calculator than a machine capable of timing events with microsecond precision. Thus, my transition from windoze to Linux is in progress. Linux is not a real time OS as well, but it's possible to launch Linux as a process by a real time kernel which can exploit the full speed of a Pentium processor. Also, Linux is open source so if one doesn't like the way that something works, it can be changed. If one has to work with windoze, then the only option is to do data acquisition with hard real time MCU's such as the Propeller chip which can then send data into the senile windoze OS which will take care of the data packets in a doddering fashion.

Sorry, no nice graphs in this post as all the data from GC2 is being acquired on my 64 bit ASUS laptop which has an 1820x1024 screen resolution and I'm too lazy to copy the graphs over to my HP tablet PC and render them in a smaller size for posting to this blog entry.

While on the subject of Geiger Counters, the new version of the Sparkfun GC doesn't suffer from the excess of short intervals which plagued the first version of their GC. This redesign was in response to a negative user comment on the flakiness of the initial GC high voltage supply. Clearly, if the short interval excess was a power supply problem, it has been fixed on the current Sparkfun GC. And thus endeth the GC topics (unless of course I detect a supernova in my basement).

Wednesday, February 20, 2013

Propeller based geiger counter

A few months ago I got one of the Sparkfun geiger counters and, despite my plans to make this unit into a portable device, it's ended up as my basement radiation sensor. So, there was only one option available to me - buy another geiger counter (GC). This device arrived a couple of weeks ago and it will be a portable device but still haven't figured out a way of protecting the delicate mica window so can also sample alpha radiation during my travels. A project for the future.

GC1, which was the first unit, is connected to my laptop via a USB to serial link. It outputs serial data in the forms of 1's and 0's but the only thing that interests me is the timing of each event. This is done via a VB6 program using the MScomm control which seems to introduce significant timing artifacts -- far more than I would have expected. Thus, GC2 is sampled via a Parallax Propeller chip. Considering that I've entered the Parallax Micromedic contest, I thought this would be good programming practice and, more importantly, practice at putting circuits which spread over my desk into boxes.

After a month of coding for the Propeller chip, I can honestly say that I've never had this much fun programming since I first discovered computers. The Propeller chip is an amazing device that runs at 20 MIPS/cog (although by changing from a 5.00 MHz to a 6.250 MHz crystal one can overclock it to 25 MIPS/cog) and it's a hardware hackers dream machine. I highly recommend this chip for any application that needs to connect to hardware. My preference is for Propeller assembly language (PASM) although there is also the option of Spin as well as PropForth which I'm just playing around with now.

The setup I'm using is shown below:

The GC is powered from the Propeller demo board +5V output (on the left) and the output of the GC (the Out pin) is connected to a Propeller pin via a 18 Kohm resistor. One can go higher as the Propeller only requires low input currents, but the 18 Kohm resistor happened to be in a box of resistors so I grabbed it, it worked, and that's all I wanted.

Propeller code is written in PASM with a bit of Spin glue code. All the Propeller code does is to look for a high-going pulse on one of its inputs and, when it finds one, it grabs the start time of the pulse and starts timing the duration. When the pulse goes low, the clock time and pulse duration are written to hub RAM on the propeller. The Spin code just executes an endless loop looking for new data and, when it finds some, formats it and sends it over a serial connection at 115,200 baud to my laptop. The resolution of the Propeller "clock" is 2 microseconds and so after some 8900 seconds, the 32 bit clock counter overflows. Considering that I'm just interested in the intervals between two GC events, there's no need for me to have an extension to this 500 KHz clock.

A histogram of the distribution of pulse widths is shown below:

So, the pulse is a rather short one and I'm likely getting the largest pulses as an examination of the Sparkfun GC schematic shows that the Out connection is just coming from a single transistor capturing the GM tube voltage. The Out connection really should use a larger resistor, as the out connection is taken at the juncture of Q2's emitter with R17 which is a 1 Mohm resistor to gnd. Likely there are some interesting GM tube phenomena happening is one was to look at some of the lower voltages, but I'd suggest sticking in a FET op-amp voltage follower to get a high enough input impedance and then connecting to the Propeller. The pulse is quite short (~280 microseconds in width) with a narrow distribution although there was a single pulse among the 6560 points which was about 470 microseconds long.

When one looks at the distribution of inter-event intervals, they form a classic exponential curve as one would expect for a stochastic radioactive decay process:

While the number of intervals is still small, it appears that the "ringing" that was visible in the early portion of the decay curve is absent thus proving this is a M$ windoze artifact. Already quite a nice curve fit and the value of C is quite similar to that obtained wtih GC1.

Next step was to compare GC1 with GC2. GC1 has been running for several months now with periodic restarts of the sampling program to keep the file size managable. Here's the last week or mores data from GC1:

A couple of things to notice about this graph:

(1) the periodic spiking throughout. This is what I believe is the windoze artifact. In a stochastic variable, any such periodicities should be eliminated with enough sampling as it's well known that the the limit of this distribution (assuming a stationary radioactive source) as N approaches infinity is a pure exponential curve

(2) Marked excess of short intervals. I commented on this anomaly when I first began my series of blog posts about the Sparkfun GC, then the short intervals went away. However, over the course of the last month or so, the mean cpm have gone from 15 to 16.9. It may be that GC1 is either picking up some of the small oscillations that result in a GM tube when an especially energetic cosmic ray passes through or it may be a more sensitive tube. Of course the only way to sort this out is to buy another GC and have two of them running in parallel monitoring my basement background radiation (does this process ever end?)

Also seen above is that the two exponential fit is marginally better than a single exponential fit (in retrospect, the color of the line for the 2 exponential fit is too close to the original red). The two exponential fit also seems to fit the data better fromabout 4000-10000 msec where the single exponential fit is seen to sag in relation to the data.

When one compares GC1 and GC2, one finds a close correspondence between the two units except in the very short interval region:

Given the small number of samples, GC2 data is obviously quite a bit noisier. The difference GC1-GC2 is shown below:

As has been the case with my previous programs, they'll be posted once they're cleaned up enough to be exposed to public gaze. The Propeller pulse finding program will be posted first to Propeller forums and will try to remember to update this blog entry when I do that. All of my code is open source but I consider it unwise to post broken code.

Sparkfun and Parallax have been great catalysts at getting me heavily involved in electronics again (I'm not sure my patients appreciate this newfound interest). Sparkfun is a very dangerous site to peruse at 03:00 if ones credit card is in easy reach (even worse I've memorized my credit card number given the amount of late night ordering of toys that I do). Sparkfun's electronics are totally open source and I get the feeling that whoever designs the circuits gets something working, has circuit boards etched, and puts it on the site for sale. I'm not denigrating the quality of their material, but rather pointing out that their designs are unfinished. OTOH, this is a tremendous impetus to improve on the design and that particular aspect of their products has gotten me programming with the solder programming language (to quote Steve Ciarcia).

Parallax is another company that has bought into the open-source concept and they produce an incredible microcontroller in the form of the Propeller1 chip (and the Propeller2, when it comes out, will be the bit-bangers ultimate wet dream). I've gotten one of the Propeller1 chips running with a 10 nsec clock speed and am finding that I have to insert NOP's for my "high speed" TTL circuitry to function correctly! This particular hardware interface to the GC took about 15 minutes of soldering and a couple of hours of programming in PASM. Teraterm is used to store the data to a text file which is then processed by a simple VB6 program and plotted out with DPlot.