« December 2011 | Main

Tuesday, July 19, 2011

Timing VB code

In the middle of a large programming project now involving using a Phidgets 8/8/8 board to sample force sensitive resistors (FSR's) in order to quantitate fingertapping speed. At some future date will post a large amount of material on the details of the project but the one thing that struck me yesterday was how damn fast modern CPU's are.

OK, I admit that the bulk of my programming was done on PDP-11's in the late 1970's and through most of the 1980's and the PDP-11 architecture is one that will always be one of my favorites. I managed to get quite high sampling rates out of the PDP-11 and have been trying to find something that's comparable since then. Phidget's interface kits are a low cost means of acquiring analog data albeit limited in performance to 1 KHz sampling rate. For the physiologic data that I'm interested in, this is adequate and, at $80/board they're a lot cheaper than a high performance A/D.

There were a number of problems getting the system running and the full details of my Phidgets sampling problems can be found at the Phidgets forum. Last night I decided to find out how long it took for my event handler code to execute in compiled VB6. The 8/8/8 board is a curious beast that sends 64 byte data packets via a USB link every 8 msec. It can sample up to 4 A/D channels at 1 KHz simultaneously but the speed it likes to sample at is 125 Hz. When a USB packet is recieved, the Phidgets OnSensorChange event handler is called rapidly with the 8 A/D values that arrived in the packet. Then it sits around waiting for the next packet. For event handlers in VB, I usually try to keep my code as minimal as possible as I worry about losing data if the code takes too long to execute. This is the code for my standard event handler:

Private Sub phidgetIFK_OnSensorChange(ByVal Index As Long, ByVal SensorValue As Long)
	Static ierr As Integer
	Static i As Long
	If (SampleFlag <> 0) And (OvfFlag = 0) Then
	  If (DataIndex(Index) <= DataMax) Then  SensorData(Index, DataIndex(Index)) = SensorValue
	  If (StoreTSC > 0) Then i = GetTSCStruct(TSC_Data(Index, DataIndex(Index)))
	  DataIndex(Index) = DataIndex(Index) + 1
	  TotSamples = TotSamples + 1
	  If (FlashLEDs > 0) Then
	    If (DoutEnable(Index) > 0) Then
	      If (SensorValue >= FlashThreshold) Then
	        If (Dout(Index) = 0) Then
	          phidgetIFK.OutputState(Index) = True
	          Dout(Index) = 1
		End If
	        If (Dout(Index) = 1) Then
phidgetIFK.OutputState(Index) = False Dout(Index) = 0 End If End if
End If End If
OvfFlag = OvfFlag + 1 End If
End If End Sub

What I didn't realize was that Thingamablog totally destroys VB indentation when I tried to paste the code into this post. I think I've got most of the indentation back but I'm not going to waste any more time tonight attempting to figure out how to deal with formatted text.

Basically, all this code does is to grab the sample SensorValue from A/D channel specified in Index and store the result in a humungous data array. The value of the timestamp counter (TSC) is obtained via GetTSCStruct which is a tiny but usefull subroutine I wrote 10 years ago. The only change I made for the debug version of the program was to call GetTSCStruct at the beginning and end of the data storage loop. The second GetTSCStruct call stored the data in yet another large array for future analysis. The last part of the code compares the sensor value against a threshold and, if the value exeeds the threshold, the appropriate LED is lit. I figured it would be a good idea to only call the 8/8/8 board when I turned on or turned off LED's and the timing proved the wisdom of this approach.

When I first saw the timing data from this routine I was astounded:

       1,     0,        .000,          .000,        .313
       2,     0,      91.922,        91.922,        .319
       3,     0,      91.868,       183.789,        .325
       4,     0,      92.138,       275.928,        .313
       5,     0,      92.806,       368.734,        .325
       6,     0,      92.379,       461.113,        .319
       7,     0,      92.854,       553.967,        .325
       8,     0,      92.463,       646.430,        .325
       9,     0,    7135.844,      7782.274,        .427
      10,     0,      81.359,      7863.633,        .391
      11,     0,     100.241,      7963.874,        .379
      12,     0,      84.138,      8048.012,        .301
      13,     0,      80.812,      8128.824,        .307
      14,     0,      79.753,      8208.577,        .367
      15,     0,      79.489,      8288.066,        .367
      16,     0,      78.665,      8366.731,        .361
      17,     0,    7357.029,     15723.759,        .319
      18,     0,     223.783,     15947.543,        .427
      19,     0,     106.177,     16053.720,        .319    
The first column is the A/D value sequence#, second is the A/D value, third is the interval from the last time the routine was entered in microseconds, fourth is total time since sampling started in microseconds and the final column, the one we're interested in, is the total execution time of the routine in microseconds. When I first saw the numbers I thought I'd been off by factor of 10 or 100, but after rechecking my calculations I realized that the code takes 0.3 to 0.4 microseconds to execute. This is if the CPU is in low frequency mode (the T2500 Intel CPU that was used for these tests runs at either 998 MHz or 1995 MHz depending on some whim of the OS - it was running at 998 MHz for the data above).
Here I was worrying about fitting all of the stuff I wanted to do into a 1 msec timeslot and I've only used up .4 microseconds of that time (well, actually a 250 microsecond time slot as I sample 4 channels at once). Where there are delays are in calling the Phidgets21 library which communicates with the board:
This next section shows the timing for the event handler to turn on the LED once A/D input value is greater than 100:
    2325,     0,      78.743,   2320153.474,        .403
    2326,     0,     374.021,   2320527.495,        .469
    2327,     0,     275.344,   2320802.839,        .445
    2328,     0,     146.105,   2320948.944,        .427
    2329,   222,    8223.044,   2329171.988,        .343
    2330,   476,      81.865,   2329253.853,      63.086
    2331,   460,     168.295,   2329422.147,        .361
    2332,   371,      81.353,   2329503.501,        .343
    2333,   295,      99.411,   2329602.911,        .331
    2334,   268,      81.125,   2329684.036,        .367
    2335,   298,      79.489,   2329763.525,        .295
    2336,   341,     130.761,   2329894.286,        .343
    2337,   365,    5914.400,   2335808.686,        .349
For those who are observant, the triggering should have occurred on 222 and I have no idea why this happens. Note that it takes 63.086 microseconds to turn on the LED via the libary call.
Maybe it's just me who gets thrilled with stuff like this as I couldn't even do a single memory access on a PDP-11 in .4 microseconds and instruction times started at about 2 microseconds and went up from there. This code was benchmarked on a non-descript laptop (HP TC4400) which is now considered to be obsolete. It works just fine for me and I guess I just hadn't done this type of detailed timing before to find out how long chunks of code take to execute.
The reason I decided to start sampling the TSC was first to ensure that all samples were being captured as I just mistrust the overly elaborate data path in contrast from a QBus data acquisition board which was deterministically sampled to RAM. Secondly, I wanted to see if the low bits of TSC could be used as a source of random numbers. This was the easiest way I could think of capturing enough of them for this purpose. The project to generate the random numbers and run them through various tests of randomness is still in the future, but it helps to have a large supply of data one can use when I do have time for that project.
Posted by Boris Gimbarzevsky at 9:40 PM

Tuesday, July 12, 2011

Russian referral spammers

For the last month or so I've been getting a huge amount of referal spam from what appears to be a Russian run botnet. These morons have latched on to one of my web pages and send Get requests with faked originating links which point back to Russian sex sites and drug sale sites. This is clearly from a botnet as the IP address of the originator keeps constantly changing. The behavior of the botnet program is interesting. The initial link was: /Computers/StateMachineHierarchy.html and I changed that filename to something else. Now, rather than getting just a Get request to the initial link, I get requests for: /StateMachineHierarchy.html which, of course, returns a 404.

That particular page was a minor aside to my 2000 era computers pages and I'll restore it when the botnet looks for other targets. Maybe if I get particularly energetic, and find the time, I'll find the locations of all the botnet machines and write a script to send emails to their ISP's regarding the infected machines on their networks. Why these Russian idiots are engaged in this referal spam is unknown. Looking through my weblogs indicates I've been the target of referal spam in the past and it's most annoying when the target is a large file which wastes my bandwidth. Right now all these idiots get back for their Get request is a 404. I do periodically look at where accesses to my webserver come from and what the most commonly requested pages are. I don't post a page with the most frequent referral sites which is what the referal spammers are hoping I'll do. In any event, I would filter the list to remove requests for non-existent web pages which also gets rid of other types of spam.

What disturbs me about this, and other attacks on my webserver and mailserver is that there are an increasing number of morons with access to computers. I miss the days when the internet was an anarchic but civilized place and it's totally different now that the criminals have moved in. Botnets are only possible if clueless users have no idea that their machines have been infected. The solution is not antiviral software but rather a knowledge of what is normal on ones computer. My personal anti-virus software is primarily wetware based supplemented with ProcessExplorer, WireShark, various file hex editor programs, TCPview, Process Monitor and Rootkit Revealer. I'll be fighting back against these morons although it's going to have to be via attacks on their proxies as they're likely safely isolated controlling their botnet from afar. The other option is to simply refuse all accesses from Russian and Chinese IP addresses although the botnet appears to be worldwide.

Posted by Boris Gimbarzevsky at 9:25 PM
Categories: Rants

Monday, July 04, 2011

Keeping track of electronics projects

One of the things I rarely get enough time for is electronics and, if medicine didn't pay so well, I'd work in the area again (at least until I got fed up with it and did something else). Still, it's something I really love doing. One of the problems I've found with my electronics projects is that I'll work on something for a weekend or two, get it partially working, and then leave it for 6 months. When I return to it I find that I haven't made any notes because everything seemed so obvious at the time and I essentially have to start over again. Then it occurred to me that I've got this great piece of software, Thingamablog, which can be used to store the information in a blog format.

There is a considerable difference in how I document stuff for when I think I'm the only person who's going to be looking at the documentation and when I document it for a project I've been asked to do by someone. In the former case, there might be a few quickly scribbled schematics and terse notes about unexpected things that happened but I still suffer from the delusion that if something is so clear in my head at the time I'm doing it, the clarity will return in months to years time. In the case of a project that I'm doing for someone else, I document obsessively as I hate being pestered with simple questions. This type of documentation I find personally much more usefull in the future when I return to the project. Hence, if I document anything I put on this blog in that manner, then I'll have produced documentation that will be much more likely to prevent me from rediscovering the same quirks about a circuit or program in the future. Also, as I'm not the only person with the same fascination for things electrical, then other people might benefit from my experiences. Hence, putting it out as a blog. I've lost track of the number of times that I've stumbled upon the solution to a problem that was perplexing me on someones personal web page after fruitless searches on the device manufacturers web site.

This blog will be updated intermittently and is basically whenever I get a chance to get away from medicine from a while and indulge in my lifes most long-lasting passion.

Posted by Boris Gimbarzevsky at 9:07 PM
Edited on: Friday, July 08, 2011 1:55 AM
Categories: Pure software