What was that?

When I was a young lad I had, shall we say, a more "carefree" attitude towards software development. This is the story of how I learned to be a bit more methodical in my approach to certain types of issues.

The project I was working on was the motion control system for a robot. Now, I should probably clarify that, so you don't get the wrong idea - we're not talking about C3PO or R2D2, here. This was an industrial robot, used for 3-d imaging. The robot itself was made out of slabs of cast iron bolted together, was probably 8 feet tall, 4 feet wide, and ten feet long, and weighed in the neighborhood of 4 tons.

The problem I was having was that the motion control was somewhat unresponsive - you'd move the joystick, and the translation table or the optical head would slowly start to move, and when you got to where you wanted to go, it'd keep on moving for a little while before coming to a stop.

As I was looking at the code, I found what I thought was the problem - I had simply put the wrong coefficient in for one of the control equations, so we weren't getting the proper exponential factor applied to the requested motion. A quick edit and recompile, and I was ready to test the new code.

I started the system up, and very slightly moved the joystick. The translation table started to creep forward. I then pushed the stick over a little farther, and the table accellerated. So far so good.

And then, something very bad happened. When I put the stick back to the rest position, the table didn't slow down. In fact, it kept speeding up. I tried pulling back on the stick, but that didn't seem to have any effect. I managed to turn off the power to the motors just before the table hit the hard stops at the end of its travel.

So this 600 pound cast-iron table slams into the rubber bumpers at the back of the machine, going something like 30 feet per second. The whole machine rings like a gong, and all work in the entire shop grinds to a halt as everybody looks over to see this multi-ton machine gently rocking back and forth. I was really worried that I'd wrecked at least part of a very expensive machine, but a later calibration run showed that the mechanical parts of the robot were just fine.

It turns out that there were two problems in the system - the incorrect exponential on the input side that I'd corrected, and an additional incorrect damping factor on the output side. The upshot of all this is that once the system was up to speed, it took a very long time to slow down, but the bug on the input side ensured that it never got up to more than a tiny fraction of the maximum speed.

The next time I needed to make a change in those calculations, I did a "dry run" with the motors disconnected first...