I started this post thinking it would be very technical and only appeal to an engineering type crowd, but it turned out the story here was about something else. Losing sight of what makes you happy, maybe. Or alternatively, how to create a really bad bonus incentive program. -Mad Ned
In 1996, I was working for Digital Equipment Corporation, at the time the second largest computer company in the world. Today most people have never heard of it. I still find it amazing how quickly such a huge company could just vanish, seemingly without a trace. Digital, or DEC as it was popularly known, had a long, storied history in the computer world. It was a leader in the transition towards making smaller, ‘mini’ systems that brought computers out of the controlled machine room, and into the direct hands of people in the office, school, and lab. DEC was a pioneer in that regard, a cool, new, disruptive company for the older mainframe-style competitors like IBM.
Sadly and somewhat ironically, in 1996 DEC itself was in the process of falling victim to disruptive competition. Smaller and quicker companies like Sun Microsystems and the PC manufacturers had eaten deeply into the business by making even smaller and even cheaper computers. Within a year of the start of this story, DEC would be sold off to it’s competitors, and cease to exist as a public company. Much has been written about that tragic fall (see link above), but that is not the cautionary tale I’m planning to tell here. This one is about what happens when money becomes your only motivator, and how the expectation of money can make you do irrational things.
So, 1996 then. I was already a ten-year veteran of the company at this point, and I was working as a hardware systems engineer in the Alpha Workstations group in Maynard, Massachusetts. My job was to design CPU cards and motherboards for Alpha NT workstations, which were a strange hybrid thing built from a proprietary company CPU technology, but which were capable of running Microsoft Windows NT, the ‘business version’ of Windows of the day.
In previous years, DEC had developed its own operating systems (VMS), and also built Unix-based workstations, but at this time the company was trying to fortress our place in the workstation market segment, which was rapidly moving towards cheaper Intel / Windows systems. Our Alpha processor was much faster than anything out there at the time, but did not run Intel-based software directly. Because of this limitation, most programs had to be emulated, which slowed things down to the point of losing much of the advantage we offered.
It was kind of a desperate, intractable situation, and we were struggling to survive. Management was pushing to build new projects faster and cheaper than had ever been done before, attempting to squeeze what used to take years down into months. So they launched a secret incentive program, available to a small group of R&D engineers on key projects, to try to achieve these goals.
In this program, two million dollars were set aside as a potential bonus fund for the participants. The maximum theoretical payout for an engineer was going to be $25,000, and the whole thing was kept secret since many, including our manufacturing and tech support staff working on the project, were not part of it. When I was told I was to be included in this program, I was in the midst of designing the CPU card for the Alpha XL366 NT Workstation. This was to be our low cost workstation that included the latest Digital Alpha processor, running at (what was at the time) a crazy-fast 366Mhz speed — more than double the speed of our competition.
$25,000 was not enough money to retire on for sure, even in 1996. I knew that making that much money would not have made me rich, but it was still a lot of money, and it was far and above any kind of bonus I had ever seen from the company. But the fine print of the incentive plan was, you made UP to $25,000. The way it was to be awarded was, you earned a certain fraction of the $25K total for every percent you shaved off the target manufacturing cost of the system, and you also earned a certain percentage for every day you shaved off the target schedule for the release of the system. So if you could build your project cheap enough and fast enough, everyone on the team could earn up to $25K. Everyone who was invited in, that is.
If you are guessing that this setup was a recipe for disaster, you would be right. Cracks began to form in the plan, almost immediately. My problem was, I was well into the design of my hardware and I had already designed a more expensive 8-layer printed circuit board, and also locked in a fair number of higher-quality but pricey components. This limited what I could do to control costs, but I could still try to get things done faster. By contrast, some of my peers were working on a parallel project which had just started, and they could immediately adjust the design to maximize their cost and schedule savings, as well as their payout.
Our product team grumbled about that, because it seemed unfair that one team could have a better potential to maximize their bonus than another. But as I will explain shortly, the real problems emerged later, once the hardware was designed and built.
The design phase of this project was easily the most stressful work environment I have ever been in. Over a span of two or three months, we designed and built a complete workstation, a development cycle that previously took six months and before that, years. I remember being in shouting matches with our signal integrity team about among other things, how many capacitors we needed. These tiny devices evened out the power of the system, but they wanted far too many to be added to allow the design cheap and easy to build. I eventually won the argument by taking a previous workstation we built and going to the lab, removing about half the capacitors on it with a soldering iron while measuring the power output, to prove that the signal integrity remained good and that our standards had become too conservative.
The Signal Integrity group relented, but they were not happy about it. They said the project would be a failure, and we would have flaky behavior of the system because we cut corners, in their opinion. Meanwhile I was waking up at 4AM on most days, in full-stress-insomnia. Sometimes I would just give up on sleep and go into work at that early hour, and while there, sometimes run into other stressed-out engineers who also could not sleep. I had shaved as much time as I could possibly get off the schedule, and a little cost. A $25K bonus was out of the question at this point, but $10K or more, still in the running.
Finally the day came when our new hardware came off of the assembly line and was ready for testing. A junior engineer named Mike who was assigned to work with me and I drove up to our Salem, New Hampshire manufacturing plant that had just built Alpha XL366 CPU card number 0001. We met our projects head manufacturing engineer as he was taking our card off the line. He plugged it into a test backplane, and connected a keyboard and monitor. A copy of Windows NT was loaded into the system, and it sprang to life. A huge wave of relief washed over me as he booted the OS, and opened the Solitaire game app.
Everything was working perfectly. Luck was with us that day, because our manufacturing friend played that game to completion, and as is the usual when you win at Windows Solitaire, the cards bounced around for the victory screen. On this machine, the speed was so fast that the animation was just a blur, a really good sign we had a performance leader on our hands.
Creating a design that works first time is not an easy thing, and I came back from that trip riding high, and the bonus situation was looking great. But the luck was not to last. A week later, we received ten complete systems to our lab in Maynard for testing, and that’s when things went down hill. Machines started crashing. At random times, for no apparent reason. Not just one or two that could be explained as a manufacturing defect, but all of them.
Out came the oscilloscopes, logic analyzers, component specs, and schematics. Mike and I spent days trying to chase down the phantom problem in the lab, without much assistance from our signal integrity team. As far they were concerned, it was because we didn’t listen to them and did not add in all the capacitors they wanted, and they were very vocal about sharing that opinion. Trouble was though, we didn’t really see any power fluctuations that would have been a telltale for that kind of design flaw. But we also couldn’t find a better explanation for why these systems were failing.
Days turned into weeks, and we watched our theoretical bonus amount tick down, ever smaller. There were whispers in the hallway about how we had blown it with our design, and I was pretty sure that I was going to be removed as project lead if the design had to be restarted. If not something worse.
One gloomy afternoon as Mike and I were sitting in the lab, he asked me to hand him the component specs for the memory chip we used for the CPU card cache. He was going to double-check (or at this point quadruple-check) our pinout and wiring. As he read the spec, he suddenly bolted up out of his seat, and thrust the book at me. “THESE ARE FIVE-VOLT CHIPS!” he shouted. He pointed excitedly to the board. “MANUFACTURING INSTALLED FIVE-VOLT RAMS!” I grabbed the book and looked, and it was true. The issue with there being five volt rams on our board was, the power supplied to these chips was three volts, not five. Our design was made to power a variant of the chip that could run on three volts, but the board apparently had the wrong chips installed on it.
The other interesting thing about this situation was, if you tried to power something that wants five volts with only three, it will work. Kind of. But it will be flaky, and eventually do weird things when stressed. Which was exactly what was happening here. So we checked the parts list of what we sent to manufacturing, and we had correctly specified three-volt components, but somehow they had picked the wrong, five-volt component stock during assembly.
I went rushing to my boss to tell him the news, and he was as relieved as I, because he was also taking considerable heat for the delays. We told Signal Integrity the good news (or for them, maybe bad news) next, and I can say I’ve never felt so vindicated about something as on that day. Manufacturing was third, and we had to send the boards back to them to be reworked with new chips.
Then we waited. Because manufacturing, as discussed, was not on the secret bonus program, did not even know it existed, and had very little incentive to speed up the rework. As far as they were concerned the project was slightly ahead of schedule, so why rush the job and make another mistake? More time ticked by. They actually did make another mistake anyway, and mounted incorrect rams a second time, which I remember being really mad about.
In the end they got it right, but it brought our project only to about a week ahead of original schedule, and maybe 5% under cost. They held a little secret bonus award meeting, and we found that we each would be receiving a bonus total of $2000. Meanwhile, our sister project had managed to minimize their cost target and beat the target schedule of their project, such that each member of that team received the full $25,000.
We were all furious. Furious about the sweat, tears, and most likely blood shed for this project over the past months, only to have what had turned from a bonus into an entitlement in our minds, taken away. Furious about the inequity of it all, about others making so much more money, even though we worked equally hard. I refused to attend the celebration event following, and was even on the phone with a tech recruiter, looking to see what our competitors would offer me if I went to work for them.
Then I stepped back, for a moment. What was happening here? I tried to imagine how an interview would go with a hypothetical new employer:
“THEM: So Mr. Utzig, why are you leaving your current company?”
“ME: They gave me a bonus.”
“THEM: You left because they gave you a bonus?”
“ME: Yeah too small a bonus. It was only $2000.”
“THEM: Only $2000? What were you expecting?”
“THEM: $25,000? Seems like a lot. Is this what you expect you will get from us as a bonus, if we ever give you one?”
I’d worked for ten years for DEC, and although it paid well, it wasn’t the reason I liked the company. I liked the job, I liked the work. The money was never the thing. I found myself surprised that it took a test like this for me to realize it. So in the end, I got over it, didn’t quit, and life went on.
A tall, blonde Irish guy (also named Mike) who was a mischievous sort, conspired with me to get one last bit of revenge on the Signal Integrity group. At yet another recognition meeting for the project, we built a special award we called the “Faraday Award”. (Farads are units used to measure capacitance, named after Michael Faraday.) The award was to commemorate the most capacitors added to a board, and it featured a giant, beer-can-sized capacitor from a 1950s TV mounted on top of an old bowling trophy. We tried to give this to the head of the Signal Integrity group, but he got wind of it and was a no-show. Not much of a sense of humor over there, we guessed.
Was it vindictive? Perhaps. Not sure the current me would have done it, but the 1996 me sure did. It brought a sense of closure. I stayed at the company another year, until it was bought out by Compaq Computer and Intel. Then I finally parted ways with my first job, and my hardware design career, forever.
This is where this story should end, but there is one small bit of Epilogue, that really puts everything in perspective for me. It turns out that management did have some concerns about creating an incentive program that not everyone on the project knew about or was part of. Not enough to include the missing people mind you, and not enough to avoid doing the bonus program in the first place.
What they did do was, reserve a small portion of the incentive fund, about $20,000, that was to be given out to ‘support staff’ on the project. These people were the lab techs, manufacturing people, PCB layout designers and others who worked on the project with us in a supporting role. The managers told the R&D staff to nominate people to get this much smaller bonus, and pick and amount for each. The numbers here were in the realm of $200-$500 a person. Again, these people were unaware of the bigger bonus program, and also unaware that they were about to get a smaller bonus from the pool.
So when we handed out this money, it was a complete surprise to them. They expected nothing further from the project, they had been paid for their time, and the project was a success by all accounts, in terms of targets and schedule. And when they received that small stipend of cash, they were delighted. They had not expected it, and it was as pure a definition of a bonus as could be, not an entitlement, just a pleasant surprise.
A few days after that, I received an email from a layout tech who had done the printed circuit routing work on my CPU card design. She thanked me for having her on the project, and for the bonus money, and said it was one of her best project experiences at DEC. The guilt I felt after that, and shame even for my behavior, well it was real.
I’ve received all sorts of different bonuses and awards from time to time in the years since, some expected, some not. It’s a common thing in the tech industry to do. But I’ve learned never to take a bonus for granted, even if it is a seemingly regular affair. Never to complain or worry about any amount given to me in this way, to always feel lucky I’m in a position to get one in the first place.
We are lucky enough to have a regular bonus program even now at work, it’s been going on for years. Every time it is handed out my boss will say something like, “so we had a good bonus this time, but don’t count on it in the future.”
And I say, “Yeah. I know.”