Something every engineer knows – often by discovering this truth at some cost – is that while computer simulations and the like are useful, they are *not* the same as reality. But while every engineer knows this, not every engineer really accepts that. And managers? Oh, they are often quite prepared to overlook this little truth. Example #3632:

Aerodynamic suits kept Olympic speed skaters from winning

The important bit:

The suits were developed in partnership with Lockheed Martin and reportedly tested in wind tunnels on fiberglass dummies. However, none of the team members had worn them, let alone trained in them, before Sochi.

That’s just… stupid. Testing the suits in wind tunnels on fiberglass dummies, is fine, as far as it goes, but these suits are meant to *move.* Any decent engineer would have been able to tell them that a dynamic system like this cannot be proven in a setup like that. Even if the dummies were actually “puppets” capable of a wide range of motion, there’s no way they would have been enough like an actual human. This would have been somewhat akin to wind tunnel testing a helicopter model with the rotors locked off, or testing a fixed, inflexible solid model of a bird and assuming that that tells you everything.

For any engineers-in-training, here’s perhaps the best advice I can impart: anything less than a full-up test of your concept is *not* adequate to prove that it’ll work. Wind tunnel analysis and especially computer simulations *are* useful, but only to a point. If they show that your design has a problem… fix the problem. If they show your design works just just… assume they’re lying bastards out to stab you in the back.  The only test you can trust is reality.

The second best advice I can give: if you have a boss/manager who wants you to go straight to operational based on nothing but simulations, point out the history of these sort of  failures. Don’t go nuts if he/she refuses to really pay attention. But note  this refusal to accept reality. And plan accordingly: this refusal not only endangers the project, but you and your career. Collect the data you need to prove to future investigators that you did try your best to warn that the path your manager set your program on was a flawed one. Make sure that this data is backed up somewhere your boss cannot find and delete it. The drive to ignore reality is sometimes caused by ignorance, sometimes arrogance, sometimes greed. None of these are good, either for the project or for you. If the manager *knows* the process is flawed and goes ahead anyway, it’s a safe bet that malice may well be involved. In which case, when the thing does almost inevitably blow up, he/she may very well have in place a plan to blame YOU. Make sure you can counter that with facts.

Of course, it’s perhaps more likely that your boss is simply a jackass than an outright villain… but even morons will try to protect themselves when they realize they’ve screwed up.

And please, don’t be that jackass yourself. We’re all stocked up, we don’t need any more.