The following simplistic and mythical example perhaps better makes my point.
Suppose there had been a period of "random" errors in tightening fasteners
in a major component (say an engine)--sometimes the torque setting is a
little high & sometimes a little low. Maybe it causes economic losses but
there's redundancy in the fasteners so no major accident. So it is decided
to automate the torquing the fasteners by designing a "smart" wrench. This
wrench senses a chip in the fastener head that tells the wrench what the
setting should be. This ensures each fastener is torqued just
right--perfect! Except one day, a batch of the wrong chips are used in the
manufacturing process....... Now every fastener is torqued wrongly--no
redundancy!! Guess what would happen?
In my experience, designers of automation systems rarely thing of what new
failure modes the new system adds, only the old ones that are being
eliminated. (I speak as an ex-designer of control systems, though not for
aviation.)
John Wreathall
The WreathWood Group.