The value of autonomous capability comes from allowing a system to respond to circumstances without requiring a human to specify every action in advance. As autonomy increases, capability does not automatically justify greater authority to act. The consequences of a decision should influence the evidence and confidence needed before responsibility is delegated.
So, how do you build confidence in a system’s decisions when the circumstances it will encounter cannot be predicted?
We asked Brian McDonald, Head of Airworthiness and Certification at SQEP Ltd, for his insight:
“We shouldn’t attempt to prove the system can handle every possible scenario. That is neither practical nor achievable. Autonomous systems have to perceive their environment, interpret what they observe, make decisions and adapt when conditions change. This introduces uncertainty, so rather than proving the system will always behave correctly, we should aim to demonstrate that it can manage uncertainty safely and predictably.”
“A useful way to think about this is through an authority envelope. Just as humans operate within delegated responsibilities and defined limits, autonomous systems should only be granted authority within clearly understood boundaries. These boundaries define what decisions the system may make, what information it requires, where intervention is required and how it should respond when confidence is low or circumstances fall outside its approved operating conditions.”
“Take an autonomous air system operating as part of a search and rescue operation. We need to consider multiple factors that may change or degrade during the rescue: the weather, the terrain being searched, the sensors used, the area to be searched, the definition of a casualty and more. Each could change unexpectedly, and that is where we need to focus our attention: how the system behaves when confronted with the unexpected.”
“The future of autonomous systems will depend on our ability to clearly define the limits of delegated authority, prove they are understood and respected, and demonstrate that when the unexpected occurs, the system remains safe. Confidence in autonomy comes not from eliminating uncertainty, but from managing it.”
At SQEP, our specialists bring together safety, engineering, human factors and operational considerations to understand how changes in one part of the system affect another. That helps establish where assurance is needed and where delegated authority needs to sit.
Greater capability does not automatically justify greater autonomy. Responsibility should be matched by the depth of understanding behind it.
:::
Understanding the autonomous challenge and the autonomous platforms we support.
Follow SQEP on Linkedin for the latest news, company updates and events.