Is AI Human-in-the-Loop a Safeguard or an Illusion?

Image/Igor Omilaev-Unsplash Alt-txt: A photo of a human hand touching the hand of a robot

We often talk about having a “human-in-the-loop” as if it guarantees safety and fairness in AI systems. But what if that human doesn’t have the right tools, authority, or understanding to intervene effectively?

Imagine a lifeguard at a swimming pool. You trust them to intervene if someone is in danger. But what if that lifeguard couldn’t swim, was short-sighted (like me), and the water was murky from poor maintenance? Would you still feel safe?

This is the reality we face with many AI systems today: opaque, complex, inexplicable, and often deployed with the assumption that a human-in-the-loop (HITL), will catch mistakes. But is that assumption fair or even realistic?

We expect humans to detect bias, hallucinations, model drift, and other AI shortcomings, and yet even developers themselves often don’t know which datasets were used to train their models or why a generative model produced a particular output. This lack of transparency and explainability makes meaningful oversight, human or otherwise, nearly impossible.

Despite this, many organisations are rolling out AI tools and Automated Decision-Making (ADM) systems that influence life-changing outcomes - from recruitment, grading, medical diagnoses to loan approvals and more. Often, with minimal human involvement.

Fairness vs Probability

As statistical inference models, LLMs are trained for probability, not inclusion. They optimise for accuracy, not fairness. While this may work for technical queries, it becomes deeply problematic for nuanced issues such as pension entitlements for women who have taken career breaks to raise children, for example.

By now, many of us have experienced or read about the biases embedded in foundation models that organisations rely on to build their AI tools – whether developed in-house or by third party suppliers. Some organisations assume a solution lies in using proprietary datasets. However, even these may not always solve the problem.

Showing off that you're using proprietary data or an extensive library dating back 50-60 years is not the flex some may think it is, given some of that data predates the time when, in certain countries, women could vote, let alone work in particular professions. That said, at least this data is known, transparent, and (mostly) explainable, and can be adjusted to more accurately reflect society or the organisation as it is today and into the future. Combined with inclusive intent, this could work far better than LLMs which scrap data far and wide.

The Safer Path: Narrow AI?

Although not perfect, one way forward could be to develop Narrow AI tools – domain-specific systems built on top of large language models but fine-tuned with high-quality, relevant data.

These AI tools offer several advantages including:

Technical Safeguards

• Operate within defined boundaries, making behaviour predictable and testable

• Easier to audit, refine, and monitor because they operate within a defined framework

• Designed for specific tasks (HR strategy, recruitment, discovery, translation, recommendations). Limited capabilities means the tool doesn’t make decisions outside its domain, reducing the risk of unintended consequences

• Can and should be tested under real-world conditions (not just in labs), before widespread deployment. ‘Customer zero’, so to speak - organisations using their own AI tools to stress test them, uncover weaknesses or use cases which developers may not have envisaged, and use this intel to improve or identify new opportunities

• Require explainability-by-design

• Include source attribution – the model sharing sources for each answer

Other safeguards: Human Oversight

• The role of the human-in-the-loop (HITL) must be an active, not symbolic one, with real authority to override problematic AI outcomes, otherwise having the HITL is nothing more than lip service

• HITL involvement should span the entire AI lifecycle: pre-deployment, real-time monitoring, post-decision audits, and model retirement

• Include diverse domain experts/teams from the design stage and throughout the AI lifecycle

• Whistleblower protection for those who challenge flawed outputs

• Inclusive leadership that meets people where they are in their AI journey - whether they have crossed the chasm or remain cautious

• AI Literacy training to raise awareness of risks and limitations, as well as to mitigate automation bias (the tendency to over-trust systems). We’ve all heard of drivers who have ended up in rivers after blindly following their SatNavs, despite seeing the risk from their car windows. Or, more recently, a tourist in Venice who fell into a canal after putting too much faith in their app

• Organisational cultures that prioritise fairness and safety over KPIs alone

• Humans should not become scapegoats for systemic failures

• Build for failure – there must always be plan B (or C)

• Rotate oversight roles to mitigate complacency and to monitor performance drift

• Seek feedback from the communities most impacted by AI

Data Integrity - Who is missing from your Data?

• Data must reflect real-world diversity. An organisation with customers worldwide that relies on an AI tool trained primarily on data from the US Midwest, for example, will fail under real-world pressure and potentially expose the organisation to reputational, legal and other risks

• Track the dataset development journey - from problem formulation stage onwards [I will be sharing an example of AI Data Development stages in an upcoming HR-AI newsletter]

• Enable intervention at every stage of AI development where bias may creep in

• Clean or challenge legacy data that reflects outdated norms

Final Thought

It should never be assumed that man and machine working together automatically deliver the best of both worlds. If the human isn’t trained to spot bias, and the machine is trained on biased data, the partnership is flawed from the start.

AI isn’t just about the end product; the development journey shapes the fairness and accuracy of the final AI system.

Responsible AI requires thoughtful design, inclusive leadership, and robust oversight - from Design to Model Retirement. Human intervention alone cannot fix fundamental flaws in systems built on exclusion.

Let’s build better. Let’s lead responsibly. Let’s see more CHROs involved in the development of AI tools.

   *****************************************************

If you’d like to learn more about embedding AI Literacy and Responsible AI in your organisation, then please free to contact me on LinkedIn or, visit ExecutiveGlobalCoaching.com to learn more about how we work

Subscribe below to receive an alert as soon as I publish new editions of my LinkedIn newsletters or, to read previous editions:

1. Responsible AI - (this one) Putting People and Culture at the heart of AI Strategy.

2. Leading with Emotional Intelligence (EQ)

3. Inclusive Leadership in the era of AI