The post has been translated automatically. Original language: English
AI is no longer just a tool in the background — it's now part of real decision chains: reviewing contracts, feeding board decisions, shaping financial calls. The question companies face today isn't whether to use AI in these processes, but who is accountable when the model is part of how a decision got made.
Two situations make this concrete. First: an AI system sits on a board in an advisory role and votes against a deal. The human directors overrule it, and the deal later fails. Second: a lawyer runs a contract through an AI review tool, the tool misses a non-standard clause, the lawyer skims and signs, and the company takes on damages. In both cases, the hard question is the same — where does responsibility sit when a model was consulted but a human made the call?
A few key ideas worth carrying into any team building or deploying AI internally:
- Responsibility doesn't transfer to the model. Whether it's a board's fiduciary duty or a lawyer's professional obligation, using an AI tool doesn't shift accountability away from the human who acted on its output. "The AI reviewed it" isn't a defense.
- Both failure directions carry risk. Trusting the model when you shouldn't, and overriding it when you shouldn't, can both lead to liability. What matters is being able to explain, after the fact, why a call was made.
- The real difference between a human and a model reviewer isn't speed — it's the failure pattern. A junior person misses things they don't recognize. A model can miss something because it looks similar enough to normal patterns it was trained on. That means human review needs to focus on checking the model's confidence and categorization, not just re-doing the same pass.
- Audit trail is the foundation, not an afterthought. Before anything else — before mapping data sources, before vendor contracts — teams need the basic ability to reconstruct what a model was shown, what it returned, and what a human did in response. Without that, nothing else is defensible.
- A short internal checklist helps more than a big framework: Can you retrieve the full input/output trail for your last AI-assisted decision? Do you have a written list of decisions that cannot be made by AI alone? Can your team clearly say what a human is checking that the model isn't? Do you know what your AI vendor's contract says about data retention and liability?
Several formal frameworks already exist to anchor this thinking — the EU AI Act (binding, risk-tiered, and relevant if you sell into the EU), the US NIST AI Risk Management Framework (voluntary but widely used as an operating model), ISO/IEC 42001 (an international AI management-system standard), and SOC 2 extended to AI workflows. None of these solve the problem on their own — together they set a baseline.
For startups and tech teams building AI-driven products, the practical takeaway is simple: don't wait until a regulator or a client asks. Start by logging what your model saw and did, decide early which decisions a model should never make alone, and write down — in plain language — what a human is responsible for catching that the AI isn't.
For a deeper dive into board governance and AI accountability, see:
👉 AI Compliance and Corporate Governance: Who Owns the Call When a Model Votes on the Board?
AI is no longer just a tool in the background — it's now part of real decision chains: reviewing contracts, feeding board decisions, shaping financial calls. The question companies face today isn't whether to use AI in these processes, but who is accountable when the model is part of how a decision got made.
Two situations make this concrete. First: an AI system sits on a board in an advisory role and votes against a deal. The human directors overrule it, and the deal later fails. Second: a lawyer runs a contract through an AI review tool, the tool misses a non-standard clause, the lawyer skims and signs, and the company takes on damages. In both cases, the hard question is the same — where does responsibility sit when a model was consulted but a human made the call?
A few key ideas worth carrying into any team building or deploying AI internally:
- Responsibility doesn't transfer to the model. Whether it's a board's fiduciary duty or a lawyer's professional obligation, using an AI tool doesn't shift accountability away from the human who acted on its output. "The AI reviewed it" isn't a defense.
- Both failure directions carry risk. Trusting the model when you shouldn't, and overriding it when you shouldn't, can both lead to liability. What matters is being able to explain, after the fact, why a call was made.
- The real difference between a human and a model reviewer isn't speed — it's the failure pattern. A junior person misses things they don't recognize. A model can miss something because it looks similar enough to normal patterns it was trained on. That means human review needs to focus on checking the model's confidence and categorization, not just re-doing the same pass.
- Audit trail is the foundation, not an afterthought. Before anything else — before mapping data sources, before vendor contracts — teams need the basic ability to reconstruct what a model was shown, what it returned, and what a human did in response. Without that, nothing else is defensible.
- A short internal checklist helps more than a big framework: Can you retrieve the full input/output trail for your last AI-assisted decision? Do you have a written list of decisions that cannot be made by AI alone? Can your team clearly say what a human is checking that the model isn't? Do you know what your AI vendor's contract says about data retention and liability?
Several formal frameworks already exist to anchor this thinking — the EU AI Act (binding, risk-tiered, and relevant if you sell into the EU), the US NIST AI Risk Management Framework (voluntary but widely used as an operating model), ISO/IEC 42001 (an international AI management-system standard), and SOC 2 extended to AI workflows. None of these solve the problem on their own — together they set a baseline.
For startups and tech teams building AI-driven products, the practical takeaway is simple: don't wait until a regulator or a client asks. Start by logging what your model saw and did, decide early which decisions a model should never make alone, and write down — in plain language — what a human is responsible for catching that the AI isn't.
For a deeper dive into board governance and AI accountability, see:
👉 AI Compliance and Corporate Governance: Who Owns the Call When a Model Votes on the Board?
Share
