AI’s existential threat poses new challenges for law and justice

This year, the Court turned that same apprehension to AI. In July, it set aside orders of the company law tribunals in the Essel insolvency proceedings because they rested on precedents that were imagined by AI.

Every legal remedy to a wrong assumes that there is a world that survives the wrong. Advanced AI is the first technology since the atom bomb to seriously test that assumption, because of the risk that it poses. “Existential risk” is not a science fiction concept but a precise idea. As captured by philosopher Nick Bostrom, it describes harm that cannot be undone: not merely extinction, but a collapse from which civilisation never recovers. Little wonder, then, that the UN Secretary-General now speaks of uncontrolled AI in the same breath as nuclear weapons.

The road there need not run through killer robots. It has three very plausible directions. The first is capability outrunning control. AI will become capable of acting in ways we did not foresee and cannot control. The second is dependence. With every year that AI absorbs more of our banking, power grids, hospitals and corporate management public administration, the cost of withdrawing it rises. The third is cascading failure. In July 2024, a single faulty update from CrowdStrike brought down approximately 8.5 million computers across airlines, hospitals and banks in the space of a morning.

Not built for this world

You Might Be Interested In

Our legal tools were forged for a world of products: objects with a settled design, an identifiable maker and a declared purpose. Frontier AI is none of these, and Indian courts are already feeling the strain of fitting this technology into old categories. In IndiaMart v. OpenAI, the Calcutta High Court observed that AI companies like OpenAI, offering ChatGPT and LLM models, are not intermediaries, and neither are they search engines; they are originators of information. The court further went on to state that their exact function was a question of law that required expert assessment. This highlighted the difficulty of fitting AI into existing statutes that were never written with such sophisticated systems in mind.