When AI Reads the Law, Part 2. Why the right source can still produce the wrong answer, and what structure, time and rules add.
In Part 1 we looked at how legal AI systems find information. Assume that part now works. The assistant searches thousands of documents, finds the right regulation and quotes the correct article. The answer is still wrong. The article was amended last year, an exception elsewhere in the act applies, or the facts of the case date from before the change.
Finding a rule and applying it are two different tasks. This article explains why, and what that means for anyone building or buying legal AI.
1. Law is a network, not a library
Legislation does not exist in isolation. National acts transpose EU directives. Later acts amend or repeal earlier ones. Secondary legislation is made under powers granted by a parent act. A single provision often depends on a definition, a condition or an exception that sits somewhere else.
On top of this sits a hierarchy. Higher-ranking law prevails over lower-ranking law, and a special rule displaces a general one. When two provisions conflict, their rank decides, not how closely each text matches the question.
Lawyers also interpret. They read a provision by its wording, its place in the system, its history and its purpose. These methods can point in different directions, and legal orders weigh them differently. A search system performs none of these steps on its own.
For IT teams: a legal question is rarely answered by one passage. Treat a provision, its definitions, its exceptions and its amendments as one unit.
2. Similar is not relevant, especially in Legal Tech
Part 1 showed that a single passage that sounds right but lacks the answer can cut accuracy by up to 25 percent, even in general question answering. In law the problem is sharper, because the decisive text often does not sound like the question at all.
Take a question about the notice period in an employment contract. Semantic search will return passages about notice periods. The text that decides the case may be a transitional clause in an amending act, stating that the new rule does not apply to contracts concluded before a certain date. It shares almost no vocabulary with the question.
A recent review of systems that combine retrieval and reasoning makes the same point in general terms: high similarity does not guarantee relevance, and one-shot retrieval struggles when the next search depends on what the previous one found. Legal questions are full of such chains. Find the provision, then its definition, then the act that changed it.
For legal teams: ask vendors how their system handles definitions, exceptions and amending acts, not only how well it finds similar text.
3. Giving legal information structure
A knowledge graph stores facts as simple statements: subject, relation, object. An ontology defines which kinds of entities and relations exist. Applied to law, acts, provisions, courts and judgments become identifiable entities. Citations, amendments and transpositions become typed links between them: this directive is transposed by that national act, this judgment interprets that article.
The EU already provides much of the vocabulary. The European Legislation Identifier (ELI) gives legislation stable web addresses and machine-readable metadata. Its address template can encode the version, language and point in time of a text, and its metadata can state amendments, consolidations and transpositions explicitly. The European Case Law Identifier (ECLI) does the same for court decisions, with a uniform five-part code used by Member State and EU courts.
Two details matter for AI. First, ELI can identify individual articles and sections, which is the level at which lawyers cite. Second, not every legal effect changes the text. ELI’s glossary notes that a provision can be “modified” or “applied” by another without a single word changing. A system that only compares text versions will miss this.
The hard part is building and maintaining the graph. Administrations once tracked which national provision transposes which directive in manually kept tables, which the ELI guidelines describe as cumbersome and not always accurate. Automation is no cure. In one automatically built knowledge graph (of AI research, not law), the final extraction pipeline reached 78.7 percent precision: roughly one in five extracted relations was wrong. The ELI guidelines draw a sensible line. Machines generate structural metadata such as titles, dates and identifiers. Humans handle subject classification, links to other legal texts and quality control.
A knowledge graph can tell you that one act amended another. It cannot tell you what the amended provision means for your case.
4. Time: which law actually applies?
Law changes, and the newest text is not always the applicable one. A contract signed in 2023 may be governed by the version in force in 2023, not the one your database shows today. Transitional provisions can keep a repealed rule alive for facts that occurred before the change.
Three dates are easily confused: when an amendment was published, when it entered into force, and when the facts of the case happened. Consolidated versions, which integrate all amendments into one text, help, but practice differs. According to the ELI guidelines, France consolidates immediately after an amending act is adopted, while Ireland does so case by case and mainly for large acts. A consolidated text found online can therefore lag behind the law. For this reason ELI names consulting legislation as it stood on a given date as a typical use case, and its address template has components for point in time and version.
Many AI architectures ignore this dimension. A large comparison of graph-based RAG methods found that all of them assume a document collection that never changes. For general knowledge that is a simplification. For law it is a serious gap.
For IT teams: store validity dates with every provision and every relationship, and make the date of the facts a required input, not an afterthought.
5. Can law be turned into code?
One response is to translate legal norms into formal rules: if conditions A, B and C are met, consequence D follows. This works well for filing deadlines, fee schedules and numerical thresholds. It works less well elsewhere, because law is full of exceptions. Special rules displace general ones, and later rules displace earlier ones.
So-called defeasible reasoning handles this by allowing a conclusion to be withdrawn when a stronger rule applies. LegalRuleML, an OASIS standard since 2021, makes such rules machine-readable, including obligations, permissions, exceptions, priorities and links back to the legal source.
This suggests a division of labour: the language model interprets text and facts, and a rule engine checks the formal requirements. The review mentioned above recommends exactly this for legal applications. Decisions should pass through deterministic rule engines that filter inadmissible outputs. Generated content should be checked against statutory requirements. And the system should never issue legally binding decisions on its own.
But the bottleneck moves. Someone has to translate the law into rules first, and that translation is already an act of interpretation. A rule engine is only as correct as the reading of the law it encodes.
6. Where judgment stays human
Some legal terms resist formalisation by design: reasonable, proportionate, legitimate interest. Their content is settled only when someone applies them to specific facts. Principles can conflict and must be weighed against each other. Opposing lawyers build competing readings of the same provision. Legal reasoning is therefore often a matter of argument rather than calculation. The open question is less what a provision says than which reading of it should prevail.
Adding reasoning steps to retrieval does not remove this. The same review notes that in legal interpretation, an early reasoning error can carry through later retrieval steps and lead the system to wrong conclusions. It also calls checking whether a chain of reasoning is complete an open problem, and describes agent-based systems as black boxes that are hard to correct while they run.
That defines a realistic role for legal AI: locating provisions, mapping amendments and relationships, flagging exceptions and preparing arguments for a lawyer to review. Where decisions affect people’s rights, someone must be able to explain the outcome, contest it and answer for it. That someone is not the model.
Our conclusion
Retrieval decides whether a system sees the right text. Structure, time and rules decide whether legal AI can use that text correctly. These are separate functions. Search finds passages. A knowledge graph records relationships and validity. A rule engine checks formal requirements. A lawyer weighs what remains open.
No single legal AI technology covers all four. The practical question is which of them a given use case really needs, and the answer depends on the questions your team asks, not on the newest architecture.
What comes next
If finding the law is not enough, how do we know whether a legal AI system reasons better? Part 3 looks at how to measure quality, reliability and traceability in practice.
Every statement in this article is based on published research and official EU documentation. If you would like the source for a specific statement, contact us through wrungel.eu and we will send you the reference.