Hallucinations, once the domain of gap year ayahuasca adventures, are now running free in our post-ChatGPT lives. Lawyers, caught between constantly evolving technology, expectations to use AI, and strict professional responsibilities, have found themselves at an uncomfortable frontier. Law is a precision profession. AI hallucinates by design. So how do we approach AI in legal practice?
On 16 April 2026, Chief Justice Mortimer issued the Federal Court's new Generative AI Practice Note (GPN-AI), setting out clear guidelines on how the Court expects lawyers to interact with AI, what constitutes unacceptable use, and the baseline expectation that practitioners understand how these tools actually work.
"What's interesting is that the Court isn't saying, 'Don't use AI,'" says Wenee Yap, Director of AI Literacy at 43° Below. "Instead, its said, 'Use it well, use it responsibly, and understand that you are still the professional accountable to the Court. Whether you used AI or not, you are responsible for what you submit."
Wenee Yap is here to unpack what the Practice Note means for your practice. As Director of AI Literacy at 43° Below, Professional Fellow at UTS, and Adjunct Lecturer at the College of Law, she works directly with firms to build genuine AI capability and implement AI safely across legal workflows. In partnership with the College of Law New Zealand and FrontTier, she developed AI Risk and Hallucination Management and Introduction to AI in Legal Practice. From July, Wenee will also deliver the Tech and Innovation modules as part of the College of Law's Legal Practice Management Course.
The Court's Position: AI Is Welcome, But Lawyers Are Accountable
The Practice Note does not prohibit Generative AI. The Court explicitly recognises that tools such as ChatGPT, Claude, Harvey, Google Gemini and Microsoft Copilot have the "potential to facilitate the just resolution of disputes by increasing efficiency in the conduct of litigation, reducing legal costs, enhancing access to justice and the quality of the administration of justice."
"This is promising, and shows a real pragmatism in the Court's view of AI,” Wenee says. “However, what the Practice Note makes clear is that the benefits of AI do not diminish the responsibilities of the lawyers who use it. As a lawyer, you are required to use AI 'appropriately, responsibly, and with due care.' Otherwise, we're putting public confidence in the law at risk."
The Practice Note applies to anyone who appears before the Court, including litigants without legal representation, witnesses, and third parties producing documents under subpoena.
It sets out three core expectations for anyone using Generative AI in connection with Federal Court proceedings.
- Understand what you are using.
The Practice Note is clear: "Any person who uses Generative AI will have a basic understanding of its capabilities, and its limitations and risks."
"You need to think of AI as a tool, like anything else,” explains Wenee. “The Court's expectations sound simple and reasonable, but given the pace at which AI is evolving, understanding how it actually works can feel like quite a task. That is exactly why we have built training around it. Understanding AI's limitations is what allows you to manage its risks, and managing the risks is what allows you to use it without compromising your obligations."
- Do not compromise the administration of justice.
The Practice Note requires that "any use of Generative AI must not adversely affect the administration of justice" and that practitioners must be "guided by, and act in accordance with, their existing legal and professional responsibilities."
"By understanding AI's limitations and risks, including hallucinations, input data, prompts and data security, you'll have a good sense of where AI use is inappropriate, or when transparency is required,” Wenee explains.
- Be prepared to disclose.
If the Court requires it, practitioners must disclose whether and how Generative AI was used in a proceeding. The Court expects practitioners to be able to explain what AI was used, how it was used, and for what purpose. Later in this article, Wenee walks through the Document Checklist System her team uses with every AI-assisted matter, a practical tool for meeting this obligation.
"Before you reach for an AI tool, ask yourself: Do I understand what this tool does? Am I using it in a way that is consistent with my duties to the client and to the Court? And am I prepared to be transparent about it if asked?
Specific Cautions: Why Filed Documents Carry Real Risk
The Practice Note is frank about what AI can do wrong. It warns that Generative AI may produce results that are "not accurate, entirely fictitious or plainly wrong," including:
- "Fictitious cases, citations or quotes, or references to legal sources that do not exist by reason of hallucinations or for any other reason"
- "Incorrect or misleading information on the law or how it might apply"
- "Factual errors"
- "Confirmation that information is accurate if asked, even when it is not"
"This is where I see the greatest risk for practitioners,” Wenee says. “AI can generate a citation that looks completely real. It has the right format, the right court, the right year. But the case does not exist. This is what’s happened in almost every case where courts have sanctioned lawyers for AI misuse.”
The Practice Note makes accountability explicit. Where a lawyer's name appears on a filed document, the Court treats that lawyer as having personally confirmed that all cited authorities exist and support the propositions stated, that evidence is admissible and in the materials before the Court, that facts in pleadings can be proved, that chronologies are accurate, and that document lists conform with the Federal Court Rules 2011 (Cth).
“If you file a document without checking its provenance, your name is on it, and the Court will hold you responsible,” Wenee warns.
How to Meet Your Disclosure Obligations: The Document Checklist System
"This doesn't mean you avoid AI. It means you need a clear system to show you're using it responsibly,” says Wenee. “At 43° Below, we provide lawyers with a Document Checklist System for every AI-assisted matter. You record what AI was used for, when verification was completed, and you build in a cooling period between AI use and sign-off."
For each document, the checklist confirms that citations have been verified in a legal database, quotes have been checked word for word, factual claims have been verified against source documents, and legal principles have been verified against primary sources. For medium and high-risk documents, a second lawyer reviews the work before filing. The checklist concludes with a personal sign-off: I have personally verified that this document meets professional standards.
"This can feel time-intensive, but if you think of the time saved in drafting as balanced by verification, you'll see this more as a shift of where you spend your energy. It is also your professional protection. If you are ever questioned about how a document was prepared, you can demonstrate exactly what you did and when."
Affidavits, Expert Reports and Evidentiary Materials
The Practice Note extends to affidavits, witness statements and expert reports. AI is not prohibited, but its use must be consistent with the requirement that "a person makes an affidavit or witness statement, they are representing that the document reflects their own recollection, knowledge and/or experience." For experts, the obligation is even more significant: an expert report must contain that expert's own opinion and process of reasoning.
Disclosure is required at the start of the document where AI was used to summarise or analyse information a witness relies upon, to create multimedia materials presented to the Court, or in "any other manner that might reasonably be expected to affect the admissibility of that evidence."
"What the Federal Court requires is in line with best practice,” Wenee explains. “In short, evidence must be genuine. AI can assist with organisation or drafting, but the substance, the opinion, the recollection, must come from the person. And if AI touched it in a meaningful way, you must say so, at the top of the document, concisely and clearly."
Confidentiality, Client Information, and Shadow AI
The Practice Note states: "If information is provided to a generally accessible Generative AI tool, it may become available to other people. Users may not know where that information is stored, how it is used, or who will have access to it."
Privileged information, material subject to suppression orders, documents obtained on discovery, and anything otherwise confidential must not be entered into a public AI tool. The Practice Note goes further, warning that even a closed or ringfenced tool may breach the implied undertaking "if outputs from the tool are later used for different purposes."
"What the Court is flagging there is the risk of outputs being taken from one tool and fed into another, perhaps a public one, through shadow AI use,” explains Wenee. “Someone processes information in your firm's secure environment, then pastes the output into ChatGPT on their phone to keep working. That's a breach, even if the first step was compliant."
"Confidentiality is also at risk from shadow AI use,” says Wenee. “Free and consumer-tier tools collect your data by default. Data security protections apply only when you have a paid licence and have explicitly disabled data sharing for training. Check the terms carefully. And ask yourself: is the AI being used here part of our secure IT environment? If it isn't, you have a problem."
The Real Cost of AI Errors, and How to Get It Right
For lawyers misusing AI, the Practice Note is clear about the consequences.
Where AI is used inconsistently with its requirements, "all persons should expect that there could be consequences including adverse costs orders and issues as to compliance with legal and professional obligations." That last phrase is a reference to disciplinary referral.
“The court is simply reinforcing how we’ve seen courts across the world respond to misuse of AI,” says Wenee.
According to the AI Hallucination Tracker, AI hallucinations have occurred in 1,397 cases worldwide. In the US, that has resulted in over $145,000 in fines across 956 cases. In Australia, court hallucinations have been recorded in 73 cases, costing over $20,000 in costs orders, warnings and disciplinary referrals.
The most significant Australian example is Mr Dayal, whose practising certificate was varied as a result of AI-related conduct. He lost the right to practise as a principal lawyer for two years, was stripped of his authorisation to handle trust money, could no longer operate his own practice, was required to practise only as an employee solicitor under supervised legal practice for two years, subject to quarterly reporting to the Victorian Legal Services Board.
"We already have 73 Australian cases involving AI hallucinations,” says Wenee. “Mr Dayal's situation should be a wake-up call for every time-poor lawyer who hopes hallucinations can be explained away by 'I didn't know' or 'I did my best.' That is not a defence the Court will accept."
Now that the Practice Note has been released, it is an opportunity for every practitioner to take stock of how AI is actually being used in their practice.
"Review what your team is actually doing, not just what your policy says. Make sure you're working within a secure IT environment, build your AI literacy, and if your firm doesn't have an AI policy yet, that needs to change today."
Wenee's 4D Framework and Document Checklist System offer a practical starting point for building the habits and systems the Court now expects.
"Lawyers who invest in genuine AI literacy will find this entirely manageable. Learn how the tools work, verify your outputs, and document what you did. The lawyers who do that will use AI confidently and well."
