Lawyers who cite fake caselaw created by ChatGPT are often mocked, but generative AI technology is unfairly blamed. The fault lies with lazy lawyers who copy and paste dubious citations without adequately reading the underlying cases. At the ILTA Evolve show, a panel discussed some of the security challenges facing lawyers who use AI. While these risks may not be entertaining for outsiders, they can be severe, as prompt injection can cause the AI tool to overlook important material or bypass valuable guardrails. AI can also compound user mistakes, and developers must constantly adjust and improve the technology, which can sometimes undermine faith in the process or even trigger catastrophic forgetting. This poses a significant challenge for anyone seeking legal advice using AI.
Source: Above the Law
Leave a Reply