@danepps @Vermeullarmine This is the first time I've seen text accurately (ish) rendered in DALL-E--kudos!
English
Jon Choi
67 posts
Jon Choi
@JonathanHChoi
Law Professor @WashULaw. Law & AI, Statutory Interpretation, Tax Law.
Katılım Aralık 2019
60 Takip Edilen553 Takipçiler
Ok, I've done originalism and living constitutionalism. What does DALL-E think of common good constitutionalism? Let's find out. (Thread.)
DALL-E, draw me a picture of a law professor who is an adherent of common good constitutionalism.
cc @Vermeullarmine
English
Jon Choi retweetledi
🚨New Paper 🚨 In the first randomized controlled trial of AI assistance’s effect on human legal analysis, we find it leads to large & consistent increases in speed on various legal tasks w/o harming quality. Check out Lawyering in the Age of AI now: papers.ssrn.com/sol3/papers.cf….
English
Incredible news! Also big kudos to @ajurowkleiman who played a big part in the Direct File IRS feasibility report.
Zachary Liscow@ZLiscow
Congrats to folks at @USTreasury like @jacobsgoldin who built the free Direct File tax-filing service -- great to save taxpayers money. cnn.com/2023/10/17/pol…
English
Jon Choi retweetledi
Very happy that my paper w/ @JonathanHChoi, AI Assistance in Legal Analysis: An Empirical Study, papers.ssrn.com/sol3/papers.cf…, is now forthcoming in the Journal of Legal Education!
English
@damienriehl @samuel_grange @NeelGuha @vlex @johnjnay I don't think they're publicly available, the concern is that if they can be scraped for training data they won't be useful as validation anymore. Contact Neel Guha for more info though!
English
@samuel_grange @NeelGuha @vlex That's a fine question — I'd bet that co-authors @johnjnay or @JonathanHChoi might know that answer: Are any of your paper's cases available — as open source?
Not finding them here:
huggingface.co/datasets/nguha…
...or here:
github.com/HazyResearch/l…
English
Are all the legal docs @NeelGuha's LegalBench available open source anywhere? For example, all the cases for the citation_prediction_classification task? @damienriehl @vlex
English
Jon Choi retweetledi
We’re releasing a guide for teachers using ChatGPT in their classroom — including suggested prompts, an explanation of how ChatGPT works and its limitations, the efficacy of AI detectors, and bias. openai.com/blog/teaching-…
English
Lots more in the paper, check it out! 4/4
papers.ssrn.com/sol3/papers.cf…
English
📜New paper!📜
The big question for professors this past year: how useful is GPT-4 for university exams? @Dschwarcz and I ran a study on law school exams, finding that GPT-4 helps with simple multiple-choice but not complex essay questions. 1/4
papers.ssrn.com/sol3/papers.cf…
English
Analysis code is here: jonathanhchoi.com/code-llms-elr. Thanks to Nina Mendelson for providing the human-coded data for this study; the methods article is a companion to a larger doctrinal project we're working on, stay tuned!
English
The article is forthcoming in the Journal of Institutional and Theoretical Economics as part of a terrific symposium on Machine Learning & Law. Thanks to Christoph Engel for organizing and to @jed_stiglitz and @eredmil1 for commenting! 4/4
English
@PaulErikVeel @adamgoldenberg @jed_stiglitz @eredmil1 @LencznerSlaght You betcha! Code available here: jonathanhchoi.com/code-llms-elr
English
Jon Choi retweetledi
It was great to return to New Haven for the Harvard/Stanford/Yale Junior Faculty Forum! Lots of engaging convos with @danepps, @Alan_Trammell, @EtienneT_Esq, @didwanias, @BrennerFissell, @HBWHBWHBW, @JonathanHChoi, @LDHerrine, and many more!
English
New draft paper on arXiv with @johnjnay @sarahlawsky and others! Explores GPT's capabilities at answering tough tax problems; finds that performance improves significantly with few-shot prompting and access to relevant sources.
AK@_akhaliq
Large Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergence paper page: huggingface.co/papers/2306.07… Better understanding of Large Language Models' (LLMs) legal analysis abilities can contribute to improving the efficiency of legal services, governing artificial intelligence, and leveraging LLMs to identify inconsistencies in law. This paper explores LLM capabilities in applying tax law. We choose this area of law because it has a structure that allows us to set up automated validation pipelines across thousands of examples, requires logical reasoning and maths skills, and enables us to test LLM capabilities in a manner relevant to real-world economic lives of citizens and companies. Our experiments demonstrate emerging legal understanding capabilities, with improved performance in each subsequent OpenAI model release. We experiment with retrieving and utilising the relevant legal authority to assess the impact of providing additional legal context to LLMs. Few-shot prompting, presenting examples of question-answer pairs, is also found to significantly enhance the performance of the most advanced model, GPT-4. The findings indicate that LLMs, particularly when combined with prompting enhancements and the correct legal texts, can perform at high levels of accuracy but not yet at expert tax lawyer levels. As LLMs continue to advance, their ability to reason about law autonomously could have significant implications for the legal profession and AI governance.
English
Some news: Excited to say that I'm joining the faculty @USCGouldLaw this fall as a Professor of Law. Looking forward to a dynamic new group of collaborators at USC, and grateful to my colleagues and students @UofMNLawSchool, which has been a wonderful place to start my career.
English