Google AI tool boosts senior lawyers but stalls juniors
A Google Research study found that while AI tools boost immediate drafting quality, they fail to help junior lawyers build the long-term professional judgment needed to work unassisted.
Researchers from Google Research and the National Bureau of Economic Research conducted a three-month randomized controlled trial involving 133 patent attorneys across 11 intellectual property firms. The study evaluated an unreleased Google Labs AI patent writing assistant, which has since been integrated into Gemini Notebook. Two-thirds of the participants received early access to the AI tool, while the remaining third served as a control group, receiving only basic AI training without hands-on access during the trial period.
The researchers evaluated the lawyers on patent drafting tasks at 10 days and 90 days. At the 10-day mark, access to the AI tool raised drafting scores by 0.34 standard deviations, representing a 10-point jump in percentile ranking. By 90 days, this improvement rose to 0.38 standard deviations, or an 11-percentile point increase. Junior lawyers using the tool also completed their 10-day drafting tasks 18 minutes faster than the control group's average time of 124 minutes. However, these quality improvements primarily shifted scores from the bottom to the middle tier, without increasing the number of exceptional scores.
To measure long-term skill development, the researchers had the lawyers perform an unassisted patent redlining task at 90 days without any AI help. On this test of independent judgment, lawyers who had used the AI tool outperformed the control group by 0.32 standard deviations overall. However, this gain was driven entirely by senior lawyers with seven or more years of experience, who outperformed their control counterparts by 0.45 standard deviations. In contrast, junior lawyers showed no average skill improvement, and their scores bifurcated into more very low and more good scores.
The study reveals that senior lawyers benefited from using the AI as a "logic auditor" to challenge their own structural edits. Meanwhile, junior lawyers struggled with rigid formalism and surface-level edits. For legal practitioners and corporate managers, these findings highlight a critical tension: while AI can immediately speed up workflows, over-reliance on these tools may disrupt the traditional on-the-job learning pathways that turn junior employees into seasoned experts.
This is our own summary of reporting by Google Research
