Shen & Tamkin (2026) · arXiv preprint, Anthropic · January 2026
AI & Skill Formation
An experiment on what AI assistance does to learning a new tool.
−17%
lower scores on a post-task knowledge quiz for developers who used AI assistance
Randomized experiment: 52 developers learned a new Python library, half with an AI assistant and half with documentation and web search only.
AI users scored 17% lower on the follow-up quiz; the gap was largest for debugging, with code reading and conceptual understanding also impaired.
On average, AI users did not finish significantly faster.
Of six observed interaction patterns, the three that preserved learning all involved active engagement: asking for explanations rather than delegating the task.
Why it matters
How people use AI, delegation versus engagement, determines whether productivity today comes at the cost of competence tomorrow.
Preprint, not peer-reviewed; a single experiment with 52 developers on one programming library. Source: Shen & Tamkin (2026), How AI Impacts Skill Formation. arXiv:2601.20245.