Seeing the impact beyond the changed file
Blast-radius recall and developer confidence in a controlled comparison involving 55 full-stack engineers.
After seeing the impact of UnvibeCode within our own company, our team was curious: how would it help other engineers understand code?
We invited 55 full-stack engineers to complete a code-understanding exercise in a controlled environment. A third party conducted the exercise to reduce our influence on the process, and participants received no incentive to give UnvibeCode a positive rating.
The results surprised us. Engineers identified more of the components affected by a change, completed the tasks faster, and felt more confident in their assessments.
Study design and participants
| Element | Reported design |
|---|---|
| Participants | 55 full-stack engineers: 30 juniors with 0–2 years of experience and 25 mid-level engineers with 2–5 years. |
| Comparison | The same developers participated in conditions without and with UnvibeCode. The two experience groups are subgroups, not separate treatment and control arms. |
| Environment | A code-understanding exercise conducted by a third party in a controlled environment. |
| Task focus | Blast-radius analysis: identifying components or behaviors affected by a code change. |
| Incentives | No incentive was offered to provide positive ratings of UnvibeCode. |
| Results | Study-reported group results; overall recall and time-reduction summaries are weighted by participant count. |
Task time reduction by experience group
| Experience group | Participants | Reported time reduction with UnvibeCode |
|---|---|---|
| Junior · 0–2 years | 30 | 55% |
| Mid-level · 2–5 years | 25 | 33% |
| Participant-weighted average | 55 | 45% |
Blast-radius recall improved in both groups
Recall measures the proportion of relevant affected items successfully identified. The reported increase was larger for juniors: 33 percentage points, compared with 27 percentage points for mid-level engineers.
| Group | n | Without | With | Absolute gain |
|---|---|---|---|---|
| Junior · 0–2 years | 30 | 42% | 75% | +33 pp |
| Mid-level · 2–5 years | 25 | 55% | 82% | +27 pp |
| Participant-weighted summary | 55 | 47.91% | 78.18% | +30.27 pp |
Precision and confidence
The study reported a modest improvement of approximately 3% in precision overall, alongside larger increases in blast-radius recall.
Average confidence rose by 0.75 points on the five-point scale. This captures participants’ confidence in their assessments; confidence is a self-reported outcome distinct from measured correctness.
Research inspiration: understanding from overview to detail
This study draws inspiration from Gao et al., Understanding Codebase like a Professional! Human–AI Collaboration for Code Comprehension (ICPC 2026). The authors investigate how code auditors understand unfamiliar repositories and use those insights to inform CodeMap, an LLM-assisted comprehension system.
Their work describes a progression from a project overview to code structure and business logic, then to local functions and variables. That progression motivates the question explored here: can structured context help developers identify consequences beyond the file directly in front of them?
Reference
Gao, J., Xue, Y., Xie, X., Cao, J., Thant, S., Lee, E., & Xu, B. (2026). Understanding Codebase like a Professional! Human–AI Collaboration for Code Comprehension. Proceedings of the 34th IEEE/ACM International Conference on Program Comprehension, 343–354. doi:10.1145/3794763.3794822. Institutional publication record.