UnvibeCode · Developer study

Seeing the impact beyond the changed file

Blast-radius recall and developer confidence in a controlled comparison involving 55 full-stack engineers.

Report prepared 21 October 2026 · Study organizer: UnvibeCode · Exercise conducted by a third party during a developer coding workshop

After seeing the impact of UnvibeCode within our own company, our team was curious: how would it help other engineers understand code?

We invited 55 full-stack engineers to complete a code-understanding exercise in a controlled environment. A third party conducted the exercise to reduce our influence on the process, and participants received no incentive to give UnvibeCode a positive rating.

The results surprised us. Engineers identified more of the components affected by a change, completed the tasks faster, and felt more confident in their assessments.

55Full-stack engineers across two experience groups
↑30.2 pointsBlast-radius recall
↑0.75 / 5Average self-reported confidence increase
↑~3%Modest improvement in precision

Study design and participants

ElementReported design
Participants55 full-stack engineers: 30 juniors with 0–2 years of experience and 25 mid-level engineers with 2–5 years.
ComparisonThe same developers participated in conditions without and with UnvibeCode. The two experience groups are subgroups, not separate treatment and control arms.
EnvironmentA code-understanding exercise conducted by a third party in a controlled environment.
Task focusBlast-radius analysis: identifying components or behaviors affected by a code change.
IncentivesNo incentive was offered to provide positive ratings of UnvibeCode.
ResultsStudy-reported group results; overall recall and time-reduction summaries are weighted by participant count.

Task time reduction by experience group

Experience groupParticipantsReported time reduction with UnvibeCode
Junior · 0–2 years3055%
Mid-level · 2–5 years2533%
Participant-weighted average5545%

Blast-radius recall improved in both groups

Recall measures the proportion of relevant affected items successfully identified. The reported increase was larger for juniors: 33 percentage points, compared with 27 percentage points for mid-level engineers.

Recall: juniors 42% without versus 75% with UnvibeCode; mid-level 55% versus 82%; participant-weighted 47.91% versus 78.18%.
Figure 1. Study-reported subgroup recall and participant-weighted overall recall.
GroupnWithoutWithAbsolute gain
Junior · 0–2 years3042%75%+33 pp
Mid-level · 2–5 years2555%82%+27 pp
Participant-weighted summary5547.91%78.18%+30.27 pp

Precision and confidence

The study reported a modest improvement of approximately 3% in precision overall, alongside larger increases in blast-radius recall.

Average confidence increased from 3.50 out of 5 without UnvibeCode to 4.25 out of 5 with UnvibeCode.
Figure 2. Mean self-reported confidence across the study. The plotted scale runs from zero to five; the questionnaire rating scale was reported as 1–5.

Average confidence rose by 0.75 points on the five-point scale. This captures participants’ confidence in their assessments; confidence is a self-reported outcome distinct from measured correctness.

Research inspiration: understanding from overview to detail

This study draws inspiration from Gao et al., Understanding Codebase like a Professional! Human–AI Collaboration for Code Comprehension (ICPC 2026). The authors investigate how code auditors understand unfamiliar repositories and use those insights to inform CodeMap, an LLM-assisted comprehension system.

Their work describes a progression from a project overview to code structure and business logic, then to local functions and variables. That progression motivates the question explored here: can structured context help developers identify consequences beyond the file directly in front of them?

Reference

Gao, J., Xue, Y., Xie, X., Cao, J., Thant, S., Lee, E., & Xu, B. (2026). Understanding Codebase like a Professional! Human–AI Collaboration for Code Comprehension. Proceedings of the 34th IEEE/ACM International Conference on Program Comprehension, 343–354. doi:10.1145/3794763.3794822. Institutional publication record.