Method

DeepSeek V4 Pro

DeepSeek-V4-Pro-0813 directly evaluates each source sample using the developer-guided zero-shot prompt adopted from Ouédraogo et al. The prompt covers structure, nesting, intent, length, action granularity, and reading flow. Scores average three independent runs, without tools or an agentic workflow; recorded explanations appear on sample pages.

Score scale: 0–20; mean of three runs.

Dataset results

DatasetCoverageSpearman ρ
MBJP17/170.74
Buse100/1000.324
Scalabrino200/2000.466
Dorn360/3600.483
Schnappinger304/3040.537
JetBrains119/1190.53
Java transformations1400/1400—
Python transformations1400/1400—