Method
DeepSeek V4 Pro
DeepSeek-V4-Pro-0813 directly evaluates each source sample using the developer-guided zero-shot prompt adopted from Ouédraogo et al. The prompt covers structure, nesting, intent, length, action granularity, and reading flow. Scores average three independent runs, without tools or an agentic workflow; recorded explanations appear on sample pages.
Score scale: 0–20; mean of three runs.
Dataset results
| Dataset | Coverage | Spearman ρ |
|---|---|---|
| MBJP | 17/17 | 0.74 |
| Buse | 100/100 | 0.324 |
| Scalabrino | 200/200 | 0.466 |
| Dorn | 360/360 | 0.483 |
| Schnappinger | 304/304 | 0.537 |
| JetBrains | 119/119 | 0.53 |
| Java transformations | 1400/1400 | — |
| Python transformations | 1400/1400 | — |