Method

GPT-6.1 Sol

GPT-6.1 Sol uses the same developer-guided zero-shot prompt adopted from Ouédraogo et al., covering structure, nesting, intent, length, action granularity, and reading flow. It uses low reasoning effort and no tools or agentic workflow. Scores average three independent runs; recorded explanations appear on sample pages.

Score scale: 0–20; mean of three runs.

Dataset results

DatasetCoverageSpearman ρ
MBJP17/170.759
Buse100/1000.206
Scalabrino200/2000.489
Dorn360/3600.521
Schnappinger304/3040.646
JetBrains119/1190.604
Java transformations1400/1400—
Python transformations1400/1400—