Code Readability Results

Spearman correlation between predicted scores and human ratings, computed separately for each dataset. Counts show evaluated samples / total samples.

Methods
Datasets

Independent Readability Interferences

Percentage of changed variants scored less readable than their matched original, reported separately for Java and Python. Counts show evaluated changed pairs.

Methods
Languages

Motivating example

A month-offset lookup adapted from Sakamoto’s compact day-of-week algorithm. All five developers rated it 1/5, yet four traditional models place it above the 74th percentile.

A lookup hidden in a string

class MonthOffsets {
    static int offset(int month) {
        int value = "-ilkDwlu2thk5".charAt(month);
        return value % 7;
    }
}

Relative readability

Posnett
86.5
Scalabrino
92.2
Dorn
76.4
Mi
74.5
DeepSeek V4 Pro
8.3
BTTF
1.9
Human
0.0

Model percentiles over the human-rated benchmark pool; lower is less readable. Human 1/5 is mapped to 0. DeepSeek averages five judgements.

Largest BTTF penalties and their source regions

Assignment-value surprisal-0.253
"-ilkDwlu2thk5".charAt(month)
Literal-tail surprisal-0.24
"-ilkDwlu2thk5"
Declaration-surprisal variation-0.073
int value = ...
Identifier-onset surprisal-0.037
value

Explainability analysis

LLM explanations and BTTF feature contributions for 30 randomly sampled programs.

Datasets and samples

DatasetSamplesEvaluationDownload
MBJP17human-ratedJSON
Buse100human-ratedOfficial source
Scalabrino200human-ratedOfficial source
Dorn360human-ratedOfficial source
Schnappinger304human-ratedJSON
JetBrains119human-ratedJSON
Java transformations1400controlledJSON
Python transformations1400controlledJSON