The setup from When R² is too good: predict each player's Sofascore rating before kickoff, using only home/away, opponent strength and the player's own history. Walk-forward validation over the last 80 matches of the 2025 Brasileirão, 2,494 appearances.
| Model | MAE | R² |
|---|---|---|
| Player's last-5 average | 0.412 | -0.06 |
| Position average | 0.398 | 0.01 |
| Player's season average | 0.389 | 0.05 |
| Position specialists | 0.375 | 0.10 |
| Generic Random Forest | 0.374 | 0.10 |
Specialists didn't help
The generic model's MAE minus the specialists' MAE was -0.0014, with a 95% confidence interval of -0.0036, 0.0008. The interval comes from a bootstrap with 2,000 resamples, drawing whole matches instead of single rows, since players in the same game aren't independent. It includes zero, so it's a tie. The decision rule was set before seeing any result: on a tie, keep the generic model, because one model is simpler than four. Specialists didn't win any position.
Recent form is noise
The player's average over the last 5 games did worse than ignoring the player entirely and guessing the position average. The full season average did better than both. Over five games, a hot or cold streak is mostly noise. Permutation importance agrees: the generic model leaned on two features, home/away and the player's season-average rating. The histories of the other 61 stats contributed close to nothing.
Goalkeepers
For goalkeepers, no model meaningfully beat the baseline (MAE 0.559 for the generic model and for the position average alike, R² slightly below zero). A keeper's rating is decided by saves and goals conceded in that match, and before kickoff there's no way to know whether the keeper will face 2 shots or 12.
Predictions hug the mean
Real ratings in the test set ran from 3.0 to 9.7, with a standard deviation of 0.54. The generic model's predictions ran from 6.63 to 7.41, with a standard deviation of 0.16. 108 appearances in the test set were rated 8.0 or higher. The model never got close to any of them. It nudges predictions up for home games and for players with a good season so far, but it can't see an unusually good or bad game coming.
The answer
Specialization by position did not reduce the error. The generic model beat the position average by 6.0% and the player's own season average by 3.9%, and that second gap is real (its confidence interval doesn't include zero). Small numbers, but that's the honest size of the signal: most of a match rating is decided during the match.
Code and full method: sofascore-rating-prediction.