[{"data":1,"prerenderedAt":234},["ShallowReactive",2],{"note-r2-too-good-en":3,"titles-en":233},{"note":4,"project":162},{"id":5,"title":6,"body":7,"date":146,"description":13,"draft":147,"extension":148,"kind":149,"meta":150,"navigation":151,"path":152,"pinned":147,"project":153,"seo":154,"stem":155,"summary":156,"tags":157,"__hash__":161,"translated":151},"notes\u002Fnotes\u002Fr2-too-good.md","When R² is too good",{"type":8,"value":9,"toc":140},"minimark",[10,14,22,76,79,84,87,90,93,97,100,113,128,131],[11,12,13],"p",{},"The thesis question: can Random Forest models specialized by position (goalkeeper, defender, midfielder, forward) predict a player's Sofascore rating better than one generic model?",[11,15,16,17,21],{},"The first version fed each model the player's stats from the match itself: shots, passes, duels, tackles, touches, 63 columns in total. The validation was clean. Train on rounds 1 to 30, test on 31 to 38, tune hyperparameters with ",[18,19,20],"code",{},"GroupKFold"," grouped by match, so players from the same game never end up on both sides of a fold. A baseline that just predicts the position's average rating set the floor.",[23,24,25,41],"table",{},[26,27,28],"thead",{},[29,30,31,35,38],"tr",{},[32,33,34],"th",{},"Model",[32,36,37],{},"MAE",[32,39,40],{},"R²",[42,43,44,55,66],"tbody",{},[29,45,46,50,53],{},[47,48,49],"td",{},"Position average",[47,51,52],{},"0.391",[47,54],{},[29,56,57,60,63],{},[47,58,59],{},"Generic",[47,61,62],{},"0.203",[47,64,65],{},"0.72",[29,67,68,71,74],{},[47,69,70],{},"Specialists",[47,72,73],{},"0.207",[47,75,65],{},[11,77,78],{},"Forwards reached R² 0.80. Cutting the baseline error in half reads great in a thesis. It also didn't add up: one match's worth of stats shouldn't explain that much of anything in football.",[80,81,83],"h2",{"id":82},"not-leakage-circularity","Not leakage. Circularity.",[11,85,86],{},"The first suspect was train\u002Ftest leakage. There wasn't any. The split and the folds kept every match whole, and no test row touched training.",[11,88,89],{},"The problem was the target. The Sofascore rating comes out of a proprietary formula fed by stats that look a lot like those 63 features: goals, passes, duels, tackles. The Random Forest wasn't learning what a good performance is. It was approximating the formula that produced the number. The high R² measured how well the model imitates Sofascore's algorithm, and nothing else.",[11,91,92],{},"That experiment still answered a narrower question (with the match stats in hand, specialists only won for goalkeepers, the most statistically distinct position). But it couldn't answer the one that matters: can you know a rating before the game?",[80,94,96],{"id":95},"the-redesign","The redesign",[11,98,99],{},"The second experiment bans every stat from the match being predicted. Each feature has to exist before kickoff:",[101,102,103,107,110],"ul",{},[104,105,106],"li",{},"home or away;",[104,108,109],{},"opponent strength: goals scored and conceded by the opponent in earlier matches;",[104,111,112],{},"the player's own history: season averages of 61 stats, plus season and last-5 averages of rating and minutes, all computed only from matches strictly before this one.",[11,114,115,116,123,124,127],{},"\"Strictly before\" is why the round ordering in ",[117,118,122],"a",{"className":119,"href":121},[120],"wikilink","\u002Fnotes\u002Fsofascore-data-traps","A season of Sofascore data, and where it lies"," stopped being a detail. Players with no history yet (14.9% of them play 3 games or fewer all season) get ",[18,125,126],{},"NaN",", which scikit-learn's Random Forest handles natively since 1.4. A count of prior matches goes in alongside, so the model can learn to trust 20 games of history more than 1.",[11,129,130],{},"Validation moved to walk-forward: train on everything before a batch of 10 matches, predict that batch, add it to the training set, move on, across the last 80 matches of the season. Hyperparameter search uses time-ordered folds inside the training window, always validating on matches that came after the training ones.",[11,132,133,134,139],{},"R² dropped to 0.10. That's the expected result, not a regression. Explaining a rating that already happened, from stats close to its own formula, is one task. Predicting it before the game is a different and much harder one. ",[117,135,138],{"className":136,"href":137},[120],"\u002Fnotes\u002Fpre-match-ratings","Most of a match rating can't be predicted"," covers what came out.",{"title":141,"searchDepth":142,"depth":142,"links":143},"",2,[144,145],{"id":82,"depth":142,"text":83},{"id":95,"depth":142,"text":96},"2026-10-22",false,"md","article",{},true,"\u002Fnotes\u002Fr2-too-good","sofascore-rating-prediction",{"title":6,"description":13},"notes\u002Fr2-too-good","The first model explained 80% of the variance in forwards' ratings. That was the problem: it was rebuilding Sofascore's formula, not predicting football.",[158,159,160],"ml","random-forest","python","CP8qnPZk8uwlUZW8_GuAtolxx3ul1AWgwUEW6IlLme8",{"id":163,"title":164,"body":165,"cover":220,"date":221,"demo":220,"description":169,"draft":147,"extension":148,"featured":151,"meta":222,"navigation":151,"order":223,"path":224,"repo":225,"seo":226,"status":227,"stem":228,"summary":229,"tags":230,"__hash__":232,"translated":151},"projects\u002Fprojects\u002Fsofascore-rating-prediction.md","Sofascore rating prediction",{"type":8,"value":166,"toc":215},[167,170,174,177,198,202,205,209,212],[11,168,169],{},"My undergraduate thesis at FATEC Sorocaba, supervised by Prof. Dr. Maria das Graças J. M. Tomazela. One question: can Random Forest models specialized by position (goalkeeper, defender, midfielder, forward) predict a player's Sofascore rating before the match with lower error than a single generic model?",[80,171,173],{"id":172},"pipeline","Pipeline",[11,175,176],{},"Three stages, each one reading only the previous stage's output from disk, so any of them can be rerun alone.",[101,178,179,186,192],{},[104,180,181,185],{},[182,183,184],"strong",{},"Collection."," A polite crawler for Sofascore's internal API: rate limiting, retries with backoff, checkpoint\u002Fresume and Pydantic validation on every row. 11,977 player-match records from all 380 matches of the 2025 Brazilian Série A.",[104,187,188,191],{},[182,189,190],{},"Cleaning."," One function per step. The interesting part is that an empty cell can mean three different things, each with its own rule.",[104,193,194,197],{},[182,195,196],{},"Modelling."," Pre-match features only: home\u002Faway, opponent strength and the player's own history up to the match being predicted. Walk-forward validation over the last 80 matches, bootstrap confidence intervals over whole matches, permutation importance.",[80,199,201],{"id":200},"result","Result",[11,203,204],{},"Specialization didn't reduce the error. The generic model beat a position-average baseline by 6% (MAE 0.374 vs 0.398), and predicting goalkeepers turned out to be hopeless before kickoff. Most of a match rating is decided during the match.",[80,206,208],{"id":207},"notes","Notes",[11,210,211],{},"The story in order: the data traps, the first model that was too good to be true, and what came out once the model could only see the past.",[11,213,214],{},"Stack: Python, httpx, tenacity, Pydantic v2, pandas, scikit-learn, pytest.",{"title":141,"searchDepth":142,"depth":142,"links":216},[217,218,219],{"id":172,"depth":142,"text":173},{"id":200,"depth":142,"text":201},{"id":207,"depth":142,"text":208},null,"2026-08-27",{},3,"\u002Fprojects\u002Fsofascore-rating-prediction","https:\u002F\u002Fgithub.com\u002Falmeidaota\u002Fsofascore-rating-prediction",{"title":164,"description":169},"active","projects\u002Fsofascore-rating-prediction","My undergraduate thesis. Can Random Forest models specialized by position predict a player's Sofascore rating before kickoff better than one generic model? The answer was no.",[160,231,158,159],"scraping","IcTcWXpYn5Kpmy9fqNl7cIFEbUs7Jrm8SMXO46pBHt8",{},1793409193544]