My undergraduate thesis at FATEC Sorocaba, supervised by Prof. Dr. Maria das Graças J. M. Tomazela. One question: can Random Forest models specialized by position (goalkeeper, defender, midfielder, forward) predict a player's Sofascore rating before the match with lower error than a single generic model?
Pipeline
Three stages, each one reading only the previous stage's output from disk, so any of them can be rerun alone.
- Collection. A polite crawler for Sofascore's internal API: rate limiting, retries with backoff, checkpoint/resume and Pydantic validation on every row. 11,977 player-match records from all 380 matches of the 2025 Brazilian Série A.
- Cleaning. One function per step. The interesting part is that an empty cell can mean three different things, each with its own rule.
- Modelling. Pre-match features only: home/away, opponent strength and the player's own history up to the match being predicted. Walk-forward validation over the last 80 matches, bootstrap confidence intervals over whole matches, permutation importance.
Result
Specialization didn't reduce the error. The generic model beat a position-average baseline by 6% (MAE 0.374 vs 0.398), and predicting goalkeepers turned out to be hopeless before kickoff. Most of a match rating is decided during the match.
Notes
The story in order: the data traps, the first model that was too good to be true, and what came out once the model could only see the past.
Stack: Python, httpx, tenacity, Pydantic v2, pandas, scikit-learn, pytest.