My thesis needed every player rating from the 2025 Brazilian Série A. Sofascore has them, behind the same internal API its website uses. It isn't an official public API, so the crawler was built to be boring: one request every 1.2 to 2.4 seconds, retries with backoff, a checkpoint after every round so a crash resumes instead of starting over, and Pydantic validation on every row before it reaches the CSV.
That pulled 11,977 player-match records out of all 380 matches, played between March 29 and December 7, 2025. Collecting was the easy part. The data had three traps in it, and none of them raised an error.
Dates in the wrong timezone
The crawler stored match dates in UTC. Brazilian night games kick off at 21:00 or later, which is already the next day in UTC, so 67 of the 380 matches had the wrong date. The fix was recomputing the date from the kickoff timestamp in America/Sao_Paulo, both in the cleaning step and in the crawler itself.
Three kinds of missing
An empty cell meant three different things, and each one got its own rule:
- Doesn't apply. Saves for a striker. Those keys never appear for outfield players, so they get
-1, a value no real count can take. - Not measured. Physical stats (distance, sprints, top speed) exist in only 349 of the 380 matches, ball carries in only 83. These stay missing. Filling them with zero would claim a player didn't run.
- Zero, omitted. Sofascore doesn't send stats that are 0. In the raw data,
goalswas empty in 11,124 rows andexpected_goalsin 6,314, exactly the rows where the player took no shot. These become 0.
Inside the physical and carries groups the rule is per player: if a player has any metric of the group, the missing ones are omitted zeros. If they have none, the group wasn't measured for them.
One edge case: goals_prevented can be negative for goalkeepers (from -3.79 to 3.12), so the -1 sentinel collides with real values there. The position column breaks the tie.
Rounds out of order
The obvious way to sort a season is by round number. It's wrong. Rounds get rescheduled around continental competitions, and most of round 34 was only played in November, after rounds 35 to 37 had been played in October. Across the season, 1,795 of the 72,010 possible pairs of matches (2.49%) have round numbers that contradict their kickoff times.
That stops being trivia as soon as you compute a player's history. Sorting by round instead of kickoff changes the order of matches for 335 of the 686 players (48.8%) with two or more games. For a model that is only allowed to look at past matches, that is leakage: a future game sneaking into the history of a past one. Everything downstream sorts by kickoff_timestamp, never by round.
After cleaning, 163 appearances without a rating were dropped, all of them 7 minutes long or shorter. That left 11,814 rows from 732 players, which is what the models in When R² is too good were trained on.