[{"data":1,"prerenderedAt":460},["ShallowReactive",2],{"log-zh":3},[4,58,222,350],{"id":5,"title":6,"body":7,"date":41,"description":42,"draft":43,"extension":44,"kind":45,"meta":46,"navigation":47,"path":48,"pinned":43,"project":49,"seo":50,"stem":52,"summary":53,"tags":54,"__hash__":57,"translated":47},"notes\u002Fnotes\u002Fva-longe-mas-va-acompanhado.md","走得远，就要结伴同行",{"type":8,"value":9,"toc":37},"minimark",[10,14,17,28,31,34],[11,12,13],"p",{},"科技是我能想象到的最有意思、也最深奥的领域之一。前几天，我花了将近一个小时去弄懂一个线性代数的概念，而它只是在我为写硕士研究计划而读的一篇论文里被顺带提了一句。",[11,15,16],{},"不过，如果只能挑一样我最喜欢这个领域的东西，我会说是人。",[11,18,19,20,27],{},"读了一些关于外熵主义者（Extropians）的资料后，我很惊叹：寥寥几个投入、有创造力、彼此分享想法的人，就能改变世界。正是他们，在多年的邮件列表讨论中，",[21,22,26],"a",{"href":23,"rel":24},"https:\u002F\u002Fcypherbrush.substack.com\u002Fp\u002Farquivo-cypherpunk-0003-hashcash",[25],"nofollow","提出了后来催生比特币的那些概念","。",[11,29,30],{},"就我个人而言，我有一位在网络安全领域工作的朋友。我们会交流怎么安排学习，也交流各种想法，而在一次随意的闲聊中，冒出了一个想法，如果我被录取，它也许会成为我的硕士课题。",[11,32,33],{},"这个领域里有很多孤僻、不太爱说话的人，他们常常是“不被理解的天才”。但对他们，我想引用一句我外婆常说的老话：",[11,35,36],{},"“独行快，众行远。”",{"title":38,"searchDepth":39,"depth":39,"links":40},"",2,[],"2026-10-30","Tech is one of the most interesting and dense fields I can imagine. The other day I spent almost an hour trying to understand a linear algebra concept that was mentioned in passing in a paper I was reading to write my study plan for my master's.",false,"md","note",{},true,"\u002Fnotes\u002Fva-longe-mas-va-acompanhado",null,{"title":51,"description":42},"Go far, but go together","notes\u002Fva-longe-mas-va-acompanhado","我最喜欢科技领域的是人：从催生比特币的邮件列表，到和朋友的一次闲聊，那次闲聊也许会成为我的硕士课题。",[55,56],"community","cypherpunk","6BWabb5zFsUU8mHYtMFIBQQlE8T3i3x5uc64sYAJXZs",{"id":59,"title":60,"body":61,"date":208,"description":209,"draft":43,"extension":44,"kind":210,"meta":211,"navigation":47,"path":212,"pinned":43,"project":200,"seo":213,"stem":215,"summary":216,"tags":217,"__hash__":221,"translated":47},"notes\u002Fnotes\u002Fpre-match-ratings.md","一场比赛的评分，大部分无法预测",{"type":8,"value":62,"toc":201},[63,73,157,161,169,172,175,178,181,184,187,190,193],[11,64,65,66,72],{},"沿用 ",[21,67,71],{"href":68,"className":69},"\u002Fnotes\u002Fr2-too-good",[70],"wikilink","当 R² 好得过头"," 中的设定：在开球前预测每名球员的 Sofascore 评分，只使用主客场、对手实力和球员自己的历史数据。在 2025 年巴西甲级联赛的最后 80 场比赛上做前向滚动验证，共 2,494 人次出场。",[74,75,76,92],"table",{},[77,78,79],"thead",{},[80,81,82,86,89],"tr",{},[83,84,85],"th",{},"模型",[83,87,88],{},"MAE",[83,90,91],{},"R²",[93,94,95,107,118,129,140],"tbody",{},[80,96,97,101,104],{},[98,99,100],"td",{},"球员近 5 场平均",[98,102,103],{},"0.412",[98,105,106],{},"-0.06",[80,108,109,112,115],{},[98,110,111],{},"位置平均",[98,113,114],{},"0.398",[98,116,117],{},"0.01",[80,119,120,123,126],{},[98,121,122],{},"球员赛季平均",[98,124,125],{},"0.389",[98,127,128],{},"0.05",[80,130,131,134,137],{},[98,132,133],{},"位置专用模型",[98,135,136],{},"0.375",[98,138,139],{},"0.10",[80,141,142,148,153],{},[98,143,144],{},[145,146,147],"strong",{},"通用随机森林",[98,149,150],{},[145,151,152],{},"0.374",[98,154,155],{},[145,156,139],{},[158,159,160],"h2",{"id":160},"位置专用模型没有帮助",[11,162,163,164,168],{},"通用模型的 MAE 减去位置专用模型的 MAE 为 -0.0014，95% 置信区间为 ",[165,166,167],"span",{},"-0.0036, 0.0008","。这个区间来自 2,000 次 bootstrap 重抽样，每次抽取的是整场比赛而不是单行数据，因为同一场比赛中的球员并不相互独立。区间包含零，所以两者打平。决策规则在看到任何结果之前就已确定：如果打平，保留通用模型，因为一个模型比四个更简单。位置专用模型在任何一个位置上都没有胜出。",[158,170,171],{"id":171},"近期状态是噪声",[11,173,174],{},"球员近 5 场的平均评分，表现甚至不如完全不看这名球员、直接猜该位置的平均值。整个赛季的平均值则比两者都好。在五场比赛的尺度上，手感好或差大多只是噪声。置换重要性也印证了这一点：通用模型主要依赖两个特征，主客场和球员本赛季的平均评分。其余 61 项数据的历史几乎没有贡献。",[158,176,177],{"id":177},"门将",[11,179,180],{},"对门将来说，没有任何模型能明显胜过基线（通用模型和位置平均的 MAE 都是 0.559，R² 略低于零）。门将的评分取决于这场比赛中的扑救和失球，而在开球前，根本无法知道他会面对 2 次射门还是 12 次。",[158,182,183],{"id":183},"预测值紧贴平均值",[11,185,186],{},"测试集中真实评分的范围是 3.0 到 9.7，标准差为 0.54。通用模型的预测值范围是 6.63 到 7.41，标准差为 0.16。测试集中有 108 人次的评分在 8.0 或以上，模型的预测从来没有接近过其中任何一个。它会在主场比赛、以及本赛季表现好的球员身上把预测值往上调一点，但无法预见一场格外好或格外差的比赛。",[158,188,189],{"id":189},"答案",[11,191,192],{},"按位置专门建模并没有降低误差。通用模型比位置平均好 6.0%，比球员自己的赛季平均好 3.9%，而第二个差距是真实存在的（它的置信区间不包含零）。数字不大，但这就是信号的真实大小：一场比赛的评分，大部分是在比赛过程中决定的。",[11,194,195,196,27],{},"代码和完整方法：",[21,197,200],{"href":198,"rel":199},"https:\u002F\u002Fgithub.com\u002Falmeidaota\u002Fsofascore-rating-prediction",[25],"sofascore-rating-prediction",{"title":38,"searchDepth":39,"depth":39,"links":202},[203,204,205,206,207],{"id":160,"depth":39,"text":160},{"id":171,"depth":39,"text":171},{"id":177,"depth":39,"text":177},{"id":183,"depth":39,"text":183},{"id":189,"depth":39,"text":189},"2026-10-22","The setup from When R² is too good: predict each player's Sofascore rating before kickoff, using only home\u002Faway, opponent strength and the player's own history. Walk-forward validation over the last 80 matches of the 2025 Brasileirão, 2,494 appearances.","article",{},"\u002Fnotes\u002Fpre-match-ratings",{"title":214,"description":209},"Most of a match rating can't be predicted","notes\u002Fpre-match-ratings","按位置专门建模没有胜过单一的通用模型，近期状态还不如完全没有信息，而门将的评分谁也预测不了。论文的答案是否定的。",[218,219,220],"ml","random-forest","football","lRXhmendunPnntPdV0Kyh7MYFU2L_Af49JqWx0-YMFc",{"id":223,"title":71,"body":224,"date":208,"description":341,"draft":43,"extension":44,"kind":210,"meta":342,"navigation":47,"path":68,"pinned":43,"project":200,"seo":343,"stem":345,"summary":346,"tags":347,"__hash__":349,"translated":47},"notes\u002Fnotes\u002Fr2-too-good.md",{"type":8,"value":225,"toc":337},[226,229,237,280,283,287,290,293,296,299,302,315,328,331],[11,227,228],{},"毕业论文的研究问题：按位置（门将、后卫、中场、前锋）分别训练的随机森林模型，能否比单一的通用模型更准确地预测球员的 Sofascore 评分？",[11,230,231,232,236],{},"第一个版本给每个模型输入的是球员在这场比赛本身的数据：射门、传球、对抗、抢断、触球，一共 63 列。验证过程是干净的：用第 1 到 30 轮训练，用第 31 到 38 轮测试，用按比赛分组的 ",[233,234,235],"code",{},"GroupKFold"," 调超参数，保证同一场比赛的球员不会同时出现在一个折的两边。一个只预测该位置平均评分的基线给出了下限。",[74,238,239,249],{},[77,240,241],{},[80,242,243,245,247],{},[83,244,85],{},[83,246,88],{},[83,248,91],{},[93,250,251,260,271],{},[80,252,253,255,258],{},[98,254,111],{},[98,256,257],{},"0.391",[98,259],{},[80,261,262,265,268],{},[98,263,264],{},"通用模型",[98,266,267],{},"0.203",[98,269,270],{},"0.72",[80,272,273,275,278],{},[98,274,133],{},[98,276,277],{},"0.207",[98,279,270],{},[11,281,282],{},"前锋的 R² 达到了 0.80。把基线误差砍掉一半，写进论文里很好看。但这说不通：在足球里，单场比赛的数据不应该能解释这么多东西。",[158,284,286],{"id":285},"不是数据泄漏是循环论证","不是数据泄漏，是循环论证",[11,288,289],{},"第一个怀疑对象是训练集和测试集之间的泄漏。结果并没有。数据划分和交叉验证的折都保持每场比赛完整，没有任何测试数据碰到训练过程。",[11,291,292],{},"问题出在预测目标上。Sofascore 评分来自一个不公开的公式，而这个公式的输入和那 63 个特征非常相似：进球、传球、对抗、抢断。随机森林学到的并不是\"什么是好的表现\"，而是在逼近产生这个数字的公式。高 R² 衡量的只是模型模仿 Sofascore 算法的程度，仅此而已。",[11,294,295],{},"这个实验仍然回答了一个更窄的问题（在已知比赛数据的情况下，位置专用模型只在门将上胜出，而门将恰好是统计特征最独特的位置）。但它回答不了真正重要的问题：能不能在比赛开始前就知道评分？",[158,297,298],{"id":298},"重新设计",[11,300,301],{},"第二个实验禁止使用被预测的那场比赛的任何数据。每个特征都必须在开球前就已经存在：",[303,304,305,309,312],"ul",{},[306,307,308],"li",{},"主场还是客场；",[306,310,311],{},"对手实力：对手在之前比赛中的进球数和失球数；",[306,313,314],{},"球员自己的历史：61 项数据的赛季平均值，以及评分和出场时间的赛季平均值和近 5 场平均值，全部只用严格早于本场的比赛计算。",[11,316,317,318,323,324,327],{},"\"严格早于\"正是 ",[21,319,322],{"className":320,"href":321},[70],"\u002Fnotes\u002Fsofascore-data-traps","一个赛季的 Sofascore 数据，以及它在哪里说谎"," 中的轮次顺序不再是小细节的原因。还没有历史数据的球员（14.9% 的球员整个赛季只踢了 3 场或更少）会得到 ",[233,325,326],{},"NaN","，scikit-learn 的随机森林从 1.4 版本起可以原生处理。同时还会加入一个此前出场次数的特征，让模型学会更相信 20 场的历史，而不是 1 场。",[11,329,330],{},"验证方式改为前向滚动：用一批 10 场比赛之前的全部数据训练，预测这一批，再把它加入训练集，继续下一批，覆盖赛季的最后 80 场比赛。超参数搜索在训练窗口内使用按时间排序的折，始终在晚于训练数据的比赛上验证。",[11,332,333,334,27],{},"R² 降到了 0.10。这是预期中的结果，不是退步。用接近评分公式本身的数据去解释一个已经产生的评分，是一回事；在比赛前预测它，是另一件难得多的事。结果见 ",[21,335,60],{"className":336,"href":212},[70],{"title":38,"searchDepth":39,"depth":39,"links":338},[339,340],{"id":285,"depth":39,"text":286},{"id":298,"depth":39,"text":298},"The thesis question: can Random Forest models specialized by position (goalkeeper, defender, midfielder, forward) predict a player's Sofascore rating better than one generic model?",{},{"title":344,"description":341},"When R² is too good","notes\u002Fr2-too-good","第一个模型解释了前锋评分 80% 的方差。问题恰恰在这里：它是在重建 Sofascore 的评分公式，而不是在预测足球。",[218,219,348],"python","CP8qnPZk8uwlUZW8_GuAtolxx3ul1AWgwUEW6IlLme8",{"id":351,"title":322,"body":352,"date":208,"description":450,"draft":43,"extension":44,"kind":210,"meta":451,"navigation":47,"path":321,"pinned":43,"project":200,"seo":452,"stem":454,"summary":455,"tags":456,"__hash__":459,"translated":47},"notes\u002Fnotes\u002Fsofascore-data-traps.md",{"type":8,"value":353,"toc":445},[354,357,360,363,370,373,376,408,411,421,424,427,438],[11,355,356],{},"我的毕业论文需要 2025 年巴西甲级联赛的全部球员评分。Sofascore 有这些数据，就在它自己网站所用的内部 API 后面。由于这不是官方的公开 API，爬虫被刻意做得很\"无聊\"：每 1.2 到 2.4 秒发一次请求，失败时带退避重试，每一轮结束后保存断点，崩溃后可以接着跑而不必从头开始，并且每一行数据在写入 CSV 之前都要经过 Pydantic 校验。",[11,358,359],{},"最终从全部 380 场比赛中抓取了 11,977 条球员单场记录，比赛时间从 2025 年 3 月 29 日到 12 月 7 日。采集是最简单的部分。数据里有三个陷阱，而且没有一个会报错。",[158,361,362],{"id":362},"时区错误的日期",[11,364,365,366,369],{},"爬虫用 UTC 存储比赛日期。巴西的夜场比赛在 21:00 或更晚开球，按 UTC 已经是第二天了，所以 380 场比赛中有 67 场的日期是错的。修正方法是根据开球时间戳在 ",[233,367,368],{},"America\u002FSao_Paulo"," 时区下重新计算日期，清洗步骤和爬虫本身都做了修改。",[158,371,372],{"id":372},"三种缺失",[11,374,375],{},"一个空单元格可能代表三种不同的含义，每种都有各自的处理规则：",[303,377,378,388,394],{},[306,379,380,383,384,387],{},[145,381,382],{},"不适用。"," 比如中锋的扑救数。这些字段对非门将球员根本不会出现，所以填 ",[233,385,386],{},"-1","，这是任何真实计数都不可能取到的值。",[306,389,390,393],{},[145,391,392],{},"未测量。"," 体能数据（跑动距离、冲刺次数、最高速度）只在 380 场中的 349 场有，带球推进数据只在 83 场有。这些保持缺失。如果填成零，就等于声称球员没有跑动。",[306,395,396,399,400,403,404,407],{},[145,397,398],{},"为零而被省略。"," Sofascore 不会发送值为 0 的数据。在原始数据中，",[233,401,402],{},"goals"," 有 11,124 行为空，",[233,405,406],{},"expected_goals"," 有 6,314 行为空，恰好都是球员没有射门的那些行。这些填为 0。",[11,409,410],{},"在体能和带球推进这两组数据中，规则按球员判断：如果某球员有该组中的任意一项指标，缺失的就是被省略的零值；如果一项都没有，说明这组数据没有为他测量。",[11,412,413,414,417,418,420],{},"一个边界情况：门将的 ",[233,415,416],{},"goals_prevented"," 可能为负（从 -3.79 到 3.12），所以在这里哨兵值 ",[233,419,386],{}," 会和真实值冲突。这时用位置列来区分。",[158,422,423],{"id":423},"打乱顺序的轮次",[11,425,426],{},"给一个赛季排序，最直观的方法是按轮次编号。但这是错的。轮次会因为洲际赛事而改期，第 34 轮的大部分比赛直到 11 月才踢，而第 35 到 37 轮早在 10 月就已经踢完了。整个赛季中，72,010 对可能的比赛组合里，有 1,795 对（2.49%）的轮次编号和实际开球时间相矛盾。",[11,428,429,430,433,434,437],{},"一旦要计算球员的历史数据，这就不再是冷知识了。按轮次而不是开球时间排序，会改变 686 名踢过两场或以上的球员中 335 人（48.8%）的比赛顺序。对于一个只能看到过去比赛的模型来说，这就是数据泄漏：一场未来的比赛混进了过去某场比赛的历史里。之后的所有步骤都按 ",[233,431,432],{},"kickoff_timestamp"," 排序，从不按 ",[233,435,436],{},"round"," 排序。",[11,439,440,441,444],{},"清洗之后，删除了 163 条没有评分的出场记录，它们的出场时间都在 7 分钟或以下。最终剩下 732 名球员的 11,814 行数据，也就是 ",[21,442,71],{"className":443,"href":68},[70]," 中模型所用的训练数据。",{"title":38,"searchDepth":39,"depth":39,"links":446},[447,448,449],{"id":362,"depth":39,"text":362},{"id":372,"depth":39,"text":372},{"id":423,"depth":39,"text":423},"My thesis needed every player rating from the 2025 Brazilian Série A. Sofascore has them, behind the same internal API its website uses. It isn't an official public API, so the crawler was built to be boring: one request every 1.2 to 2.4 seconds, retries with backoff, a checkpoint after every round so a crash resumes instead of starting over, and Pydantic validation on every row before it reaches the CSV.",{},{"title":453,"description":450},"A season of Sofascore data, and where it lies","notes\u002Fsofascore-data-traps","抓取 2025 年巴西甲级联赛的全部球员评分，以及数据里的三个陷阱：时区错误的日期、从不发送的零值，以及打乱顺序的轮次。",[348,457,458],"scraping","data","eXQcuImFjV98ThxZr0pIfb9o9ar0a4y7Y8iJLRS93iQ",1793409193109]