Opinion

How Go Players Disempower Themselves to AI

​Written as part of the MATS 9.1 extension program, mentored by Richard Ngo.From March 9th to 15th 2016, Go players around the world stayed up to watch their game fall to AI. Google DeepMind’s AlphaGo defeated Lee Sedol, commonly understood to be the world’s strongest player at the time, with a convincing 4-1 score.This event “rocked” the Go world, but its impact on the culture was initially unclear. In Chess, for instance, computers have not meaningfully automated away human jobs. Human Chess flourished as a pseudo-Esport in the internet era whereas the yearly Computer Chess Championship is followed concurrently by no more than a few hundred nerds online. It turns out that the game’s cultural and economic value comes not from the abstract beauty of top-end performance, but instead from human drama and engagement. Indeed, Go has appeared to replicate this. A commentary stream might feature a complementary AI evaluation bar to give the viewers context. A Go teacher might include some new intriguing AI variations in their lesson materials. But the cultural practice of Go seemed to remain largely unaffected.  Nascent signs of disharmony in Europe became nevertheless visible in early 2018, when the online European Team Championship’s referee accused a player, Carlo Metta, of illicit AI use during a game. His results were voided and he was banned from further participation in the event. At the time the offending game was played, open-source engines based on the AlphaGo paper, such as Leela Zero, had only been around for about a month. However, a predecessor called Leela 0.11 was already widely available and was known to match the level of the top Europeans that Metta was facing. Metta’s accusers claimed that his play was too similar to this AI’s preferred moves. It was moreover considered suspicious that his Over-The-Board (OTB) play agreed significantly less with the AI than his online moves did.Unfortunately for the prosecution, their results were reported in intransparent and sloppy ways. This is evidenced by the fact that the best compilation of their findings is the slapdash facebook thread I linked above. This, along with the circumstantial nature of the evidence, was criticised in the same thread by community members. Teammates and friends of Metta’s also stepped up to publicly defend him. One way in which their rhetoric proved effective involved the public stigma and disdain against AI cheaters; this ironically made the case against Metta seem unfair and disproportionate due to the perceived gravity of the accusation. Ultimately, the Italian team appealed the decision and they won. Carlo Metta was officially exonerated.Among non-Italian European Go players, the claim that Metta used AI in almost every game in the ETC since 2018 has become barely disputable, especially considering how things developed. In the 2017/2018 season, he scored wins roughly half the time, likely using Leela 0.11 against opponents who were roughly the bot’s level. That same year, the Italian team was relegated to a lower league where no-one powerful in European Go politics cares to look anyway. This coincided with the popularisation of Leela Zero, a properly superhuman open-source go engine. Metta went on a 9-0 streak against opponents matching his OTB level in the 2018/2019 season, scored 9-1 in the 2019/2020 season, and then won 25 out of 26 games in the following years[1]. His only loss in this last streak was in a match where he was forced to play under camera control. During this time, his OTB level remained stagnant.At this point, considering Metta “innocent” represents a near-categorical rejection of convictions based on circumstantial evidence. I am not here to litigate that question, but am nevertheless comfortable assuming here that Metta was regularly using AI for these games. However, this is only the very start of our story because it illustrates some key points about the sociology of AI use in Go. First, the public announcement of his disqualification and the ensuing discourse vilified AI cheaters (incorrectly as it turns out) as being unusually dishonorable and evil. Second, he set the precedent that AI users would basically never get punished, no matter how obvious their cheating was even while under investigation. They could always just get their allies to kick up a fuss and pressure organisers into reversing the decision. These features made accusing people of cheating socially costly, and gave tournament organisers and fair-play committees an expectation of futility. Cheating in online European events thus became trivially easy due to a near complete lack of functional mechanisms for retribution.I started my career as a Go teacher in 2020, producing technical game reviews for a newly re-established online Go school set up to meet pandemic demand. We had not planned for cheating to be a major issue in our school. Whereas illicit AI use was already a well-known problem for the growing ecosystem of online tournaments, we didn’t expect it to affect our unrated, prizeless teaching league. To the contrary, we soon became cognisant of how some of our students were outputting better games than we, their teachers, could ever hope to play. Occasionally, AI use was unmistakably blatant because both sides played top AI moves for the entirety of the game. I now estimate that about half our students had used AI in at least one game and one in ten were chronic users. We were originally baffled by our observations. It didn’t make sense that players would just throw away their practice games to have AI win on their behalf. We also struggled to decide what to do about the problem and were reluctant to address it for roughly the same reasons that most tournament organisers were.Around the same time, I was asked to look into the online games of a promising young player that a friend suspected of using AI in a youth league. Like at the Go school, I was surprised at how easy cheating was to detect since nearly all the kids regularly used AI against each other. This incident and other similar ones made me gradually realise that illicit AI use was entirely endemic to the Go world. It fortunately turned out that this pattern didn’t generalise to the really important or prestigious tournaments that were held online during COVID. The symbolic camera controls – which would be easy to circumvent for a dedicated cheater – seemed sufficient to curb almost all cheating in a way that threats or impotent references to “fair-play committees” were failing to. This reminded me of how Metta tended only to lose in online tournaments[2] when playing under a camera (or when facing another AI user).Back to my hapless colleagues and I at the Go school, we initially settled for drily implying that suspicious games were “too good to review” and emphasising how we couldn’t help students who were playing “at such levels”. Our students caught on, and we were subsequently lucky to get some private confessions of cheating; over the years I was able to follow up with and interview many students that used AI, including some that hadn’t originally come forward. The appealing, exciting archetype of a cheater is one that uses covert, elaborate methods to get outside information and fraudulently obtain prize money or prestigious titles. Instead, we learned from the many examples of cheating and player confessions that idle curiosity and laziness were the dominant reasons for AI use in our school. Our students would often set out to play a normal game of Go, but would get stuck on a particularly difficult or annoying move; eventually, their curious eyes would drift to their second monitor — where they usually had their AI software running anyway — and they would check the answer as one would sheepishly side-eye the solution to an interesting puzzle or homework problem. Another reason people cited for using AI was an emotional investment in preserving or improving their image within the school community. Some wanted to avoid appearing incompetent and would employ strategies such as only playing moves that lost “n” points or less in expected value according to their computer.None of these reasons were surprising to us; we had already thought of most of them while shadowboxing our pupils’ strange behaviour. What personally shocked me, however, was the way our students conceptualised their AI use. In this, Carlo Metta was also a surprisingly predictive case. The original reddit thread discussing his ban featured a comment from a user called “carlo_metta”, which read:I never let Leela choose move. I just decide myself which one is better, for this reason i think i can find my own style with Leela. Go is an art and Leela help me tyo [sic] express my skillThat account was a burner, quite possibly a troll. However, I couldn’t help but recall the comment when I heard identical arguments coming from our cheating students’ accounts. A central part of every student’s retelling was that despite their AI use, they retained artistic control over their output and could exercise agency to think and improve for themselves. The AI felt to them like a tool that helped them fulfill latent potential or artistic sensibilities.AI users never find out they haven’t “got it”.Continental European math undergraduate degrees have a deserved reputation for their brutality, with completion rates of 10-15% being relatively common. Many of the 90% drop out nearly immediately, but some stick out the entire first year. These can often follow along with the proofs and exercise corrections’ atomic steps, which gives them false hope. However, they tend to struggle to see the “big picture” motivations of the material and are likely to have their hopes unravelled eventually. I was accidentally privy to a collective unravelling at the end-of-year third sitting of an exam on some basics of algebra and matrix calculations. I was retaking it to boost my grade from earlier in the year, but no other remotely competent person had bothered to do the same. Outside the exam hall, I listened to some other forty students’ chatter and had my blood internally curdle at phrases such as “I hate the proofs but I can do the exercises” or “I memorised all the matrix multiplication laws for this one”. The exam itself was quite unconventional; the professor clearly figured we would have had enough of manipulating matrices and instead asked an eclectic mix of simple algebra questions that to me vibed as “these are fundamental exercises you should be able to do if you have learned to think like a mathematician by now”.The atmosphere on exit mixed depression with vitriol. People complained on the object level about the exam, usually about how it was too niche or off-topic with respect to the material. However, there was something more fundamental going on. People had shown up with bags of half-baked heuristics and hand-copied exercises and proofs. That exam had put them face to face with the fact that their memory aids were never going to help them “get it”. I don’t think I saw any of them ever again.The population of Go AI users – both those who cheat in online games and those who simply review their games with AI post-hoc – is one on the perpetual eve of that exam. They fire up their computer out of idle curiosity and nod along passively as the truths of the universe float by them. They register the insights not one bit more because they can click the sublime moves. People consistently underestimate just how lost they will be when the solution is no longer right in front of them. This perspective of AI use to me explains why camera controls proved so effective against online cheating. Since AI use is usually an act of self-debasement and disempowerment – a subjection of oneself to ambient incentive gradients – it fundamentally contradicts the aesthetics of resourcefully overcoming a minor obstacle.The illusion of control that AI users have reliably shown interacts in an insidious way with their disempowerment. It contributes to a society of Go players that allow their participation in culture to be automated away. They are moreover so disempowered about it that they have built-in psychological mechanisms to keep them from ever recognising their own obsolescence. This mechanism even works to sabotage the detection of AI use in others. People tend to give overly conservative estimates of the chances a given game involves AI. I think this happens because they usually consult their own AI to check a suspected game. In doing so, they also come around to the machine’s point of view and conclude that playing the correct AI move was the “natural” thing to do anyway in that situation.My view of AI use (especially cheating) in Go originally manifested as disgust for its practitioners. I switched eventually to an attitude of compassion and pragmatism towards a habit that was clearly much more vulgar and weak than it was evil. Over time, I have progressed to feeling deep sadness for a group that surrenders much of what it claims to value. The thing I want to impress with this article is the consistency with which we as a species underestimate our own willingness to give up our culture, economy and autonomy to AI, even without monetary incentives. For this to happen, AI does not even need to be superhuman. Indeed, Go AI automates human players’ role in culture as shallow simulacra. All an AI needs to do is be passably good at a task and that may well be enough for people to volunteer their own replacement.Appendix A: No, Go players aren’t getting strongerOne of the objections I can anticipate to this pessimistic monologue is that expert Go players seem to have improved since AI became widely available. There’s a modest body of research in the field of Cultural Evolution advocating this, including this paper and related ones from the same group of authors. These views have been promoted by blogs in the techno-optimist orbit and one of the associated graphs was recently making the rounds on Twitter. I have already written a post analysing the data used for the research, where I concluded that it is being misinterpreted. The tl;dr is that all improvement in the quality of play comes before move 60, when humans can mimic memorised AI policies. Play after move 60, in the pivotal parts of the game, shows no improvement. For me to think there’s any meaningful change in human play from pre-AI times, I would have to be convinced that players understand the AI moves they copy well enough to keep a heightened level when they go off-policy after the opening. There is no evidence of this.Appendix B: Why this article existsThis piece is not meant to rigorously justify that Go players are disempowered or to carefully explore the shape of that disempowerment. It is instead designed to communicate a vibe from anecdotal experiences in the Go community that I think can give useful intuitions about Gradual Disempowerment as a general phenomenon.^I scraped the data from the tournament website using a vibecoded script (ironic!) and manually verified most of it.^Metta also regularly played (and used AI) in online events outside of the ETC during the COVID eraDiscuss ​Read More

​Written as part of the MATS 9.1 extension program, mentored by Richard Ngo.From March 9th to 15th 2016, Go players around the world stayed up to watch their game fall to AI. Google DeepMind’s AlphaGo defeated Lee Sedol, commonly understood to be the world’s strongest player at the time, with a convincing 4-1 score.This event “rocked” the Go world, but its impact on the culture was initially unclear. In Chess, for instance, computers have not meaningfully automated away human jobs. Human Chess flourished as a pseudo-Esport in the internet era whereas the yearly Computer Chess Championship is followed concurrently by no more than a few hundred nerds online. It turns out that the game’s cultural and economic value comes not from the abstract beauty of top-end performance, but instead from human drama and engagement. Indeed, Go has appeared to replicate this. A commentary stream might feature a complementary AI evaluation bar to give the viewers context. A Go teacher might include some new intriguing AI variations in their lesson materials. But the cultural practice of Go seemed to remain largely unaffected.  Nascent signs of disharmony in Europe became nevertheless visible in early 2018, when the online European Team Championship’s referee accused a player, Carlo Metta, of illicit AI use during a game. His results were voided and he was banned from further participation in the event. At the time the offending game was played, open-source engines based on the AlphaGo paper, such as Leela Zero, had only been around for about a month. However, a predecessor called Leela 0.11 was already widely available and was known to match the level of the top Europeans that Metta was facing. Metta’s accusers claimed that his play was too similar to this AI’s preferred moves. It was moreover considered suspicious that his Over-The-Board (OTB) play agreed significantly less with the AI than his online moves did.Unfortunately for the prosecution, their results were reported in intransparent and sloppy ways. This is evidenced by the fact that the best compilation of their findings is the slapdash facebook thread I linked above. This, along with the circumstantial nature of the evidence, was criticised in the same thread by community members. Teammates and friends of Metta’s also stepped up to publicly defend him. One way in which their rhetoric proved effective involved the public stigma and disdain against AI cheaters; this ironically made the case against Metta seem unfair and disproportionate due to the perceived gravity of the accusation. Ultimately, the Italian team appealed the decision and they won. Carlo Metta was officially exonerated.Among non-Italian European Go players, the claim that Metta used AI in almost every game in the ETC since 2018 has become barely disputable, especially considering how things developed. In the 2017/2018 season, he scored wins roughly half the time, likely using Leela 0.11 against opponents who were roughly the bot’s level. That same year, the Italian team was relegated to a lower league where no-one powerful in European Go politics cares to look anyway. This coincided with the popularisation of Leela Zero, a properly superhuman open-source go engine. Metta went on a 9-0 streak against opponents matching his OTB level in the 2018/2019 season, scored 9-1 in the 2019/2020 season, and then won 25 out of 26 games in the following years[1]. His only loss in this last streak was in a match where he was forced to play under camera control. During this time, his OTB level remained stagnant.At this point, considering Metta “innocent” represents a near-categorical rejection of convictions based on circumstantial evidence. I am not here to litigate that question, but am nevertheless comfortable assuming here that Metta was regularly using AI for these games. However, this is only the very start of our story because it illustrates some key points about the sociology of AI use in Go. First, the public announcement of his disqualification and the ensuing discourse vilified AI cheaters (incorrectly as it turns out) as being unusually dishonorable and evil. Second, he set the precedent that AI users would basically never get punished, no matter how obvious their cheating was even while under investigation. They could always just get their allies to kick up a fuss and pressure organisers into reversing the decision. These features made accusing people of cheating socially costly, and gave tournament organisers and fair-play committees an expectation of futility. Cheating in online European events thus became trivially easy due to a near complete lack of functional mechanisms for retribution.I started my career as a Go teacher in 2020, producing technical game reviews for a newly re-established online Go school set up to meet pandemic demand. We had not planned for cheating to be a major issue in our school. Whereas illicit AI use was already a well-known problem for the growing ecosystem of online tournaments, we didn’t expect it to affect our unrated, prizeless teaching league. To the contrary, we soon became cognisant of how some of our students were outputting better games than we, their teachers, could ever hope to play. Occasionally, AI use was unmistakably blatant because both sides played top AI moves for the entirety of the game. I now estimate that about half our students had used AI in at least one game and one in ten were chronic users. We were originally baffled by our observations. It didn’t make sense that players would just throw away their practice games to have AI win on their behalf. We also struggled to decide what to do about the problem and were reluctant to address it for roughly the same reasons that most tournament organisers were.Around the same time, I was asked to look into the online games of a promising young player that a friend suspected of using AI in a youth league. Like at the Go school, I was surprised at how easy cheating was to detect since nearly all the kids regularly used AI against each other. This incident and other similar ones made me gradually realise that illicit AI use was entirely endemic to the Go world. It fortunately turned out that this pattern didn’t generalise to the really important or prestigious tournaments that were held online during COVID. The symbolic camera controls – which would be easy to circumvent for a dedicated cheater – seemed sufficient to curb almost all cheating in a way that threats or impotent references to “fair-play committees” were failing to. This reminded me of how Metta tended only to lose in online tournaments[2] when playing under a camera (or when facing another AI user).Back to my hapless colleagues and I at the Go school, we initially settled for drily implying that suspicious games were “too good to review” and emphasising how we couldn’t help students who were playing “at such levels”. Our students caught on, and we were subsequently lucky to get some private confessions of cheating; over the years I was able to follow up with and interview many students that used AI, including some that hadn’t originally come forward. The appealing, exciting archetype of a cheater is one that uses covert, elaborate methods to get outside information and fraudulently obtain prize money or prestigious titles. Instead, we learned from the many examples of cheating and player confessions that idle curiosity and laziness were the dominant reasons for AI use in our school. Our students would often set out to play a normal game of Go, but would get stuck on a particularly difficult or annoying move; eventually, their curious eyes would drift to their second monitor — where they usually had their AI software running anyway — and they would check the answer as one would sheepishly side-eye the solution to an interesting puzzle or homework problem. Another reason people cited for using AI was an emotional investment in preserving or improving their image within the school community. Some wanted to avoid appearing incompetent and would employ strategies such as only playing moves that lost “n” points or less in expected value according to their computer.None of these reasons were surprising to us; we had already thought of most of them while shadowboxing our pupils’ strange behaviour. What personally shocked me, however, was the way our students conceptualised their AI use. In this, Carlo Metta was also a surprisingly predictive case. The original reddit thread discussing his ban featured a comment from a user called “carlo_metta”, which read:I never let Leela choose move. I just decide myself which one is better, for this reason i think i can find my own style with Leela. Go is an art and Leela help me tyo [sic] express my skillThat account was a burner, quite possibly a troll. However, I couldn’t help but recall the comment when I heard identical arguments coming from our cheating students’ accounts. A central part of every student’s retelling was that despite their AI use, they retained artistic control over their output and could exercise agency to think and improve for themselves. The AI felt to them like a tool that helped them fulfill latent potential or artistic sensibilities.AI users never find out they haven’t “got it”.Continental European math undergraduate degrees have a deserved reputation for their brutality, with completion rates of 10-15% being relatively common. Many of the 90% drop out nearly immediately, but some stick out the entire first year. These can often follow along with the proofs and exercise corrections’ atomic steps, which gives them false hope. However, they tend to struggle to see the “big picture” motivations of the material and are likely to have their hopes unravelled eventually. I was accidentally privy to a collective unravelling at the end-of-year third sitting of an exam on some basics of algebra and matrix calculations. I was retaking it to boost my grade from earlier in the year, but no other remotely competent person had bothered to do the same. Outside the exam hall, I listened to some other forty students’ chatter and had my blood internally curdle at phrases such as “I hate the proofs but I can do the exercises” or “I memorised all the matrix multiplication laws for this one”. The exam itself was quite unconventional; the professor clearly figured we would have had enough of manipulating matrices and instead asked an eclectic mix of simple algebra questions that to me vibed as “these are fundamental exercises you should be able to do if you have learned to think like a mathematician by now”.The atmosphere on exit mixed depression with vitriol. People complained on the object level about the exam, usually about how it was too niche or off-topic with respect to the material. However, there was something more fundamental going on. People had shown up with bags of half-baked heuristics and hand-copied exercises and proofs. That exam had put them face to face with the fact that their memory aids were never going to help them “get it”. I don’t think I saw any of them ever again.The population of Go AI users – both those who cheat in online games and those who simply review their games with AI post-hoc – is one on the perpetual eve of that exam. They fire up their computer out of idle curiosity and nod along passively as the truths of the universe float by them. They register the insights not one bit more because they can click the sublime moves. People consistently underestimate just how lost they will be when the solution is no longer right in front of them. This perspective of AI use to me explains why camera controls proved so effective against online cheating. Since AI use is usually an act of self-debasement and disempowerment – a subjection of oneself to ambient incentive gradients – it fundamentally contradicts the aesthetics of resourcefully overcoming a minor obstacle.The illusion of control that AI users have reliably shown interacts in an insidious way with their disempowerment. It contributes to a society of Go players that allow their participation in culture to be automated away. They are moreover so disempowered about it that they have built-in psychological mechanisms to keep them from ever recognising their own obsolescence. This mechanism even works to sabotage the detection of AI use in others. People tend to give overly conservative estimates of the chances a given game involves AI. I think this happens because they usually consult their own AI to check a suspected game. In doing so, they also come around to the machine’s point of view and conclude that playing the correct AI move was the “natural” thing to do anyway in that situation.My view of AI use (especially cheating) in Go originally manifested as disgust for its practitioners. I switched eventually to an attitude of compassion and pragmatism towards a habit that was clearly much more vulgar and weak than it was evil. Over time, I have progressed to feeling deep sadness for a group that surrenders much of what it claims to value. The thing I want to impress with this article is the consistency with which we as a species underestimate our own willingness to give up our culture, economy and autonomy to AI, even without monetary incentives. For this to happen, AI does not even need to be superhuman. Indeed, Go AI automates human players’ role in culture as shallow simulacra. All an AI needs to do is be passably good at a task and that may well be enough for people to volunteer their own replacement.Appendix A: No, Go players aren’t getting strongerOne of the objections I can anticipate to this pessimistic monologue is that expert Go players seem to have improved since AI became widely available. There’s a modest body of research in the field of Cultural Evolution advocating this, including this paper and related ones from the same group of authors. These views have been promoted by blogs in the techno-optimist orbit and one of the associated graphs was recently making the rounds on Twitter. I have already written a post analysing the data used for the research, where I concluded that it is being misinterpreted. The tl;dr is that all improvement in the quality of play comes before move 60, when humans can mimic memorised AI policies. Play after move 60, in the pivotal parts of the game, shows no improvement. For me to think there’s any meaningful change in human play from pre-AI times, I would have to be convinced that players understand the AI moves they copy well enough to keep a heightened level when they go off-policy after the opening. There is no evidence of this.Appendix B: Why this article existsThis piece is not meant to rigorously justify that Go players are disempowered or to carefully explore the shape of that disempowerment. It is instead designed to communicate a vibe from anecdotal experiences in the Go community that I think can give useful intuitions about Gradual Disempowerment as a general phenomenon.^I scraped the data from the tournament website using a vibecoded script (ironic!) and manually verified most of it.^Metta also regularly played (and used AI) in online events outside of the ETC during the COVID eraDiscuss ​Read More

Related Posts

Leave a Reply

Your email address will not be published. Required fields are marked *