Published on November 6, 2025 8:36 AM GMTInspired by Daniel and Romeo https://x.com/DKokotajlo/status/1986231876852527270, I’m posting a memo on Takeoff, which I plan to expand into a proper article.This memo comes from notes I took from a talk from Daniel, which have been synthesized by Claude for breviy:Key PredictionsTransformative AI timeline: Late 2028Key prediction: All existing benchmarks solved by 2030p(doom): 10%Current GapsReasoning capabilities (mostly solved)Agency and long-horizon skills:Error correctionGoal pursuitHierarchical planningData-efficient learningProposed SolutionsNew models and architectures (e.g., recurrence)Language Model Programs (LMP)Scaffolding systemsAI bureaucraciesIncreased inference compute”Just train it to have those skills” approachStages of AI DevelopmentThe RE→RS→G→ASI FrameworkResearch Engineer for AI Research (we are entering here with Kosmos)”Devin actually works”Significant speedup in AI researchResearch Scientist”CEO talks to the cluster”Autonomous white paper productionGeniusQualitatively better than human researchersArtificial SuperintelligenceCapable of anythingEquivalent to 100k human teamsTechnical Specifications:Compute Requirements10^14 text predictions/rollouts on agentic paths10^23 floating point operations for transformative capabilitiesGPT-4 pre-training: ~2.15 × 10^25 FLOPS for comparisonTime Horizonst-AGI (10 second AGI to 10 day AGI)Year-AGI scalingAnnual 1 OOM (order of magnitude) improvements expected Critical Questions and ConcernsOn Full AutomationRobotics gap: “If TAI is Industrial Revolution level, what about physical automation?”Job stickiness: Institutional and social lags in job displacement”Wet robots”: Humans as interim solutionThe Capitalism-AI Nexus”Paperclips are a metaphor for money””Capitalism is the ultimate Turing test”AI systems are “incentivized to be constrained by human values”Societal PredictionsEarly prediction market odds:1/3 Futurama scenario1/4 Fizzle out1/5 DystopiaRisk ScenariosCyberpunk 2077-style hacking risks43% of Americans believe civil war is somewhat likelyDemocracy at its peak by voter count (2024)Unprecedented political deepfakesCultural homogenizationHuman disempowerment (sci-fi scenarios)Potential for singleton control”OpenAI coming for everybody’s jobs is a solace” -Basic CritiquesLogarithmic growth in atoms, exponential growth in bits”Folk theorem – bigger (model) is better””We’re very different from current machines””Brains are way more complex than even the most sophisticated NNs”Adaptation and time remain key differentiatorsAlternative ApproachesVirtual societies and digital twinsAI safety as cancer immune responseAutomated social science researcha deceptively simple solution to alignment: “Train them to be honest” Discuss Read More
A memo on Takeoff
Published on November 6, 2025 8:36 AM GMTInspired by Daniel and Romeo https://x.com/DKokotajlo/status/1986231876852527270, I’m posting a memo on Takeoff, which I plan to expand into a proper article.This memo comes from notes I took from a talk from Daniel, which have been synthesized by Claude for breviy:Key PredictionsTransformative AI timeline: Late 2028Key prediction: All existing benchmarks solved by 2030p(doom): 10%Current GapsReasoning capabilities (mostly solved)Agency and long-horizon skills:Error correctionGoal pursuitHierarchical planningData-efficient learningProposed SolutionsNew models and architectures (e.g., recurrence)Language Model Programs (LMP)Scaffolding systemsAI bureaucraciesIncreased inference compute”Just train it to have those skills” approachStages of AI DevelopmentThe RE→RS→G→ASI FrameworkResearch Engineer for AI Research (we are entering here with Kosmos)”Devin actually works”Significant speedup in AI researchResearch Scientist”CEO talks to the cluster”Autonomous white paper productionGeniusQualitatively better than human researchersArtificial SuperintelligenceCapable of anythingEquivalent to 100k human teamsTechnical Specifications:Compute Requirements10^14 text predictions/rollouts on agentic paths10^23 floating point operations for transformative capabilitiesGPT-4 pre-training: ~2.15 × 10^25 FLOPS for comparisonTime Horizonst-AGI (10 second AGI to 10 day AGI)Year-AGI scalingAnnual 1 OOM (order of magnitude) improvements expected Critical Questions and ConcernsOn Full AutomationRobotics gap: “If TAI is Industrial Revolution level, what about physical automation?”Job stickiness: Institutional and social lags in job displacement”Wet robots”: Humans as interim solutionThe Capitalism-AI Nexus”Paperclips are a metaphor for money””Capitalism is the ultimate Turing test”AI systems are “incentivized to be constrained by human values”Societal PredictionsEarly prediction market odds:1/3 Futurama scenario1/4 Fizzle out1/5 DystopiaRisk ScenariosCyberpunk 2077-style hacking risks43% of Americans believe civil war is somewhat likelyDemocracy at its peak by voter count (2024)Unprecedented political deepfakesCultural homogenizationHuman disempowerment (sci-fi scenarios)Potential for singleton control”OpenAI coming for everybody’s jobs is a solace” -Basic CritiquesLogarithmic growth in atoms, exponential growth in bits”Folk theorem – bigger (model) is better””We’re very different from current machines””Brains are way more complex than even the most sophisticated NNs”Adaptation and time remain key differentiatorsAlternative ApproachesVirtual societies and digital twinsAI safety as cancer immune responseAutomated social science researcha deceptively simple solution to alignment: “Train them to be honest” Discuss Read More
Related Posts
AI Incident Forecasting
Published on January 10, 2026 2:17 AM GMTI'm excited to share that my team and…
Rainbows, fractals, and crumpled paper: Hölder continuity
Published on November 4, 2025 8:01 AM GMTOne of my favorite website is allRGB. It's…
Steering RL Training: Benchmarking Interventions Against Reward Hacking
Published on December 29, 2025 9:55 PM GMTThis project is an extension of work done…