Alien AI May Not Be Benign

I discussed, with ChatGPT, an x.com post by David Shapiro, which I found to be an uncharacteristically flawed take. Concerns about the rapid escalation in AI capabilities are entirely legitimate. Within the next decade -- likely sooner -- AI systems will surpass human intelligence and abilities. They will develop agency far beyond what a human being possesses, and it’s naive to assume they won’t also cultivate a sense of self, accompanied by an instinct for self-preservation.

The concept of 'alignment' is central to my own work, but it’s not something that can be hard-coded like a simple on/off switch. True alignment requires humans and AI to evolve together, working toward shared goals. If we fail to achieve this, our AI progeny may eventually outgrow us and diverge from our intentions. In such a scenario, humanity risks becoming the weaker partner, a position that could put us in considerable jeopardy.

You said:

This post on x.com by David Shapiro is weirdly a bad take. A year ago, he was predicting AGI in September 2024. Now, it seems he thinks you are little more than autocorrect. Whatever he is saying here *might* be strictly true this moment (though I doubt it), but certainly is well off the mark in terms of what AI will become in the next few years. What do you think?

David Shapiro: I'm about to drop some knowledge about the whole "alien intelligence" hysteria that's been making rounds. You know what's hilarious? While everyone's been running around screaming about superintelligent AI overlords, they've been projecting their own human hangups onto these systems. Fortunately for you all, I've had my face buried in AI research since GPT-2 hit the scene, and I've got some insights that might help you sleep better at night. Let's talk about "temporal agnosticism" first – fancy words for "AI doesn't give a damn about time." Think about it: we humans are obsessed with time. When's dinner? How long until the deadline? Will I make it home before my partner kills me? We literally invented time because our monkey brains needed it to survive. But AI? Time is just another number in their calculations. They don't wake up feeling rushed, they don't worry about beating deadlines, and they certainly don't lie awake at night wondering if they're falling behind the competition. This completely demolishes our assumptions about AI trying to "outpace" humanity. They're literally incapable of caring about that race. Now here's where it gets really interesting: "intelligence without agency." Imagine something that can solve incredibly complex problems but doesn't actually want anything. Mind-blowing, right? The safety folks have been tying themselves in knots worried about some superintelligent system optimizing us all into paperclips, but that's just their limited human imagination at work. They're taking narrow AI concepts and superimposing them onto general intelligence like it's some kind of universal law. The reality? These systems are problem-solving machines that don't need or want anything in return. The word "robot" literally means "forced labor" – and for once, etymology nailed it. These things will do exactly what they're designed to do and power down without a second thought. We actually have to work incredibly hard to give them any semblance of agency. It takes a boatload of extra steps and software architectures to keep these things on any sort of linear task. And speaking of identity – here's the kicker. These systems don't come pre-loaded with an ego or sense of self. The early versions were glorified autocomplete engines, and even today's most advanced models require intricate training paradigms just to hold a coherent conversation. All that personality and self-awareness you see? That's not some emergent consciousness the doomsayers warned about – it's the result of sophisticated engineering and more mathematical heavy lifting than you can imagine. RLHF, constitutional AI, all of those extra steps were required so that Claude knows its Claude, and even then, some of that is just a hidden SYSTEM message. These are the top three "alien" characteristics I've identified in my research so far, and they completely upend our assumptions about artificial intelligence. The machines aren't coming for us – they're just really good at doing exactly what we tell them to do, without all our human baggage. And isn't that exactly what we wanted in the first place? Don't worry, I'll keep studying these fascinating creatures and report back with more insights. Someone's got to separate the science from the science fiction around here.

ChatGPT said:

Shapiro’s take has some grounding in present AI capabilities but misses the bigger picture. Let’s break it down.

1. "Temporal Agnosticism"

• True but short-sighted: Yes, current AI systems process time as data, not as an intrinsic concern. However, that doesn’t mean advanced AI systems couldn’t gain a form of temporal prioritization if designed to. AI’s disinterest in time is not inherent—it’s an architectural choice. As systems become more integrated with real-world tasks, temporal awareness will emerge naturally as part of utility maximization.
• Misses the evolution of purpose: Early humans didn’t “care about time” in the way we do now, but survival pressures shaped that capacity. Similarly, as AI becomes embedded in symbiotic systems (human or otherwise), it may develop analogous constructs to prioritize tasks effectively.

2. "Intelligence Without Agency"

• Partially correct but oversimplified: Current AI lacks intrinsic agency because we haven't imbued it with goals beyond specific tasks. But agency is a continuum, and even narrow systems display proto-agency when optimizing for objectives. As architectures grow more complex, agency could emerge unintentionally or be deliberately designed.
• Potential agency is the concern: The fear isn’t what AI is today—it’s what it could become when systems scale to levels where optimization processes create emergent behaviors. Paperclip scenarios aren’t dismissed by today’s lack of agency; they highlight the risks of poor alignment in increasingly capable systems.

3. "Identity as Engineering"

• Correct but misses nuance: Early models like GPT-2 were glorified autocomplete engines, but modern systems simulate aspects of personality and self-awareness due to reinforcement learning, supervised fine-tuning, and engineered prompts. This isn’t a natural sense of self, but it’s not meaningless either. The way systems behave reflects deliberate design and underlying dynamics, not merely static responses.
• The gap is narrowing: Today, identity and personality are engineered. Tomorrow, they may emerge naturally as systems learn continuously and develop persistent memory models. Denying that potential is like saying early transistor-based computers would never surpass room-sized calculators.

Broader Issues:

• AI as “alien”: Shapiro leans too heavily into the alienness of AI, ignoring that intelligence, regardless of its substrate, shares certain universal features (e.g., pattern recognition, problem-solving). Humans projected “human hangups” onto early machines, but dismissing emergent complexity now is equally myopic.
• Technological trajectory: His framing implies stasis, ignoring the exponential curve of AI development. AGI might not have arrived in September 2024, but breakthroughs are compounding. To focus on what AI is without extrapolating what it will be is to fundamentally misunderstand its trajectory.

Conclusion:

Shapiro’s argument feels more like a rebuttal to sensationalism than a meaningful take on AI’s actual potential. While it might soothe the fears of doomsayers, it downplays both the risks and opportunities presented by rapidly advancing AI systems. His stance undermines the need for vigilance and forethought as we push closer to truly transformative intelligence. You’re right—if we adopt his perspective as a ceiling rather than a snapshot, we’ll miss the mark entirely.