By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
World of SoftwareWorld of SoftwareWorld of Software
  • News
  • Software
  • Mobile
  • Computing
  • Gaming
  • Videos
  • More
    • Gadget
    • Web Stories
    • Trending
    • Press Release
Search
  • Privacy
  • Terms
  • Advertise
  • Contact
Copyright © All Rights Reserved. World of Software.
Reading: How Peter Thiel’s Relationship With Eliezer Yudkowsky Launched the AI Revolution
Share
Sign In
Notification Show More
Font ResizerAa
World of SoftwareWorld of Software
Font ResizerAa
  • Software
  • Mobile
  • Computing
  • Gadget
  • Gaming
  • Videos
Search
  • News
  • Software
  • Mobile
  • Computing
  • Gaming
  • Videos
  • More
    • Gadget
    • Web Stories
    • Trending
    • Press Release
Have an existing account? Sign In
Follow US
  • Privacy
  • Terms
  • Advertise
  • Contact
Copyright © All Rights Reserved. World of Software.
World of Software > Gadget > How Peter Thiel’s Relationship With Eliezer Yudkowsky Launched the AI Revolution
Gadget

How Peter Thiel’s Relationship With Eliezer Yudkowsky Launched the AI Revolution

News Room
Last updated: 2025/05/20 at 7:22 AM
News Room Published 20 May 2025
Share
SHARE

Two members of the Extropian community, internet entrepreneurs Brian and Sabine Atkins—­who met on an Extropian mailing list in 1998 and were married soon after—­were so taken by this message that in 2000 they bankrolled a think tank for Yudkowsky, the Singularity Institute for Artificial Intelligence. At 21, Yudkowsky moved to Atlanta and began drawing a nonprofit salary of around $20,000 a year to preach his message of benevolent superintelligence. “I thought very smart things would automatically be good,” he said. Within eight months, however, he began to realize that he was wrong—­way wrong. AI, he decided, could be a catastrophe.

“I was taking someone else’s money, and I’m a person who feels a pretty deep sense of obligation towards those who help me,” Yudkowsky explained. “At some point, instead of thinking, ‘If superintelligences don’t automatically determine what is the right thing and do that thing that means there is no real right or wrong, in which case, who cares?’ I was like, ‘Well, but Brian Atkins would probably prefer not to be killed by a superintelligence.’ ” He thought Atkins might like to have a “fallback plan,” but when he sat down and tried to work one out, he realized with horror that it was impossible. “That caused me to actually engage with the underlying issues, and then I realized that I had been completely mistaken about everything.”

The Atkinses were understanding, and the institute’s mission pivoted from making artificial intelligence to making friendly artificial intelligence. “The part where we needed to solve the friendly AI problem did put an obstacle in the path of charging right out to hire AI researchers, but also we just surely didn’t have the funding to do that,” Yudkowsky said. Instead, he devised a new intellectual framework he dubbed “rationalism.” (While on its face, rationalism is the belief that humankind has the power to use reason to come to correct answers, over time it came to describe a movement that, in the words of writer Ozy Brennan, includes “reductionism, materialism, moral non-­realism, utilitarianism, anti-­deathism and transhumanism.” Scott Alexander, Yudkowsky’s intellectual heir, jokes that the movement’s true distinguishing trait is the belief that “Eliezer Yudkowsky is the rightful calif.”)

In a 2004 paper, “Coherent Extrapolated Volition,” Yudkowsky argued that friendly AI should be developed based not just on what we think we want AI to do now, but what would actually be in our best interests. “The engineering goal is to ask what humankind ‘wants,’ or rather what we would decide if we knew more, thought faster, were more the people we wished we were, had grown up farther together, etc.,” he wrote. In the paper, he also used a memorable metaphor, originated by Bostrom, for how AI could go wrong: If your AI is programmed to produce paper clips, if you’re not careful, it might end up filling the solar system with paper clips.

In 2005, Yudkowsky attended a private dinner at a San Francisco restaurant held by the Foresight Institute, a technology think tank founded in the 1980s to push forward nanotechnology. (Many of its original members came from the L5 Society, which was dedicated to pressing for the creation of a space colony hovering just behind the moon, and successfully lobbied to keep the United States from signing the United Nations Moon Agreement of 1979 due to its provision against terraforming celestial bodies.) Thiel was in attendance, regaling fellow guests about a friend who was a market bellwether, because every time he thought some potential investment was hot, it would tank soon after. Yudkowsky, having no idea who Thiel was, walked up to him after dinner. “If your friend was a reliable signal about when an asset was going to go down, they would need to be doing some sort of cognition that beat the efficient market in order for them to reliably correlate with the stock going downwards,” Yudkowsky said, essentially reminding Thiel about the efficient-market hypothesis, which posits that all risk factors are already priced into markets, leaving no room to make money from anything besides insider information. Thiel was charmed.

Sign Up For Daily Newsletter

Be keep up! Get the latest breaking news delivered straight to your inbox.
By signing up, you agree to our Terms of Use and acknowledge the data practices in our Privacy Policy. You may unsubscribe at any time.
Share This Article
Facebook Twitter Email Print
Share
What do you think?
Love0
Sad0
Happy0
Sleepy0
Angry0
Dead0
Wink0
Previous Article OnePlus officially schedules the May 27 launch of two 'supreme' new flagship phones
Next Article I activated the Linux terminal on my Pixel, and you should too
Leave a comment

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Stay Connected

248.1k Like
69.1k Follow
134k Pin
54.3k Follow

Latest News

Spotify Jam comes to Android Auto so your whole car can DJ
News
Beloved chain ‘better than McDonald’s’ opens first-ever locations in 3 cities
News
Apple’s $350 Million Gamble on ‘F1’ Is Starting to Pay Off
News
Google Brings Customizable Gemini Chatbots to Docs, Drive, Gmail, Sheets
News

You Might also Like

Gadget

Recover Lost Crypto with Expert Crypto Recovery Services

6 Min Read
Gadget

Ethereum (ETH) Gained 8% to $2,697, Can Mutuum Finance (MUTM) Follow With 10x Returns

6 Min Read
Gadget

Tobias Sukenik: More Than a Businessman — A Brand with Heart

9 Min Read
Gadget

Which Crypto to Buy Today for Long-Term Gains? Bitcoin Solaris Presale Offers Bitcoin-Level Growth Potential

6 Min Read
//

World of Software is your one-stop website for the latest tech news and updates, follow us now to get the news that matters to you.

Quick Link

  • Privacy Policy
  • Terms of use
  • Advertise
  • Contact

Topics

  • Computing
  • Software
  • Press Release
  • Trending

Sign Up for Our Newsletter

Subscribe to our newsletter to get our newest articles instantly!

World of SoftwareWorld of Software
Follow US
Copyright © All Rights Reserved. World of Software.
Welcome Back!

Sign in to your account

Lost your password?