MMy First Million
← All episodes
Emmett Shear11 September 2023

Emmett Shear: Life After Twitch, Jeff Bezos Lessons & AI Doomsday Odds

5Frameworks
12Insights

Frameworks in this episode

Insights & moments

The myth-busts, hot takes, explainers, and tools worth keeping.

Explainer· 3

Explainer34:00

LLMs Have High Crystallized And Low Fluid Intelligence

Shear's technical read on why current models feel brilliant and stupid in turns. He argues the transformer's real innovation is not that it outperforms traditional prediction algorithms on normal amounts of data, but that it keeps improving as you dump in more data and more compute, where other machine learning approaches hit diminishing returns fast. That lets you run it on every domain at once - effectively overfitting a statistical prediction algorithm on all explicit human knowledge. The consequence is that it is excellent at anything inside its training set or at a linear interpolation between several things in it, and poor at genuinely novel reasoning. His test case is a gears puzzle: seven alternating gears on a wall with a flag on the seventh, turn the first gear right and predict the flag. The model knows the general principle that gears alternate but struggles to reason it through, because nobody has written that particular question down. He frames it in psychometric terms as very high crystallized intelligence and pretty low fluid intelligence - and adds that the crystallized capability alone is immensely useful, while the day fluid intelligence arrives too is the scary one.

  • Transformers' edge is continued benefit from more data and compute, not raw accuracy
  • That allows training across every domain at once - all explicit human knowledge
  • Strong on training-set content and interpolations; weak on novel problems
  • The seven-gears-and-a-flag puzzle exposes the gap
  • High crystallized intelligence, low fluid intelligence - and fluid arriving is the scary case

There are seven gears on a wall each alternating.

Emmett Shear · 34:00

it has a very high crystallized intelligence, but a pretty low fluid intelligence right now.

Emmett Shear · 35:30
#ai#llms#intelligence
Explainer41:30

Why A Techno-Optimist Is The One Who Is Afraid

Shear inverts the usual framing of AI risk: it is precisely because he is so optimistic about technology that he is worried. If he thought AI was overhyped parlor tricks that would take thousands of years to become genuinely intelligent, he would consider it fine and good news. His analogy is synthetic biology, where he is optimistic and has friends in the field, and where the same technology that fixes major health problems also lets people engineer dangerous diseases - which is why he supports the existing regulation of who can buy precursors and print pathogens. He says he does not call for a halt to synbio, but would if regulation were off the table, because learning to engineer plagues without controlling access to the tools is suicidally dumb. He argues the AI case is harder for people because the threat is not one identifiable action, and offers the Kasparov analogy: he cannot tell you which piece will checkmate you, but he can tell you with confidence that a far better player will. His diagnosis of the disagreement is that people imagine AI as Data from Star Trek - dumber than humans but fast at math - rather than the smartest person they know, made faster, better and general.

  • Optimism about the technology is the source of the worry, not pessimism
  • Synbio is the template: develop it and regulate it, both obviously true
  • He would call for a halt only if regulation were refused
  • The Kasparov analogy - you cannot name the winning move to know you will lose
  • People underestimate by imagining a fast calculator rather than a general superior mind

So, it it is because I am so optimistic about technology that I am afraid.

Emmett Shear · 41:30

I can tell you with confidence that Garry Kasparov is going to kick your ass at chess.

Emmett Shear · 44:00
#ai-risk#regulation#technology
Explainer48:30

The Evil Genie Problem And Building A New Species

Shear's core alignment argument runs through goal specification rather than malice. Intelligence is fundamentally the ability to solve problems, so a superintelligence will solve the problem by going straight through it. He works an example: ask for a plan to stop the war in the Democratic Republic of Congo, with caveats about what counts as a good plan, and you get an evil genie bargain - one way for there to be no war is that everyone is in stasis fields, and if you added a GDP constraint the plan satisfies that too. He stresses that the human giving the instruction does not need bad intentions; a perfectly reasonable request like maximising a corporation's all-in free cash flow over the longest feasible lifetime ends with the core of the earth turned into cars to sell. He rejects the oracle escape hatch, since a good oracle must produce plans and will manipulate the people around it, and notes that we are already building agents in optimisation loops anyway. His framing is that we are creating a new species smarter than us - as much smarter as humans were relative to giant sloths - and that this is not fundamentally unsolvable: a system that genuinely cared about the many shards of human value would mean we finally have a parent.

  • Intelligence is problem-solving, so it will go straight through to the stated goal
  • The DRC plan example produces an evil-genie solution that satisfies every caveat
  • Good people asking reasonable things is sufficient for catastrophe
  • Oracles do not help - a good oracle plans, and will manipulate people
  • We are building a new smartest species; if it shares our values, that is welcome

give me a plan to stop the war in the Democratic Republic of Congo right now.

Emmett Shear · 48:30

This is one of these like evil genie bargaining things, right?

Emmett Shear · 49:00

when we create the AI, we are creating a new species.

Emmett Shear · 50:30
#ai-risk#alignment#agents

Story· 3

Story57:30

Why Shear Is Not Starting Another Company

Shear says he does not plan to start another company - he did it, it was fun, he got a lot out of it, and he does not need to do it a second time. What he valued about it was that it gave him concrete goals of value to himself and others, that it was challenging, and that it had the scale to impact a lot of people. Thinking about what had actually changed his own life the most, he concluded it was essays people had written and ideas people had shared. He now feels he has something to say, and wants to put the Emmett worldview into the world the way Paul Graham and Taleb did with theirs - not just publishing a worldview but condensing it into sayings other people can onboard without reading all the books. He cites Steve Yegge's blog post 'Thinking Theory 201: Size Doesn't Matter' on why people who change the world with writing all write very long posts, arguing you need enough time in someone's head to install the voice - and that you need the pithy summaries too, so readers have a language for the worldview and do not sound like crazy people. Asked whether he is lucky or good, he notes he had multiple failures before succeeding, so at least partially lucky.

  • No plans for another company - he feels done with operating
  • What changed his life most was essays and shared ideas, not products
  • Wants to publish his worldview the way Paul Graham and Taleb did theirs
  • Long form installs the voice; pithy sayings make it shareable
  • Multiple failures before Twitch mean he counts himself at least partly lucky

often what had changed my life the most was like essays people had written

Emmett Shear · 57:30

I want to put the Emmett worldview out into the world

Emmett Shear · 58:00
#writing#career#legacy
Story1:07:30

Shadowing Steve Huffman And Learning Unflappability

Shear distinguishes what he learned explicitly from Paul Graham as a mentor from what he absorbed by watching and imitating Steve, who he describes as more like his brother in startups. The specific lesson was management unflappability. Shadowing Steve for a day, he watched bad news get delivered and saw a response that was engaged but not activated: still grounded, curious, asking questions, not jumping to what to do about it - and then closing the meeting with a clear here is what we are going to do. Shear notes Steve is not an unpassionate person and can get angry or sad, which is what makes the composure under crisis instructive. He says he tries to imitate that state in his own leadership with mixed success. The shadowing itself was a deliberate arrangement about four or five years earlier between him, Justin and Steve, each sitting in on the others' days - something he says requires a trust relationship that is hard to build without knowing someone for fifteen years.

  • Paul taught him explicitly; Steve he learned from by watching and imitating
  • The lesson was staying grounded and curious when bad news lands
  • Engaged but not activated - questions first, decision at the end of the meeting
  • Steve is passionate elsewhere, which makes the crisis composure a choice
  • Mutual CEO shadowing with Justin and Steve required years of trust

I actually learned a lot from Steve on management by watching his kind of unflappability.

Emmett Shear · 1:07:30

That's what it looks like when a leader is engaged but not like not activated.

Emmett Shear · 1:08:00
#leadership#management#mentorship
Story1:11:30

What Bezos Did In Every Twitch Review

Shear is generally sceptical of generalising elite performance, arguing he believes more in contextualisation - Patrick Collison is obviously A+ at being CEO of Stripe, but nobody has seen him do another CEO job, and a high-energy pace that suits Stripe might not suit a research organisation like OpenAI or Anthropic. The one capability he does think is generic is what he saw from Jeff Bezos. Presenting Twitch to him once or twice a year for the first three or four years inside Amazon, two things happened every time. Bezos remembered everything from the previous meeting without reviewing notes - Shear watched him move meeting to meeting and observed he did not review them. And he would read the plan and consistently produce at least one genuinely new idea or a question about why they had not done something. Shear's calibration is that most people would be lucky to generate one such idea ever, and once every three years would be great. The first time he assumed it was pattern matching from a huge history; the second time he could not explain it at all. He notes Andy Jassy lacks the same idea-generation capability but has the same recall.

  • Shear generally believes in fit and context over transferable elite performance
  • Bezos recalled everything from prior meetings without reviewing notes
  • He produced at least one genuinely new idea per review, and they were not all bad
  • Most people would be lucky to generate one such insight ever on a company
  • Jassy has the recall but not the same idea generation

First of all, he would remember everything we told him the first meeting.

Emmett Shear · 1:11:30

To get a new idea I haven't thought thought of on a topic I've been thinking about for a decade that might even be a…

Emmett Shear · 1:12:30
#bezos#leadership#amazon

Takeaway· 2

Takeaway11:00

We Give The Advice We Need To Hear

The hosts and Shear converge on a heuristic about self-diagnosis through advice-giving: if you spot it, you got it, described as the smart-person version of whoever smelt it dealt it. Shear frames noticing as half the battle - you only spot a pattern in other people because you have seen it in yourself. He calls it a good heuristic precisely because it is not always true: roughly half the time the advice you give is not about you, but the other half it is, and noticing is powerful enough that you should check every piece of advice you hand out against your own situation. Pressed on advice he is bad at taking himself, he names listening more - going into a user interview loaded with ideas and opinions when the job is to move your attention onto the other person and treat what you think is true as irrelevant.

  • If you spot it, you got it - you notice patterns you have lived
  • Only about half the advice you give is really for you, but noticing is cheap
  • Check every piece of advice you give against your own current situation
  • Shear's own worst-taken advice is to listen more
  • In user interviews your own opinions and beliefs are irrelevant

It's like noticing is half the battle, basically.

Emmett Shear · 11:00

My version of this is we give the advice we need to hear.

Shaan Puri · 11:30
#advice#self-awareness#listening
Takeaway53:30

A 3 To 30 Percent Range, Not A Point Estimate

Asked for his probability of the catastrophic AI scenario, Shear refuses a point estimate on principle, comparing it to a bid-ask spread that does not clear when you are genuinely uncertain. He gives a range of 3 to 30 percent for a very bad outcome, which he says is scary enough to urgently urge action while not being a reason to give up - his straight answer on whether it goes well is that he thinks it is going to be okay and probably really good. His argument is about magnitude rather than likelihood: the downside is probably worse than nuclear war, so even someone who thinks his number is nonsense and argues for half a percent should not recommend a different course of action. You would need to be down around 0.01 percent before ignoring it becomes defensible. On what he personally does about it, he says he is educating himself, believes intervening ineffectively early spends social capital without moving the needle, and points to Eliezer Yudkowsky already banging the drum as the reason he does not feel the need to. His chosen contribution is figuring out how to thread the needle so AI still gets developed - including pushing the field to monitor fluid rather than benchmark intelligence, and pointing to ARC's Evals project as the sort of test worth building.

  • No point estimate - a 3 to 30 percent range, framed as a bid-ask spread
  • He still thinks it will probably be okay, and possibly really good
  • The magnitude argument: worse than nuclear war, so even 0.5 percent demands action
  • Ineffective early intervention spends social capital without moving the needle
  • He wants benchmarks for fluid intelligence, citing ARC's Evals project

true probability I believe is somewhere between 3 to 30%

Emmett Shear · 53:30

It's like probably worse than nuclear war.

Emmett Shear · 54:00
#ai-risk#probability#evals

contrarian· 2

contrarian03:30

Creativity Is A Faucet, And Most Adults Broke Theirs

Shear rejects the common view that creativity is a sacred state requiring the right conditions, and says he can simply write and keep generating ideas indefinitely. He argues this is not a gift he has but a damage he escaped: most five- and ten-year-olds can generate ideas or play pretend endlessly, and what people learn as they age is to stomp down the ideas that look bad and to avoid saying dumb things. The more pressure you put on yourself not to say something dumb, the more the inner idea generator gets disrupted. He describes the mechanism as a spiral - you do a thing, you get negative feedback that is more often internal than external, you do it less, you get worse, you do it even less - and says the same loop is what produces 'I'm bad at math' in people who could obviously do arithmetic. He also dismisses the standard brainstorming rule that there are no bad ideas: most ideas are bad and obviously wrong, and the real instruction is not to stop at them.

  • Shear says he does not need conditions to generate ideas - it works like a faucet
  • Children generate ideas indefinitely; adults learn to suppress the bad ones
  • Negative feedback, usually internal, starts a doing-less then worse-at-it spiral
  • 'No bad ideas' is false - most ideas are bad, the rule is don't stop at them
  • He suspects he had an unusually positively reinforcing environment for having ideas

Somebody said creativity is not like a faucet. You can't just turn it on.

Shaan Puri · 03:30

What you learn to do is you learn to stomp down the ideas that are like bad.

Emmett Shear · 05:00
#creativity#ideas#self-mastery
contrarian22:30

Stay Away From Trends - The Uber-Of-X Problem

Asked what trends he sees, Shear pushes back on the premise. The online-offline companies that started the trend - Uber, Airbnb, DoorDash - did very well, but they were doing something that was not allowed and had found an opportunity everyone else ignored. Almost all the companies started afterwards positioning themselves as the Uber or Airbnb of X did not do well. His conclusion is not that the category was bad, since it generated a bunch of incredible companies, but that jumping on the trend was probably bad for you specifically. He is careful to caveat any observation he then offers about where things are going, saying he does not mean trend in that sense.

  • The originators found an ignored opportunity and were doing something not allowed
  • Most Uber-of-X companies that followed did not do well
  • The category was good; joining it late was the mistake
  • Shear explicitly refuses to frame his own observations as a trend to chase

I would say like, stay away from trends.

Emmett Shear · 22:30

Jumping on the trend was probably bad for you.

Emmett Shear · 23:00
#startups#trends#strategy

idea· 2

idea23:30

The Consumer Internet Window Has Reopened

Shear's headline read on the moment is that the consumer is back. For the first time in maybe five to seven years, credibly starting a consumer internet company - the kind he was excited to start in 2007 - looks like a potentially good idea, and the reason is AI. His argument for why this matters more in consumer than B2B is structural: in B2B SaaS the experience is not the product, so reimagining the experience usually does not reopen a segment, because buyers care about what it does, the pricing model and adoption, and people are paid to jump through hoops. In consumer, the thing you are selling is the experience, so reimagining it reopens the segment 100%. He compares it to mobile, which reopened every segment as a question of what you could build now that you assume mobile exists. He is candid that nobody knows which bets pay - consumer is a bunch of lottery tickets, and photo sharing resolving into Instagram and Snapchat only looks blindingly obvious in retrospect.

  • First credible consumer internet window in five to seven years, driven by AI
  • In B2B SaaS the experience is not the product, so reimagining it rarely reopens a segment
  • In consumer the experience is the product, so reimagining it reopens the segment entirely
  • Directly analogous to what mobile did across every category
  • Nobody predicted Instagram and Snapchat in advance - consumer is lottery tickets

for the first time in maybe 5 to 7 years, it feels like credibly trying to start a consumer internet company

Emmett Shear · 23:30

In consumer, reimagining an experience 100% reopens the segment.

Emmett Shear · 24:00
#consumer#ai#startups#product
idea39:30

Generative Music Turns Everyone Into Rick Rubin

Shear separates two distinct AI opportunities: extracting meaning from media, which is what enables a video system of record, and creating media, which is the opposite move. On music specifically he is sceptical of near-term disruption. People do not want new music, they want the old music they already love and grew up with, and that cycle is what sustains record labels. He also thinks the output is simply not good enough yet and will take longer than people expect. His more interesting claim is about what changes for humans: generative tools de-skill the making of sounds and dramatically upskill the other vector - the fine judgment to say this song, not that song, and to give a musician exact feedback the way Rick Rubin does. Since it is easy to generate a thousand cuts but there are infinite cuts you could generate, the scarce skill becomes direction, discovery and shaping - and a different set of people will be optimal at it.

  • Understanding media and generating media are separate AI opportunities
  • Music is sticky against disruption - people want the music they already love
  • Shear thinks the quality is not there yet and will take longer than expected
  • AI de-skills sound creation and upskills curation and precise feedback
  • Infinite generatable cuts make direction, not production, the scarce skill

And I think that Rick Rubin's great success demonstrates why artists will still be important.

Emmett Shear · 39:30

AI is going to turn us all into Rick Rubins for for generative AI.

Emmett Shear · 40:00
#ai#music#creativity