You need to form your own opinion about AI

This is not pro-AI nor anti-AI. I don't want to tell you how to feel about it, everybody's already doing that. I've been keeping up with the AI heavily since last September, and it feels like most people who talk about it are rehashing a second-hand opinion without evaluating any of the core ideas themselves.

And I get it! It's a new, weird technology that people are saying is going to be your therapist, your boss, and your girlfriend. So it's very easy to hear an opinion from someone you generally agree with, then adopt it if it passes the sniff test. But the real picture is more confusing and unclear than I think anybody wants to accept.

There are many opinions that you can safely just inherit. I inherit lots of opinions about low-stakes choices without further examination e.g. products to buy, people to interact with. In our information-firehose world, you need to take shortcuts. I want to convince you here that AI is worth taking the time to form an opinion on that's really yours.

mental blocks

I need to address these immediately. A lot of people have mental blocks that keep them from critically thinking about AI on their own. I want to flag a few of these then try to ease the splinters out of your brain.

“It's scary to think about”

No matter what is scary about it, the flight response is inappropriate here. While AI may very well be cause for fear, conversations about AI are not going to jump out of the screen and bite you. This is a good fear to step into; it's better to be prepared.

“Interacting with AI is a moral vice”

I think this is a defensible position. But even if conflated, there is a major difference between using AI systems and thinking about AI systems. So many blog posts, conversations, and papers are freely available on the internet and you can always raise the topic in-person. The people who build AI regularly put out multi-hour discussions on how they're made. You can go watch some of these without paying OpenAI.

“It's so awesome!!!”

Lots of AI boosters will just mindlessly push it online. But just using it a lot doesn't mean your opinion on AI is self-formed. And if you mostly talk to people who also love AI and chide anybody who strays from total praise, you're not going to have a well-informed opinion. Avoid echo chambers.

“It's a fad”

Even if you think the technology will stall at its current capabilities, we're still far from figuring out how to integrate it properly into our systems. There's this idea that there are no AI shaped holes in the world, and I think that's true. Many companies are trying to just throw a chatbot at you, but they're going to keep iterating on these AI-interactions. We've seen all the ways companies try to quickly integrate AI systems, but over time we'll start to see integrations that took longer to plan and execute. Even if you believe it sucks, these companies are too invested already.

“I hate tech bros”

I largely feel the same. I get irrationally angry whenever I see people post janky-ass clips of AI-made-videos and say that "the movie industry is cooked." And even outside of the obvious grifters, I hate the culture that idolizes those who are "cracked." But if you look hard enough and block aggressively you can curate better vibes.

“It's too hard to catch up”

Just keep reading. I hope to be accessible to anybody. If you have questions, you can legit just text or email me.

basic facts about AI

I want this piece to be agnostic to most existing opinions about AI, but there are some baseline facts that are important to establish. The "AI" I'm referring to here refers to big, black-box models trained on tons of data. Mostly I'm talking about large language models (LLMs) like ChatGPT. Click the pink triangles to hear more specifics on each point.

AI substantially surpassed most well-informed expectations
1. You can look at the 2023 Expert Survey on Progress in AI to see that ChatGPT's release alone sped up timelines (compared to the 2022 survey) by over a decade on average.
People are spending a lot of money to try to continue our recent AI progress
Some people think it will be transformative on the scale of the industrial revolution
1. Many of these people would scoff at the above statement. Some would say the industrial revolution is an insanely high bar, others would say that's reductive of AI's potential.
2. Some of these people think that intelligence will continue to scale to the point of solving all of the world's problems.
Some of those people think this transformation could happen very quickly
1. A core idea here is that sufficiently intelligent AI can improve itself. If this starts happen there might be some fast takeoff where seemingly overnight capbilities have jumped without human input.
2. A recent popular, yet controversial, description of a scenario like this was AI 2027.
Nobody knows exactly how it works
1. While AI research is going a mile a minute, the same can't be said for interpretability.
2. Part of my sense of this is that while making models faster or smarter are pretty agreed upon endpoints, the aims of interpretability are less clear.
3. Regardless, these models are black boxes at their core; closer to a brain than an algorithm.

The "Some people think" claims are not endorsed, but the fact that they're commonly believed among relevant stakeholders heavily informs the conversation around AI.

information minefields

Everybody has an angle. Even academics with no industry ties, rationalist/postrationalist types, and lay commentators. You shouldn't be trying to find the "right" source of discussion around AI, you must seek out many different perspectives and try to synthesize them into a self-consistent opinion. I want to talk through some of the obvious groups who are talking about it and what I see as their angle.

people in AI

You can probably intuit that people who work in AI tend to be optimistic about AI progress and tend to greatly discount present and future risks due to AI.

One of the present risks of AI is that of taking people's jobs. If you listen to enough podcasts with AI researchers you will hear the same response between different researchers and CEOS: "AI also has the potential to create jobs!"

Yes, sure. I buy that. For example, Mark Zuckerberg was recently on a podcast and when asked about this issue gave an example about Meta currently seeing it infeasible to staff call centers, but in a human-supervising-AI situation might now find it profitable. I don't doubt that situations like this exist and that some jobs will get created by AI. But nobody who asks this question earnestly thinks that AI won't create some jobs; it obviously created a lot of AI research jobs. It's the net impact that people are anxious about. And no AI leader has provided a clear picture of how AI can be as powerful as they're saying without it being similarly disruptive.

In the same interview when asked about whether personalized content and excess time spent on social media could have any negative effects, Zuck brushes it aside and says that he trusts people know their own tastes pretty well.

I'm not marking these people as bad sources, but optimism on capability and discounting of risk is what you have to watch out for. However, they're also some of the most important people to hear from! Most frontier AI labs are pretty dodgy about internal details. So anything they say gives us a bit more information about how the leading systems work. Also, it's important to have some model in your head of how the people in charge of the leading AI labs are thinking about the problems.

public thinkers

This class of person is kind of vague. Their job title is usually something in the intellectual sphere. They make a lot of blog posts, tweets, and salivate at the concept of getting asked to be on a podcast. Luckily, these public thinkers are on all side of the aisle regarding AI capabilities, future capabilities, and the utility of AI. However, this group tends to be overconfident in their claims is also biased away from simple explanations.

Their value as public thinkers is to provide new perspectives, which can be very helpful to the conversation around this technology! But they are incentivized to sound slick rather than be correct. So you get these statements that sound good when confined to 140 characters but don't hold up to so much scrutiny.

NFT-esque shills

These guys annoy me the most, and if you have any sort of bullshit detector then you're probably already avoiding them. These are the guys who take some AI advancement out of context and say that a whole class of jobs is now obsoleted. I think a good portion of these people literally did jump ship from shilling NFTs to making these inflationary posts on Twitter. I hope they're jumping to memecoins.

In any case, be wary of anybody making extremely overconfident claims about AI based on a single output that AI produced. These systems are probabilistic and very context-sensitive such that no one output can say much. Even so, none of these people ever give a real argument for why the janky Burger King ad they made is destroying the advertising industry.

people in creative work

It's fair that people in creative work feel chafed by AI. These models have been trained on intellectual property without time for society to litigate these issues, and the hunger for data to train the next generation of models is keeping this out of the conversation for the most part. But I think that well-earned frustration gives some creative people a cognitive blind spot when it comes to AI.

A common response from an artist to an AI generated image is to say that it sucks for some reason. Often, I agree with these reasons. But the same types of comment were being made a year ago when fingers were a huge issue for image generation. Any specific issue they point out with AI is likely to be solved as we feed more data and compute to the shoggoth. I'm sympathetic to the creative voice here, so I wish more people pointed out the very real issues rather than wasting their energy nitpicking stuff the shills post.

So this group's statements typically understate AI's capability and misplace (not necessarily overstate) risks.

why it's important to have an opinion

(Expand via pink triangle)

You can argue with people without sounding dumb
1. There's this predictable problem where people with distaste for AI often have some key misunderstandings about the technology.
2. It makes sense! You don't like it so you don't use it so you don't know how it works. I don't spend a lot of time studying my enemies.
3. But whether it's in conversation or not, people's opinions about AI will work their way into your life and it's much better to have an internal opinion you can defend with your own values than trying to patch together some opinions that an ally had.
You'll know if it's coming for your job
1. Whether it literally takes your job title or shifts your employer's incentives to no longer have a need for you, most people should be keeping a tab on this. It is not sufficient to believe that your job can't be done by AI to feel confident that the effects of AI won't propagate into making your job undesired.
You need to know what makes humans special
1. AI can do a lot of things today that only humans could do 5 years ago
2. I don't think that inherently devalues what we do, but social or economic notions of value will certainly shift. This will naturally change what I value
  1. Regardless of whether you think AI writing a shitty poem directly affects the value you place in your own ability to do that, it will change society's perception of that activity which should change the way that you think about these activities
3. Keeping a tab on what AI is good and bad at in the short-term keeps you from facing some social shock
It's going to be everywhere
1. It doesn't matter if you think it should be; it will be
People you know will become dependent on it
1. It might be a family member, it might be a friend, it might be an intern that you're supervising
  1. Forming an opinion on it informs whether you steer people to (1) not become dependent, (2) use it even more, or (3) just use it differently
2. If they don't tell you, then it might be hard to know that they're dependent on this technology without knowing what the technology is like, a necessary prereq to forming an opinion
It can do really bad things
It can do really good things

starter pack

If you're sufficiently convinced that it's a good idea to form your own opinion on these things, here's my proposed starter pack for "knowing about AI stuff." There're lots of starter packs to understand the math and engineering behind LLMs, but there are few comprehensive resources I've found that get you up to speed on the conversation surrounding AI.

background on the AI scene

A confusing thing that leads people to downplay the capability of AI systems is that most labs are not focusing on making the best possible product for users right now. Rather, most of the past year has been characterized by AI labs putting out bigger models that do better on academic benchmarks. This is probably best explained by the fact that high-level goals in AI have largely been around pursuing AGI (artificial general intelligence) or ASI (artificial super-intelligence). The former basically refers to AI that is intellectually at the level of humans, and the latter refers to AI that far exceeds our capabilities. You might be hoping for a clearer definition, but there really isn't one. I don't put a lot of stock into thinking about whether or not we have AGI or ASI, but it's this nebulous and always-out-of-reach goal that frontier labs seem to be aiming at. It feels always-out-of-reach because conversations around AGI or ASI often have the flavor that "once we get there, everything is solved." And obviously it's not all solved yet.

One of the recent major developments in AI models is the advent of reasoning models. These models "think before they speak" and can do much better in standardized-test type settings than their predecessors. Right now, many major AI labs are pouring money into scaling up reasoning-like training and it's likely that this will continue to be a major source of improvements for at least another year.

A nice feature of the AI scene right now is that you can count the major players on both hands (an AI generated person from 2023 could count them on one hand!). By major players I mean labs that are training LLMs aiming to be at the frontier.

OpenAI
1. The incumbent. They created ChatGPT and continue to have the premiere models.
2. My biggest worry following their recent "CEO of Applications" hire is that they start hyper optimizing on retention-type metrics and become the TikTok of AI.
Google
1. Considered by many to have zoomed to at least second place in recent times. Their line of LLMs are called Gemini, and their flagship Gemini 2.5 Pro matches OpenAI's models' performances for many tasks. Nothing flashy, just solid, reliable models.
2. It's worth noting that Google acquired DeepMind in 2014, a UK-based AI lab that made (and maybe continues to make) the most innovative non-chatbot AI technology.
Anthropic
1. The former obvious number two, founded by former OpenAI people. For a while, their model Claude was seen as the tasteful LLM choice. Lots of people still feel that way and cite Claude's personality as being far superior to ChatGPT. They haven't been as lightning fast with consumer-facing releases as OpenAI and seem to really be focusing on making models good at coding right now.
2. It's unclear how much this holds up, but for a long time they were the lab that cared about safety. They still do put out lots of important interpretability-type research.
DeepSeek
1. I bet you heard of the "DeepSeek moment" earlier this year. DeepSeek-R1 dropped as the first non-US model to achieve performance at or near the frontier. They are notably the first company on this list to release their model weights, meaning you can run DeepSeek-R1 (more practically, a weaker version of it) on your own computer without paying DeepSeek anything.
2. Many are awaiting their R2 model to see if they continue to be at the frontier. DeepSeek is the de facto representative of Chinese AI progress, which many people view as an extremely geopolitically important detail of this whole thing.
xAI
1. Elon Musk's AI company. They got to the frontier because Elon built the biggest datacenter in the world extremely quickly. Their flagship model, Grok, is marketed as being pro-free speech and less restrictive than other models.
2. I would consider them the lowest innovation lab, but based on raw firepower and distribution via Twitter, they can't be discounted as unimportant
Meta
1. Meta is notable for releasing all of their Llama models' weights. They don't really perform at the frontier in most things, but lots of community projects are built on Llama models.
2. It seems like a mess over there though. A bunch of the team for their recent Llama 4 release resigned, and they continue to delay important releases.
Alibaba
1. Alibaba releases the Qwen series of models, some of the best open-weight models right now. Similar to Llama, many community or research projects are based on Qwen models. They're the major non-DeepSeek AI player coming out of China.

There are other labs on the way! Over the course of OpenAI's history a bunch of people have left, and some of those people go on to start foundation labs. Anthropic is one example of this, and two other major ones are SSI (led by Ilya Sutskever, former Chief Scientist at OpenAI) and Thinking Machines (led by Mira Murati, former CTO of OpenAI). Those two are the stern balding man and woman in the photo at the top. People are keeping an eye out on these two, which are already valued near $10B and $2B respectively with no products released.

A key idea surrounding AI right now is scaling. There are these scaling hypotheses that generally state that we will continue to see improvements in AI capabilities as our models get bigger and bigger. For the models to get bigger and bigger, they definitely need more compute and they likely need more data. They've already been trained on pretty much the whole internet, so AI companies are paying top dollar for experts to create more data for them as well as trying to turn over every last stone (scanning never before scanned books etc.). For companies to get more compute, they need access to big datacenter with lots of GPUs.

GPUs are the computer chips that you need to run video games with good graphics. In the past decade or so these chips have been leveraged to do certain math operations in really efficient ways, so that's what we use to train AI models. As a result of the AI boom, NVIDIA was the wealthiest company in the world for some time. While NVIDIA designs GPUs, the chips themselves are largely made in Taiwan by TSMC. This complicates the geopolitical conversation surrounding AI especially considering the role of China.

The talk of the town right now is "Agents." It's hard to exactly define it, and I think the word "Agent" obscures the underlying concept to the point of muddling public understanding, but it's the word everyone's using. Simply put, "AI that goes out and does stuff on its own," acting agentically. OpenAI just released their software engineering agent `codex` that can go out and edit, run, and create code in many instances in parallel. Their $200 tier gets you their `Operator` agent that can browse its own computer window. Anthropic released "Claude Plays Pokemon" to demonstrate their model's ability to navigate a video game world. But even tools like OpenAI's Deep Research, which goes out and produces a report for you after doing some extensive searches can be considered an agent.

good AI content

My favorite source for in-depth conversations around AI is The Dwarkesh Podcast. Dwarkesh hosts many AI researchers, lab leaders, and public thinkers for multi-hour in-depth conversations. If you know who Lex Fridman is, he's the actually good version of that.

Dwarkesh, like everyone, has his own leanings and agenda. In podcasts he mostly takes a passive role, but is clearly very pro-AI proliferation and seems to believe that a fast takeoff of AI capabilities is possible. Some of my favorite interviews from him are:

If you can tolerate being kind of confused but slowly picking things up, listening to these interviews then Googling (or asking ChatGPT) things I was unclear about is pretty much how I got up to speed. I also follow a lot of AI-related accounts on Twitter. These are hit or miss but provide a bit more of a fine-grained and quickly updating look at what's happening in AI. Some hits are:

I think the right balance to strike between noisy tweets and infrequent podcasts is probably found by reading Substack. There's a lot of very thoughtful blogs that cover AI developments:

If you want to understand how the models actually work, there are a lot of resources for that:

It's moving too fast for any good textbook to exist (other than Nathan Lambert's RLHF book)

questions you develop opinions about

Depending on your depth/breadth preferences, expand at will.

Should I use AI in my daily life?
1. If so, for what?
  1. Should I use it in the same ways I would use Google?
  2. Should I use it for administrative tasks like drafting emails?
  3. Should I use it for harder intellectual tasks required by my job?
  4. Should I use it casually to think through ideas?
  5. Should I use it for life advice?
2. If so, which one (ChatGPT, Claude, Gemini, Grok)?
  1. Should I pay for a subscription?
    1. If so, should I pay for the extremely expensive plans from OpenAI or Anthropic (~$200/mo) or the normal plans that they have (~$20/mo)?
    2. If not, should I just use their free tiers?
    3. Otherwise, should I download LMStudio and use some weaker LLMs that can run locally on my computer?
3. If not, why not?
  1. Should I avoid paying these companies because they're bad?
  2. Should I avoid giving any of my personal data to model companies to be used in their next round of training?
  3. Should I avoid becoming reliant on these tools?
Will AI be able to do my job?
1. If so, when will that happen?
  1. Maybe more importantly, when will the suits at my company realize this?
  2. Can I pivot before then to an AI-safe job?
    1. Will there be any AI-safe jobs?
2. If that happens, will my employer still keep me hired?
3. If not, what is the fundamental part of what I do that can't be replaced by AI?
  1. Is it fundamental to the task, or will human norms never let an AI do this job?
What role do governments play in the AI story?
1. Should US government regulate AI labs in America?
  1. If not why not?
    1. Do you trust AI labs to self-regulate?
    2. Do you think the government would be inefficient in doing so?
    3. You don't think there's anything too dangerous that could come out of advancing AI capabilities?
  2. If so, how?
    1. What forms of regulation make sense to you?
    2. What political party do you think will get it right?
    3. If AI capabilities advance rapidly, should the US government nationalize labs?
2. Is America vs. China an important part of the AI story?
  1. How far behind (or ahead) do you think Chinese AI is compared to American AI?
    1. Do you think DeepSeek-R2 will have comparable capability to whatever OpenAI's best model will be at the time of its release?
  2. If China vastly surpasses the US, do you think that's an issue?
3. Do you think AI will have important military uses?
Are safety concerns due to AI an important thing to worry about right now?
1. Do we face existential risk due to rapidly advancing AI?
  1. If so, what should we do about it today?
  2. If not, is it because models will not advance that far, or because our likely safety measures are sufficient?
2. Do we face social risk due to rapidly advancing AI?
  1. Will it be used to deceive people?
  2. Will it be used to steal intellectual property?
  3. Will traditional education systems crumble due to AI?
  4. Will people become too reliant on AI, WALL-E style?
  5. Is it already doing any of these things?
3. What other kind of risks might AI pose?
  1. Are these risks likely to manifest?
  2. Is anybody talking about them?

my opinions

I've been super fascinated in this stuff for a while. When ChatGPT first came out, I remember playing with it for hours because it was so novel to get a robot that responds to anything you say. But then I kind of forgot about it for two years.

In September 2024 when OpenAI released ChatGPT o1-preview I was pretty amazed at its ability to seemingly "reason." I started reading up on the scene and trying to really understand the underlying tech. It was convenient timing because I enrolled in an LLM course that semester. So I started really becoming invested in the story behind how this all came to be and where it was going. The enthusiasm and intrigue I started to feel at this time came in large part from the thought that nobody knows exactly where it's going. I hope to instill some of that in whoever's reading, even if spurred on by more negative feelings toward the whole enterprise.

I use ChatGPT a lot (and Gemini sometimes).

It replaced close to 90% of my Google searches. I would say relatively surface-level "information seeking" is probably 50% of my usage. If you haven't used it in a while, the web-search capabilities make it pretty much better than Google, and certainly better than Google AI overview. And all the models can read images, so it's pretty common for me to just screenshot and ask it something I'm curious about. I think the thing that makes ChatGPT a better interface for this kind of thing is the ability to always get an answer to the exact question you're asking rather than a somewhat relevant website.

The line is blurred, but another 35% of my usage comes from open-ended speculation that I wouldn't have done otherwise. Having someone to join me on any thought-rabbit-hole has made me a lot more curious. Often it's about my future, "what would it take to pivot to X industry?" -> "give me a 5 year plan for what that would look like" -> "would i likely find success?" -> "what is work/life balance like for people who do X?" I say the line is blurred between this and Googling because some of these rabbit holes have a flavor closer to "what exactly is it that lawyers do?" or "how hard would it be for me to rig up a raspberry pi to do a cool project?" The ability to develop these thoughts over one conversation feels entirely different than what would be Wikipedia rabbit holes in the past. It really is an approximation to talking to an expert.

Maybe I suck at it, but I would say only 10% of my ChatGPT usage is producing code. It does greatly speed up what used to be (Google -> Stack OverFlow), and Copilot tabbing speeds up the boiler plate, but it's not really one-shotting files. I just don't feel like it's good enough at understanding the purpose of code to be faster than me in a project I'm familiar with. It's pretty much only helpful to make code in languages I'm unfamiliar with. But since I want to know how my code works, I don't really trust just pasting it in and moving on. As such, the best use I've found is for web development type stuff where what you see is what you get.

5% is me testing whether or not ChatGPT could solve some problem, typically when a new model comes out.

Despite paying $20/mo for ChatGPT Plus and using it pretty often, I'm far less convinced about its moral sanctity than most people I see discussing AI. I'm really worried about the social issues it raises. I'm pretty sketched out about the non-consensual aspect by which all of society's conversations, thoughts, and expression on the internet were just vacuumed up into some guy Rufus who Amazon wants to sell you Protein Powder. I think the way that modern AI labs continue to scale without, in my opinion, a meaningful commitment to AI safety is pretty horrendous. But it's too good. And consequentialist arguments that my usage has no marginal effect on anything are sufficiently convincing to keep using it.

I do think that it has the potential to bring lots of good into the world by empowering people to do things they couldn't before and distribute personalized information that would have been previously price-locked. It may sound corny, but it does make me feel like I can do anything. Not because it does those things for me, but because it's been probably the single most transformative learning tool I've ever used. That's not because it unlocks doors that were previously locked; a motivated person can learn just as much just as fast as I can with ChatGPT. But effort is limited, and traditional forms of learning face so many "blocker" moments. I've got a little bit of Gen-Z brain. A break in the flow of trying to answer some question just takes me out of it (e.g. looking for some theorem online but I can only find a weaker version). But that's not true anymore! I can always just ask for follow-ups till I'm satisfied. The biggest difficulty is in knowing which moments are "blocker" moments and which ones are opportunities for growth at the hand of unassisted struggle.

I'm pretty unsure about timelines for when I think important milestones will happen. The Singularity happening this year seems on par plausibility-wise with AI's 40-year impact being less than the internet. I think my actions post-AI-obsession are relatively consistent with this? I dropped out of grad school and started working in an AI-safety adjacent role, but the relation between these choices and the AI revolution is mostly post-hoc identified.

I think academic jobs like my own will get largely downsized due to AI but not replaced wholesale. Someone needs to take accountability for the veracity of statistical analysis, even if in the future it looks more like checking an AI's work than doing it all on your own. The slowness with which academia adopts things gives me comfort that I'm safe for some time.

I do think there is a place for sensible AI regulation, but I don't think I trust the federal government to come up with it. I naively hope that enough high-influence policy people become fascinated by the technology and dig into it at a deeper level. The US-China thing is a bit unclear to me. I'm skeptical that there will be an intelligence explosion/fast takeoff, so I think capabilities are likely to be in relatively parity for the foreseeable future.

I'm really interested to know what all of you think. I wrote all of this in hopes of identifying friends as or converting others into "people who think a lot about AI." Even if you're interested in parts that I didn't talk about here (for example the actual math behind it!), let me know. Among those in this space, it's common to think "why isn't this way more talked about?" While in net I'm certainly positive-feeling toward AI, I think the conversation around it would get so much better if more diverse thinkers joined the conversation. I don't care what your opinion is, but you should form your own. And once you've done that, let's talk.