The AI Boom Inside Silicon Valley Start

The California Issue

In Silicon Valley's hacker houses, the latest crop of young entrepreneurs is partying, innovating — and hoping not to get crushed by the big guys.

Emily Liu (center) and Dave Fontenot, two co-founders of the start-up accelerator HF0, with Marylin Ma, a member of its latest batch of fellows, in San Francisco.Credit...Laura Morton for The New York Times

Supported by

Send any friend a story

As a subscriber, you have 10 gift articles to give each month. Anyone can read what you share.

By Yiren Lu

The archbishop's mansion in San Francisco, built in 1904, is now a stately hotel at the northeast corner of Alamo Square Park. Since February, it has been rented out entirely to HF0, or Hacker Fellowship Zero, a start-up accelerator that provides 12-week residencies for batches of fellows from 10 different start-ups. Their experience, which culminates in a demonstration day, is supposed to be the most productive three months of the fellows’ lives. Dave Fontenot, one of HF0's founders, was inspired by the two years he spent living in monasteries in his 20s: While monastery life was materially ascetic, he found that it was luxurious in the freedom it gave residents to focus on the things that really mattered. And at the Archbishop's Mansion this year, almost everyone has been monastically focused on what has become San Francisco's newest religion: artificial intelligence.

For more audio journalism and storytelling, download New York Times Audio, a new iOS app available for news subscribers.

The A.I. gospel had not yet spread in 2021, when Fontenot and his two co-founders, Emily Liu and Evan Stites-Clayton, started the accelerator. Even a year ago, when HF0 hosted a batch of fellows at a hotel in Miami, six out of the eight companies represented were cryptocurrency start-ups. But at the mansion in San Francisco, eight of the 10 companies in HF0's first batch this year were working on A.I.-based apps, and the lone crypto start-up — focused on what happens to your Bitcoin when you die — was worried, they told me, about whether the investors who showed up at this spring's demo day would actually want to invest in them.

That generative A.I. has largely supplanted crypto in the eyes of founders and venture capitalists alike is not exactly surprising. When OpenAI released ChatGPT late last year, it sparked a new craze at a time when the collapsing crypto and tech markets had left many investors and would-be entrepreneurs adrift, unsure of where to put their capital and time. Suddenly users everywhere were realizing that A.I. could now respond to verbal queries with a startling degree of humanlike fluency. "Large language models have been around for a long time, but their uses were limited," says Robert Nishihara, a co-founder of Anyscale, a start-up for machine-learning infrastructure. "But there's a threshold where they become dramatically more useful, and I think now it's crossed that."

One appeal of generative A.I. is that it offers something for every would-be entrepreneur. For the technically minded, there is research to be done. For the business types, it's easy to create applications on top of the OpenAI platforms. For the philosophically inclined, A.I. offers interesting avenues through which to explore what it means to be conscious and human. And unlike crypto, especially now, A.I. is a more credible field to be in for mainstream techies. Its products have already achieved significant traction among consumers — ChatGPT is believed to be the fastest app ever to hit 100 million users — and some of the figures at its forefront are familiar faces, now in their second acts, like Sam Altman, formerly the president of the start-up accelerator Y Combinator, and Greg Brockman, formerly the chief technology officer at Stripe, the payments-processing company. In short, you can't help thinking that, as one friend recently proclaimed to me, "Everyone in S.F. is either starting or running an A.I. company or starting or running an A.I. fund."

A.I., in turn, seems to love San Francisco back. During the pandemic, as tech workers went remote and Twitter pundits evangelized the tax benefits of being in Austin or Miami, the San Francisco area seemed poised to cede its start-up primacy. But recently that trend has reversed. There's a sense that if you want to be working in A.I., this is still the place to be. "We actually first considered doing the batch in New York, but when I went to New York and asked people what they thought of GitHub Copilot" — an A.I.-powered coding assistant — "people told me they maybe tried it once," Fontenot said. "On the other hand, people in S.F. told me they were using it to write 50 percent of their code."

Fontenot's anecdote gets at one of Silicon Valley's enduring qualities: the willingness, even eagerness, to embrace new technology. Out in the rest of the world, A.I. is triggering nerves — fears and even predictions of wiped-out jobs, of existential doom — and endless commentary. In San Francisco, it has kindled all these things too, but also a question just as powerful: How do you get a piece of it?

During the day, the Archbishop's Mansion often feels surprisingly empty and quiet, perhaps because it's so large. There are four floors and a grand staircase that winds up through the middle of the building, lit by a giant skylight. Many of the teams work in their rooms upstairs; some teams work in the "hackspace" in the basement, with its whiteboards and rows of standing desks. When I was visiting this spring, one wall displayed some ChatGPT-generated poetry: "In HF0, the hackers work and play/With laughter and fun, throughout the day./They’re a community of techies, with a heart of gold,/And their humor and hacking skills, never grow old."

But on a spring Friday night — the one night of the week when the broader tech community is welcomed in — the A.I. party was in full swing. Fontenot and Liu bounded around the common spaces at the front of the mansion, giving out effusive hellos and introductions.

In one back room, a bar served elixirs. (The mansion is a no-alcohol zone.) In another, an A.I. rap battle raged. (It wasn't much of a battle, actually — while the A.I. is no Eminem, it was still destroying everyone.) In a third, Jonathan Shobrook, a fellow, was demoing his product, Adrenaline, a tool that lets you ask natural-language questions of your code base. He had the interface up on a monitor and a small cluster of spectators around him, seemingly riveted.

"Can you ask it to implement ReLU?" Sasha Sheng asked. Sheng, a former software engineer at Facebook, is now working on her own app; in dyed pigtails and a baseball cap, she is something of a personality in the community.

"Oh, yeah, that's a hard question," Shobrook responded. On his keyboard, he typed, "Which neural networks use ReLU?"

The right answer flashed on the screen, the cursor blinking as characters appeared. Someone asked how it worked.

"I just basically chunked up all the files into functions and classes and groups of code, generated summaries of those code chunks and then recursively summarized the file," Shobrook said.

"Do you use an abstract syntax tree?"

Only in San Francisco would people be talking about abstract syntax trees at 9 p.m. on a Friday night.

Out in the main entryway, someone introduced himself as Bruno. I asked him whether he was in A.I. "My first two companies were in A.I., but now I’m in crypto," he said jovially. Fontenot came up from behind and slung an arm over his shoulders. "I’m not popular anymore, no one wants to talk to me," Bruno fake-moaned. But he didn't seem deeply bothered by crypto's abrupt comedown and A.I.'s ascendance. Bruno, it turned out, was Bruno Faviero, a well-known investor and entrepreneur. He and Fontenot have been buddies since they met as college students, when each of them was organizing hackathons. After Fontenot left school in 2013 — he dropped out of the University of Michigan, where he was studying computer science — he continued to run hackathons and networked in the tech world as Faviero built his first company.

"Four years ago, he called me to say that he was raising a fund," Faviero told me. "I was like, ‘Yeah, whatever, everyone is raising a fund.’ A week later, he calls me and is like, ‘Hey, the fund is oversubscribed, do you still want to put in a check?’ If Dave says he's going to do something, he does it."

Or, as Emily Liu put it to me, "You show up to one of Dave's things as a friend, and 10 minutes later you’re wearing a staff shirt."

Fontenot is charismatic, a forceful speaker with wild hair. Like all good venture capitalists — he is a general partner in the investment firm Backend Capital — he has an unerring nose for the thing of the moment, be it blockchain or A.I. That he seems agnostic about whether it's blockchain or A.I., or some other underlying technology, is almost beside the point. In many ways, he personifies the modern Silicon Valley dichotomy between spirituality and hustle, between monasticism and flamboyance. His expertise, he believes, is people.

"We look for three things — grit, storytelling ability and product sense," he said, describing the selection process for the fellows. Notably absent from this list, I pointed out, was a background in machine learning. Fontenot shrugged. This generation of start-ups doesn't have to come up with its own cutting-edge research. Big companies like OpenAI and Google will provide that. Instead, he said, the fellows need the ability to build prototypes quickly on top of the new models.

And, in fact, the unifying thread among the first batch of 2023 fellows was their experience at that sort of enterprise. The average age was 28 (Fontenot is 30), and several of them were second-time founders. Adam Reis is a founder of Candid Health, which provides medical-billing software. Emma Salinas founded an online community called Gen Z Mafia. While various fellows often talk about how they’ve long been interested in A.I., it's clear that some of the "why now" is opportunistic.

But if the people at hackathons and programs like HF0 tend to be newcomers to A.I., this doesn't preclude their success: The consensus is that building things in the A.I. field isn't as complex as working in biology, say; you don't need to get a separate Ph.D. in it. If you’re already good at math and good at engineering and good at business, there are few limits to what you can do.

A few themes characterize the sorts of projects the HF0 fellows have been working on. On the one hand, there are applications to automate tedious business tasks like copywriting or spreadsheet wrangling. A company called Fileread falls into this category. Its law-firm customers upload all the documents relevant to a particular case into an online portal; Fileread indexes those documents into a special database that enables users to search the documents not only for exact terms like "truck" or "James," but also for broader questions like "who made the transaction?" or "what are the relevant cases?" Under the hood, Fileread first fetches the most relevant documents from its database, then adds those documents to a user's question and sends the whole, long query to the OpenAI application programming interface, or A.P.I. Fileread then spits out an answer, powered by the same large language models behind ChatGPT.

Without A.I., identifying and crafting a legal narrative by piecing together textual evidence from thousands of sources is a painstakingly manual process. Most of Fileread's customers specialize in business litigation, including antitrust and liability cases. Sometimes they are paid on contingency, which means when they succeed, they typically get a percentage of the award or settlement, but when they lose, they get nothing. Firms need the A.I. to efficiently search for evidence in the documents that might, for example, either establish or refute liability. "They don't have the manpower or the budget to do unlimited document review," says Chan Koh, a Fileread founder and an HF0 fellow. "They want to spend the minimal amount of effort in order to win the case."

Other HF0 fellows have been creating applications that lean into A.I.'s seemingly human affect in order to tackle some psychological need. For instance, Brian Basham, who has worked in Google's Brain division and since 2018 has been a life coach in California, is working on Thyself, a subscription service for "guided emotional inquiry" that currently uses A.I. and human coaches but will eventually transition fully to A.I. I met him and his employee Maverick Kuhn over dinner one night at the Archbishop's Mansion. After Kuhn waxed rhapsodic about a four-week-long retreat he attended last summer, called Sleepawake, I asked him whether the experience would have been as great if the facilitators had said and done all the same things but been A.I.'s. "Probably not," he conceded. "That would very much be a disembodied head."

One-on-one life coaching from the current human-A.I. hybrid version of Thyself costs $50 an hour. Once the service is fully automated, Basham expects to be able to offer unlimited sessions for $30 a month. At that price, he believes, it would be broadly accessible.

A few days later, I did my first Thyself session. Mostly it consisted of the bot asking me to visualize scenarios — remembered or imagined scenes — and then to describe the physical sensations and emotions that resulted from "surfing the emotional wave." I didn't feel much of an emotional wave, but I was impressed by how natural it felt to speak to an A.I.-filtered guide. Compared with calling, say, the automated hotline for a cell-service provider, it was a vast improvement, although it did tend to talk over me.

Evan Stites-Clayton, an HF0 founder (and a fellow in the accelerator's inaugural batch), has come up with a similarly intimate product, an A.I. assistant called Consort. To try it out, I had to go through a 15-minute quasi-therapy session, where I was asked about my childhood, my relationship to my parents and my favorite books. A few hours after my responses were fed into the A.I., Stites-Clayton — who was a founder of Teespring (now Spring), a platform that sells custom-made T-shirts and other merchandise — sent me a link to my "consort," which I could then text. Over the course of the next couple of days, it texted me a daily message at midnight, reminding me to wind down for the night. On the weekends, it asked me whether I was planning to go out. The texts included appropriately casual spelling and (lack of) punctuation. I found myself warming to it, despite an earlier prejudice against becoming friends with A.I.

A.I. and emotional regulation might seem like an odd juxtaposition, but it makes sense that emotional labor — at the end of the day, just another form of labor — could be one of the first job categories to be transformed by automation. And yet, setting aside its effectiveness, there's something odd about using A.I. to manage our human brains when it's not clear that the A.I. brain is at all similar to ours. "We’re obviously trying to anthropomorphize A.I., make it in our image," said Matthew Rastovac, the founder of Respell, a tool that lets you create A.I. apps without doing any coding. "Because we don't really know how else to build and understand a new kind of intelligence. But I think it's much more likely that it's going to be like a reptile, in that it has its instincts, but we can't understand what's going on inside its brain and listen to its actual thoughts." We were sitting on the roof of Atmosphere, a hacker house in Nob Hill that he helped found; all around us, San Francisco was enchanting in the afternoon light. Earlier, he paraphrased for me some lines that he liked from Season 2 of "Westworld" that spoke to how early we still are, and how blinkered, when it comes to understanding this technology: "Sanity is a very narrow sliver of the possibilities of mind. Because we have culturally accepted norms, we have a certain way of acting and thinking and speaking, and if you deviate from that a little too much, then you’re, at best, weird, and at worst, clinically insane."

During the week I spent staying at HF0, everyone told me I had to make it down to the South Bay for the A.G.I. House GPT-4 hackathon. The organizers asked me to come after 6 p.m., though, so as to not distract the hackers before it started.

A.G.I. stands for artificial general intelligence, a phrase that has come to represent a potential dream goal for A.I.: a machine intelligence with the flexibility to handle any intellectual task that humans can. A.G.I. House, it turns out, is a $68 million mansion in the small town of Hillsborough, 25 minutes from downtown Palo Alto. The mansion has a long thoroughfare of ferns out front, a pool and a barbecue pit in the back. Rocky Yu, previously the chief executive of an augmented-reality start-up, runs A.G.I. House, overseeing both its 10 residents and a raft of community events. He is warm and smiley and exceedingly well connected in the local A.I. community.

The crowd at that night's GPT-4 hackathon was so large as to render the Wi-Fi basically nonfunctional. Every room overflowed with hackers crowding around whiteboards. In the kitchen, Chinese takeout was laid out on a table. A smattering of investors were present to check out the demos, which started at 8 p.m. with short speeches from the organizers. The speeches were all variations on a theme: We are living in a momentous time. Maybe in a few decades from now, we’ll look back at all these seminal A.I. achievements and see that they all came from this house in Hillsborough.

As at HF0, the demos here alternated between business uses and personal applications — a chatbot that impersonates business gurus like Mark Cuban, the owner of the Dallas Mavericks basketball team and a judge on "Shark Tank," the business-reality TV show, and that allows you to ask for business advice; or an A.I. sommelier that will take your dinner menu and suggest an appropriate wine pairing. Six months ago, any one of these projects might have seemed remarkable, but the arrival of ChatGPT has remade expectations. "The one pattern I’m starting to see is that ChatGPT is the killer app," the technologist Diego Basch has written on Twitter. "None of the tools built on top of the A.P.I. have been as useful to me." Indeed, if you are building something on top of OpenAI's A.P.I., it does seem as though your app's marginal value has to be extremely high if it is to avoid being bulldozed by either OpenAI itself or one of the big tech companies like Google and Microsoft (or even later-stage start-ups that are rapidly rolling out A.I.-enabled features in their products).

As two analysts at N.E.A., an investment firm, put it in a recent report, generative A.I. may not be as disruptive to established businesses, and beneficial to start-ups, as previous big shifts in tech platforms. "Unlike with the prior shifts, incumbents do not need to re-architect their entire products to adopt this new platform shift," the analysts wrote. "In addition, this shift favors companies with bigger, proprietary data sets which can give an edge to more established companies."

During previous tech eras, start-ups could introduce a superior technology or interface and then race to build market share before entrenched competitors could match them. But with large language models, incumbents like Google and Microsoft have had a huge head start in both developing the technology and acquiring market share among consumers. The situation risks becoming like that of the pharmaceutical industry, in which research and development is outsourced to start-ups and many of the benefits ultimately accrue to the parent company. Moreover, the capital-intensive nature of training large language models means that smaller companies like OpenAI and Anthropic creating their own large language models have few alternatives beyond making Faustian "partnerships" with tech giants.

This doesn't mean that generative A.I. isn't going to transform industries or eliminate jobs. Beyond the incumbents, one beneficiary might well be the indie hacker, the kind of coder for hire who does niche A.I. projects to fill specific needs in specific industries. Problems that have been too esoteric to solve or work flows that have been too complicated to improve might become easily automated with the help of ChatGPT. As the Gumroad founder Sahil Lavingia recently put it on a podcast, "If I had a friend who's like, ‘I want to make 200K a year, building something in S.A.S.’" — software as a service — " ‘building some A.I. tool,’ I would basically tell them to walk around their neighborhood, go to as many businesses as possible and see what manual thing, what piece of paper you sign, and figure out how to automate that."

That the era of V.C.-backed, outsize returns might be coming to an end is, of course, a source of anxiety. Everyone in Silicon Valley knows someone who worked at the last generation of successful start-ups, start-ups whose growth followed the proverbial hockey-stick line on a graph, and reaped the benefits. Who doesn't want to get their share — whether money, status, fame — before it all runs out? This anxiety is compounded by the fact that it feels as if the deterioration of the physical world is happening roughly at a pace with the flourishing of the virtual one, and the only way to insulate yourself is by achieving vast financial success. In some ways, San Francisco perfectly embodies this tension: A.I. is moving toward A.G.I., but outside the high-end tech offices, there is rampant homelessness, house prices are so high that even couples with two tech incomes can't afford to buy property and children are sufficiently rare as to make them a spectacle.

But the anxiety runs deeper than concerns about success, prestige or even material safety. Change has always been accompanied by hand-wringing, and the pace of change in A.I. right now is mind-bending. The resulting mood is perhaps best encapsulated by tweets from Tiago Forte, the productivity guru known for a self-help system called Second Brain. "I’m feeling a broad loss of motivation for many projects and goals that used to excite me due to what I’m seeing with A.I.," Forte posted in April. "It's not fear of A.I. apocalypse or fear that I’ll lose my job or anything like that. ...More like a feeling of grief that many of the personal skills & qualities I’ve spent a lot of time developing have suddenly been devalued."

This, of course, is not a new kind of ennui. It has always been a bewildering experience to lose your livelihood because of technological change; Silicon Valley has just generally been on the right side of it. For the first time, such change heralds an era in which software engineers themselves may be less well compensated and less in demand. After years of disrupting other industries, Silicon Valley has disrupted itself.

Back at the Archbishop's Mansion, 52 general partners from the Valley's top venture-capital funds were in attendance as the year's first batch of HF0 fellows made their presentations at demo day on April 4. A few days later, Fontenot posted a behind-the-scenes video of the event. It started with drone footage zooming in on the mansion, then cut to a close-up of Adam Reis, a fellow, jittery with nerves before his presentation. "Y’all, this room is [expletive] stacked," Fontenot said in the video. "Sequoia's here, Benchmark's here, [expletive] a16z is here. Everyone's here. So anyone you would want to meet, they’re here. And they’re excited." He was wearing a pink, woolly winter hat with a pom-pom, inexplicably, and he sounded like Caesar rallying the troops.

Two weeks after demo day, all 10 teams had received initial offers from outside investors and some had chosen lead investors. Fontenot was optimistic. HF0's next batch, to be hosted again at the mansion, was set to begin in May, and as he was reviewing applications, he texted me, "The talent coming in now is insane."

A few days later, I saw on Twitter that a friend of mine, Travis Fischer, would be joining the next HF0 batch. He and I last hung out in real life two years ago. At the time, the hot thing was the "creator economy," and he was looking to develop tools that would enable people, particularly open-source software developers, to monetize their work. While that effort, in the end, wasn't fruitful, last year he started a series of side projects in A.I. These included coming up with a way for other developers to use the ChatGPT A.P.I. so they can more easily incorporate large language models into their products.

Travis no longer talks as much about the creator economy; at HF0, he is now working on an open-source framework for building reliable A.I. agents that do things such as booking airplane tickets or submitting tax documents. But despite the shifts in theme, my sense is that what he's passionate about — making tools for the open-source community — hasn't changed. He has just found a way to come at it from a different angle. And in that adaptability, that ability to reinvent himself while coming out on top, he resembles Silicon Valley itself.

Yiren Lu is the chief executive of Frindle, a technical writing agency. She last wrote for the magazine about researchers’ designing and mass-producing genetic material. Laura Morton is a photographer based in San Francisco. A Pierre & Alexandra Boulat Grant recipient, she has been documenting tech start-up culture since 2014.

Advertisement