
Interviews
Zach Lloyd is the founder and CEO of Warp. the agentic development platform backed by Sequoia and GV (Google Ventures) and used by developers at more than half of the Fortune 500.

|
Founder & CEO @ Warp
Takeaways:
From $40 a Night to Google: At 25, Zach was prepping tracks at a recording studio from midnight to 4 a.m. for about $40 a night. A part-time programming job he found on Craigslist reminded him he loved building things, and it led him to Google and to Sheets.
You Can’t Win on Product: At Google, every launch comes with built-in distribution. At a startup, by default no one uses what you build. “A good product is necessary, but it’s not sufficient,” and in software, copying a product is very cheap.
Churn Will Catch Up With You: SelfMade could sell its product, but customers didn’t stick around. A growth curve going up and to the right can hide a churn problem. His lesson for Warp: don’t try to grow until you have evidence people love what you’ve built.
Out-Competed by Cursor and Claude Code: Warp was early to AI agents in the terminal. Then Cursor won developers with autocomplete, and Claude Code won on coding quality. Because Warp has to buy tokens from the model providers, it couldn’t match their cost structure, so it pivoted.
Building Like Switzerland: Warp Factories is deliberately neutral. Companies can use any model and any agent harness, and they own their own data. Zach argues there’s a conflict of interest when the same company sells you tokens and advises you on how to optimize your software spend.
Inside a Software Factory: Agents handle triage, specs, implementation, code review, verification and monitoring, all defined in code like Terraform. An AI judge scores batches of past runs, spots recurring mistakes, and proposes a fix to the factory’s own definition for a human to review.
Code Review Becomes Risk Management: Humans can’t review every AI-generated PR. Let front-end UI changes ride, but keep a person on anything touching user data, security or production config. Engineers will have two jobs: building the product, and engineering the factory that builds it.
References
Zach's LinkedIn – https://www.linkedin.com/in/zachlloyd/
Zach's X – https://x.com/zachlloydtweets
Warp – https://www.warp.dev/
Jing Conan Wang (Host) – https://www.linkedin.com/in/jingconan/
FounderCoHo – https://www.linkedin.com/company/foundercoho
Timestamps
00:00 Highlights
01:36 The long way into engineering
05:31 Lost at 25: a recording studio and a Craigslist job
08:11 Joining Google and building Google Sheets
10:52 Unlearning Google: why a good product isn't enough
14:56 Leaving Google: SelfMade and TIME
18:21 The SelfMade postmortem: lessons from a first startup
23:18 Why Warp started with the terminal
26:52 Warp's three phases and the AI coding race
32:36 You can't win on product: building like Switzerland
36:39 What is a software factory?
40:42 The new role of the software engineer
45:13 How a factory improves itself
48:01 200 open PRs: code review as risk management
52:52 AI anxiety, optimism, and what's next
Full Transcript
Jing Conan Wang (09:36.374)
Hi Zach welcome to the show. It's so great pleasure to have you.
Zach (09:47.424)
I'm excited to be here. Thanks for having me on.
Jing Conan Wang (09:49.955)
You have a somewhat unusual path in your career journey. you did symbolic system in Stanford and you get a master of philosophy of science from LSE, spent some time in NASA, and even spent some time as a JD in Yale Law. And how did that actually combine and also point it toward engineering?
Zach (10:19.521)
It took me a while to figure out what I wanted to do is kind of the honest answer. And so I had a mix of interests. Like when I was in college, symbolic systems is like an interdisciplinary major. so it's partly philosophy, part math, part computer science. I just kind of enjoyed all of those things. And when I left college,
I spent some time doing kind of like engineering, technical stuff. worked at Amazon, worked at JPL. And then I was like, you know, I don't know if I really want to do this. And so I went and I got a philosophy degree. And then I was like, you know, I don't really know if I want to do that. And so what I did, what a lot of people, you know, my generation did when they didn't know what they wanted to do, they would go to law school. so I went to
to law school for a year and I was like, no, it's not this. not, this isn't it. I actually think I do enjoy building stuff. And, you know, after that, I, you know, it's just took me a while, but I worked in a music studio for a bit. And then to make money, I got back into engineering and I was like, actually kind of like building stuff. It's fun to do software engineering. And then I ended up,
You know, pursuing that with like real focus over the next whatever 15, 20 years, but it took me, it just took me a little bit. I was interested in a lot of different things.
Jing Conan Wang (11:48.227)
If you're looking back at all this experience early on, for example, when you studied the symbolic system. Do you think the learning all you had over there is useful to what you're doing later when you become engineer?
Zach (12:22.973)
I do. like, I always recommend, I think it's useful to have like a, like a breadth of experiences and not just do one different thing. So for instance, the education I got in philosophy, I think is generally useful, even though I don't, I'm not.
I'm not doing philosophy everyday, but it does, it teaches you, for instance, I think how to think pretty clearly and think critically. It teaches you how to write well. And I think writing is actually a kind of a big part of my current job. Like it's communicating about what our company is doing. It's talking to agents in clear language. It's a really, really important skill. So I think there's something to be said, especially
these days just like go for an education or set of experiences that really helps you with critical thinking. And, you know, the particular things that I did, even the ones that weren't in engineering, we're helpful for that. And like, I don't, I don't really like look back and have a bunch of regrets about not doing engineering sooner. if I had one regret, I was admitted to a
PhD program in logic. And I turned that down to go to law school. And I kind of wish I'd done, I think I would have gotten good, like even further like good like training on how to think if I had done that program. That's my only kind of like regret of a path not taken. But I also think I wouldn't have known that the law school path wasn't for me unless I actually went and experienced it. So Yeah, that's how I, I did think there's a lot of value trying different things and not just having like a straight line path.
Jing Conan Wang (14:14.314)
It's all about exploration, exploitation. How do you find path? Yeah.
Zach (14:19.24)
Exactly, yeah, that's right. That's the right analogy. Exactly.
Jing Conan Wang (14:22.678)
how did you like realize you actually love building?
Zach (14:34.997)
I really was kind of lost, just to be clear. And I think that's I don't mind admitting that I was working basically as like an intern at a recording studio. I love music. I play guitar and I was like, you know, this will be cool. I'll learn the music trade, but I made, I would get up and go to the studio at like midnight and be there from like midnight to 4 a.m. because that's when the producer wanted me to like, I was basically like preparing these tracks to be mixed. It's probably something that can be done by AI right now, but I would go there in the middle of night and he would pay me like 40 bucks or something. And I was living with my dad.
And that's just not a like awesome mental space to be like you're 25, you're making 40 bucks for working in the middle of the night. And so I was like, well, what can I do?
to get more money so that can not live with my dad and I can have a social life. And I did, what else? I tried, I tutored the LSAT for a little bit. was like, I don't really like that. And so then I eventually literally looked at Craigslist and there was a part-time programming job.
And the reason I looked at Craigslist, cause I was like, I don't know if I fully want to do it yet. I was like looking for something for part-time. And then, so I started going and it was, I think 20 hours a week to start. And it was not at a proper office. It was in some guy's apartment. And I went in, I got the job because I still knew how to program well enough from college and from my time at Amazon JPL. And there was a consultant who was working there too, who was really, really good at engineering, who I was like out of practice, but this person, his name was Hany, like really helped me actually become, you know, good at it again. And the part-time thing turned into like, you know what, I actually really liked doing this. It was fun. It was building something.
It turned into like a full-time thing and they were like, they made me the lead engineer for their business. It was actually a business that worked pretty well. It was an online car rental business. I didn't do that for very long before realizing it. You know, actually I really like this. I want to do this for real. I applied to Google, which is where I spent most of my career.
But it took like, I need to make some money in like a good situation where I was learning to be like, I should actually spend more time doing this.
Jing Conan Wang (18:39.948)
after that you land the job and Google you you rolled very rapidly to be a principal engineer at Google. I'm also curious, like, I would imagine because you are just reconnecting to your programming skills in your first job and Google, I think that was two thousand and seven. Was that right time? It was like the hottest startup in the world. And it's like millions of people apply it. And any secret you use to get into a Google job.
Zach (19:15.528)
Not really. I mean, I guess I had some advantage. I went to Stanford and I had a lot of classmates who had Google was kind of born out of Stanford. So I had my college roommate work there and he like referred me, but I had to go through the Google, their crazy interview thing. And like, I feel very fortunate to have passed that.
The entire time at Google, I worked on Google Docs, specifically I worked on the spreadsheet part of it. I started on that when it was about a five person team, when, you know, it's still in like Google labs it wasn't even like a fully launched thing.
there's a bunch of luck when you go into Google, you could be put on some project that gets canceled and our project grew a lot. And I got to be one of the first people on it and I got to grow with it. And so as we hired more people, I got more responsibility and they put a bunch of trust in me. And, Google is where I really learned how to be a good engineer for sure.
Zach (20:56.788)
When I got to Google and started working on Google Sheets, I couldn't believe that spreadsheets could even work in the browser. It was kind of crazy. It really was a hard technology problem at the time. I know it's like you take it for granted now. But the browsers were just barely capable of doing that, like a spreadsheet. And so there was all sorts of interesting challenges around.
Can you make this thing feel like a real spreadsheet? Can you make it not be a toy? More than 5,000 cells. Can it do all these formulas? And so when I was there, we rewrote the thing completely from where it started to be entirely work in the browser, work offline, do all the calculation locally. We did it all through web workers. so I really felt like everything became canvas-based.
It was all cross compiled so you could use it on different platforms like iOS and Android and like on the server. It felt like a super interesting set of technical challenges. And then it also, you know, really scaled. So when we started, it was, I don't know how many, was in like the hundreds of thousands of people using it. And then, you know, when I left, it was probably like a hundred million.
Jing Conan Wang (23:49.231)
And so building at a scale at Google is very different than building a startup. So looking back, what are the main lessons you learned in this journey?
Zach (24:13.406)
so one of the things is actually to sort of unlearn the way of doing things from Google if you're going to go do a startup. And this took me a while. Like, I've been doing startups now for 10 years, give or take. So when you're at a place like Google, if you launch a new product,
you have built-in distribution. And so you're going to get, you're going to get people using it just by virtue of it being listed amongst Google things. it's Google's going to have marketing for it. It might be linked off of Google site. Whereas if you're at a startup, you're going to by default have no one using your thing. And so you need to really think hard about how do you get people to try it? How do you get people to care? working as an engineer at Google, I just didn't think about that. All I thought about was how do you make the product work well? And so you kind of have this assumption of if you build a really good product, then people will use it. And that's just not quite how it works. A good product is necessary, but it's not sufficient. And so in the startup world, it's like,
you need the good product and then you need some strategy for marketing it or selling it or getting people to care about it. So that's one sort of big lesson of like, if you're coming out of big tech, what you have to do differently. Another like, sort of lesson is, you know, at Google, it's like some of the great engineers and like, you're always building for scale.
But when you're about a startup, that doesn't matter. so at Google, it's like, you can't build anything without it being in 40 languages and being able to scale infinitely and scale horizontally. Does it have like multi-homing? So it's in multiple like geos. And it's like, if you're a startup, you have no users, it doesn't matter. And so you have to be pragmatic in terms of the engineering trade-offs.
Zach (26:20.24)
and you really have to favor velocity over everything else. Some of the good things are if you do go to a place like Google and stuff I learned, I think you really learn what does great engineering look like. And so when it comes to sort of hiring or evaluating talent, like,
I had some sense just of like the bar of what a really, really awesome software engineer looked like. so, and then I had some great networks. So like some of the first people who I hired at my startup were people who I worked with and thought highly from Google. So there are definitely some pros. You know, it gives you some credit with investors. It's like, it's a good, it's like, there's definitely.
good parts to it, but it's, it's a, you have to be careful about extrapolating what will make you successful at Google to a startup. Cause it's, it generally won't.
Jing Conan Wang (27:23.98)
And I think it's applicable for anyone who worked in a big company, big corp, or build a big product before and now want to do startup. It's completely, new world. I think finding the users who care about the product is very, very important. Most most important thing, for the stuff.
Zach (27:39.807)
Yeah. I think so. I think probably the most common way that it sort of fails is building a thing that no one cares about. I knew plenty of people who worked on failed projects at Google, but were able to translate that into a promotion still because it was like, well, they did a good job on the failed
project and the promotion committee would look at that and be like, let's promote them. And so it's a different optimization function than if you're at a startup. There's no reward for building a thing that no one uses when you're at a startup where it's a place like Google. know, you're kind of to an extent doing what you've been told to do and being judged against how well you did that.
Jing Conan Wang (28:33.71)
Yeah. Like I mentioned, it's a different reward function as well optimize. Yeah, and you have a very successful careers at Google and be spent seven years there, become a principal engineer of of Google Sheet. And then you left. And you be become you become you started to join and the startup world again. What made you decide to leave this comfortable, great, awesome Google World to the Startup land.
Zach (29:05.727)
I thought I would have more like growth learning potential, like upside if I went out on my own. and I'd always want to do something entrepreneurial even before Google. I kind of thought I would say Google max of like two years I had like, I had a bunch of side projects that I was doing around like trying to start my own thing. And I thought I would go do that. So it was kind of always part of my plan. I don't know. I wanted to see how I would do.
in a sort of more pure, like, can I start something on my own? from like a growth perspective, from like a competitive perspective, I'm a competitive person. I was like, I want to challenge myself by doing this. you know, you're working at like a big organization, can get slow you're working within the confines of like, even though it's principle engineer, it's like, I have like 12,
There's like 10 levels above me or something. So it's not like I'm running Google. I have to do what people want me to do. And I was like, I think I'll, I think I will personally grow more and learn more by trying something where I might fail more. I wanted to just like grow in a bunch of dimensions where if you're like on the engineering ladder at Google, you're going to grow for sure. But you know, it's going to be within some constrained boundaries. And so I want to try something different.
Jing Conan Wang (31:18.008)
after that you actually spend a year in as an interim CTO at time.
Zach (31:35.933)
I'll tell you the story. So I left Google. It took me a little while to figure out what company to work on. So I've done two startups now. So the first one was a company called Selfmade that worked on for three years. Candidly, it wasn't like an abject failure, but not like a big success either. It was like a series A company. We grew some revenue. We had some customers.
It was in like a space that I would honestly say I don't really care about, which was like social media and marketing is like, I ended up, If I had a time machine, you know, I don't know that I would have worked on that particular thing again, but I, I learned a ton of stuff from it. And again, I don't really regret it. Like I leave Google after being there a long time, you don't really know what you're doing at a startup. And so I, kind of went through like a bunch of learning there.
After that startup, I knew I wanted to found another company and basically take what I had learned in the first experience. But I needed some time in between and I ended up at Time. I'll be totally honest. my cousin is Mark Benioff, who is a Salesforce founder who owns Time. And so he was looking for basically like his could I offer some engineering help there. And I don't know anything about technology and media, but I was like, I'm down to go and like learn and, try and actually a really good experience there. Was like glorified consultant with like a fancy title, helping their technology team. but I did that. And while I was doing that, I was also thinking a bunch around like, what do I, you know, what's a company where I really feel like I could invest in the longterm where I feel very mission aligned about it, where I thought that would be a really good fit for me as a founder. And so I was thinking through all the lessons in my last startup while was also helping out at time.
Jing Conan Wang (33:39.595)
And related to that, because for all of us doing startup just leave a Google and just shoot that up, it's really the first time we hit the real world and there's so many new things. the world works like completely different, it's like that. I think that probably happens in your first startup selfmade. And if you think about that experience, I I I'm pretty sure you probably spent a whole year to reflect on that. What are the main takeaways that you learn from that experience and advice you would give other people who want to go like who are in a same region?
Zach (34:27.712)
Yeah, so I actually wrote up like a postmortem, like Google style, you know, after an outage. some things went well, but some things didn't. I I wanted while it was fresh to be like, okay, what are all the things that I've learned from this first experience? And so I'll just give you a few. So one was really, you know, pick a thing that you care about to work on. that just like starting a company is so hard. The default state of any startup is going to fail. And one of the advantages that you can give yourself is if you really actually care about the thing and enjoy it, and are doing it because it's like a thing that you, you know, you have like a real point of view, you want to exist, then I think that can help you get through it more when it's not going well. So I think that's like,
really important. also helps just that makes everything easier. It makes it easier to convince people to invest in you. makes it easier to convince people to come work with you if they sense that you truly care about it. So that was like the really big lesson. More tactically, I also think like really taking an intentional approach to sort of building the company in terms of like who's going to join the team. I came out of it being one to be like
really, really sort of like picking people who I thought would be absolutely awesome in their roles in terms of like trying to build the culture. And so, you know, before I started Warp, which is the company that I run now, I wrote up a whole bunch of things around like, what are like the principles of running the company? What are the processes? Like, how do you, how are we going to communicate with each other?
What are our values? And I like, I want to make that really explicit. Cause if you're not explicit about it, culture will still form, but it might not be like the culture that you want. And so by being intentional, you can be like, okay, this is, this is how I think that this is the, I want the DNA of the company to be. And I made it public. And so people could look at it and decide if it was a good fit for them or not. And so.
that's been like a really good thing. It like helps people decide whether or not warp is a good fit for them. Should they apply to work there? And so that was the second thing. a third thing more like just more tactical, more just like, don't make this mistake was at selfmade we, in my opinion, I don't know, other people might disagree, but in my opinion, what we, we had a thing that we could sell and we could.
Zach (37:21.385)
by selling it kind of like temporarily get revenue, but we didn't really have a thing that people would stick with. And so we had a very high churn rate. And so when that happens, you can kind of create the illusion that you have something that's really good. You can kind of make a curve that goes up to the right, but the churn will really catch up with you and make it like harder and harder to grow. And so you really don't want to try to
grow a thing until you have a bunch of evidence that it's providing value and that people love it. And so that was like another kind of lesson that I've taken into warp. And by the way, I've, just made new mistakes at warp. Like I just have a whole new set of things that I've kind of, you know, I'm learning, but those were, those were a few things that, you know, I took away from the last experience. I also was like,
I'll one more. So for, for me, the last company was very, it wasn't like a pure software company. So it was like software enabled services. we had like, 150 people working there and like people were doing operational things and I don't, I'm not good at that. I'm not good at managing a big operational company. My strength is really in like building a software experience. And so I was like, I'm not doing that again. And so it was like, it really clarified a bunch of things around.
Zach (38:49.98)
What am I good at? What do I care about? How should this work? And so it was great in that respect, but also, you know, challenging.
Jing Conan Wang (40:49.858)
so let's talk about the warp. Actually, one thing I want to share before we have the warp experience is actually WARP is one of the very first agentic development environments I use. the first time I used a warp and it was starting to like figure out itself and just like do all this agentic step.
Zach (41:40.232)
It's pretty incredible experience. Now there's a lot of ways today that you can get something like that experience. We were actually really early to it. For folks who don't know, we started as a terminal. then before Cloud Code, had basically an agent built into it where instead of running terminal commands, you could ask warp to do something in English. And it was primarily for like terminal use cases like Git or Docker, that kind of stuff. But it's become for coding use cases too.
Jing Conan Wang (42:23.246)
it looks like you spent a year in time, think about what is the what is the next step. And then you during the pandemic you decided to start the new company. what is the thinking process on that? Why you pick the warp as your idea.
Zach (42:52.35)
going back to like what I learned from the last company. was like, I really wanted to pick something that I'm interested in and could work on for a long time. To me that meant I was either going to work on something in the developer space, cause that's like, it's like what I do, or it could be something in the knowledge workspace, more similar to Google Sheets. I felt like I wanted to pick something where I had some
advantage from personal experience and cared about it. In the developer space, I looked at a few different ideas. I looked at rebuilding an IDE. I just ended up thinking that there was the terminal, which is what warp initially was focused on, was like a more underdeveloped product where I personally, I...
I've always used terminals and I've always worked with people who are really good at using terminals and been like, if I could use this better and it was easier, I would probably get a lot more done. And so the more I thought about that, I was like, okay, we could change XYZ aspect around the terminal to make it a more like something that I would want to use more. And that was kind of the initial idea. so the, the feature set like in warp that we had out of the gate was like,
the mouse works in the terminal. There's like a real text editor for entering commands. it knows what the output of a command is. And so I actually, I'm really proud of it. I feel like we completely changed what a terminal could be. like even from the start, it had a bunch of like user traction. So that was like the original, that was as simple as that really. was like, I think that there's like a lot that you can do with the product.
I think it's something that could impact a lot of developers because pretty much every developer uses the terminal. And then for the business model, it was kind of a vague idea. It was going to be around collaboration in the terminal. The business model was the weakest part of the initial idea, to be honest. But that ended up not really mattering because AI changed everything anyways. the initial thing was like, pick a product I care about.
Zach (45:22.898)
Make sure there's like wide impact. wanted it to be technically ambitious. I wanted it to be like something that I was uniquely positioned to do a good job building and understood the user pain of. And so I kind of thought of a bunch of different ideas and landed on that as being a good one.
Jing Conan Wang (45:40.793)
you mentioned also early on AI changed everything and the whole space later becomes so competitive. There's like all sorts of Agentic development environment. I'm curious from your experience, how does that journey look like being
Zach (46:59.9)
the journey of warp, that was a pretty good description of it. The way that I think of it is like we've actually had, we've been around for six years and we've had three products. The first product predated AI and it was like, just like we took the user experience of the terminal and we really tried to just from first principles be like,
How do we make this thing more usable and more powerful? And that was successful, not from a revenue standpoint, but from a user adoption standpoint, right from the start. Like the first day we launched that, we had 10,000 people sign up on Hacker News. people were very, like developers were very interested in that. The second iteration of the product, roughly speaking, is what you're describing. So we started progressively putting
more AI features into warp. we realized like actually the terminal is an incredible place for this. even before chat GPT, we had the ability in warp where you could like basically translate English into a command. So that was the first AI iteration. so if you wanted to like, you could be like, what's the command for finding all rust files greater than a thousand lines or whatever, it would translate that to like find dash whatever. And it would, it was just great. It's like so annoying to do that in the terminal.
When ChatGPT came along, realized actually there's, you know, we should, we did, first we put a chat panel into warp and then we're like, this is stupid. The whole terminal is like a chat interface. And so then we basically had it. So you had the claude code experience in a sense of like, you could either type a terminal command or you could type in English. And if you typed in English, it would invoke an agent and that agent would do things.
Like I said earlier, the things that it would do at that point in time and call this 2024, it wasn't so good at coding, but it could be like, you know, fix your like Git merge conflicts, or it could like debug why your Docker container was failing or like, you know, do all these like DevOps type stuff. And so was really, really good for that. And we were pretty pioneering there. Like we had the first, I think we really had the first interface for doing this.
Zach (49:22.28)
We actually coined a couple of terms in this space that are pretty cool that I'm proud of. Like we coined the term agent mode. That's like warp. And then we could, coined the term in 2025, agentic development environment. ADE, like if you ask chat GPT, like we invented that I'm on video, like saying it's not an ID, it's an ADE. but the, the, we, and to be clear, we grew our revenue off of that, like pretty well, but
There were a couple of things that competitively have made it really hard. So one is you have cursor coming at the space from the IDE. Cursor had an awesome product of just the autocomplete, which is better for coding at the start. And so cursor just like blew up, went like crazy. And then you had claude code do it in the terminal in a way that was like,
better at coding than our thing, just like frankly, like so they had coding support and coding is the killer use case. And then they had a good product of credit to them, but they also have a massive cost advantage compared to us, right? So we have to pay them for their tokens and they take a big margin on that. Whereas they could...
what they did to like really grab a lot of the market was like they offered these highly, I don't know if subsidize is the right word, cause it's like, it's their product, but they offered a much better deal to customers to get onto these like, you know, $200 a month plans or whatever. And so we, we just can't compete from a cost structure perspective with that. so, you know, we've still grown that business. Okay. But not like claude code is probably like,
$10 billion plus business or something. We’re not like that. And so we've had to sort of change our positioning to the third iteration of warp, which is, is all about automation and it's a product we call warp factories. And it's actually, it's where the market's going anyway. So it's like, actually think continuing to invest in the interactive claude code style thing is like,
It's just not what companies want. it's not the future of development is a hundred percent going to be about automation. And so we've taken, you know, a bunch of stuff that we've had and we've basically pivoted into this new direction where we have a ton of traction all of a sudden, which is super cool. And so it's been, it's, it's really, really interesting to see the evolution of the whole thing. but yeah, that's like the third sort of phase of what we're doing. And it's, it's like,
It's the best phase. We really thought through, I can talk about what factories are, but we really thought through like, how do we not put ourselves in a position where we're just going to be undercut on cost and try to have like other differentiators. Cause you know, again, it's like, there's so much that has to go right in order to win in this sort of like competitive space. So we're really thinking hard about that.
Jing Conan Wang (54:00.001)
for me the question I have is that I think as a Google engineer you probably also put some thoughts on that journey, postmortern. So what is the biggest takeaway from that phase of warp?
Zach (54:14.845)
certainly you can't win on product. So that's like one takeaway. Like product, would say having a great product is a, go back to my philosophy, it's like a necessary but not sufficient condition, in my opinion. Like, it's certainly very, very helpful to have a great product, but that's not, that alone will not get you a win, especially now because
In software, it's very, very cheap to copy a product. The time to sort of clone something or copy it is very low. And so I'll just tell you how we're thinking about it now. I won't claim that this is the answer, but we're really trying to find a defensible position that is something more structural than just having a really good product. And so for us, for warp factories,
WarpFactories is infrastructure for companies that want to stand up their own internal software factories. And by software factory, I mean an automation loop where you have agents that are doing all the different parts of the software lifecycle. So everything from triage to spec writing.
Zach (55:42.77)
to implementation, code review, verification, monitoring. So basically setting up a system that you can deploy that lives in the cloud. And the other really important aspect of it is that it's a measurable system that you as an engineering organization can try to improve over time from both like a quality perspective, a cost perspective and a velocity perspective. So think of like,
I don't really love the analogy, but it's like an assembly line for software where agents are doing all this. But the part of the analogy that's good for the assembly line is that you measure everything, you test everything, you optimize it, and then you make it work better over time. so we can go more detailed into how that works, but at a high level, that's what it is. And then to answer your business question, this is also a competitive space for sure, but our approach to it is to
basically help companies that, you know, they want the option of using different underlying models. And so they want the option of using different harnesses. They want the ability to own their data. They want super flexible deployment. And so we are trying to build something that is explicitly like Switzerland, like it's neutral. It's like, and we're trying to do it in a way where we
Zach (57:07.27)
aren't necessarily building our business off of like selling you tokens. Cause we believe there's like a bit of a conflict of interest if we're in the token selling business and the software optimization business. So, that's like, that's like something that like Anthropic, might, unless they're going to be like really going all in on like serving you open weight models, which maybe they will, but I don't think they just have like a conflict of interest. And so we,
come into conversations with companies that are like, don't want lock-in, I want the ability to use different models, I don't want to have to buy inference through you with like a structural advantage where we can credibly say that something that these other bigger players aren't going to do. Will that work? We'll see. It's really resonating. Like there's a lot of desire right now amongst enterprises to not be locked into one model provider and to not be like at the
mercy of their pricing power to have access to open weight models, to be able to try different harnesses, to be able to measure everything and to own all their own data. So it's a really great sales pitch at the moment, but we explicitly try to do something that's not just a product advantage, but like a kind of like sustainable moat from the business sense. That's our strategy. I'm very transparent about it.
Jing Conan Wang (58:25.866)
you mentioned briefly about the software factory. I think for us we have some idea what a factory means. But software factory is more abstract. Can you give like a very concrete example? What does it mean for software or factory?
Zach (59:23.771)
the best analogy here for developers who are listening is think of like, you want to create a bunch of agents that are doing different things that humans in this process might have traditionally done. So you might want agents that do issue triage, agents that implement things, agents that use computer use to verify your code, agents that like respond to server crashes.
And think of this as like a loop, right? That runs in the cloud. The way that we think of factories is like as an actual product primitive, you define them in code. So if you're familiar with something like Terraform, you basically say, okay, this is the definition of the factory. You could think of it almost like a graph or a flow. These are the skills that drive the agents.
These are the MCPs they have access to. These are the models they use. These are the harnesses that they use. These are all the skill, like the anything that might provide context, you define it in code. And then you basically can take that code, you deploy it and think of it almost as like the new version of like CI/CD issues start coming into it. And these agents will start doing work on it.
And the agents, you can encode in this factory that you want the agents to loop in humans at certain points, but you can also encode that you want something that's much more automated. Like for simple front end issues, just have the agent make a PR, make a video of it, ship it. So you get this loop going. And then the really key aspect, the thing that we lean into is like, you set up a whole bunch of measurement around how this works. And so the things that you want to measure are like, well, what's the...
Zach (01:01:20.104)
velocity at which workflows through it, what's the cost, what's the pick a dimension that you care about. And then you can set up basically LLM as a judge type scoring across how that whole system works. And so those scores could be things that are like general, like efficiency or verbosity, but they can also be very specific to your organization, which you could be like, is this thing, is this
Zach (01:01:48.402)
factory working in the right way for our front-end code base. And so you set this up and then you have the ability on top of that to have agents that look at the scores and suggest improvements to the factory definition. And since the factory is defined as code, you basically have agents that are able to change like what we call self-improve. Like you have like a self-improvement loop that goes over the software development process. And then the final thing you can do is actually test different
configurations. And so you can, if you're familiar with things like sweet bench or terminal bench, you can basically make benchmarks off of your own internal coding workflows and then run them against different mixes of models and different mixes of harnesses and generate the type of like Pareto graph where you can say, okay, okay, maybe an open weight model is cheaper, but not as high quality. And maybe like a frontier model is,
Zach (01:02:46.598)
is higher quality and then feed that back into model routing so that you can prove to yourself that the way that you're doing software development is like as efficient as possible. And this like measurement aspect is really appealing because companies are spending a ton of money on these coding agents now. It's like it's starting to compete with the actual budget for software engineers and they want a way of believing or understanding that they're deploying these agents.
to get like the most business value out of them. And so that's like in a nutshell what we're doing.
Jing Conan Wang (01:03:22.432)
Yeah, that's that's super cool. Is my understanding correct that like basically what Claude Code and Code is doing is to to produce a new one like AI developer, but you pretty much produce a AI software team.
Zach (01:03:37.822)
Yeah, you could think of it in that analogy. I think the CI analogy is pretty good too, like CI-CD, if you're familiar with that, just creating more of a system, building your system to build software. And a lot of companies are trying to do this internally on their own. And so we're providing them with infrastructure where they can get a real shortcut by adopting it.
and get to this like end state of automating some of the development more quickly.
Jing Conan Wang (01:04:11.022)
you mentioned early on that basically this well works like a loop that self improve. And I can imagine the role of human now will be completely different with this new kind of framework or new factory. And what do you think will be the new role of a software engineer? Was it does the the definition of engineer will be different?
Zach (01:05:06.139)
Yeah, man, it's changed so much what it means to be a software engineer in the last couple of years. I think it's going to keep changing. So the way that I can see with this is like, if you look back to whatever three years ago, we're all writing code by hand. Two years ago, we started using autocomplete. A year ago, I think everyone starts moving to this model of like, you're just like on your laptop prompting an agent, right? And so
you're still in this factory world gonna be doing some prompting. Like you might prompt from Slack, you might open an issue. But your job, I basically think engineers are gonna have two jobs. So the first job is the traditional thing of product engineering, where the engineer's job is still to basically build the product by prompting an agent, make sure the product works well, make sure it solves user problems. The second job of the engineer is like to be a
what I call like a factory engineer. And so that means to be trying to improve the factory system that helps you ship the product more effectively. And so those are, it's kind of like meta engineering. It's like you're engineering how all the coding agents work, but that's gonna be a big part of your job. if you're an engineer, I really do think it pays to learn
how harnesses work, how to think about what models to use, how do you provide context efficiently to these agents, how do you set up these self-improvement loops? so engineers, that's going to be a part of their job, in my opinion, where it's like, they're going to have to engineer how all of these agents work together to ship software.
Jing Conan Wang (01:06:57.024)
A principal engineer at Google usually they wouldn't code themselves. They would do all these architectural decisions, like weighing the or pros and cons, communicate to the team, make sure things are working, do some correction. Do you feel like we're entering to an era where everyone becomes a principal engineer?
Zach (01:07:24.189)
there's two analogies that are common here. So one is like the like everyone becomes a tech lead or principal engineer. I favor a different analogy, which is that everyone is becoming like a almost like a dev prod or like like a like a platform engineer where
Zach (01:07:48.559)
It's easy to personify these agents and think of them as junior teammates. I don't really like that analogy that much. think it makes it, I think it's more fun to think of them as like automation engines that have like, they have like non-determinism, but ultimately like you should think of how do I apply intelligence to a task in a system? And it's like a big systems engineering challenge. I prefer that analogy to the like,
Everyone is like, I'm like managing a team at junior engineers doing stuff. Cause like the first one is like a management challenge, which sounds like drudgery to me, just to be honest. The second one is like an engineering challenge, which sounds like, fun. Like I have this new automation primitive and it's my job to figure out how do I most effectively engineer a system using that primitive.
Jing Conan Wang (01:08:45.55)
I totally get it. Actually the two points different things why which the manager usually you need a lot of soft skills and all sorts of things. And a platform engineer is like you how to think in the principles and how to leverage all the tools, automation tools to make things happen. And it you generally more fun to build
Zach (01:08:55.91)
Yeah.
Zach (01:09:07.281)
That's how I think that that will appeal much more to engineers. I know it does to me. I'm not dying to manage more people, but I do like the idea of, there's this new cool automation primitive where I can engineer how to get the most out of it.
Jing Conan Wang (01:09:24.898)
can you tell us a little bit more about how did you build this self improvement mechanisms in your software?
Zach (01:09:47.579)
so when you're working in a factory approach, again, this is specific to warp factories. This is how I think of it. One of the things that's primitive that's built into the system is every time an agent does something, the whole agent trace is recorded someplace. So you get a complete record.
of what was the prompting, what was the context, what were the tool calls, what was the output in terms of the PR on any artifact. as the raw material of doing this self-improvement, you have all the past things that folks have done. And so to make this concrete, let's say you've had an agent build a feature. As part of that, it built something, and you went and reviewed the code, and you were like,
no, this isn't done according to our coding conventions. This isn't how we do static constants here. Pick some random thing that this is wrong. This is wrong thing is like a very useful signal to a future agent if you capture it. What we do is then we would have a scorer that is looking at all of those past runs and
It doesn't go like one run at a time. It basically it does them in batches because you want to look for patterns because there's like a risk of sort of overfitting to single examples. So you look at a batch of them and then, you know, we use LLM as a judge. I could see you could, you could use this to fine tune something in the future, but it's like, we basically look at it and the, the judge model will say, okay, I've seen a pattern now where there's like six PRs where it's like the agent is making the wrong
suggestion in its PR reviews, let's then go ahead and create a change to the skill that drives code reviews so that the next time an agent runs in the future, it will know not to repeat the same mistake. And what it will do is because all the factory is defined in code, it will make a PR against the factory definition itself, which will say like, hey, here's how constants are used at our company.
Zach (01:12:07.985)
someone will review that PR, it'll get merged and then going forward, it will not make that same mistake. And like, if you really want to do this, I think provably right, you have to then, you might want to like AB test this. There's lots of things you could do to make sure that it's actually improving stuff. You might want to like rerun the prior runs with that skill adjustment. And so, but that's like a concrete example of how the system works.
Jing Conan Wang (01:12:37.93)
one follow up question on that because as a as a very technical startup founders, I also get a pitched by many companies talking. I'm having a lot of like AI that can write PR so you can review. And then I got a ton of review PR. I actually have two hundred PRs open right now. so the question I have here is like, from your perspective, because the AI is like pushing code changes and so many PRs, how like humans almost cannot catch up on the AI speed nowadays. How do we solve that problem?
Zach (01:13:38.545)
Yeah, so my answer here is like, I think the industry is going to have to start moving to a model of like PR review is an exercise in risk management, essentially. And so I don't think it's going to make sense to have humans review every PR. Now we still do this at Warp, like we still do human PR review or like
Sock 2 compliant, have various reasons for doing this. But we've moved from a mode of like every PR used to have like, like basically one author, one human author, one human reviewer. Now we just have like the ability to have an agent author and a human reviewer. So you're not like taking two people's time. And I think in the future, part of like the process of building with agents is you're going to do some risk assessment.
on the PR and you're going to want to like decide this might not be risky. And then the other thing is like, there's stuff you can do before the PR and after the PR to basically get a better sense that the PR is right. And so the, before the PR, the more that you can check that you have a spec and that that you can, you can check that the PR adheres to the spec. You can get confidence that the PR is.
less risky. And after the PR, depending on the change, we do this at warp is built into our factory, it will like make videos of a computer using the like the thing and take screenshots and exercise these cases. And like, that's the kind of thing I used to ask engineers on the team to like, make me videos of them doing it. And so there are these things where you can gain more confidence. And then I think you're just gonna have to accept
from a business trade-off perspective that some types of things are gonna need more code review and some less. Like front-end UI changes, you're probably gonna be like, let it ride. But like changes that touch user data or have security implications or production configuration, you're probably gonna want a person looking at for longer. So that's like my sense of how this will play out. But I do agree, it's a real problem of just like, it's very, very easy to have an agent generate a PR.
Jing Conan Wang (01:15:53.027)
And on the other side of the coin is that there was also a team trying to explore about best practice of agentic development. So they will be on the loose end of letting PR. So I have I personally experienced myself. I also have Friend’s company who have almost approved most of the PRs. to point out risk management are leaning more to the velocity than the risk. And after a while, sometimes we will find that engineers couldn't talk about their own code base. They couldn't develop it, couldn't talk. and then how what would be your advice for those situations? Well, do we accept that of the new norm?
Zach (01:16:37.178)
I think people need to be able to talk about their code base at an increasingly higher level of abstraction is kind of my answer where you certainly need to understand the system, the components. This is where it does become more like you need to be more like a senior engineer or staff engineer. But then you can probably start to think about like the code is a bit more of an implementation detail of the system. Cause it's just, it's not going to be feasible for people to read and understand and truly internalize every line of agent code. It's just not like the best you can do in my opinion is make sure that you don't lose grasp on how the pieces fit together and how the system works and be able to make sure that the architecture is right.
That is one place I think that the agents still kind of struggle because they have like a limited context window. I think they're better at a local change and they are understanding the system and definitely understanding the user problem space. So your team should focus on that and there's just not that much need to focus at the level of a function, for instance.
Jing Conan Wang (01:18:01.378)
I'm also very curious from what you see What are the most exciting things you see in the startup world, in the spaces you are in the next six months?
Zach (01:19:00.689)
AI has so much like capability to induce anxiety around everyone's, rightfully so, everyone's jobs, like every software engineer is doing something totally different now and it's in, I think other professions are going to be disrupted too. I, as someone who like, I'll give you my big zoomed out, like as someone who studied philosophy and who, you know, was really interested in always like logic and knowledge and like what, how does, how does thinking work? To me, it's like incredible what we have achieved collectively as people in the past couple of years to create,
something that really like can think. It's really interesting how tied into language thought is. I didn't, you can do so much with just predicting like what the next token is. It's a crazy, it's crazy how that generalizes into intelligence. And so I don't know, to me, it's like a very, very exciting, unbelievable technological advance. It's like on par or probably.
That's more impressive than like sending a person to the moon. And I think like, it really is. I think like, everyone has so much locally tied up with this, but just to step back and be like, this is incredible what has been achieved here. Now, hopefully I'm not like, this doesn't come back to haunt me when, you know, terminators arise in whatever five years, but I don't know. I think take a minute and be like, this is really incredible.
and you know, the, the things that I would be most excited about honestly here are not things in software development. There are things that related to like healthcare and medicine and like, can we actually put this technology to use to do some stuff that's really good for people is like, like it's one thing to be able to write more code and like, that's cool. Like I'm excited for that. But like, when I, when I step back and I'm like, what am I really excited about? I think the world all of a sudden has this.
Zach (01:21:21.82)
You know, the marginal cost of like intelligence, which I've always thought is kind of a very scarce thing. The cost has gotten way down. And I really hope that we find some ways to apply that are like truly economically beneficial way beyond what we have so far. So that's like my very zoomed out answer.
Jing Conan Wang (01:21:42.935)
we need to be appreciative that we are born in this era.
Zach (01:21:59.74)
That is so cool.
Jing Conan Wang
I think the intelligence probably only invented twice. One by the fact that we human beings exist in the world and takes hundreds of millions of years, right?
Zach
Yeah, the revolution.
Jing Conan Wang (01:22:12.076)
And then second time yeah, for us to reach revolution. And second time it's just right here.
Zach (01:22:18.705)
We're here. We're living in it. I think it's like the coolest. It's obviously it's very, very scary and anxiety inducing. And I don't want to be dismissive of that at all. Like I have a whole bunch of anxiety around it. But I also think it's like to your point, it's like evolution did this once and it took five billion years and we just happen to be living at a time when we've done it again is pretty remarkable. So I'm pretty psyched.
Jing Conan Wang (01:22:43.03)
And it's a wonderful time to be builders to to really see what is possible. Back to what I mentioned, I think you have spentl like the early part of a career exploring what is interesting, what is fun. I think to some extent that we builders are all exploring what is possible ahead. Like we're so fortunate to be involved in such incredible time. What will be the world look like in three years? We don't know.
Zach (01:23:11.804)
if you'd asked me that question 10 years ago, I would have been like, I don't know, you can probably, you'll be able to like buy a car on your phone or something like, like who cares? Like, but in the last few years, it's like, I don't know, we might have like robots walking around our houses or it's like, maybe it'll be some amazing like advance in medicine. So it's like, I, if you're not getting excited about that, I don't know.
Zach (01:23:40.988)
It feels really exciting to me, personally. Because it's totally changed, yeah.
Jing Conan Wang (01:23:42.893)
Zach, it's so much pleasure to chat with you and thank you so much for sharing your own personal journey, learning, insights, postmortems, everything you learned and it's so incredibly useful to our community. I'm pretty sure also for me. means a lot for me to be able to have this conversation with you. Thank you.
Zach (01:24:27.8)
I appreciate you having me on. hope this was a great conversation. I hope folks found it interesting. So thanks so much.