Fireworks CEO: AI Startups Are Scaling Into Bankruptcy (And How to Fix It)

As we all know, starting from this year, a lot of companies will build AI into bankruptcy. This concern was never even a topic to discuss in the SaaS era, because product-market fit and durable business almost means one thing.

Because the business operation, the cost of business operation, is quite low. But in the AI era, product-market fit would mean your ICP just loves your product. But it doesn't mean you'll have a durable business, because your cost of operating using AI could be much bigger than your income, and then when you scale, you literally scale into bankruptcy.

This is a real issue. So we came here to solve both issues. I'm talking about 5 to 10 times cheaper using.

This is Lin Qiao, co-founder and CEO of Fireworks AI. She helped build the go-to platform that helps some of the biggest companies customize and run production AI models at a fraction of the cost. And now she's revealing why generic models destroy your unit economics, and how to turn your private data into the ultimate moat.

I think everything boils down to data. Product development has been disrupted. Product as-is is not the moat. Because of coding agents. Products cannot be easier to copy. So then what is the real moat,right? So every company, they are solving a unique problem in a special way.

Because they're solving a special problem exceptionally well, they will have a unique product taste judgment and design. And what makes them continue to be unique is the data they have accumulated from the product surface area interacting with the customer.

That is their moat. That's the foundational value of why they exist. And those companies will continue to thrive by building their own specialized intelligence. Using your private data, we can tune a highly intelligent model towards SOTA quality, but with much better unit of economics.

That means for the same budget, you can grow your customer base 10 times bigger. There's a framework we use to think about that. Basically.

Welcome to NEW ECONOMIES.

Hey Lin, welcome to NEW ECONOMIES. Thank you so much for joining us today.

Thank you for having me.

I'm so excited to have you here. Jensen has recently said you're one of the most interesting and most exciting companies to watch. Why did he say that, and why is Fireworks so excitingright now?

We started Fireworks about 4 years ago. We can talk about our background and history, but we have been working in AI for more than 10 years. I think what's interesting is our bet in the future of AI. As we all know, the Frontier Labs, they're betting on general intelligence.

They're building this one model that are aiming to solve common tasks very well, and offer those models as black box API for all applications to build on top of. We think differently. We think the world will be kind of much more diversified, and every company, they are solving a unique problem in a special way, exceptionally well.

That's the foundational value of why they exist. Those companies will continue to thrive by building their own specialized intelligence. And this specialized intelligence is built on their unique data from their product surface area, capture their product taste judgment and design.

And those models will continue to grow deep and deep, deeper and deeper into their domain, and really be the anchor of their moat. So Fireworks, we execute on this specialized intelligence vision and help all application companies and enterprises who has huge amount of private data to customize their model, customize model deployment for the highest quality, speed, and cost efficiency.

So that's why I feel like that's extremely exciting for the whole entire industry. And that comment from Jensen came up in recent conversation I had with him when we talk about specialized intelligence.

You've known Jensen, I think, for a while. What makes Jensen and NVIDIA such an incredible and enduring company, do you think?

It has a lot to do with how Jensen operates the company. I think he has a very unique philosophy of company operation, which I share a lot of value. I think to me, leadership is not privilege. Leadership is judgment.

And our job as leaders in the company is to deliver the best judgment to move the company forward. Those judgment is reflected in our vision and our operation charts. And in order to get a strong and correct vision and operation charts, it requires us to really understand a lot of context everywhere across the company.

And before it's hard, because without AI, pre-AI,right, pre-GenAI, we are designing the organization, relying on organization hierarchical structure to operate. And there has been well-established communication flow and management structure up and down the chain in the past few decades.

But I would say Jensen's operation is completely different from a conventional company operation. He knows everything across the company. I'm really shocked how much detail he understands, from capacity allocation to big conference, all the topics, all the technical details, and all the kind of operation direction, where it's going.

And because of that, he can make the best judgment for the company. Obviously, that is based on huge amount of personal capacity of operating that way. It's very unique. It's not for everybody. But I would say with AI, this type of operation will be more and more common across a broader set of companies because information flow is going to be much easier.

And we do not need a deeply hierarchical organization to operate. And as the information flow much smoother using all sorts of AI tools and have people have access to all sorts of data. And then with good judgment, it enables a lot more leader.

They can operate easily. So this is how I think about our company as well. Our company is pretty flat. We do not like to introduce deep hierarchy because it just slow down the information flow, slow down a lot of decision making, add redundancy in kind of reporting communication.

And whereas a startup, we thrive in moving extremely fast and making very decisive decisions with quick information collection and access to what's going on and all the context. So I think that's extremely unique part of NVIDIA, but I think a lot of companies are starting to operate in a similar way.

You mentioned that human judgment. You know, I've had a lot of founders and heads of AI from the leading companies on the show recently who all mentioned this thing around human judgment taste. What does human judgment actually mean when you're running a company of the size of the likes of Fireworks, do you think?

So we actually are a very small company, given where we are. End of Q1, we were 150 people. End of Q2, we're 300 people. So we're doubling every quarter. We're growing very fast, but still, compared to where the business is, we're quite small.

So one of the kind of philosophy is we really care about the quality of people. I interview everyone before they join Fireworks. And now as a company scaling much bigger, we do have leaders kind of managing all different areas of recruiting.

And they are spending a lot of personal time to make sure everyone we hire into the company is the highest quality. And here, quality means different things for different company. As founders, I really value the fire of founder in the candidates, in every single person joining us, because I really believe people choose to join Fireworks is not because they want to just be an employee in a company.

They really want to build a company with me together. And we view them as every single one of them as a pillar in the company growth, in the company operation, in where we're pushing the boundaries, in where we bring in innovation, creative ideas, and seeing through things end to end.

So every single one of us in the company are taking huge amount of responsibility. And one of the key tenets of our culture is extreme ownership. So extreme ownership is actually a military term from Navy SEAL. In the army, it's extremely hierarchical, and no one cross the boundary.

You're sticking your line. And that's how army operate. But in a battlefield, it changes because it's life and death situation. And the thing here is it's important for the Navy SEAL leaders to watch each other's back. And here, boundary doesn't mean anything.

If it's a life and death situation, you just go ahead and take over responsibility and, like, save a life,right? And make the call and watch each other's back. So there's a lot of similarity to how startup operate is no problem is anyone else's problem.

We are actually part of our company growing to the next phase, and we really value what really matters is seeing is kind of calling out, hey, we have a gap here. And not just calling it out and actually seeing through and fix those problem.

It doesn't matter who wrote the code. It doesn't matter who are the primary person. It's kind of more important to solve the problem than just wait and think about someone else is going to solve the problem. And this is not unique to startups.

I think I learned a lot from the company I worked last in the previous gig from Meta. I learned a term from Meta, how to kind of think about ownership. And I've seen tremendous amount of results operating with extreme ownership.

That's kind of one of the key thing we value from people joining us.

I totally hear you on the ownership and also getting involved in the hiring. But also respectively, there is only so many people you can spend time with and hire until the organization becomes so big. At one point, do you stop interviewing?

Is it 450 people, 600 people? Because you can only do so much.

First of all, I started the company with six other co-founders. So we have a very big founding team. And they are every single one of them are a pillar operating the companyright now. And along the way, I brought in great leaders across the industry.

And they are building a pillar, a bigger pillar of the company under their realm. So for example, recently I brought in George Huo. He was a legendary operator in Silicon Valley. He worked with Mark Benioff at Salesforce, a Salesforce CEO, and CEO at Twilio as well.

So he is driving, like, building the whole entire GTM engine for Fireworks. And that part is growing extremely fast. And he is managing all the hiring for the GTM side. So I don't need to spend too much time there.

And recently I brought in Dirk Rajagopal. So he is also a very experienced technical operator that has been working on almost all stacks in AI. And so he came in and took over a lot of deep hiring work on the engine side.

So I will continue to hire researchers, our leadership team, and the various different and the product team in various different areas. So I think obviously it doesn't make sense for the entire company to report to me,right? So I do have very strong leaders I'm working very closely with to expand the company.

And those people are also carrying the same value and the culture norm that will help us build a consistent company over time.

You mentioned earlier around you were at Meta. Why did you want to join such a large company and then later, obviously some years later, go and find found your own company? You know, building a company is so hard.

You know, when you're working at a big corporate, you have a nice comfy lifestyle, you know, a good salary, and then you want to go into the trenches and actually go and build. Why did you want to go and build your own company?

I didn't join Meta for the stability of a big company. I joined Meta to learn. At that time, it's still called Facebook. That's 2015. I want to join a company I can learn how to build a startup. I want to look at a company that is at a later stage, but still with a startup culture for me to absorb, like, what does managing and, you know, high people talent density look like.

So that's my motivation to join Meta. And I did learn a lot. I would say at that time of

it is still in hypergrowth mode. Everything is quite chaotic. It is not that structured. I think when I joined, there's already 2,000 people. But the sense of ownership was off the chart. I was shocked. I completely don't have the, like, the expectation of anything surprising me.

I know I'm going to go there and learn, but I'm like, maybe it's not that different from my prior companies, but I'm going to go there and see. But I was shocked. The most important thing I learned is the sense of ownership.

Everyone felt deeply in love with the company. They really treat the company as their own. And that just kind of unleashed superpower within everyone. So I think these carry on into Fireworks is I deeply believe everyone choose to join Fireworks at our current stage has a fire of a founder in their heart.

They are not just adventurous. They are not just seeking for challenges or end opportunities. They are ambitious to manage uncertainty and pave, find, walk an unpaved path to excellence. So that's audacity. That's ambition. That is courage. But also it requires perseverance for us to kind of get to continue to innovate and penetrate through blockers and always find like, I talk to the team.

I describe our spirit like water. It doesn't matter how many rocks are there along the way. We always flow through around the rock and keep moving forward. Nothing will block us. And so that's the spirit where I believe every single one of us, like, joining this company, joining Fireworks, is carrying.

And that's part I deeply appreciate. And I feel excited, very exciting waking up every morning, working with them, solving all sorts of problems. So that's my personal style is I'm not a leader that just sit on my seat, receiving reports, giving just giving high level instructions and directions and the chill.

I'm an operator. My job is solving all sorts of problems within company. And I solve problem very quickly. And there are a lot of interesting thing we can talk about as a founder. And I'm pretty sure this audience has a lot of founders that is solving all kind of challenges.

Happy to kind of pick an area you're interested in diving to.

I love that company philosophy around we're all in this together and whatever the challenges, we're going to get through this and we're going to absolutely win and hopefully own itright. But I also think building a company is all about the founders.

You have six fellow co-founders, which I think is a pretty incredible, but also quite unusual because most companies only have maybe two, three co-founders at most. What is Fireworks all about? We should touch upon that. And the second part of the question is around how do you manage six incredible co-founding relationships at the same time?

Yeah. So I would say I'm very extremely lucky to be able to start this company with six of them. And they are each one of them are incredibly brilliant. And

at the same time, we are different. We're not all the same. So it's not like we're echo chamber of each other. That's why we get along. The interesting thing is we all have different perspectives. And one thing that really bound us together is we are extremely intellectually honest.

So that defines us is we're brutally intellectually honest with each other. And we rationalize company strategic decision or operational steps through logic. And because we are all engineering background, it's very easy. So that's our common language. But also at the same time, it's very important to have different perspectives.

So we have a much broader range of design space to rationalize and for us to make the best decision. And oftentimes we're sitting in a room, we do a lot of pre-mortem. We do a lot of post-mortem. We do a lot of reflection together.

We constantly discuss. So one of the challenge, I would say, the biggest contrast between how we operate at hyperscale, we all came from hyperscale. So that's kind of one interesting thing I can talk more about the funny conversation I had with my first investor.

We all came from hyperscale. The biggest difference is at hyperscaler, there's abundance of data, abundance of product information that you can make extremely precise calculation and decision of the outcome of a particular path. Operating a startup, we have the opposite problem.

We do not have abundance of the data because if it's that obvious, if we have a lot of data to analyze, anyone can do it. This is not a gig for startups. So startup is basically pave a path where there's no path.

And obviously, there's no data. So we have to operate a lot with strategic hypothesis and assumptions to build a map in front of us. And that's where we are brutally honest with each other and critique different ideas and make a decision.

So usually we go very broad and we shrink down. We cut all the branches and shrink down. And we are not fixated with a particular decision we have made. There's no ego. And we will, when new data comes in, we will change, we will validate the assumption we made and be honest with each other whether we made theright decision or whether we should continue to kind of iterate and adjust and tweak the direction we're heading towards.

So that feedback loop and that trust in holding each other accountable and truth seeking is the connective tissue binding us together. It's the deep respect we have to each other to operate together for four years. And we are going very strong into the next 10 years.

So yeah, so I think that's a very unique setting. I understand what the question you're asking because when I raised the first round talking with Eric Rischwart, he's the partner from Benchmark, when we first met, he was surprised.

I have so many co-founders. He asked me, are you sure you're going to start a company with seven of you? That's unusual. Obviously, the subtext is, you know, usually those founders don't get along and we'll have all kind of drama.

And there's so many cases that companies split because of the founder dynamics. But I think we are just an unusual collection of people. We are each other's strength. And we back into each other and create a much bigger force than every individual one of us.

How do you manage conflicts or disagreements when there are seven of you? You know, if you and I are on a team and we've got five others, I love something, you don't love something, how do you come to a common ground where you then all finally agree?

Usually when there are different opinions, it's not black and white. It is all about trade-offs. So we will like to put different opinion on the board and analyzing what are the underlying trade-offs we're talking about to leading to these different opinions.

And the trade-off has to be based on what is more important for the company to grow, what is more important for the product to be more loved by our customers, which customer segment is more important. And those kind of goes back to the foundation of product decisions.

And once we bisect that, then it's very easy to rationalize it. So that's the beauty of working with a group of people who are very, very rational. And we basically put everything on the table. So those trade-off analysis usually leading to, hey, we don't have data for certain assumptions.

And then it becomes simple. Not making a decision is not an option. Not making a decision is a bad decision. So I will make a decision when there's no clear data to or hypothesis we can support one or another, then I'm going to pick one based on my intuition and taste and move forward.

So that's kind of, but we will constantly reflect. So that's kind of the extremely strong muscle of the team. We'll constantly reflect and decide whether we should think differently. So yeah, so we have a lot of kind of stories along the way that we have changed the initial decision and pick one that which we understand better.

So I think a lot of startup founders would have to go through similar exercise as they seek bigger and bigger goals as they grow the company. I think that nature shouldn't be a foreign concept to all the founders.

If you're enjoying today's episode, hit subscribe at the bottom of your screen. The guests on the show are getting bigger and bigger, and we need your help to grow this channel. So if you're enjoying today's guests or any previous episodes, hit subscribe, and I promise you it'll be worth it.

Now back to today's episode. So let's talk about Fireworks. We're going through this incredible inflection point. I think '25 was all about the year of code. I think this year is all about the year of cowork. We should talk around, you know, what does Fireworks actually do and what is the current state of the company?

Yeah. So last year is absolutely, so it's interesting. We are observing the whole entire industry building on top of our platform. So that's one of the most fun parts of my job is to see the creativity and innovation going off the chart from all different kinds of application.

So you'reright. We can see the whole market. And last year is the year of coding. Coding is the biggest application, most successful segment last year. And now all coding companies are running on Fireworks. We started journey with Cursor.

We started to work with Cursor when they were 2 million AR. Now it's a thousand times over the past three years. I'm so thrilled being in their journey of hypergrowth. I think coding is a fundamental building block of all sorts of software development,right?

And it is a foundational skill that AI brought, a foundational skill that AI brought in to automate. And based on coding, the next step is knowledge worker because knowledge worker use a lot of digital assets. And those digital assets are also all sorts of software to help them being productive.

And because coding liberate the software development, and then the next step is the software used by knowledge workers are going through next generation of innovation. And many of those knowledge workspace, another interesting thing is coding is more kind of universal or more consolidated, but knowledge worker is very diversified.

Every profession has a unique depth of their own domain. You know, if you look at dentists,right, how many

different dental professionals are there? So it's kind of different categories. It's enormous number. So you can think about knowledge worker as a very bushy tree where coding is the trunk,right? So it's just kind of software engineer is most likely there.

Very few categories of differences. But knowledge work is a very broad bushy tree. And then each of the tip of the bushy tree is, you can imagine one AI native startups. They are trying to kind of automate. They are trying to kind of build smarter agents assistant to make a particular professional domain so much more productive, so much more creative, innovative.

So naturally, this year we start to see huge diversification of all sorts of application that comes into picture all the way from legal. But by the way, I start to learn legal has so many things. Legal has defender side, defense, offense, and legal research somewhere in the middle.

So it's very broad. Finance is very broad space. Recruiting, all sorts of recruiting, marketing, sales, customer support, cybersecurity, manufacturing, you name it,right? So supply chain management. There's so many all sorts of agents being built. And one interesting thing is it's not just the startups.

There is significant movement from the enterprise side as well. You can think about many of those enterprise, they were startups a decade ago, two decades ago, few decades ago. And they were the winner from the competition at that time.

And they deeply understand this wave is a defining wave for their future. They are all top down from board level investing in AI to compete with the rising startups because they have seen those movies happening all the time.

If you are not staying on top of your gig, then this is time to change the landscape. So the startups and enterprise, they're all moving. But then they face two big existential challenges. One challenge is lack of control of their IP.

The other challenge is control of their cost. So control IP is because of multiple megatrend. One is product development has been significantly disrupted software product development because of coding agent. It cannot be easier. In the past, product itself is a moat because it requires creative idea and ability to implement it with top-notch product engineers and PMs over multiple quarters that is considered to be fast and scale in production.

Now with coding agent, without knowing how to write a single line of code, it's good visionary person with great ideas can launch the product within few weeks and reach huge scale. And that collapsing of resource required and timeline required is completely shifting the competitive dynamics in the product space.

In addition, products cannot be easier to copy because even if people don't have original idea, it's now kind of through the coding agents very easy to replicate an idea. So then you will say, imagine a product built on top of which is very easy to replicate, built on top of a black box API.

That combo, that moat is very, very thin. So then what is the real moat,right? And where is the IP? That goes back to first principle. We think about in the past 100 years, recent 100 year, capitalism thrive because I know we have, I don't know how many, we have more than a million companies across the whole globe,right?

And those companies thrive because they solve a unique problem in a special way, exceptionally well. And the diversification of specialty is the value. And that's kind of the, that's the foundation of how our current economy works. And I deeply believe in that.

I do not believe in the future there will be one company across everything. I do not believe in that. Well, we can argue philosophically. There's these two future, one company versus millions or maybe tens of even, you know, a lot more diversified specialty companies.

I believe in the other side, not one company. So let's continue this train of thought. Then each company, because they're solving a special problem exceptionally well, they will have unique product taste judgment and design, unique for anyone else.

And now product development becomes so easy. And their uniqueness may not be that unique. And what makes them continue to be unique is the data they have accumulated from the product surface area interacting with the customer. And that's where the customer's preferences, judgment, and feedback is being collected.

That is their moat. And today, those are the private data will never get shared with any other company because this is their alpha. But interestingly, if we take a big step back, that alpha, that extremely valuable data is not being activated to create any intelligence.

We believe the future of the intelligence lies inside of those deep pockets of product development. And that's what we call specialized intelligence. And we believe every product team, app, enterprise companies, they should own that critical supply chain of intelligence of their own, not depending on other companies because then your moat is going to be very, very thin.

So that goes back to control of their IP. The control of their IP is to turn their private data into their customized model and continuously evolve the model together with the product. Second is control of cost. As we all know, starting from this year, it is a well-established sentiment across the industry.

This AI is going to, you know, a lot of company will build AI into bankruptcy.

So this is very interesting phenomenon as it was never exist. This concern was never even a topic to discuss in the SaaS era because product market fit and durable business almost means one thing. Once you hit the product market fit in the SaaS era, you just scale as fast as possible because the business operation, the cost of business operation is quite low.

It is built on top of commoditized CPU and storage. And then probably the biggest cost is people,right? So during SaaS era is one concept, but in the AI era, these are two concept. Product market fit would mean, hey, your ICP just loves your product and they're willing, they want to use it day to day, day and night, and pay money to you.

And they want to keep going. But it doesn't mean you will have a durable business because your cost of operating using AI could be much bigger than your income. And then when you scale, you literally scale into bankruptcy.

This is a real problem for startups because startups has to raise money, use their equity. And once they burn all of their money and they cannot grow their business fast enough, they cannot raise the next round, then they're pretty much dead.

But this is even bigger problem with enterprises who are the winner from the past decade or two. As they grow from a startup, they won the war of consumer-procurement developers. They have huge amount of traffic running on their existing product.

And when they roll out AI features, they're going to roll out AI features to all their customers. And they cannot afford that. Their CFO is literally going to look at their cost forecasting and say, I'm sorry, I cannot approve this because we just cannot, it will destroy our book in the coming quarter or even multiple quarters down the road.

So this is a real issue. It's the cost of, it's control of cost. So we came here to solve both issues through specialized intelligence. Using your private data, we can tune a highly intelligent model towards SOTA quality, but with much better unit of economics.

Here I'm talking about five to 10 times cheaper in the cost side. And that means you, for the same budget, you can grow your customer base 10 times bigger or with the same size, you can reduce your cost 10 times lower.

So this is while you don't send your alpha out, you keep that deep moat within the company itself. So I think it resonates with every company I talk to, with every company I talk to. The only question they have is how much it sounds like it's an investment.

How should we do that? We don't have talent or I'm not sure how to think about this budget. And I am happy to talk more about that because there's a framework we use to think about that.

Yeah, let's talk about that. What does that framework look like?

Oh, so basically for the company, they want to own their own intelligence. They should think about that as their investment,right? And this customization training is not one time. You don't train a model once and use it forever because your product keeps evolving and improving.

And the baseline model quality also gets better and better pretty fast. We launch new model every week, literally. So you want to continuously train this model as a continuous investment. But think about this investment under the umbrella of total cost of ownership.

And think about you investing 10% to 20% of your overall AI spend into training and the remaining 80% to 90% as your investment in inference. Because of training that we're helping you customize your own model and your overall inference cost will reduce by five to 10 times.

And your overall total cost of ownership will still get significant reduced even with the training investment by four to eight times,right? So that's how you rationalize this. And it's a very sound framework we've been using with many, many other companies.

You know, if we think about the opportunity today, the number of companies who are now trying to build their own models is only going to get bigger and bigger as the trust with these closed models potentially becomes a bit opaque.

For the likes of Cursor, Harvey, Legora, who are coming to you potentially and saying, hey, we would love to build our own open model. What does that process actually look like? And how long does it take to get a first version deployed?

So because we are in an early technology adoption curve, the customer profile has a range.

The range are the following in kind of rough three buckets. One bucket is the likes of Cursor. They have deep researchers in-house and they want to control every single knob to tweak and get the best quality. So we give them a very low level API so they can change anything as they want and be productive.

And oftentimes they also operate with very large fleet. And there we design our training system for GPU poor. GPU poor doesn't mean it requires less GPU to train. Oftentimes the training job could start with tens of thousands of GPUs.

It is a large fleet, but expecting these tens of thousands of GPU to be in one cluster, RDMA interconnected, high-speed interconnect for big training run is very expensive. Very expensive cluster, also very hard to find. Good luck finding those clusters.

And we design our system to be able to run through a fully tiered, disaggregated, regional, smaller cluster. We can assemble into a big virtual cluster to drive those big training runs. And that requires a lot of deep technical design because once it's a very large distributed system, then we introduce a lot of synchronization and kind of weight transformation that will introduce delay.

And during training time, introduce delay means there's a trade-off with quality. But we have innovation to really control and keep pushing the quality to the best. So this system can be used for all different customers, but we offer the lowest level of abstraction to the power users.

And the second category are the many companies, existing companies, if they have more than 500 people, usually they have a machine learning team. And those teams are there before, even before GenAI to drive machine learning algorithms or deep learning, even deep learning training job.

But those are much smaller model. They are transitioned to manage GenAI models. And they also want to control, have a lot of control over the training job, but they don't want to have all the knobs. That's going to be too overwhelming.

So we give them a middle level abstraction that is our training SDK. And they can use the training SDK to still control loss functions, control training algorithms, control how they want to drive, set up various different hyperparameters, and monitor how things are going and decide how they want to explore and do experimentation.

And the other category of customers is they do not have anyone yet in the company who can run training. So we will just, we will send our researchers to work with them, to pair with them and help them set up the initial run and bring that to a really good stage.

Along the way, we will educate and teach them how to do it themselves because we believe a strong partnership is to kind of really make them self-sufficient along the way as we help them solve the problem. And we believe, and those are extremely, extremely smart founders or product engineers.

They pick up really quickly. So I think it's going to work because this is an area, a lot of cutting edge companies, they can learn and do it themselves. So that's our third mode. And across the board, those different modes represent different level of product abstraction and different level of engagement for us.

But the beauty is they stack on each other. We don't build different systems for different modes. It's all kind of built on top of each other and it's kind of fully streamlined. So I'm very happy with that design.

I think that that is reflection of where the industry are and we always want to meet the customer where they are.

I imagine there's an additional mode there. If we see what's happened in the last few weeks, few months, you know, we've had a bunch of security hacks. Maybe there's going to be a bunch of national security concerns at some point.

As a company who is helping all of these companies build their own AI models, I imagine one of the biggest questions these founders have to you is, how do we trust you and how do we make sure our data and security is fully protected?

This is really deeply rooted in our company operation is first of all, we really respect privacy. So our company operate with ZDR. ZDR means zero data retention for both training and inference product. So that means data in and out gone.

So if a customer self-serve training, we don't even see anything. The platform, they just directly use our platform and do their job. And we also deeply understand training process, a lot of data. So you can imagine model is the new form of database.

It absorbs the data and kind of the knowledge can be used in many different forms. So we have various different secured data access implemented, including customer can bring their own bucket of storage where they have full control and we can read from that storage with their approved authentication.

It can be accessed through deep encryption. They can roll the encryption key. So anytime we will not have control of when and how we get access. They also need to manage, it's not just data security working with us, but also internally different roles from their company will have different type of data access.

Some will not have access to those data, training data. Some will have read-only access and will have write access. And we give them like role-based access control that they can manage data themselves. So those are areas we really care about and that is area also build deep trust between us and our customer.

You know, we talked about modes. So when all of these companies have good taste, good judgment, they have their own AI models, which is what we've been talking about, and they have security, what is going to be the next mode companies should be thinking about to stay ahead and to stay relevant?

I think everything boils down to data. So here's my extremely simplistic view. I think product is a vehicle for deeper customer understanding. So okay, so here I think product as is is not the moat. Product as is is a vehicle to start customer engagement and that customer feedback and deeper understanding is the moat.

So once you get the, you get the wheel rolling, then you keep rolling faster and faster. And then that understanding to turn into flying wheel,right? That flying wheel is better product design, it's better intelligence to power the product.

So I do see a trend that's happening in the industryright now is all these innovative product company, they are evolving to be a hybrid. The product managers or product engineers, they're also researchers, training models. And researchers also carry, they naturally think about product.

These two are becoming one thing. So for example, whether you train a model or not, every product company, if they use AI, they need to write evaluation,right? Evaluation is judgment. And evaluation is needed if you want to switch models.

Just kind of use API as is. You want to switch from GPT models, from Claude models to also open models or your own tuned models. Even for you to make that judgment call, which models to switch to, should you catch up with the new release or not, you need those evals.

And typically people think about researchers creating evals, but that's actually product judgment. It's a product team create evals. And once you have evals, then you can start the training and you can start the prompt tuning, you can start the context tuning, you can start a lot of things.

So, but we see the companies that move the fastest are the companies who invest in evolving people to be hybrid or they hire people who can be the hybrid. And we start to see the convergence of product people also doing research.

So that's a kind of new trend, but I'm happily surprised. I've seen so many people who are capable of doing both. That is amazing.

You know, when I think about the huge opportunity for Fireworks, you're providing all of these incredible tech companies with their own infrastructure and with all their models. But then when you think about the GPUs and data centers, is this potentially a market where Fireworks would enter at some point?

I think I will refer back to one thing Jensen mentioned. I think he's, I don't remember the exact framing, but the idea is to do the minimal thing that we are best at doing. So I would say, I don't think Fireworks will be the best building data centers.

We can, we can do anything if needed, but we are really, I think we're the best building a specialized intelligence platform and we want to double down on that. So in that regard, we would love to partner with companies who are the best building data centers, who are the best operating GPU fleet.

And there could be an intersection point as we work a lot with hyperscalers, we work a lot with NeoClouds. We start to kind of give each other feedback. We start to kind of influence their roadmap. We start to be more engaged in helping them design things and design kind of very robust operational cadence of thinking through how to put pieces together.

So we start to get more involved because we want them to thrive as our partner. We want to teach them like from our vantage point of view, what does good quality of service look like? And if they're successful, we'll be successful.

So we were more thinking about a co-development mode where we double down on the strength from both sides and really provide multiplicative outcome. So that's our current thinking. And I really respect a lot of very creative, innovative founders building NeoClouds these days.

And they're really outstanding ones.

You know, when we also look at the opportunity between closed and open models, if these enterprise customers are preferring to partner with the likes of Fireworks with open models versus the likes of Anthropic and OpenAI with closed models, where does the value lie with these closed models if companies preferring to partner with people like you on the open model side?

I think I will call it as this. So first of all, I really, I deeply admire these two companies, OpenAI and Anthropic. They are brilliant companies, but we are executing on two different strategies. They're executing on the general intelligence strategy.

We execute on specialized intelligence strategy. I think both strategy will coexist into the future. But I do not believe the world will be dominated by a few models, a few generalized intelligence models. I think the world will be millions of specialized intelligence models, continuous updating and improving, owned by every single company and every single use cases.

So again, I think the generalized intelligence will more evolve into become a utility and it will be the baseline to power the future economics. But we are building into the essence of capitalism where diversification of specialization is the bedrock of the next phase of economics built out.

And we want to give everyone the tool to own their own intelligence.

You know, one of the best things about doing a podcast show is, A, you get to speak to the most incredible AI leaders, but you also get to learn what a week is actually like with them. So one of the questions that we love to ask on the show, if I was to join you as Chief of Staff for a week, what would that week look like and what would we be doing together?

Actually, this is not just about joining as Chief of Staff. For all the leaders joining me, initial few weeks, they're going to shadow me. So basically how they're going to go to the meetings I go to. And here's how I spend time.

I spend time meeting customers. I spend time recruiting. I spend time solving specific problems. For example, I will solve problems of, hey, here's a recruiting bottleneck and here's how we're going to restructuring the flow of recruiting and approach of recruiting.

So we got to kind of be efficient and still tap into the highest talent. They're going to spend time with me to drive capacity. There are a lot of capacity prioritization discussion and capacity procurement, future planning, growth discussion.

They will join me on product discussions in terms of, you know, product priority, people who we're going to have to accelerate what part of product, part of reflection, is this experiment successful or not? Are we going to double down or are we going to shut it down?

It will be a wide range of strategic, operational, even tactic things. A lot of problem solving we're going to do together.

You know, it'll be silly not to ask, but how do you actually use AI personally on a daily basis?

So first of all, my whole entire company is AI-built. And the whole entire company actually built on top of our own stack. So we have a, we have a product called Nexus. Nexus is an intelligent router. It's multiple models.

Our we train also ourselves. It can route across different models. Our own model, the model we hosted in the Frontier Labs model to solve all sorts of problems from software engineering to GTM to finance, even legal. So that actually significantly increased the productivity across whole entire team.

And I myself also use that. And that is kind of one of my way to stay very close to every day what's happening because those AI tools are very good at summarizing and distill the harder bits of what's going on.

But more importantly, I would like to observe where are the bottlenecks? So remember, we all operate, I want the whole entire company operate like water. Our job is flow around rocks, roadblocks. And my job is flow the company around the biggest roadblocks and keep pushing forward.

So understanding where's the biggest roadblocks and I'm pretty effective in unblock through various different resources and vehicles and getting a very close tap to what is going on and it's very important. So that is very helpful. I think through AI tools, I got a very good understanding of what's happening across the company.

And my leaders, my team across the board, they're also using AI tools to kind of make themselves very productive.

And the productivity shows, I think you're processing 40 trillion tokens a day. If we were to come back in 16, 17 months at the end of next year, how many tokens do you think you'll be processing a day?

Oh, that's really hard. I think, so let's move forward a year. I think there are multiple factors. One is the demand is definitely, my prediction is the demand is going to go up. Two is the token efficiency will go up too.

As in model preserving the same intelligence is going to be smaller. And the model to deliver high quality result requires less thinking token. So the token volume to reach a good quality result is also going to reduce. So there are multiple compounding effect.

I believe in a year, reaching a hundred times more token is possible. So that's kind of where things are. Obviously, there are a lot of assumptions I make here, but that's through my observation of how the AI space is moving.

It's moving so quickly. I'm super excited for Fireworks. What you guys have just done, you know, in the last three, three and a half years, I think it's incredible. And hopefully, you know, you can come back in 12 months and share the latest, but really appreciate you for coming on the show and thank you so much.

Thank you for having me.