[0:05]Today we're going to talk about AI at uh Ramp and uh I'm going to give an intro quick introduction into what Ramp is.
[0:13]Um really briefly, we're going to walk through the simplest possible expense use case that you guys can all resonate because I see everybody's drinking coffee.
[0:22]And um then we're going to uh talk quickly about a lesson that we learned uh this year why we were building agents.
[0:33]Um and sort of the pivot in the paradigm that's happening, especially after February 6.
[0:38]And um then we're going to double click onto how we built one of our most popular agents, the Policy agent.
[0:44]Um and then finally, we'll dig in into the infrastructure built that this is requiring requiring to do on our side.
[0:53]And in my mind, most importantly, the culture shift that needs to happen on everyone's teams in order to be able to operate in a way that delivers products into the hands of your customers in the fastest and most impactful way.
[1:07]Uh so without further ado, uh quick intro about Ramp.
[1:10]Uh we are number one finance platform for modern businesses. We have 50,000 plus customers and we're in the business of saving you time and money.
[1:21]Uh we have uh I've seen some of the uh some of those names on the on the name text here, so thank you for being around customers. Uh
[1:32]Really exciting. Uh really quickly, so a cup of coffee uh takes uh usually about 15 minutes of your time. Um because you got to do this three simple things which unfortunately take minutes.
[1:45]Um this compounds through the company.
[1:48]And uh what Ramp does in the simplest possible way, we just condense time and uh return money back.
[1:56]Uh so a simple story of a transaction from tap in the card to write in a memo, to classifying the transaction according to your GL to sourcing the receipt, attaching the receipt.
[2:12]Um normalize the merchant to your um inventory or merchants uh is all done at Ramp. And this was our first foray, uh probably by now uh you guys still hear me? Yeah. Probably by now about three years ago, we started doing this one-shot things with AI.
[2:24]Uh normalize merchant, write a memo and it's been working really, really well as the models get better.
[2:29]Uh what else is going on at the company? Well, literally every persona uh at the company is wasting time on a lot of manual work.
[2:40]Uh so from AP clerks to a finance team, from your purchasing teams. Uh keep going to more finance work, your data teams. Uh at Ramp, we used to have a channel called help data where somebody will ask for a CSV and a poor person will go and write a SQL query.
[2:59]Uh we replaced it about a year and a half ago. Uh so a lot of time being spent and the complexity has a Ramp shape. It only increases as you go through different jobs to be done.
[3:10]Um so if you guys watch Super Bowl, uh you might familiar with Brian, uh our agent. Uh so we've been writing a lot of agents, literally for every job to be done to cover the entirety in the end state, the entirety of what admins, employees, and finance teams uh doing that is not directly related to making the money.
[3:32]We want you all to be making money and focus on your customers, not on how to close the books.
[3:37]Uh but what's been happening for the past few weeks is uh they're living through the most exciting paradigm shift in software.
[3:45]Um and it requires complete rethink and with rethink, simplification of your stack.
[3:54]Um so what we learned is you don't need to build a thousand agents. We intentionally last year allowed each individual team to go and experiment.
[4:02]And we ended up maybe with four different ways of doing the same thing, both for synchronous agents as well as for background agents.
[4:10]Um but instead, you want to drive your framework towards a single agent with a thousand skills.
[4:20]Uh so let's talk about what the software traditionally used to focus on. So every process, uh especially in the modern modern AI stack, boils down to having an event.
[4:30]Um so a Ramp, you can receive an invoice and you want to pay it. Um some prompt instructions of what you want to do with it and some guard rails, like a policy, uh like an expense policy or your payables policy.
[4:42]Um context, what is the data that the agent should consider? And then finally, tools, these are APIs and actions that you can do.
[4:54]And traditionally software would focus on only four and five. Uh in the new paradigm, software is doing everything.
[5:00]So you want to focus on building an autonomous system of action that can react, reason, and act without a human or with very little human supervision.
[5:10]Um so what does it mean in terms of what we're building?
[5:16]So first, uh we decided we're going to consolidate the interactions, um verbal interactions uh with the agents to a single conversational UX.
[5:30]Uh we literally at the end of last year, we had about five different conversational UX's. We now have consolidated it into what we call an Omni chat.
[5:37]Omni meaning for omnipresent. It is now being deployed to every surface of the product and it works well with the traditional UX because you still need tables and buttons and uh you don't always want to be talking uh to your software.
[5:51]Um but this is a good example of what Omni chat looks like. Uh please on board a new employee. Uh Omni chat can resolve an employee to an employee ID and look up through an HRES tool their corporate structure.
[6:05]And it found a workflow, a gentic workflow that we created previously called the new higher playbook. And the agent is asking, would you like me to to onboard the person using this playbook.
[6:14]How is this possible? We built a in-house lightweight agent framework uh that provides orchestration with uh tools that engineers are very quickly building and most recently, uh we have one product manager vibe coded about 20 tools, so engineers are no longer needed to build those tools.
[6:33]Um and sometimes your workflows are involved, such as employee onboarding consists of four steps. So you can just go and ramp and describe what do you want to happen when a new employee joins, give them a card, uh make sure they get receipts for every transaction.
[6:49]Congratulate them on slack and check in with them in two weeks. We now are able to compile this into a runnable, deterministic workflow. Um and then give it to the agent to execute.
[7:01]Playbooks make use of tools. And how this all comes together, um this is an example which Viral is going to double click next is uh upon swipe in the card, uh there's a real-time policy review that's happening uh directly in the software.
[7:18]And policy agent enforces your company requirements with regard to spend. Um therefore, it's very safe to give ramp cards to literally every employee in your company. And there's a handoff happening with an accounting coding agent that uh classifies this transaction.
[7:35]Applies the rules of your uh back office team, of your finance team. As an employee, I have no idea how certain transactions should match to our GL and that's what typical traditional products would do, they will expose it to you.
[7:47]Um so the agent is much better doing it because it has the full context of your chart of accounts, it understands your ERP. And then it can either auto approve or in the worst case scenario, it will involve uh the human and the loop to review materiality or notify that there's an out of policy spend.
[8:05]Uh with that, uh please welcome Viral, who will uh dive deeper into the policy agent.
[8:13]Thanks, Nick. Awesome. So, a lot of finance teams are looking at receipts like this basically every day.
[8:26]And maybe they might have hundreds or thousands of these. If you told me to look at this and decide if I should approve or reject this transaction, I'm probably going to make a mistake.
[8:35]So, Policy agent basically reasons on this image and all the transaction data that we have and told me that there were eight guests in the receipt. I could barely see that when I was looking at it.
[8:44]Uh it was below the $80 per person cap that we have internally. Uh they were going for a team welcome dinner. Uh and so because the amount was verified as well and the merchant, uh Policy agent told me to approve this transaction.
[8:58]Similarly, for this OpenAI transaction, Anan was testing out uh some some ChatGBT features and so Policy agent told me this was a valid uh business expense and told me to approve it.
[9:10]And then this $3 bakery charge was told uh was was uh rejected because uh it wasn't uh part of an overtime purchase and it didn't happen on the weekend.
[9:22]So, really, we looked at this as an opportunity to rethink how Ramp was set up.
[9:28]Um controllers and finance teams are looking at transactions like these and and making these decisions every day. And a Fortune 500 company that is one of our customers was coming to us and saying, hey, can you uh make sure that you approve these types of expenses and reject these types of expenses?
[9:44]And they basically had a list of all the rules that uh Ramp uh should should follow. And we kind of saw this as an opportunity not to kind of add more incremental deterministic rules that kind of defined our product and I worked on some of the first versions of these.
[10:00]Um but actually kind of turn the expense policy into the rules themselves. So, um you can you can see Ramp's expense policy on the left and and this is a screenshot from our production environment.
[10:16]But we are seeing really great uh use out of our policy agent product and it kind of needed to start it kind of needed to start really um organically.
[10:25]So we kind of operated like an early stage startup. We're already very incremental and and fast at Ramp, but uh we found some design partners like that Fortune 500 company.
[10:35]We iterated really quickly and we had weekly meetings with all of them to kind of understand exactly what uh feedback we wanted to hear and what what we can improve.
[10:46]I think one of the main important um, I guess, things that we realized across uh Ramp is that we really needed to lean into the fact that AI products cannot be one-shotted.
[10:57]You need to start with something simple and so as long as everyone on your team, PMs, designers, engineers are aligned that you're not going to have perfection on day one, I think that was actually one of the main like cultural learnings.
[11:11]Um and so we dog-footed a lot of this work internally and uh started with an even more constrained problem of trying to decide whether our coffee with a colleague transaction should be approved or rejected.
[11:21]These are single uh uh dollar amount transactions that are low risk um uh according to our finance team and so we started uh with these transactions and uh one of the early learnings, especially as we kind of release this uh into production was that a lot of the reason that Policy agent would be wrong would be less on the models themselves and more about the context that we were giving uh to to LLMs themselves.
[11:45]So, uh we we could have sat down and thought about all the context in the beginning before we even kicked off any engineering work, uh but we realized actually the best thing would be to learn from some of our live internal data.
[11:57]And so, uh for example, we learned that the role and the title of an employee is super important when looking at expense policy docs or in levels. C-suite, for example, might have higher limits, maybe they can fly on first class for for certain flights.
[12:09]And so we started extracting more information from receipts, started pulling in information from HRS fields that are already on Ramp and so, um Will is going to kind of talk you through exactly the iterations that you went through to implement Policy agent and and uh some of the learnings along the way.
[12:37]All right, cool. Um awesome. Um so when we first started building the Policy agent internally, um we dreamed big. We're like, hey, let's automate all of finance, let's automate all reviews.
[12:48]But when it came down to it, we actually had to start small. Um is that cup of coffee, you know, in your expense policy. And the reason that we did that was because even though the problem sounds simple.
[12:59]To automate, you know, is this a simple question, is this in policy or not. Um it was going to grow to be complex.
[13:05]Kind of like Viral said, we could have gone down and we could have figured out what context do we have, how can we add it, how can we put it all together in a way that LM can understand and, you know, put it all together from the get-go.
[13:16]But we knew that even if we aimed and got everything right the first time, it was probably going to be wrong once you apply and generalize it and then to another business.
[13:27]Um, so the simpler the system, I think the easier it is to iterate on top of it. And once you iterate, you know what's going to work, you know what's not, and you can kind of layer complexity on top of that.
[13:33]And I think that's pretty important to um keep in mind when you're building a um LM or an agent.
[13:41]So for us, we started really simple, very, very um kind of the classic, you know, we have an expense coming, we retrieve the context around it.
[13:48]We pass it through a series of LM calls that are very well defined of like, hey, is this in policy? Why is it in policy? How can we show the user that's in policy? And then given an output that um makes sense in this way to the user.
[14:01]Eventually, we learned that each expense is kind of different. We can classify an expense based on is it travel, is it a meal, is it entertainment, do conditional prompting.
[14:09]And then retrieve context based on that and then pass it through a series of LM calls and give it some tools so that you can also autonomously decide, hey, um I need flight information actually or I need this employee's level um and kind of layered that on top.
[14:22]And a few iterations later, we came to a full on agentic workflow. Um we ended up with um complex tools to read across all of our platforms.
[14:31]And these tools are shared across our all of our agents. It's not just for Policy agent. We have a company internal toolbox that all of our agents are easily can, you know, reach into and use. And we gave it the um we gave it the um capability to write as well.
[14:45]So it's now writing decisions, it's writing uh reasoning, it's writing auto-approving expenses on user's behalf. Um and it goes in a loop. So, um you know, now it's more of a black box and that's kind of the trade-off you get.
[15:00]Um as you go from simple to complex systems, um your capability goes up, your uh autonomy goes up, your agent's able to do more, your AI can do more, your AI seems smarter. But in exchange, you're going to be able to, you're losing traceability and explainability.
[15:13]Um we look at it now. We can kind of look at the reasoning tokens that the LLM gives us, but in the end we have no control over it.
[15:19]It's going to do what it thinks it's right. It's going to make the tool calls, it's going to tell you it's right or wrong. So, a smaller black box becomes a bigger black box as the system becomes more complex.
[15:29]So, one thing that is really important, um when doing something like this, is from the beginning, you need really good auditability.
[15:35]Um assume even if you know how it works, assume that your inputs and outputs are all you know and make sure that it's correct.
[15:43]Um so, if it was a black box system and you only saw the input and output, can you verify that it did the right thing? And even if that black box changes, you should be able to reason about whether the output is correct.
[15:55]Um as with many products that we built at Ramp and across, you know, other companies, we thought that the users were correct. Uh you know, if the user says approve, the agent should approve.
[16:03]If the user says reject, the agent should reject. But, turns out, the users are actually incorrect. They're wrong. They are sometimes, you know, they don't know the expense policy, you know, they trust their employees, they're lazy, it's a Sunday, who knows.
[16:16]Um so, turns out, we can't always do what the users are doing because sometimes that's where uh finance teams come back to you and are like, hey, this is wrong. This shouldn't be on the uh company card.
[16:27]So, we have to define our own definition of correctness. Um and to do that, um we had a weekly labeling session with across functions that are working on this product.
[16:36]Um and that had two um kind of really good um outcomes. One was that we had a ground truth dataset that we could always test against and we knew that this was correct.
[16:44]And two was that everyone was on the same page. If our agent got something wrong, everyone knew that it got it wrong or, you know, our agent is missing context. Everyone knew that it's missing that context, so there was less communication.
[16:55]Everyone's on the same page and um we could focus on what's really priority and kind of have alignment on that.
[17:03]Initially, um getting all these people together in the room every week, giving their homework to label 100 data points, it's expensive. You know, everyone, everyone has things to do. Sometimes they don't come back with their homework done and it's it's kind of like come almost becomes tedious, even though it's so important.
[17:18]So, we wanted to make it as simple as possible. And the way we did that was that we looked for third-party vendors that could provide us the tools to label data and collect the data.
[17:27]But turns out, some tools are too specific to our use case, some tools are too general and we could have spent weeks trying out different tools, but we decided, let's just build our own.
[17:34]Um so we used cloud code using Streamlit. We basically one-shotted all of this. And the greatest part of it all is that it's low maintenance, um low risk. It's a particle base that if it breaks, we can fix it right away.
[17:47]The deploys are happening like instant seconds. And non-engineers can go and personalize it. They can, they can vibe code it, they can cloud code it. And this was in Opus 4. So now it opens 4.6. I expect it's even better and um something like that it's definitely easier and cheaper sometimes to do something one-off like this.
[18:04]And with that, with the ground truth dataset, we were able to make quick iterations. We're able to find out, hey, we need employee levels, add that, how does that work, run it against this dataset, does it actually catch it and now say accept or approve.
[18:18]Um and we're able to make really quick iterations and that was kind of the key, uh that was actually kind of a key point in developing this.
[18:24]Uh we had really early confidence that this could actually work and we were able to actually by get a lot of buy in, um get a lot of customers on board and that kind of try it out as a design partner.
[18:36]Um and as part of like doing that iteration with the dataset, you had Evals and I feel like Evals are really, you know, obviously everyone, I think can assume that knows about Evals and what they mean.
[18:47]But, um it's pretty important to have them early on. I wouldn't say that, you know, don't let perfectionism, you know, get in the way. You don't need a full dataset of 1000 data points you're testing against every iteration.
[18:55]We started with five, you know, and we knew that those five, we're not going to fail. We kept adding and adding and adding. And, you know, make sure it's easy to run.
[19:03]Anyone could go and just run that command. And then make sure that the results are really easy to understand. Um they are able to look at it, get instant, you know, output, like and understand, like, hey, this is what the model's doing.
[19:11]This is like good, this is bad. And like if you run it as part of your CI, then everyone now can safely uh merge in code, um because whenever, um whenever you think you're doing something right, for the LLMs or agent, give it more context, give it more tools. More likely than not, it's probably going to have some kind of bad, you know, consequence that you didn't see happening.
[19:33]Context rot um whether it be the tool instructions are wrong or maybe the doc string was like a little bit confusing and conflicting. Um so it might have consequences you just want to you just want to make sure you're catching against those.
[19:43]Um and then I'll touch on it briefly, but online Evals are also great. So these are offline, you have a dataset, it's historical, you're testing it, but if you can, online Evals can be a little more confusing and uh harder to kind of measure.
[19:53]But if you can measure anything that as your users are interacting with the system, definitely as a leading metric, go set them up. And for us, part of that was, hey, how many are rates of like decisions?
[20:04]We had an unsure decision which is which just meant that the agent didn't have enough information, so we can measure that online. You know, it's a much simple Eval, but that also gave us a pretty good health check um as our system was running.
[20:16]Cool. And another great part about Evals is that with Evals, you can make confident model changes. Uh whenever a new model comes out, Opus 4.6, GPT 5.3, you want to make sure that you can leverage those new models because sometimes that could be the difference between, you know, your system getting one part of the problem right to wrong, but it could also be the opposite.
[20:34]It could have, it could actually be not good without any prompt changes or changing how your system works. So, um having Evals really set up and being able to benchmark really helps um make confident model changes.
[20:46]Cool. Um so now that Policy agent, we've been developing this for a while, it's available for everyone on the Ram platform. Some of the things that we learned along the way is that um cloud code as engineers is very exciting.
[20:57]We have full control, we get to modify our cloud MD, we get to make sure you know, tell it to not leave comments, it won't leave comments, hopefully. Um turns out, it's not just us. Um finance people also really like to have, you know, modified their cloud MD, which is their expense policy.
[21:10]So if something went wrong with the decision, then we just like tell them, hey, go update your policy doc, which to them, it's a little scary concept to begin, like this is a document.
[21:20]Like, you know, you don't mess with that. Um you have to go through a lot of hoops if you want to mess with that. But then turns out if you give them really excited about the feedback loop, hey, change that, you'll see it right away.
[21:26]Turns out they would be like really excited to do this. Um and then trust builds over time. So some of the earlier customers that we had were some of the Fortune 500s.
[21:35]We actually started with a really big, you know, um enterprise customers that we had because we thought that they would have the most value. They have the most expenses coming in. They have the most time spent on reviewing coffee expenses.
[21:49]Um so, you know, roll it out to them. Let them have the trust. Don't we didn't do any autonomous action. We're just like, hey, we're going to give you a suggestion. That's just how that's kind of how we phrased it. Suggestions.
[21:58]And then eventually, they came to us and we're like, okay, you know what? I want to go from suggestions to auto approvals. Like anything under $200, you guys are mostly right.
[22:04]I don't care about this. Let me just go auto approve it. So we gave them the autonomy slider. We gave them a way to just be like turn it on and then they actually could do it themselves.
[22:15]And then last but not least, um similar to LLMs, users thrive with, you know, in product feedback loops. Um so, you know, when you're building a AI product and you have a full way of like LLMs can test if its code was right and then took a iterate.
[22:29]Users are the same way. Um give them in product ways to improve the expense policy dot, improve the agent and how it operates. And um they're more than likely to kind of take it over themselves and um kind of improve it and personalize it for them.
[22:42]So, um from here, I'll pass it on to Ian who's going to kind of talk about the infrastructure and the culture that we have at Ramp that kind of, you know, led us to building the Policy agent.
[22:56]Hey everybody. So you've heard a little a little bit about like how we're kind of getting leverage to all of the different finance teams as we operate on top of their financial infrastructure and really try to get leverage for our customers.
[23:07]Um but I think a big thing that we also spent a lot of time thinking about is how can we get leverage for Ramp itself, the engineers, our XFN works, all the people that we work with um every single day.
[23:20]Um and this slide is this section is pretty intentionally named AI infrastructure and culture because we think that this is both like a really challenging infrastructure problem, but it's also really challenging culture problem and changing how you work as well is a big part of the story.
[23:35]And so to kind of start on the infrastructure side, the core of how most of applied AI happens at Ramp is our applied AI service.
[23:44]And at like a 10,000 foot view, this looks something kind of like an LM proxy or something like light LM.
[23:51]But there's really three kind of main extensions that we've invested in to make this a lot more powerful for a lot of our use cases.
[23:57]The first is like structured output and consistent API and SDKs across different model providers. This can be pretty tricky to do, especially with how quickly the APIs are changing, but it's a problem that we don't want downstream product teams to have to think about.
[24:10]So if you have an idea of I want to switch from uh GPT 5.3 to Opus or I want to try Gemini uh 3 Pro, you should be able to do that with a config change and really quickly be able to iterate on semantic similarity and trying to do a bunch of different um you know, code sandboxing and structured output calls that way.
[24:28]The other thing that we've spent a ton of time thinking about is kind of batch processing and workflow handling.
[24:34]This is really useful for Evals or if you're doing like bulk for us, bulk document or data analysis. Um and that's something that we also don't want teams to have to spend a bunch of time on of how do you want to batch this and handle it with rate limits and do we want to do this on an offline or online job with something like Anthropic.
[24:47]We just want to handle that for downstream consumers, so they can just focus on providing value for downstream customers. And then the last, which is a pretty big deal is the ability to trace different costs across teams and against products as well.
[25:00]And this allows us to kind of identify the paradol, you know, curve of like what is the best kind of model performance for cost, how are these evolving over time, what teams are actually not, you know, building something that's going to be sustainable long term for different product surfaces.
[25:11]And this can be really, really important to remove all this work from internal teams having to think about this.
[25:20]And the last thing that's kind of, I think, funny to think about, we often joke about that, that, you know, our customers are actually using the front, more of a frontier model than they may even know even is out yet.
[25:28]Is it allows us to see at the frontier when a new model comes out, it's a one line config change that impacts every single SDK downstream. And so rather than teams having to learn the SDK or go into 12 or dozens of different call sites, they can just change it in one place for their specific team.
[25:44]And they now get the benefit of being on the latest and greatest models, um that we've kind of vetted and built into the rest of the system.
[25:53]Our product, as you've kind of heard earlier, earlier, works on a lot of like very sensitive data and very sensitive workflows.
[26:01]And I think oftentimes, uh, you know, something that I hear from engineers in the space is this kind of concept of hallucination and safety and how are you actually going to be able to produce a lot of these things to have benefits to downstream finance teams.
[26:12]And we're pretty big believers that it all comes down to the catalog of tools that teams are building and integrating with on a daily basis.
[26:19]And so what you're seeing here is our internal tool catalog. So an example would be like get a policy snippet or a perdium rate or a recent transactions.
[26:30]And these are built alongside of product teams to really understand a lot of the nuances in the data and the use case. And what's really cool about this is not only can you see where there's gaps in our offering that, oh, we actually don't have a tool for this specific use case.
[26:40]These can be used both in internal repos and our core product. And so if you have an idea of I want to do a cool reimbursement agent idea, here are the different ways to integrate the tools, the different APIs and systems that they integrate with and now you can prototype that on a totally new product and vibe code surface area without having to worry about like learning all of these things from scratch or building the tools on your own.
[27:09]So, uh, thank you. So it's been pretty interesting to see the impact that Ramp inspector has had.
[27:19]We're over 50% of PRs that we merge on a weekly basis goes through this system. And so with all this time not spent on thinking about these really low level fire fighting tasks or really low level small fixes or tweaks that can be kind of democratized across the company.
[27:36]We're really rethinking like how our engineering teams operate and think about their job and how they can actually be really impactful in this new kind of AI native future.
[27:48]And so as a thought experiment, um we let's pretend we have two different teams.
[27:51]I'm sure everyone in this room has worked with like their handful of extraordinary teams, maybe teams that are finding their footing. And you'll notice that there's like a couple of different qualities that may sound that may resonate.
[28:03]So we have Team A on the left here and let's say that they really care about impact, they handle ambiguous problems, they understand the product, business and data, they adopt new tools. They can find creative solutions and they obsess over like the user experience.
[28:16]And then Team B may also resonate with some people. You know, they debate libraries, they add process when things things start to feel chaotic, they constantly complain about head count.
[28:25]They bikeshit the details instead of actually focusing on the user experience. Like, hey, should we use, you know, functional programming paradigm here or what version of, you know, different TypeScript libraries do we want to use?
[28:37]And then they build before understanding the problem, right? They just say, hey, we're going to just vibe code this bro, don't worry or they focus on, you know, performative code quality or nitpicks that may not actually, they may be very much like a subjective kind of matter of fact as well.
[28:50]I've worked on both of these teams. And I think the argument that I'm going to make today is that there's going to be a divergence, I think, depending on what side of the aisle you land there.
[28:59]This is a study from Harvard that was out, I think, uh the end of last year. And it was very much geared towards juniors and and seniors in terms of what's actually happening with hiring trends in uh in engineering since AI tools have accelerated.
[29:11]And I think what this gloses over is I don't think it's just a years of experience problem. I actually think it's very much um all of the different qualities that I said in Team A versus Team B that really make it apparent that like coding was never really the hardest part of a lot of jobs for a lot for a long time.
[29:26]And there's all these other engineering principles that become really important than just raw coding speed.
[29:33]And so when you think about like a staff or a staff plus engineer, you're really compensating those people more for a lot of the judgment that they bring to the table, the context, the ability to see around corners, all the learning that they have, the actual like scar tissue.
[29:48]And so if, you know, you ask Opus 4.6 to do something, they'll have the knowledge to actually know if that is not going to work or that's actually a bad idea. And I think one thing that a lot of the narratives that we see in the media gets get wrong about coding agents is they don't really identify the pack that you could still build the wrong thing just a lot faster.
[30:04]And you can build like bigger messes. And I think that having a lot of these skills of a team A and really focusing on like what is the context and reason behind this, we'll only become more important in AI.
[30:16]And so what does that actually look like? We hit on some of these things. Figuring out what to build and understanding users well enough to know what they actually need. Selling an idea to skeptical stakeholders.
[30:26]Making good design decisions with incomplete information. And maintaining momentum through the long middle of a project, which can be really gnarly.
[30:33]And I think this last bit, you know, everyone in this room, I'm sure is painfully aware of, you know, the conversation around SAS and and the stock market and things like that.
[30:46]And I think this is like a big element that they glossed over, which is that, yes, it's easy to vibe code something, but actually going through that middle process is like why you need really good engineers to actually get something deployed that has product market fit, that people are really excited about. Um and I think not enough people recognize that.
[31:10]And so where does that leave us? Personally, I think there's a lot of kind of doomism and and scariness around a lot of the AI narratives.
[31:17]But I think it's also a really exciting time to be building. Unlike maybe factory work or farming, software is never done.
[31:25]We have this uh really kind of like meme internally where we say, you know, job's not finished. You've probably seen in the marketing as well.
[31:32]And I think software is perpetually not finished. And so with all this extra capacity, with people focusing less on this kind of low-level work and more on high leverage engineering tasks, I think four things are going to really happen.
[31:46]I think companies are just going to chase opportunities they couldn't afford to pursue. I don't know if we would be chasing these like agentic workflows and really thinking about bigger scale problems in the financial stack, if this technology didn't exist.
[31:58]People are going to enter adjacent markets. They're going to try to stitch together more value for customers. It's not going to be like because everyone's two X more productive, you need two less or half the people.
[32:08]You're going to rebuild systems that are too expensive to touch. I think building an internal background coding agent, uh for a company that does financial operations, software felt like probably a pretty crazy idea, but now that makes a ton of sense.
[32:21]And raise the bar for what good enough means. I think being able to kind of build more mind-blowing experiences for users, provide a lot more value is going to be the narrative of the next decade. And I'm super excited to be able to build some of these things and see what everyone else in this room is going to build too. So, thank you.
