Luke: [00:00:00] You’re listening to a new episode of The Brave Technologist, and this one features Andy McMahon, who is the Principal AI Engineer at Barclays, and author of the book Machine Learning Engineering with Python, and a guest lecturer at Oxford University. He trained as a theoretical physicist, but has spent his career making production AI systems that are safe, governed, and uniquely valuable in some of the most regulated and risk-controlled environments in the world.

In this episode, we discussed how evaluation and telemetry is key for agentic operations, importance of building internally first where it’s easy to play, and how factoring the value of what you’re building is as important as being able to build the thing itself. And now for this week’s episode of The Brave Technologist.

​

Luke: Andy, uh, w- welcome to The Brave Technologist. How are you doing today? I’m great. Thank you. Excellent. Excellent. We’re at the, uh, AI Summit in London. I think you had a talk this morning about, agent operations. what do you hope people took away from that, and kind of what did you cover?

Andy: Yeah. I

covered three different core concerns- Mm-hmm … for productionizing agent systems. So one [00:01:00] was observability.

Luke Mulks: Okay.

Andy: So, you know, how do you capture telemetry, metrics, logs, traces for your agent and agentic solutions?

Luke: Mm-hmm.

Andy: Um, one was about safety, so how do you make sure what you’re deploying is safe, and you can understand how it’s going to behave in the wild?

How can you employ different levels of control over your agents? How do you do things like kill switches- Yeah … programmatically, things like that. and then the final one was, you know, other aspects of, of control around, like, how you do evals, how you design your agents with different agentic topologies- Mm-hmm

to sort of impart different levels of control and structure. So it’s all these kind of technical concepts, but showing how you employ them in the real world. Yeah. Sort of real code examples, but then say, “This is why this is important.” So I hope people took away that we can do these things at scale properly.

it’s not an unknown thing. Right. We, we have the tools and capabilities. It’s just about learning and sticking them all together in the right places.

Luke: Getting them to work together- Yeah, exactly … and getting the people together.

Andy: that is a different type of multi-agent challenge, but yes.

Luke Mulks: I mean, and there’s a lot of [00:02:00] noise now too about AI agents and autonomous systems, and, uh, what, what conditions actually have to be true before you can trust an agent to act with real autonomy? Are we even there yet?

Andy: I think we are. I think though there is, there is definitely a case to be made that it’s really autonomy within bounds.

Luke Mulks: Mm.

Andy: So especially in like, say, a regulated environment like banking,you can’t just give an agent, like, carte blanche to do what it wants. Right. You can’t give it, like, access to all of your tools, all of your systems. Yeah. You can’t just say, “Eh, OpenClaw YOLO. Let’s go.”

Luke Mulks: Hey- Right … you’d be surprised what I’ve heard at crypto conferences, okay?

Andy: Well, uh, I can imagine, I can imagine that, but not, not in a, not in a couple of hundred-year-old British bank, no. No, no, no, no. Um, but, but, but you can give them autonomy, and I mean, that’s, that’s the power of agents, right? And, and the power of AI generally is that it’s so extensible and flexible. We should be leveraging that.

Mm-hmm. But you can bound these systems and em- employ that control I was talking about. So, you know, you can say agents can only access their own tools. There’s a lot you can do, and I spoke about this this morning around agentic identity- Mm-hmm … auth flows, [00:03:00] you know- Mm on-behalf auth flows. There’s different permission structures you can employ.

There’s just lots of ways of giving control so that, the agent can act autonomously- Mm … but within bounds- Mm-hmm … and that’s important. But it’s like humans. Like, I don’t have access to all the systems in Barclays- Right … quite rightly, do you know? But I still hopefully act with some autonomy.

Luke Mulks: Right. Sure,

Andy: sure. You know? So it’s, it’s really about treating it like that. These are… They’re still software systems.

Luke Mulks: Yeah.

Andy: It’s not a completely new animal. Mm-hmm. It’s just got some differences. Like, it’s, it’s a bit more dynamic. They dynamically orchestrate. They can do planning. They can kind of go deeper in some things.

But yeah, I think, we are there. It’s just making sure we don’t forget- how we want to make these things safe and controlled- Yeah … and then just, you know, just go through the motions to do that. So I th- I think they, I think we are there.

Luke Mulks: That makes sense.

Andy: Yeah.

Luke Mulks: That’s interesting. Yeah, like, I mean, how does your background…

I mean, and, and for folks that don’t know, you have a background as a theoretical physicist, right? Like, uh, uh, uh, how does that shape your mental models with, uh, building AI systems?

Andy: Yeah, it’s kind of way, way back. So I did a PhD in physics down here in London actually- Oh … Imperial College. So, um, yeah, I [00:04:00] think I was always passionate about physics because it was always about how you can define the sort of the axioms, the ground rules, and then build everything up from that.

Luke Mulks: Mm-hmm.

Andy: So it’s really good for systems thinking.

Luke Mulks: Mm-hmm.

Andy: So I feel like that really helps me as a sort of an architect and a systems thinker. That’s kind of my strength really, is someone can give me a really complex problem and I can break it down into its core pieces-

Luke Mulks: Mm-hmm …

Andy: and say, “Okay, you’ve got these three or four concerns.

Here’s how they should all speak to each other, and here’s how they work together in a flow.” And then, you know, some people will sort of maybe take a few more steps together, but I’m used to just breaking it down into bottom axioms and- Yeah … deriving things up.

Luke Mulks: Yeah.

Andy: It, it used to help me in exams because I don’t have a good memory.

Yeah. So I didn’t have to remember lots of stuff. I just had to remember, like, four or five rules- Right … and then you could rederive everything up.

Luke Mulks: Yeah, yeah.

Andy: So it’s, it’s the same idea. You know, what’s the core principles and the core sort of rules you have to follow- Mm-hmm … and then from that layer up. So I think being a good systems thinker is the best thing, and then I’m good at thinking of, uh, I think edge cases and sort- Mm

of trending things out, you know? And you can, you can just think in a bit more abstract [00:05:00] terms about certain things. So it helps me rationalize about AI systems and traditional software in that sort of way, which I think- Yeah … I think is quite good. I like it. It’s a different perspective, I think.

Luke Mulks: Well, it’s interesting too ‘cause there’s like, there’s a, we’re at this point where you have, AI and then traditional software, but then also, banking, which is just like- Yeah

you know, these three worlds kinda converging and- Yeah … and, and a lot of, like, okay, uh, how can I have agents that are making payments for things, or how can I do all these things? It’s all kinda converging too and, and- Yeah … just balancing all that, you know, I think that… I talk to a lot of people that are in AI and, the one thing that really gives me hope about things is just that it’s a lot of different people from walks of life that, different types of thinkers, right?

Yeah. Like, that are, it’s not like, “Oh, I went to college for AI and this is what I’m doing.” It’s like, “No, I came from this world and it helps me in this way.” It seems like there’s a good mix-up of people doing that. Yeah.

Luke: The Brave Technologist is brought to you by the Brave Search API. Access billions of indexed web results from a simple API call with the Brave Search API. Join the leading names in AI and tech using the Brave Search API to [00:06:00] power agenic search, keep LLMs current with real-time data, train foundational models, and bring the best of the web directly to the leading edge.

Get started today at brave.com/api.

Andy: I think you’re seeing more of that-

Luke Mulks: Yeah …

Andy: because it’s so, it’s so much easier now to break into-

Luke Mulks: Yeah

Andy: start building, for example- Yeah … you know, with, like, Claude Code and stuff- Oh, totally … so anyone can build.

Luke Mulks: Yeah.

Andy: I still think you need deep software expertise in the mix, but it is really good. So it’s like, you know, I was, I was speaking to some people at a conference recently- They were like, they were psychologists and they’re like, “We’re trying to apply psychology techniques to agents.”

And I was like, “That’s really cool.”

Luke Mulks: Yeah, yeah.

Andy: And I can totally see how that is useful. Like, I don’t want psychopathic agents- Yeah. … and stuff, you know? So it’s kind of, it’s, it’s really, it’s, it’s amazing. And then people, uh, you know, there’s a lot of philosophers- Oh, yeah, yeah … kind of getting involved. It’s, it’s very rich, and I agree with you, it’s sort of a variety of perspectives always makes the sum kind of better.

Luke Mulks: Yeah, yeah. So

Andy: I always try and do that with teams I build or with projects you work on. You know, you don’t… You’re right, you don’t want sort of a homogeneous-

Luke Mulks: Yeah,

Andy: yeah … if everyone’s came [00:07:00] from the same background, always worked on the same stack-

Luke Mulks: Yeah …

Andy: they’ll always come up with the same answer.

Luke Mulks: Oh, and, and, and I think, like, you mentioned earlier for edge cases, right? Yeah. Like, that’s where I think having a good diversity and thinking is like, I’m not gonna think of every edge case ‘cause I haven’t been through every one. No, exactly. People go through some weird edge cases, you know?

Andy: No, they, no they do.

Yeah. Yeah, yeah. that’s a really good point, actually. Sometimes, though, it can be too much, kind of, uh- Oh, really? … too whack it. Yeah. Yeah, yeah. I think, I think, you know, some people, because of the hype-

Luke Mulks: Uh-huh …

Andy: some people go, there’s always this question, what if an agent goes rogue?

Luke Mulks: Yeah. And

Andy: if you start breaking down that into its constituent parts, you’re actually… You know, it’s not really possible in a lot of- … a lot of systems. if you’re building these as traditional software, they still have, you know, there’s still authentication, authorization you go through.

It’s still bounded environments and things. You know, it’s, it’s not going to go rogue in the way that, like, Terminator or something is. Yeah. So there, there’s a bit of kinda, it’s good to think of edge cases, but some edge cases I think are more hype than reality sometimes. Sure, sure, sure. So it’s a g- But it’s a good mix.

It’s a

Luke Mulks: good conversation. No, no. I mean, like- Yeah … I think, and, and there’s different types of considerations, right? Like, I mean, like if,you’re thinking too human with [00:08:00] something and, you know, a machine’s not gonna necessarily operate the same way, right? Like, or, or- Right … it’s gonna go hit the hammer on things in a different way, engineering’s, like, old and not, so new, right? Yeah.

Why does getting that distinction wrong cause problems for people that are trying to build?

Andy: yeah, the, the biggest thing for me is everyone thinking this is such a revolution-

Luke Mulks: Right

Andy: that they forget there’s a huge piece of evolution as well. Yeah. So a lot of the stuff we learned before still applies. All of the practices from DevOps, all of the stuff from MLOps, all of the stuff from data science- Yeah … all the stuff from just traditional goods app development, right? All of that is still there.

Right. Like, we’re still, we’re still containerizing stuff. We’re still running in Kubernetes. It’s true. We’re still using identity providers. We’re still using observability solutions. You know, it’s, a lot of that is just, there’s new things you need to think about. So like in the IDP side, for example, you know, now I have a concept of an agent identity, and I have to think how does that, you know, have certain permissions, and how does it act on behalf of other things?

Okay, that’s fine, but it’s still identity provision. Yeah. It’s still OS 2.0. It’s [00:09:00] still, you know, it’s the same technologies with new spins.

Luke Mulks: Yeah.

Andy: I think the biggest danger is everyone thinks I have to throw everything out and start again.

Luke Mulks: Yeah, yeah.

Andy: Um, and that, that I think can waste a lot of time and energy, and people thinking they have to like, you know, re-architect everything from scratch.

Right. And we’re like, no, actually, it’s just, there’s just some new concepts we add on the previous stuff.

Luke Mulks: Uh-huh.

Andy: Um, and as well, there’s still, there’s still a lot of traditional builds happening, right? There’s still traditional app development.

Luke Mulks: Oh, yeah.

Andy: There’s still traditional machine learning going on.

So it’s like all of that doesn’t go away.

Luke Mulks: Yeah,

Andy: yeah. It’s just there’s a new, there’s a new layer on top- Yeah … is, is the way I like to think of it. Yeah. You know, there’s a bit, a new layer of sediment, if you like- Yeah,

Luke Mulks: exactly …

Andy: the story. Um, but yeah, that, that’s the biggest danger I see, is everyone thinks, you know, “Oh, everything has to be reinvented.

This won’t work.” And you’re like, “No, actually it’s just a small addition.”

Luke Mulks: Yeah.

Andy: Uh, and just identifying any new concerns and how the previous concerns have to evolve.

Luke Mulks: No, that’s what we deal with too. Yeah. It’s like if you’re, if you’re, if you’re playing off good first principles, like you’re- Yes … gonna apply those with AI in the equation i- and, you know, it’ll be relative to what- Yeah

you’re normally doing, and it makes a lot of sense. I [00:10:00] mean, so your work with Barclays, I mean, it’s a heavily regulated financial environment. and making AI systems that are safe and kind of governed by necessity, like not by choice, like what does that teach you about the AI safety conversation in general that’s happening, in the broader tech industry right now?

Andy: Yeah, it’s, it’s interesting because safety in the environment I operate in is very narrow- Yeah … in the sense of like, is it safe to have an agent that can see your, your bank statements- Right … and sort of, you know, make payments on your behalf and- Yeah … these sort of questions are very specific. The wider safety conversation is, you know, going all the way up to are these things gonna destroy the world and- Right

take every job and stuff. So it’s, it is interesting, but they, where they meet in the middle is the concerns are always, you know, just what I was saying before, how do you create these control structures? Um, and it’s not as grandiose as like, you know, Anthropic talking about constitutional AI and stuff.

Yeah. We’re more, we’re more like sort of further downstream in the sense of like what are the guardrails we need to put in place?

Luke Mulks: Right.

Andy: Right. Uh, what are the, again, policies, so things like on your AI gateway-

Luke Mulks: [00:11:00] Yeah …

Andy: what sort of, you know, policies are you employing to say you can allow these certain types of actions with these certain users, or how do you, how do you have kill switches and stuff?

So it’s kind of, it is really interesting, and it’s constantly evolving. And what people don’t appreciate sometimes is in regulated industries, that’s where a lot of innovation happens.

Luke Mulks: Oh, really?

Andy: It’s like how do you, how do you meet these controls- Yeah … with the technologies we have available? Yeah, yeah, yeah.

There’s, there’s a lot of- The problem sets. Yeah … yeah, there’s a lot of cool problems there. Yeah, yeah. Everyone… And to be, to be fair, I, I complain about it a lot, and others do, right? But there’s a lot of kind of thinking, oh, it’s, it’s all just there to slow you down, but actually it does at the end make a better product.

Luke Mulks: Yeah, yeah.

Andy: Yeah. Because you know, you’re going to production with this thing, you’re like, “This has been battle tested.” Yeah, yeah. This is not gonna do something totally crazy. Yeah. It will fail at some point, but-

Luke Mulks: Yeah …

Andy: you know, it just, it just by its nature- Yeah … the type of products we’re building and putting out into the world, they, they have to go through so much scrutiny that you’re sort of, you’re really sure.

Luke Mulks: Yeah.

Andy: And then we were just chatting before the pod- … cast about once you do that, you create massive value.

Luke Mulks: Yeah. '

Andy: Cause you have huge customer bases, [00:12:00] really impactful stuff. You know, it’s, it’s your money at the end of the day. it’s really core to people’s lives. Yeah. So if you can bring this technology to bear there, it’s amazing.

But I think, yeah, it just all, all kind of blends together nicely to make a really interesting environment.

Luke Mulks: Yeah. Well, and, and I think, like, too, like doing a little bit of the homework on, like, why the rules are there, right? Yes. Like,what was the purpose for that? Like, even if they change or whatever- Yeah

I think it’s like a, it’s agood thing, even if you’re gonna break the rules. Like it do- not that you’re going to. Yeah. but I mean, just understanding why they’re there, like- Yeah … it’ll help with things, too. Ab-

Andy: absolutely.

Luke Mulks: I mean, ‘cause we deal with this too, and, and it’s kind of a follow-on to that.

Like, I mean, everyone’s talking about agents and agentic and everything like that, and I feel like you’ve done a good job, too, of explaining how, like, you know, okay, narrow the parameters and having a specific things. Like, we’re thinking at Brave, too, about, like, we have to think about it like we’re seeing browsers that are applying agents across everything, and your whole life’s in the browser, right?

Yeah, yeah. So, like, there’s lots of rooms for, like, injection and certain other attacks and things like that. Like, um, and, and so it’s kind of this containered or profiled approach. Like- Mm … where do you see this going with agents? Do you see it, like, [00:13:00] having a one kind of agent you’re, that delegates off to other agents?

Or, or w- well, just like long, bigger picture.

Andy: Bigger picture, everything has to be as federated as possible, I think. Yeah. So it’s just like we just have to really standardize on the protocols like we are doing with, you know, MCP A2A and stuff. There’s a lot of great innovation happening there. I think it is, it is definitely becoming a challenge, like I mentioned that IDP point before.

Yeah. You know, like if you’ve got- Different systems that all run different agents. So like, you know, you’ve got your, your CRM system has its own- Mm-hmm … build an agent and run an agent here, and then your, your own hyperscaler sort of accounts, you’ve got your agents running there, and then it’ll, you know, up to end times.

Luke Mulks: Yeah.

Andy: How do you consistently make them share information about the permissions and the levels of access? I think that’s really interesting.

Luke Mulks: Yeah.

Andy: Um, I don’t really know the answer. And then I kind of, uh, a- another question that keeps coming up is how do they effectively share context?

Luke Mulks: Yeah, yeah, yeah.

Andy: ‘Cause, you know, we’ve, we’ve seen already in even smaller use cases, the real power comes when, okay, you can [00:14:00] delegate and solve sub-tasks- Yeah … but sometimes having an agent, an LLM, see quite a broad picture-

Luke Mulks: Right …

Andy: it comes up with the right answer.

Luke Mulks: Yeah, yeah.

Andy: But that’s hard to do if you’re calling like- It’s very hard

10 sub-agents. Yeah. You know, they can’t, they can’t all shovel, like, thousands and thousands of tokens back and forward all the time. Yeah. It’s not gonna work. So it’s like, you know, thinking about things like how do you have shared context layers?

Luke Mulks: Yeah.

Andy: but I think that’s gonna be very hard to do with such disparate systems.

So it- it’s just, it’s super interesting. it’s kind of, there’s all these sort of unsolved problems. Yeah. That’s why it’s juicy for engineers to get into, right? Yeah. There’s lots to solve and think about.

Luke Mulks: Totally. we talk about, like, we’re very focused on privacy too a- at Brave.

Yep. And banks obviously have to f- deal with this too. do you think that there are things like lessons from how the banking sector has to handle user privacy or more broad compliance, that other parts of the tech industry could learn from with AI?

Andy: I think yeah, just the, the flexibility and- The kind of, all the integrations we’re talking about, they’ll all have to pick up some of those technologies and skills.

Yeah. ‘Cause, ‘cause it is just gonna be such a, a key concern. [00:15:00] ‘Cause you’re right, you’re gonna be in the browser, right?

Luke Mulks: Yeah.

Andy: Initially you’re like, “Okay, I’ll just… I’ll let you find stuff.” But soon it’ll be, go to the browser, find my banking website- Yeah … then, then pick up the fact, oh, that’s my bank. Yeah. Now I wanna actually just make a payment.

Luke Mulks: Yeah.

Andy: Or like go, go and, you know, pay via a shop or something. And I know there’s, there’s integrations, there’s APIs we can use for that, but yeah, just doing all that in a kind of controlled way is gonna be super interesting. Um, we’re, we’re very much taking an approach, I would say, where it is, it’s always crawl, walk, run-

Luke Mulks: Yeah

Andy: I would say. Yeah. So it is very much, things in our environment are quite bounded just now in the sense we’re sort of making sure that it’s not, like going after the, the North Star straight away. We are very much saying, “Okay, I’ve got this multi-agent system. It’s all running in, say, AWS.

It’s all got quite constrained permission structures. what happens? What can we do within that?” And then you’re just gradually iterating and building another integration, another integration.

Luke Mulks: Yeah.

Andy: but no, it’s, it’s super interesting. And then there’s lots of challenges even, not even on the safety side, but just how things scale.

Luke Mulks: Yeah.

Andy: Yeah. Um, I was listening to like a, a pod the other day, and it was the fast MCP [00:16:00] guys- Uh-huh … from, from Prefect, and they were talking about how MCPs are having blow in the number of tools they’ve got.

Luke Mulks: Yeah.

Andy: So they’re talk- they’ve got introduced this thing like code mode and stuff where- Mm-hmm … um, a call, a client agent can sort of write a little program to, uh, search through the tools efficiently and run it server side in a little subprocess.

Luke Mulks: Yeah, yeah.

Andy: It’s getting really wacky. Yeah. It’s like

Luke Mulks: a real- It’s like the exhibit meme, right? Like

Andy: It’s real- Yeah, it’s, it’s really, really crazy stuff happening, but it’s like- Yeah … it all sort of makes sense, but it’s, it’s, yeah, there’s just, there’s a lot, there’s a lot happening in terms of the pure functional innovation.

Yeah. And then having it catch up with it is all the stuff, like you say, privacy preservation, identity, permissions. Yeah. So it’s, it’s a really interesting space.

Luke Mulks: Part of the challenge is too is just there’s so many things and priorities. from what you guys are seeing, where are your biggest wins from this?

is it on like internal use for like optimizing process? Is it stuff- with customers? Is it, you know, where, where are you guys seeing the wins from this early?

Andy: I think internal use is always gonna be where you start.

Luke Mulks: Yeah.

Andy: Because it’s the easiest place to play.

Luke Mulks: Sure.

Andy: You know? Uh, [00:17:00] it’s the, it’s the least risky.

It’s kind of, it’s the most understood in terms of the use cases. We are now starting to push some stuff out- Yeah … uh, customer facing, which is really exciting. and I think to be honest- It’s gonna, it’s gonna be a balance, but I, I do think we’re, we’re being very good at not just saying, you know, “This is not just a productivity play.”

Luke Mulks: Yeah. '

Andy: Cause I, I, I think that’s dramatically underestimating the potential. Um, Ethan Mollick, who a lot of kinda people have heard of, you know, the guy who wrote Co-Intelligence, right? Yes. Yeah,

Luke Mulks: yeah.

Andy: I saw a thing the other day, he was saying, you know, everyone’s like, “Oh, I can use AI and fire half my staff.”

But then you don’t wanna do that. You wanna be like, if you were a Near Provider, you wanna do a Guinness ad- Right … said, “We’re gonna go take over the world,” right?

Luke Mulks: Right.

Andy: Right.

Luke Mulks: And that- Yeah, yeah,

Andy: yeah, yeah … we need more of that, and I think, I think we’re being quite good at thinking like that. Yeah. We’re sort of l- seeing it more as a, as a value-generating opportunity- Yeah

rather than a value-preserving opportunity. Yeah. But you do have to, you do have to start kind of where it’s easiest. Totally. And where, where it’s easiest is internal use, personal productivity, but we are, doing a lot of kind of operational stuff. in my talk today, I was talking about, like, the stuff we’re doing in the operation space, like AI ops agents and stuff.

Yeah, yeah. It’s [00:18:00] really cool.

Luke Mulks: Yeah.

Andy: Um, and that’s already helping us, you know, get through more incidents, triage more stuff more quickly. It’s all… So it’s kind of, it’s, it’s a good mix, but I think you do have to, you start simple, you build on that, you build some trust- Yeah, yeah … uh, earn kind of further investment- Yeah

sort of bootstrap it up.

Luke Mulks: Yeah.

Andy: And then eventually go after the quite, uh, the quite ambitious stuff.

Luke Mulks: And it can make believers out of people too internally- 100% … and then working together with those people, right?

Andy: 100%. Like,

Luke Mulks: yeah, yeah. And,

Andy: and in a big, in a big sort of historic organization, that sometimes is the most important thing.

I can imagine. Yeah, it’s like, you know, because, because people are understandably like, you know, “We can’t just unleash-

Luke Mulks: Right …

Andy: this technology everywhere.” Yeah. So it’s like, “Prove it to me.”

Luke Mulks: Yeah.

Andy: Which is a good challenge- Yeah … ‘cause you’re like, “Uh, I should prove it?”

Luke Mulks: Yeah.

Andy: Um, and I think that’s a challenge we’re also seeing generally in the industry, not just about the tech- the technical capability, proving that works, but proving that it works as a value case.

Luke Mulks: Yeah.

Andy: So, like, you know, everyone’s getting excited, building lots of prototypes, but there’s this big gap between the value realized-

Luke Mulks: Oh, my God … you know. It’s huge.

Andy: It’s huge. Yeah. Every, every- It’s huge …

Luke Mulks: and I- I, I really feel like everybody’s just, uh, marketing to developers right now, you know? Like, and, and it’s like- ‘Cause

Andy: that’s the easiest thing [00:19:00] to tick off, right?

Yeah,

Luke Mulks: yeah. Yeah,

Andy: yeah. And you’re like, you know, but, but you’re seeing all this stuff now where people are saying, “But the writing the code was never the bottleneck.”

Luke Mulks: Right, right, right. You

Andy: know? Yes. I di- the pr- you know, we weren’t, we weren’t sitting there as developers going, “I, I…” Well, I, we probably were.

Like, “I wish I could write 10X faster,” right? Sure. Okay, but ultimately we’re not seeing that translate to, like, 10X more revenue.

Luke Mulks: No.

Andy: No. And it’s because- Yeah … there’s all this other stuff that has to happen, right? Yeah, yeah. There’s all the, the integrations, there’s all deployment, there’s all the design- Yeah

there’s all the review. You know, there’s all the test cycles. So it’s, it’s just interesting to see where we branch out and-

Luke Mulks: Well, and it’s like AI’s something where, I, I, I don’t know, maybe it’s been, like, one in a 20-year thing where you’ve got all of this, like, top-down C-level pressure- Mm-hmm … from top Fortune 2000 companies to integrate this everywhere.

Yeah. And, and you’ve got a guy that maybe was like, “Oh, I’m just building, you know, customer software. now I gotta be an AI guy.” Yeah. You know? And they’re like- We’re all, we’re all- … “How do we make this happen?”

Andy: Yeah, we’re all… Yeah, exactly. But that, that is it, and it’s kind of, We had an interesting discussion in our roundtable yesterday, and I think someone made a good point that- The good place to get to is where we stop [00:20:00] saying AI.

Luke Mulks: Yeah,

Andy: exactly. And it’s just like, it’s just part of it.

Luke Mulks: Exactly. Yeah.

Andy: You know? It’s like we don’t, we don’t run around saying, “I’m using a database.” Yeah. “Aren’t I so innovative,” right? Yeah, yeah, yeah. We kinda want that to be the point where everyone’s like, it’s, it’s table stakes. Yeah. It’s not, it’s not kind of a unique selling point.

Subtracted. Uh, yeah. I think that will come as the sort of the industry matures.

Luke Mulks: Yeah. So what do you think, I mean, is the most important question an engineer should ask when they see a promising AI capability, and they need to decide if it’s, uh, i- if it’s something that’s actually deployable?

Andy: Yeah, I think the biggest question before… They al- I think a developer and engineer, scientist sort of mind will always ask, “Can I build this?”

Luke Mulks: Yeah. Ah.

Andy: But they should ask before that, “Should I build this?” Right? Especially now, because the barrier to entry is so low- Yeah … like we were saying, right? Absolutely.

You know, everyone’s like, “Oh, I can build that. I can build a bot that reads Jira tickets.” Yeah. So like, but what you end up with is like 100 bots that read Jira tickets. Yeah. You’re like, “Is that really the best use of your time?” Should

Luke Mulks: we

Andy: do it? Yeah. Yeah, exactly. Yeah, yeah. Should, should I do that? Whereas, whereas you do a bit of digging, and you go, “Actually, where’s the [00:21:00] value?”

You often find it’s not the easy problems, right? Right. And then, and then you sort of say then the could I build this question becomes a lot more interesting.

Luke Mulks: Right. Right.

Andy: Because it’s not like, “Could I build this?” Obviously, yes. Yeah. It’s more, “Oh, this is quite valuable and no one’s gone after it.” The reason they’ve not gone after it is actually because it’s a bit complicated.

Luke Mulks: Yeah.

Andy: So can I build this? Could I build this becomes far in- more interesting. Yeah. ‘Cause you have to break it down and, you know, try and work out how you’d tackle it. So I think, I think that’s the big question people should ask themselves, especially now. Yeah. It is so easy to just build.

Luke Mulks: Yeah.

Andy: But what you end up with is, like, you know, 100 self-built apps-

Luke Mulks: Yeah

Andy: doing lots of weird stuff on your laptop.

Luke Mulks: Yeah, yeah, yeah.

Andy: And you’re like, “Again? Is that, is that even a good use of your CPU cycles?” Like- Probably not, no. No, exactly. So just, like, should I, or is this the best use of my time resource tokens?

Luke Mulks: Yeah, yeah, yeah, yeah.

Andy: Rather than, you know, can I?

Luke Mulks: Yeah. Yeah. A- a- and this might be more of a, a personal and less of a, a banking question, but there’s a lot of talk around, like, agentic payments and kind of- Yeah … you’ve got these things with stable coins happening, and then you’ve got these things with agents happening. Like- Mm … where [00:22:00] do you see…

Do you see that as actually something that has some legs to it or, or is it still too early to tell?

Andy: I think it’s very early. Yeah. I think, um, the things, the things I’ve seen about, you know, like, agentic payments and stuff, it’s, it’s kind of more focused on taking the traditional API interactions and just allowing agents to sort of trigger those.

Yeah. So it’s less about doing anything completely different. Yeah. I think coming at it from, you know, the other angles, like the other technologies, blockchain and all those, the stable coin, et cetera, I th- I think that will build up and gather a lot of momentum. I’ve just kind of… I’ve not, I’ve not seen a super compelling case that that is, like, fully thought through yet.

Luke Mulks: Yeah, yeah, yeah.

Andy: But I think, I think it’s coming. For- It’s also not the space I operate too much in, but it’s kind of- Yeah, yeah … I think it does have legs, and I think it… But it’s also very hard to tell what’s coming.

Luke Mulks: It’s hard to tell. Yeah. And, and sometimes it’s almost like it’s just a forcing function for other projects that might be on a shelf that, to, to wake those up and, and, and get those out

Andy: too.

Yeah, 100%. I, I totally see that as well. Yeah. And then all it takes is, like, you know, [00:23:00] another breakthrough or change in the text that can… We’ll be sort of revisiting everything again. Yeah. So I, I place no bets in the future anymore. I have no idea, right? I have no idea. I’m basically like, if I wake up every day and I’m like f- I kind of can rationalize how the world is, I’m like, “I’m in a good place.”

That’s

Luke Mulks: healthy. That’s healthy.

Andy: Yeah, it’s, it’s healthy.

Luke Mulks: I, I’m with you. I’m with you. Yeah. It’s like, uh, uh, you know, you throw some kids in the mix and it’s like, “Gosh, there’s enough complexity in the world.” No,

Andy: exactly. Exactly. Exactly, so.

Luke Mulks: so I mean, really appreciate your time.

It’s been, like, super interesting. is there something in all this race to innovate, a conversation that you think we’re not having enough of?

Andy: Yeah, I think, that conversation about value- Yeah … like I mentioned, um, I’m finding that just super interesting. Yeah. ‘Cause that, that is like, that is the big question mark- Yeah

of all the things. Um, I think as well is just we’re not having enough conversation, I think, about some of those core concerns that I mentioned before. You know, it was interesting, a lot of people came up to me after my talk and says, “Oh, we’re now… We’ve deployed a lot of agents, we’re now starting to think about observability.”

You know, and you’re like, that’s in- that’s interesting because [00:24:00] people just haven’t… They’ve been in the race to innovate, like you said- Right. Right … into like, I can do this thing with agents, and it’s gonna drive, like, this benefit. but I’m not actually tracking what it’s doing or something, you know what I mean?

Yes. Yeah. And, or actually my evals were actually quite thin. Yes.

Luke Mulks: I’ve

Andy: kind of not done- Yeah … a really rigorous batch of testing on it, because the race was to get something out there. And I can totally understand that. Yeah. You know, I’ve, I’ve done that for years. It’s

Luke Mulks: tech,

Andy: right? It’s tech, exactly. so I think, I think those sort of conversations, just bringing it back to what’s gonna make these sustainable innovations.

Yeah. And it was the same… I’ve sort of, I keep telling people I, I’ve seen all this before in a way. We’ve all seen this- Uh-huh … with, like, the machine learning push, and then with the cloud push. Yeah,

Luke Mulks: yeah.

Andy: There was huge race to innovate, do things using a specific sort of set of technologies, and then what happens is big rush, and then we, we sort of acclimatize and equilibrate.

Yep, yep. And that often comes down to, like, you know, are you doing things actually scalably? Are you doing things actually kind of in this controlled way? Are you monitoring the cost? Are you being sensible? Are you being frugal? You know, all this talk over the past few years about frugal architecture and stuff, that came years [00:25:00] after cloud.

Luke Mulks: Yeah, yeah, yeah, yeah, yeah.

Andy: Right? So there’s always this race to do in any way possible, and then it’s about how do we optimize it? Yeah. I, I think, I think just anything you can do in your builds, in your projects, in your organization to short-circuit that means you’re gonna be leading the pack. 100%. ‘Cause you’re not gonna panic at the end.

Yeah. You’re not, you’re not gonna be like, “I’ve got 100 agents, but now I need to make them all kind of emit telemetry in a specific way.” Whereas what you could do is actually if I think about that now, I’ve got 100 agents, but I can add another thousand. Yeah. And I know all of that infrastructure will just scale.

Luke Mulks: Yeah.

Andy: So it’s, it’s kind of, it’s, it’s that. It’s, it’s just, it’s just forgetting some of those core concerns- Yeah … and having that value conversation as well. No, I

Luke Mulks: think it makes a lot of sense. Yeah. yeah, no, that’s great. Well,Andy, I really appreciate you making the time. It’s been really awesome conversation.

Where can people find… Are you putting your work out there, or like, The

Andy: main… Yeah, the main place, I’m, I’m very bad on kind of social media, but main thing is LinkedIn. Okay. So just, uh, reach out to me on LinkedIn, and then I usually share lots of stuff there. Awesome. Um, I have a, have a little podcast.

We’ve now got, um, a Substack as well called Forward Deploy. Nice. Me and a friend of mine, we’ve started. We’re starting to write some stuff there. So yeah, it’s a few things, [00:26:00] but we’ll, we’ll share that out, yeah.

Luke Mulks: Cool. We’ll put it in the show notes too. Awesome. So, well, thanks again, Andy, really appreciate it.

Love to have you back too, to check on how things are going.

Andy: Amazing. Thank you so much.

Luke Mulks: Great. Thanks.

Luke: Thanks for listening to the Brave Technologist Podcast. To never miss an episode, make sure you hit follow in your podcast app. If you haven’t already made the switch to the Brave browser, you can download it for free today at brave.com and start using Brave Search, which enables you to search the web privately.

Brave also shields you from the ads, trackers, and other creepy stuff following you across the web.