Natalia: [00:00:00] You’re listening to a new episode of “The Brave Technologist,” and this one features Natalia Konstantinova, Lead Enterprise Architect in AI for NatWest. With a PhD in information and language processing and a background spanning academia, industry, and innovation, she specializes in AI strategy, enterprise architecture, governance, and responsible AI.

Natalia is passionate about helping organizations translate complex AI technologies into scalable business value and is a recognized voice on the future of AI in highly regulated industries. In this episode, we discussed:

why so many enterprise AI projects collapse before they reach scale, and what the ones that succeed actually have in common. How to know when an AI tool needs full governance, why responsible AI and scalable AI have become the same thing, and what a risk-tiering approach actually looks like in practice.

This was a really great interview. I really enjoyed the conversation with Natalia, and it was great that she made it out to the AI Summit. And now for this week’s episode of “The [00:01:00] Brave Technologist.”

​

**Luke: Natalia, welcome to The Brave Technologist. How are you doing today?

Natalia: Amazing.

**Luke: We’re here at the, uh, AI Summit in London. you were one of the speakers, you gave a keynote today. can you give us a little bit of background on what the keynote was about?

Natalia: So I’m still to speak today, and, I have two topics.

So one is about, price of intelligence, so how we measure, the cost of intelligence, how we measure how much money we spend, and when we don’t overspend. And second one is about, all the AI tools appearing and all, promises of coding that is accessible to everyone. So I will be discussing with a panel, is it true-

**Luke: Yeah

Natalia: and, how many organizations actually manage to achieve using those tools and bringing value.

**Luke: Interesting. Interesting. yeah, and I know you’ve kind of moved through academia, industry, and now one of the UK’s largest financial institutions. what was the turning point where you stopped thinking about, uh, AI as a [00:02:00] research problem and started seeing it as an enterprise architecture problem?

Natalia: I think my move mo- wasmotivated by the fact that I saw that there is a lot of potential in AI, but this potential is only realized when it’s done properly. Mm. So I think my whole idea of moving from research into enterprise was driven by I want to see it being used- Mm-hmm … not just, like, on the paper.

And, I think pivotal was my joining my first company, BP, so where I was able to actually look large scale and how we use AI, and identify when we can’t use it. And usually it’s less about technology, it’s more about change management, it’s more about, like, making the right decision at the right time. So building things really in mind with the scale.

Mm-hmm. So now scale is, is my passion. So because I’ve seen there is a huge difference between, building small POCs and building small things in AI, and, looking at them scaling across, and really bringing value, and being really, really stable, [00:03:00] something that big enterprise like. So my previous one was, say, BP.

my current company is also a huge one, and we can’t allow to have something floppy and non-reliable, so it needs to be just working, you know?

**Luke: Mm-hmm. Yeah. Do you… how far do you think… Do you think people’s perception of, how much more efficient or cost eff-effect… Like, do you think that people’s perception’s off with that w- compared to what they’re seeing in reality right now?

Natalia: I, I think now obviously we moved, So last year everyone and, like, two years ago was talking about gen AI, so now, like, this year is all about agentic.

Yeah. So if agentic last year was more startups trying to push for it, this year, enterprises are saying, “Yes, we’re doing this.” Mm. I think, where we see a lot of catch- … is, uh, agents needs to be controlled not to, uh, go with the spend. Mm-hmm. And also people need to think-in advance what the value they’re delivering.

Because it’s really easy now to have a technological project and to have all this AI. Mm. So you need to think where you need to direct it to bring actually value. And, it’s a skill.

**Luke: Yeah.

Natalia: I don’t think, many people know, [00:04:00] how to do it, so you need to have some technology background, some bit of business background, and, a vision for what AI can actually do.

**Luke: Mm-hmm.

Natalia: but I think I’m seeing, like, so technology was before a blocker, you know? Now it’s, really an enabler, and now business just needs to catch up and say- Right … “Okay, it can be done, so what can I do with it?”

**Luke: Yeah, yeah. I know, it’s, one of those things where it’s really interesting in how much pressure there seems to have been from top-down in organizations and kind of across, like, you know, the Fortune 2000s, right?

Like, you don’t really see that so often with technology where, like, there’s all this pressure to… And all of a sudden somebody that’s a product manager, on some SaaS thing is now, like, an AI person, and they’re like, “I didn’t even know if I was gonna be into this.” So it’s, it is interesting where it’s almost like cart and horse, you know, problem, but, now we’re at the point where, yeah, we actually have to make it solve problems and do things, right?

Natalia: Yeah. It needs to be scalable. It needs to be working for you, and, it needs to be reliable. I think, we have a lot of people who come especially from software engineering background. They don’t understand this small bit of [00:05:00] reliability, because in software engineer you build once, you know, you tested it, and you’re pretty sure that it doesn’t have bugs to a certain.

So AI is more fluid. It’s changing quite often, and, there is a skill needed to understand what are these challenges and to build accordingly. Mm-hmm. So to have additional checks, to have additional observability, all this becomes,of paramount importance. So if AI was important, but, given that we’re just scaling, scaling, scaling, all these things become important, uh, thinking before you start even building.

Luke: The Brave Technologist is brought to you by the Brave Search API. Access billions of indexed web results from a simple API call with the Brave Search API. Join the leading names in AI and tech using the Brave Search API to power agenic search, keep LLMs current with real-time data, train foundational models, and bring the best of the web directly to the leading edge.

Get started today at brave.com/api.

**Luke: Yeah, and it sounds like it’s kind of like, having discipline around this as part of the structure, and standards is, [00:06:00] really important. But, you know, given how fast and messy AI is kinda moving along, how do you reconcile those two worlds without stifling innovation or losing control?

Natalia: So, You just said that it’s really difficult with AI, but I think AI and especially let’s think about agentic and, let’s say, we will building stuff and there will be agents building stuff. Agents, still follow the guidelines- Mm-hmm … and, it’s a really good time for us to come together, have all these brains of great people we work with, and formulate them and put into standards.

So it’s actually pushing us to think, harder about governance and, it becomes, let’s say, of paramount importance. So before it was good to have, so now many organizations understand that they have to do it for it to be scalable. And, I believe that governance done right is for the better. So it’s not stelly innovation, it just tells- What are the boundaries?

Safe boundaries- Mm-hmm … where you don’t want to build something and then throw it away, you know- Right … in half a year because you understand that [00:07:00] it will never go through governance. So- Right … you better have all this, defined in advance so that your innovation can do a lot of things, but in clear boundaries.

And also just not having this ambiguity, what is possible, is allowed, what is not, is, much easier. Mm-hmm. So it breaks conversations, like it speeds up, the delivery as well. So if you have all the controls and all the standards in one place, and, you don’t need to go and connect to many, many, many stakeholders, it’s really speeding up the process.

**Luke: Do you see this happening kind of at the commercial, like, , at the corporate or, at that level as opposed to, more, standardized, like more of a global kind of standard

Natalia: so we are part, for example, of, an open source, initiative called FINIS where there is a thinking and there is a push to do some standard things and to share across, still,

I won’t say, uh, it’s, it’s kind of like in- initial steps. Mm-hmm. So there is a need and there is understanding there should be something done, but I [00:08:00] think, given it’s like open source, it’s, all about people willing to collaborate and come together. when it’s commercial setting, it’s almost like you have to do it- Yeah

for it to be operational. So I think it will start, like what, what’s happening in many organizations, you do it, yourself, and then when you reach maturity, and you have all these, things working together, you might realize, you, for example, collaborate with other financial institutions, and they need to be secure, so they’re your partners, so then it’s better to come together and share all these things.

**Luke: That makes sense. I think especially, too, with like, where the parameters are and all the different parties involved with a lot of these things, with data governance and just, like, operations and all this stuff, and making sure that the right people are at the table, right? Yeah. Like, um, I, I think, you know, kind of switching gears a little bit, I mean, we’re at this time where it seems like every quarter is a new trend and, and you’re seeing a lot of, like, like developers are a lot of the target market, right?

And there’s a lot of experimentation. and in these conditions, we see things, a lot of projects never pa- make it past proof of concept, right? from your [00:09:00] experience kind of what’s the real reason that so many of these AI initiatives seem to fail at scale?

Natalia: so there are several of them. So first one, I think,as we said, barrier to entry in technology became lower.

I won’t say really low, but lower. So many use cases and many things people start building are not connected to the real need of value.

**Luke: Mm.

Natalia: So they’re tech for tech. So it’s possible, why not to build it, without checking whether anyone will be using it if, if it’s really needed. another one, not everyone thinking about the scalability upfront.

Because, for example, you start building stuff you didn’t think will, will you get, like, all the data you need? will you be able to provide this in a safe manner? And, even, like, technological scale, like for example, if you have five users and then tomorrow you will have, like, 1,000, did you think upfront?

so all those things that needs to be a foundations and they need to be considered before you start building, because in some cases also can you evaluate, [00:10:00] reliably that it works? Mm-hmm. It’s really important to give something to your customers that is reliable. So to build a POC you can Have it and look at it.

your colleagues looked at it and said like, “Oh, it’s, it’s brilliant. I love it.” But if you want to deliver something to your customers, you don’t want to end up in papers saying like this failed.

**Luke: Right.

Natalia: Not everything is easy to relate, so, when you start thinking about upfront, you might change how you create your solution.

You can add additional checks. You can, make it maybe not so ambitious. Mm-hmm. And this is, like, from my research, if you create an experiment and you have no way to evaluate an experiment, you won’t have a research paper. Right. You could just drop it. Yeah, just like, don’t do it. A- and here as well, if you don’t know how reliably understand if it’s working for your customers, just rethink about it.

Yeah. Rethink your approach.

**Luke: yeah, like not just doing something to do something. Like, what’s the actual purpose around it?

Natalia: Yeah, so it’s all about, yeah, can you control it, and is it something valuable? Will you have some users? And did you think about actual [00:11:00] proposition as

**Luke: well? Yeah.

Yeah. So, I mean, there’s a lot of talk, too, about responsible AI. but in practice, a lot of this can just feel like kind of checkboxes with compliance and, rather than kind of genuine design principles. how do you make responsibility something that’s actually built into the architecture?

Natalia: So, what we just, discussed before, I think it’s kind of like responsible and scalable becomes almost, uh, I won’t say synonymous, but if we don’t think responsible about things, they will fail later- so before we would say we need these controls because, to make sure that things are working. So I, I think without having these controls and principles, your entrance will build something which will be completely non-usable.

**Luke: Hmm.

Natalia: And obviously, things will fail if,they’re not built, with responsibility in mind.

So as I say, sooner or later you will have someone coming and saying, you can’t be doing this." Right. Given the investment and the pace of, how we’re moving, we need to think about it upfront because otherwise it will be a lot of wasted, investment.

**Luke: Makes sense. Makes sense. yeah, I know at Brave we’re really [00:12:00] focused on data privacy and user consent and, financial services sit on a lot of sensitive and personal data.

what do you think about the tension between using data to make AI smarter and protecting people that the data belongs to?

Natalia: So it’s, again, a responsible one, so, educating your people about this, really early is extremely important. I think what happened, like, when gen AI, I can remember when gen AI appeared, people were not aware of all the risks- Mm

which are inherent. They didn’t understand the whole concept of before you would build a model. It would sit in your premises, you would know exactly who is using, and now you’re sending your data somewhere.

**Luke: Mm-hmm.

Natalia: It might be Chinese government, who knows? Right, right, right. Yeah. So yeah, and all this kind of shifting and understanding all these risks, I think helps people to be really responsive what they do.

but it came with education, so raising this risk, understanding why it’s not, it’s quite naughty to do this- Mm-hmm … helped, to make us aware of, what are the additional controls, what are the things we need to think about. [00:13:00] and, yeah, so data is what is feeding AI. Without data, it can’t work, but I think if it’s, like, your personal data, you need to decide whether you want to make AI smarter.

**Luke: Yeah.

Natalia: I’m not sure looking at big vendors if it’s always the case. yeah. But, uh, we obviously, for example, as an organization re- recognize that it’s, like, the most sensitive data. It needs to be protected at all costs, and, it’s, also deciding, so will it bring value to our customers if this data is used somehow to help them?

Mm-hmm. or can it be done somehow completely different without even looking this direction? I see. And in many cases, answer is this data is not needed. you can- … help people without it.

**Luke: Yeah. No, that makes sense. I think, you know, we saw a lot of that with, like, advertising, too, where it’s like, look, like, you don’t need to know every single thing whether, you know, to get somebody to buy a pair of Nikes, right?

Like, they just have to want the shoes or something. Um, there’s a lot of growing concern around, AI governance frameworks and how some of these things, whether that’s with [00:14:00] regulators or with, even compliance side within the orgs are being, being written by not lawyers, but people that aren’t technologists, right?

Like, that aren’t building or maybe not have the vision for what the technology will end up doing. you know, how do you bridge the gap in a large organization between make, I know we’re talking about responsibility, but, between, like, kind of- Making sure that these things are built responsibly, but having people that might not be up to speed on i- what the real risks are around these things.

Natalia: It’s all about, many problems are being solved by good communication, bringing these people quite often, in the inside the same room and, having these discussions. I think it’s about also upskilling, so we’re having these people, like… And I had really brilliant colleagues in legal domain who upskilled themselves to understand more about AI, to understand how it works, and actually really deep dive in, what it does, how it operates, where things are sitting.

And I’ve seen them actually, going for some additional courses to actually understand. They don’t need to build it. They don’t need to know, like, [00:15:00] the small things, but a foundation and basics. And I would argue that, leaders of the future, people for… You said every leader wants AI, so I think for them, understanding at least, like, foundationally what are the risks, what are specific things about AI which is different from whatever we’ve seen before- Mm-hmm

is extremely important in making the right decisions. And the same for technologists. I think what we said, like, if you know the risks, no one wants to do something completely, like, risky and something which is not really well calculated and will bring you into risk of Yeah. It’s usually comes from, not understanding fully what are the boundaries, what things are well done, for example.

And we had initially loads of discussions about EU Act, and I can remember all this, like, you know, producer, consumer, or, like, difficult ones. And yeah, you have to, one, one team, uh, goes deeper into AI. Another team goes deeper into legal domain. But, what is good, we have all the big AI, chatbots to help you- Mm-hmm, mm-hmm

so, and upskill yourself, [00:16:00] and obviously communicate better.

**Luke: Do you think that the, y- you think that the delta is, like, tighter or farther than people seem to… Like, are, are people generally, not that far off the curve, you know, when you’re talking to them about these things? I mean, ‘cause I’m just guessing from, your point of view, you’re, you’re encountering a lot of different people with this.

are people generally pretty tuned into this stuff, or from what you’re observing?

Natalia: So it greatly depends, but I can see a lot of people who are non-technical, who are really keen to understand and are trying to, do more in this domain. So, I can see more knowledgeable people appearing. And I won’t discount still having, technology people.

I think, like, all these unicorns, who understand both ways and, can abstract themselves from technological details, it’s still really difficult to find those. but, we have more leaders, that are building some applications with AI over the weekend and sharing this knowledge. so obviously they won’t train models themselves, but they at least understand what are the limitations.

So yeah, I can see this, gap being bridged. As for technologists learning [00:17:00] more about regulations, I’m not sure about this. But that’s why we have, you know… We, we still have, ChatGPT and all, other ones which are upskilling in legal domain.

**Luke: Yeah,

Natalia: yeah. Uh, so I, I think, yeah, I’ve seen less tech people into regulations.

**Luke: Yeah. that’s a good point. so kind of looking out in the next three to five years, like, what do organizations need to be doing right now to stay competitive i- in this AI world?

Natalia: identifying the ways to make, user AI transformational. So what we see, and we’ve seen many times, people come, and they change the process which was inherently bad process.

So now it’s, speeding up it by using AI. But, having this knowledge, what we just said, like, this understanding what AI can do, stepping back and empowering this transformational change is, really important. So I was in a panel just, I think two weeks ago. And, who was it? It’s AWS. They said they’ve seen some business doing.

They said it came from executive sponsorship. So executive sponsorship allowed them to [00:18:00] spend more time and run the, those smaller project to look at the whole process and see how it can transform things rather than- Having smaller things done here and there, just like go, go back, to the board and say, “What do we achieve?”

Mm-hmm. So I think those businesses who will be able to have this transformation rather than just a- additional digital technology will win. building foundations, what we said, we need some controls. We need all this documented. We need them identified because we’re building a lot of AI Which doesn’t have any, clear understanding of ethics, security maybe inherently.

So we need to make sure that we have all this information structured, we have all these guardrails, observability, and we know exactly what we’re doing without just relying for it to work. Yeah.

**Luke: Makes sense. Uh, kinda changing gears a little bit, - what are you finding, like, really interesting?

It could be unrelated to, to work or whatever, but, like, just in the field right now, is there anything that you’re, you’re kind of rabbit-holing into that you’re finding super interesting that people might wanna tune into?

Natalia: [00:19:00] Oh, so interesting. Oh, it’s, it’s difficult, yeah. So I’m all about, like, so for me, I see a lot of these things.

I’m interested how it will connect, to become, like, more ecosystems- Mm-hmm … rather than separate ones. So you have all these tools and, for ages, ages we talked about this experience where you just, like, have, I don’t know, one entry point, and you do all this stuff. So, and over the years, like recent, you had a lot of these startups that came and said, I don’t know, “You will have a pin.

It will be talking to you.”

**Luke: Yes, yeah, yes.

Natalia: And it’s kind of like this research domain. I, I’ve seen papers from, I don’t know, 15, 20 years ago, people answering, “You have a phone, you move around, and it just talks to you. You sit in a car.” Mm-hmm. So I’m curious obviously to see this one. And second one, which completely not connect to what I do, but I’m, interested, is, how AI will transform medical domain.

Mm. Yeah. Because I think that’s where, they, it accelerates what we do, and, for personal reasons obviously- Yeah … I want to live a long -

**Luke: Sure, sure …

Natalia: healthy life. So I think it’s, like, one of the [00:20:00] topics where I’m really interested how they will actually use all of this AI power to make things appear quicker and, cure people much

**Luke: faster.

Yeah. Are you seeing a… Is there a specific discipline within the medical field that, that, that seems more interesting? for me, I’m looking into it, but, on, like, okay, like how can, you know, machine learning look at all the imaging, right? Like, and start- Mm-hmm … to do, early detection and things like that.

Is there something that, that-

Natalia: So, uh, drug discovery and testing. Yeah. So, like, they do a lot of stuff, understanding personalized drugs. I think that’s the one which is super interesting because even, like, for cancer they identified that, cancer cells, all of them are slightly different, so if you can formulate something specific that will be treatable, like treating your cancer, it will be much more powerful.

And there are a lot of, like, interesting research, so which is, say, not related to what I do. yeah. But, whenever I see some podcast or something about, like, talking about how medicine’s progressing because of AI, it’s like-

**Luke: Super interesting. Yeah, it is. Yeah.

Natalia: It’s super interesting. Yeah. If you, if you find some, yeah, share my way.

**Luke: Will do. Will do. Yeah. if people wanna follow along with, what you’re doing or your work or, or anything like that, where should they go?

Natalia: [00:21:00] Uh, LinkedIn. So I think I, I tried a bit Instagram to see what’s… But I understood LinkedIn has all the great minds, and there are a lot of interesting posts there.

So I try to post about things I find relevant and interesting and share my experience of, working in enterprise, what are the problems. So, yeah.

**Luke: Excellent. Well, Natalia, thank you so much for taking the time out of the day to g- spend it with us here, and, and it was a great conversation. And, I’d love to have you back, to touch back on these subjects in the future.

Natalia: Brilliant. Yeah. Enjoy your day. Thanks.

Luke: Thanks for listening to the Brave Technologist Podcast. To never miss an episode, make sure you hit follow in your podcast app. If you haven’t already made the switch to the Brave browser, you can download it for free today at brave.com and start using Brave Search, which enables you to search the web privately.

Brave also shields you from the ads, trackers, and other creepy stuff following you across the web.