Luke: [00:00:00] You’re listening to a new episode of The Brave Technologist. This one features Margi Sheth, head of AI governance and enablement at Novo Nordisk. She’s an AI ethics champion with the pharmaceutical industry, previously serving as senior director of data policy and AI governance at AstraZeneca.

She specializes in building operational governance frameworks for large global organizations, navigating the practical and ethical complexities of AI adoption. In this episode, we discuss the importance of AI governance and ethics in the organization, how having a diverse population from all walks of life in governance is important, and what other industries can learn from how pharma handles privacy and scale.

And now for this week’s episode of The Brave Technologist.

​

Luke: Margi, welcome to The Brave Technologist. How are you?

Margi: Really good. Yeah. Thank you. Yeah. Excited to be here.

Yeah,

Luke: likewise. Likewise. I’m excited to do this. Um, so you’re speaking, we’re at the AI Summit in London. Yeah. you have a, a keynote, right, like at the conference today.

what did you cover?

Margi: Yeah, so I was moderating a panel, actually. Oh, cool.

Awesome.

Yeah. So the panel was on, governing agentic AI [00:01:00] and where you need to let it be autonomous and where you need to have some, oversight as well. So it’s a new area. A lot of different companies are starting to move into agentic AI, so how exactly do you let the agents operate, versus, where do you need some control, and what level of oversight is required?

and was there kind of a specific moment when you realized that AI governance was an idea that you know, you felt like you wanted to make an impact in?

I kind of stumbled into it-

Uh-huh …

actually. So I, was working in data policy at a company, about data access and how we can share patient data that was anonymized or how to anonymize it and that kind of thing. And, I read the European Commission had put out a white paper back in 2019-

Luke: Uh-huh …

Margi: on their seven principles on AI ethics.

Okay.

And so I read that and I said, “Hi, what? Hmm, I wonder how this maps to our, principles on data sharing.” Right. And I did a little analysis, showed my manager, and she said, “Right. You’re gonna be leading our AI ethics movement in the company.” Okay. So that’ll take like, what? One day a [00:02:00] week? And she’s like, “No.”

Oh, wow. "

It will be a full-time new job for you." Okay. All right. Yeah. So, uh, I took it from there and, took my biomedical R&D background and data policy expertise and, and channeled it into something new.

That’s awesome. It’s pretty relative, too, right? Like, I mean, data is kind of powering all this AI and, and, you know, data privacy and kind of governance over data priva- it was, you know, when GDPR kind of landed- Yeah

it was a lot of new things for people to learn, but I guess in the medical field you have like HIPAA and other things that were around, too, to kind of set the tone or whatever. How is that transition now into AI ethics and governance? What else are you having to factor in beside from the data privacy side?

Luke: Yeah. So you’re very right that we are a data-driven industry by design in all healthcare, not just pharmaceuticals, right? Mm-hmm. We’re data-driven. We have a hypothesis. We test it. We collect data from patients, which is very precious to them- Yeah, yeah … right? And, we use that to demonstrate to a health authority that products are safe and effective to go to market.

And so we’re used to health authorities. We’re used to things like [00:03:00] the, HIPAA and, and other laws around the world, right? ‘Cause we operate globally. But, when it comes to AI governance, with AI privacy it’s like the, the effects if something goes wrong can get magnified.

Right.

Margi: But there’s also so much more potential, too.

So you’re balancing risks and benefits. There’s also cybersecurity considerations. There’s just general ethical considerations, right? I’ve spoken to tech ethicists, and we have bioethics is a very strong driver- Yeah … in all we do-

Yeah …

here as well in, in pharmaceuticals and healthcare. So it’s really a convergence of many different fields.

Mm-hmm. Mm-hmm. You have a lot to manage, too, right? Like- Yeah … ‘cause you, you also got like the pace of like people wanting to ship new technology and, and all of these things you have to worry about, especially like from field to field, right? Like in medicine and in pharmaceuticals, right, you got a lot of like actual laws you gotta deal with outside of these things around distribution and licensing and all of these things- Yeah

too. Yeah, it’s really interesting. I mean, what does that… Like you, you have enablement kind of in your title, too. Like what does that mean, and why does that matter, like w- with regard to like this field?

Yeah, so simply put, um, [00:04:00] enablement is there for a couple reasons. One, it’s not just about putting rules and, and stop signs on what you cannot do.

It’s about saying, “All right, maybe you cannot do this, or maybe you need to be cautious about doing this.” And this is the way you can use AI because people are innovative, they’re curious. You know, I work with a lot of scientists.

Mm-hmm.

they’re not holding back. they wanna apply new things.

They’re excited to try new tech. , We wanna encourage their creativity, their curiosity, and, channel that towards their work as well. So how can we bring them a path to be able to use different modalities of AI, and even with their vendors, their partners, whoever else they’re working outside of, their firm or their company, we wanna help them find ways, this is what we can do, and this is how we can do it.

Mm-hmm. So that’s where the enablement is. A second piece of that is about considering risks and benefits and balancing them.

Luke: The Brave Technologist is brought to you by the Brave Search API. Access billions of indexed web results from a simple API call with the Brave Search API. Join the leading names in AI and tech using the Brave Search [00:05:00] API to power agenic search, keep LLMs current with real-time data, train foundational models, and bring the best of the web directly to the leading edge.

Get started today at brave.com/api.

Margi: So there’s a law saying that you can’t do, social scoring, for example- Mm-hmm … right, using AI. How do we define what is social scoring? You know, the law says that, but what does that mean in our institution, in a industry?

You know, it’s always gonna be a little bit nuanced, and so you can take a very, uh, l- nothing that has the word social scoring, just ban it completely. That’s a little bit too extreme, and you’re probably preventing yourselves from doing stuff that you were doing all along.

Right.

So how can you- Interpret that in a more progressive way that still keeps the safeguards that matter and the ones that are relevant to our industry, while still being able to do, you know, you have to, you have to sort patients by certain categories and stuff in a clinical trial.

Yeah.

If you decide that’s social scoring and then just complete ban it, you can’t really run a clinical trial, right? Yeah. This is a very [00:06:00] simple example, but, it’s about making sure you have those right interpretations and you apply that in a way that you can bring the benefits to society that are grounded, again, in bioethics, in AI ethics- Yeah

in governance, privacy, all those sort of things, without hindering innovation. I

think it’s

a great, it’s a- It’s not a easy thing to do.

Luke: No, it’s a great point, ‘cause it’s like, to thread that needle, I mean, what I’m getting from what you’re telling me is like human judgment in AI governance is like super important.

Yes. Like, and, and I feel like perception is out there where people are just like really concerned about like humans getting out of the loop. But like with these things- Mm-hmm … it’s like this is where it really matters, where you need to have people that know, know the nuance and know, you know, okay, here’s how it’s practically applied, here’s where we have to watch out for legal, here’s where it could go wrong, right?

Mm-hmm. Like, because if those things are not, the false positives and things like that that can happen with this stuff is pretty, pretty wild. yeah, I mean, like speaking of AI governance, right, like, I feel like with compliance and with these things, it can often feel like a checkbox kind of exercise.

Like, how do you instill AI, [00:07:00] good AI governance into organizations where it doesn’t feel like a checkbox exercise and gets kinda taken a little more seriously in process?

Margi: Yeah. So remember big firms, they already have so many layers of governance, right? They’ve got procurement governance. Mm-hmm.

They’ve got portfolio governance. They’ve got data governance, whatever else they have. What you don’t want is just another set of forms to fill out that’s completely divorced from the others, right?

Right.

It’s, just becomes cumbersome, and the worst thing that can happen is someone says, “Oh, this is too much.

I can’t handle it. I’m just gonna bypass it.” Mm-hmm. Then you’re not being governed at all, right? Right, right. Then your governance regime essentially is ineffective.

Right.

So, the way to fold it in, it- there’s no beautiful, perfect answer here, but the way, that we’ve tr- done it at a few different companies I’ve worked at is to build a, a scale, like the EU AI Act has a scale, right, of low, medium, and high risk.

Mm-hmm. What does that mean in your company, and where do you not really need much governance because it’s pretty low risk, it’s everyday work? An example in pharma is, uh, you [00:08:00] know, an, a machine learning tool that’s maybe predicting how different molecule configurations can work and how they may interact-

Interesting

in your body. That sort of algorithm has been running for like 25 years or more.

Right.

Right? We’ve been using stuff like that. It is so far upstream of touching a patient. It’s validated so many times. It’s a financial commitment, too, for a company to say, “Right, this is its molecule structure we want.

Let’s go build it. Let’s go test it.”

Mm-hmm.

Getting it wrong, over-governing it at that stage, it’s gonna be governed already by a drug discovery process. Right. There’s no need to do that.

Right.

so finding that threshold, those thresholds, and even in your tooling, even if you’re using large LLMs across your company or something like that, what sort of level of governance do you need for different types of things?

Mm-hmm. If it’s someone’s using a tool to, say, summarize their emails-

Mm-hmm …

I got eight emails from this guy. There’s a whole thread, I can’t read it. Please summarize it for me. It’s a reality, right? Oh,

totally.

Yeah,

yeah, yeah.

It’s personal efficiency.

Yeah, yeah.

So if you’re doing that, I don’t really wanna govern that too much.

[00:09:00] Right. Right? Because if it’s just me saying, “I got a thread of emails, summarize it for me so I can figure out what to do next,” and I do something silly based on that, that’s on me.

Right, right,

right. Like, literally I’m fully accountable for that.

Right.

Now, if I make that sort of, I use that LM, I make a tool, make an agent, make a chatbot like that that says, “This is an email summarizer,” and then I say, “Right, 1,000 employees, I’m gonna share it with everybody.

This is great. It’s worked for me,” maybe there’s a little bit more repercussions. Right. So how do you know when something needs to be governed almost none to just a little bit to full risk assessment? Mm-hmm, mm-hmm. Like, what’s the impact? What are the consequences? How many people are impacted? And, and does it reach, uh, things like manufacturing process or, or patients, or product quality, that sort of thing?

At least in pharma, that’s when it may have a little bit more, uh, need some more governance and- Yeah … take more process there.

Yeah. Well, how is it balancing all this with, like, I mean, I know Novo Nordisk is, like, a global company. and- Mm-hmm … and you’ve got all these [00:10:00] different jurisdictions too, right?

Yeah. Like, when you’re trying to kinda balance out, governance, policy, and process with all of those kind of regional, you know, jurisdictional considerations, how is it managing that?

Yeah. So you kind of need to approach this with a split of activities that are occurring only in one jurisdiction and those that are occurring on the global scale- Mm-hmm

and affecting global operations. Mm-hmm. I mean, you have to think about the reality that data centers can be, you know, you can, you cannot just assume it’s a Danish company, all data centers are in Europe. You can’t necessarily- Right … assume that. So you can have data centers, you can have teams, analysis teams, access all around.

So you need to have your global rules and guidelines for the company on what can and cannot be done, things for cross-border, data and AI movement, agent movement, you have to think about that too, across jurisdictions as well. But then there are the setups that you need to consider where, well, this is operating in this one country alone.

we don’t need to worry about a certain European regulation or certain American regulation. It’s [00:11:00] not quite fitting there, so we can maybe be a little bit more open and, looser with our policies, as long as it does not violate the, the company values, the ethics, the ground rules for the company.

Luke: Yeah. No, that makes sense.

Margi: , So there’s always this kind of tension between, like, breakthroughs in pharmaceuticals, can have such, like, potential, to be accelerative , with balancing that with the need to move carefully, right?

how do you kind of hold those truths at the same time without kind of stifling the process or slowing things down?

Yeah, that’s a great question. So there are, again, I talked about there’s a risks and benefits balance that you need to consider. And, so the risks are things like violating compliance rules, and as a company, you don’t wanna be having fines or breaches or lose, patient trust and that kind of thing.

but then there’s also, and there’s also the consideration of patient safety, product safety, that kind of thing. You can’t compromise on certain things. However, the benefits, you need to be able to quantify them, too. [00:12:00] Mm-hmm. And the benefits to society, to advancing science and medicine are real when you use AI.

Mm-hmm. And it is a responsibility as well to take advantage of the technology that’s available to advance medicine and human health-

Yeah …

if you have the means and you have the, and the capability, you could say it’s a risk of not doing, right? in a corporate sense, yeah, you’ll fall behind. There’s competition everywhere.

Big Pharma may not be the only rivals that Big Pharma has- Right … in a few years, right?

Luke: Right.

So there is a reality of, the competitive nature of the world and even of, of hiring and keeping good talent.

Mm-hmm.

Margi: So you need to consider that. but you also need to consider the benefit to society that you could be bringing.

And again, being grounded in bioethics, being grounded in a long history of medical ethics as well, too, I think that gives us a very strong compass in navigating what we ought to do-

Mm-hmm …

and where we don’t cross lines to be able to deliver our promise.

Yeah. It seems like if you can kind of build that [00:13:00] into the ethos of, like, the company, right?

Like, uh, a lot of those strong kind of ethical, practices and it- just like this is how we kind of operate as an org, that it can be pretty effective. Is, is that fair?

Yeah, yeah, and that takes time, too, right? Yeah. Even if you have a mature company with strong values and it’s kind of ingrained in the way people act, it’s a almost like an extension of those values and how they’re applied when you’re using human and AI working together, whether it be machine learning or chatbots or an agentic workforce- Mm-hmm

as, many companies are trying to adopt now.

Luke: Yeah. how do you see decentralization as a meaningful force in how health data gets governed?

Margi: Can you expand a little bit?

Luke: Like, uh, you know,we’re kind of in this space where you’ve got different types of, providers, different types of competition, people trying different things out, right?

Like, just kind of the nature of things is getting a lot more decentralized in a lot of ways, right? and thinking about how that end area of the world is adapting, does it even factor in, right? is it even [00:14:00] something where you guys have enough, compliance things in place to where it’s not l- as much of a reaction concern or something like that?

Margi: Well, laws and regulations globally are constantly evolving in- Right … in the AI space, right? Yeah. Maybe not in the health authorities space- Sure … as much. there’s more stability there. But They’re also adopting what does it mean to use AI to prove your products are safe and effective, right?

So this is changing fast, changing globally. there is a level of unity that you need as a company in how you operate and how you take things forward, but the reality is also that AI is not just the IT department’s problem. Right. It’s not for us to solve. It’s not for us to, uh, speaking as a person who works in IT right now, it’s not for us to solve for everything, and we n- cannot be accountable for everything that happens when you use AI.

Right. I mean, people are using computers all the time. Yes, it’s the IT department supplies the computers, and makes sure things work, and makes sure programs work, but if you do something silly with your computer, or against your values or something, the IT department’s not to blame, right?

Right. So the same way, there is a [00:15:00] level of accountability that goes across an organization and to the individuals, too. And so there’s a level of governance that it may be prudent to have some kind of AI governance, and policy, and oversight from, an area of the business- Mm-hmm … especially because different areas of business, at least in my industry, are also adopting AI in different ways- Right

and different speeds-

Yeah …

different approaches. Right. And even different leaders will have, variance in their own aptitude and, Willingness to take on AI.

Yeah, yeah.

I mean, I can speak, because I’ve spent much of my career in biomedical R&D, it’s a lot of scientists. It’s a lot of computational biologists and informaticians, and that kind of career path.

You’re gonna be taking on AI much faster than- Right … maybe some other areas where, it’s not part of the job description.

Sure, sure. No, that’s fair. That’s fair.

Well, it’s now becoming part of the job description, I should say.

It’s gonna be coming everywhere, right? Like- It

wasn’t there 10 years ago.

Luke: Totally. what’s your honest read on kind of regulation keeping [00:16:00] pace with technology? Like, Is there a lot of catch-up? Is there a big delta there?

Margi: There’s always a delta there.

Yeah, yeah, yeah, yeah.

There’s always a delta between regulation and technology. and so if you have your guiding star as a firm, as an organization, you have your values, that kind of thing, the best you can do is try and be prepared for upcoming regulations and say, “We’ll apply these sort of policies, these sort of rules, ground those in our values and the way we wanna operate, and the way we want our vendors and partners who work with us to operate with us too.”

And that can try and keep you, you know, maybe you’re 80% there when the law comes out, because many of these laws, especially the European ones, which I tend to look at most in my own career, they have white papers, they have call for, feedback, that sort of thing. You can see where it’s going. Mm-hmm.

And then they give you some time before it’s effective.

Mm-hmm.

So, that’s a good thing, because you have some foresight. But, rather than being, waiting for the day to go effective and say, “Scramble, what do I need to do to continue, being able to operate?” You can’t afford that in [00:17:00] pharmaceuticals. You can’t afford to say, “Well, we didn’t necessarily follow this, and now this patient might be at risk.”

Right? the stakes are not, just fines.

Luke: Right. Right.

Margi: Right? The stakes can be much more. Yeah. And so it’s prudent to try and prepare for these regulations, and even if you think there’s gonna be some wiggle room, maybe you can be 70% there and then say, “All right, the law has, this is how it came about.

Let’s make some tweaks, but we were mostly ready.” That would be my recommendation.

Luke: Yeah, yeah, no, it makes sense. It makes sense. Is there anything we didn’t

cover that you want people to know more about?

Margi: Yeah, I’ll say that, the field of AI ethics, AI governance, responsible AI, whatever phrase you wanna use, they’re all pretty, pretty close to each other. It’s a multidisciplinary field, and that means it is beneficial to have people from all kinds of backgrounds, all kinds of walks of life, to join this field, to join this conversation, and have a diverse set of perspectives in the room when these decisions are being made.

So I encourage you. I’m not a lawyer- Mm-hmm … so I don’t have a law degree. I’m not a [00:18:00] developer. I didn’t come up with a computer science background, and I’m not a philosopher. none of those are my degree. My degree is in biotechnology and I have been doing this for several years now.

and I think that brings a benefit because it’s a multidisciplinary field. It has different angles, that all form that complete picture. So anyone that’s interested in this field and wants to come into it, whatever your background is, whatever your education is, your voice is important-

because you’re also a data subject.

Mm-hmm.

Every person on this planet, their data, their life is gonna be impacted because they are becoming data in some form or another, right? So it’s to your advantage to be able to, to advocate for yourselves as a, by joining this field and joining the conversation. So I encourage people from all walks of life, of experience, if you’re interested, I think it’s a great field to get into because it’s opened my mind so much and I would love to, to see more people from different backgrounds in this field.

That’s fantastic. That is a great, great, great point to end on, too, and, where can people, uh, go to, to follow [00:19:00] your work to learn more or, or, you know, say hello?

Probably LinkedIn is the best place to go.

Luke: Okay, cool. Awesome, awesome. Well, Mari, thank you so much for making the time out to have a conversation.

Really appreciate it- Thank you so much … and I think our audience will, and, and love to have you back sometime to kinda check back in on things.

Margi: Definitely. Thank you so much. Awesome.

Thanks.

Thank you.

Luke: Thanks for listening to the Brave Technologist Podcast. To never miss an episode, make sure you hit follow in your podcast app. If you haven’t already made the switch to the Brave browser, you can download it for free today at brave.com and start using Brave Search, which enables you to search the web privately.

Brave also shields you from the ads, trackers, and other creepy stuff following you across the web.