How Microsoft is Making AI Trustworthy

[00:00:00] This is AI Inside, episode 39, recorded Wednesday, October 16th, 2024. How Microsoft is Making AI Trustworthy.

[00:00:11] This episode of AI Inside is made possible by our wonderful patrons at patreon.com slash AI Inside Show.

[00:00:17] If you like what you hear, head on over and support us directly. And thank you for making independent podcasting possible.

[00:00:30] What's going on, everybody? Welcome to another episode of AI Inside, the show where we take a look at the AI that's layered throughout so much of the world of technology.

[00:00:39] We've got some really interesting stuff to talk about today. I've got my dog standing next to me here, wishing that I had taken him outside.

[00:00:46] Hopefully, I don't pay the price for that. I'm one of your hosts, Jason Howell, joined as always by my co-host, Jeff Jarvis.

[00:00:51] How you doing, Jeff?

[00:00:52] Hey, how are you, boss?

[00:00:54] Excellent. Doing fantastic and excited for today. We've got a really cool interview lined up with Sarah Bird, Microsoft's global lead for responsible AI engineering.

[00:01:06] There's some news from middle of last month that's kind of tied to that as well.

[00:01:11] And this is an interview we wanted to do earlier, and then the technology didn't work.

[00:01:15] StreamYard's fault.

[00:01:15] Crossing fingers. Yeah, crossing fingers that it works today. Super excited to talk with Sarah.

[00:01:22] And we have a lot of news to get to before Sarah joins us.

[00:01:26] So we're going to just jump right in.

[00:01:28] But before we get started, big thank you to those who support us directly on Patreon.

[00:01:32] You know who you are.

[00:01:34] And even if you don't, I'm going to call one of you out.

[00:01:37] Keith Harrison is one of you.

[00:01:39] And you guys are great.

[00:01:40] Thank you so much for your support.

[00:01:41] Patreon.com slash AI Inside Show.

[00:01:44] It helps keep the lights on.

[00:01:46] Studio lights, you know what?

[00:01:48] They don't pay for themselves.

[00:01:49] So there you go.

[00:01:51] Appreciate you guys supporting.

[00:01:52] Also, if you happen to be watching live, well, hey, it's great to have you here.

[00:01:57] And you should, you know, go to AIinside.show and subscribe while you're at it.

[00:02:02] That way, if you miss the live stream, you catch it each and every week in the form of a podcast download.

[00:02:09] And that's all there is about that.

[00:02:12] Why don't we just jump right into news?

[00:02:13] And I put, you put in some really interesting and great stories, as you usually do.

[00:02:18] But the top story today, I wouldn't say is necessarily my, you know, it's not like top of the news zeitgeist stuff.

[00:02:25] But I find it really fascinating.

[00:02:26] It's a paper that introduces GSM symbolic research, which is a benchmark for at least my understanding, because I did some reading through this, evaluating the mathematical reasoning ability.

[00:02:42] So it's all about reasoning within LLM models.

[00:02:45] And this paper that you threw in here, the couple of supporting links, is kind of a different approach to analyzing and looking at open source models like LLAMA, like PHY, GEMMA, Mistral.

[00:03:01] Also, closed source models like OpenAI's GPT-4-0, the O1 series, all that kind of stuff.

[00:03:09] And take a look to see how well artificial intelligence can do reasoning.

[00:03:17] In particular, this seems to be focused on math and how it approaches those math problems, right?

[00:03:24] Right.

[00:03:24] Right.

[00:03:25] The headline here, and it's from four or three or four Apple computer scientists, say that they are not as good.

[00:03:33] And after OpenAI came out and said, our new model, 4.0, is good at reasoning.

[00:03:38] Look what it all can do.

[00:03:40] They throw water on the party by saying that they just simply aren't as good at reasoning, mathematical reasoning, as you would imagine in large language models.

[00:03:50] Right.

[00:03:51] And it's beyond my ken about how to explain exactly how they went about the methodology of the paper.

[00:03:56] But they took some problems.

[00:03:58] They believed that the structure for judging whether or not it was good at reasoning was somewhat flawed.

[00:04:06] And so they came up with their additional one.

[00:04:08] And this one tweet from Merhad Farajtabar, one of the scientists, in his thread, I think made a thread to me.

[00:04:20] He said, the fragility of supposed LLM reasoning.

[00:04:23] LLMs remain sensitive to changes in proper names, people, food, objects, and even more so when numbers are altered.

[00:04:32] Would a grade school student's math test score vary by 10% if only we changed the names?

[00:04:39] Right.

[00:04:39] Susie has four apples and Jason has three.

[00:04:43] And they cut them up and eat them, right?

[00:04:45] So you have a problem.

[00:04:46] And then if they change Jason to Sam and they change Jason's three apples to five apples, this threw the model off.

[00:04:56] And so what they saw was that the model was good and so good at mimicking things, right?

[00:05:02] The model was good at mimicking reasoning because of prior art in the training set.

[00:05:10] But when you threw it off, and what also fascinates me about AI all the time, generative AI, is the effect of randomness.

[00:05:18] The effect that you don't get the same answer twice.

[00:05:20] Right.

[00:05:21] And what effect that has.

[00:05:22] So they created this symbolic template where, in other words, the symbol when Sophie watches became when Brackett's name watches.

[00:05:31] Her nephew, which became Brackett's family.

[00:05:34] So then they could throw in other variables under those symbols, whether they were numbers or players, if you will, in the mathematical drama here.

[00:05:44] And the fact that that threw it made them say that, you know, we're not so sure that it's really so good at reasoning as it's been portrayed.

[00:05:54] So I found this – I just think it's so important not to throw water on all the parties here.

[00:06:01] Generative AI is amazing and does amazing and phenomenal things.

[00:06:04] But let's not get carried away.

[00:06:06] Let's be realistic about it.

[00:06:08] And as this mad rush of saying, we're – AGI is around the corner.

[00:06:13] Let's pull back and say, actually, it's about 1,000 miles away.

[00:06:17] Yeah.

[00:06:18] And that's okay because then we have a more realistic sense of what it can do and what its value is.

[00:06:24] I saw another story today, which is not in the rundown, about how the world's energy consumption, electric consumption, is going way up.

[00:06:32] Now, part of that is climate change and air conditioning.

[00:06:34] But part of that is also probably the impact of computing power and all the technology we use.

[00:06:39] Is the power of what LLMs can do so valuable that we're willing to pay other prices for it?

[00:06:46] Well, then we've got to know what it can do and what it can't do.

[00:06:48] So having a realistic discussion about reasoning without the hype I think is so important.

[00:06:53] So that's why I put this up here.

[00:06:54] That's why I'm so glad you led the show with it because even though I can't fully understand everything they did,

[00:07:00] I think it's an important discussion.

[00:07:03] Yeah.

[00:07:04] I mean, it also illustrates some of the things that we've come to expect and almost excuse to a certain degree about text-based LLMs,

[00:07:17] the fact that you can give it a prompt and you'll always get a different outcome and sometimes it's not going to be accurate and everything.

[00:07:24] There's a little bit of wiggle room when we're talking about text, I think, and understanding in that regard.

[00:07:29] When you're talking about straight up math, there's a right answer and there's a wrong answer.

[00:07:34] Right, right.

[00:07:35] And if you're having similar challenges with something like this when it comes to those mathematical problems,

[00:07:41] and like you said, something as simple as like, you know what?

[00:07:44] We're not changing the structure of the actual math equation at all.

[00:07:49] All we're doing is we're changing the label of the person at the heart of it as one example.

[00:07:54] And that being enough to throw off the system really illustrates the distance that we are from getting to the point that you're talking about as far as AGI is concerned.

[00:08:06] There's all this talk that throwing more resources at these systems is going to get us to AGI faster.

[00:08:14] And yet this is a really like simple problem.

[00:08:18] Like maybe the mathematical formula isn't a simple problem, but the swapping out of names and totally confusing it and sending it in a completely different track as a result, that's like grade school level stuff.

[00:08:30] Yeah, and it's about reliability too.

[00:08:32] It's not, I mean, when you change Sally to Jason, it was wrong 100% of the time.

[00:08:37] It was just wrong a percentage of the time.

[00:08:39] Right.

[00:08:39] But it threw it off to that extent.

[00:08:40] So as we've talked about often, the next stage after queried AI is probably agent of AI.

[00:08:47] Well, if you're going to deputize an agent to go do something for you, you need it to have reliability.

[00:08:51] You need to know that if I ask you to get a pizza for me and the person who answers the phone says, hi, Jason talking when you're used to it being, hi, Sally talking is going to throw the machine and you get the wrong pizza.

[00:09:03] You're not going to trust the agent.

[00:09:04] Stupid example.

[00:09:05] Sorry.

[00:09:06] But this discussion is important.

[00:09:08] So yes.

[00:09:09] So good.

[00:09:10] Yeah.

[00:09:11] Yeah.

[00:09:12] Yeah.

[00:09:12] Yeah.

[00:09:12] Interesting.

[00:09:12] I mean, it really seems to make the point that reasoning isn't happening nearly as much as, you know, because that word has been thrown around a lot in recent months.

[00:09:25] And, you know, that isn't happening nearly as much as, you know, matching patterns, let's say.

[00:09:31] It's magnificent mimicry.

[00:09:33] Yeah.

[00:09:34] And it matches what we've done in the past.

[00:09:36] Impressive mimicry, yeah.

[00:09:36] Which is really amazing, but it has its limitations.

[00:09:40] Mm-hmm.

[00:09:40] Mm-hmm.

[00:09:40] Which leads to the next story where Christopher Mims at the Wall Street Journal, who he and I have disagreed in the past.

[00:09:48] He blocks me on Twitter.

[00:09:49] I don't know why, what I did.

[00:09:51] But I think he did a good piece here with Jan LeCun reminding us that he thinks that cats, I'm sorry, that AI is dumber than a cat.

[00:10:01] And again, this is a leavening of the yeast around AI that I think is so important.

[00:10:07] And I think Jan LeCun is doing great work.

[00:10:09] He's releasing open source.

[00:10:12] He's got a sensible voice about this.

[00:10:15] He shares openly.

[00:10:17] So I was really glad to see Christopher give LeCun some ink.

[00:10:22] Yeah, I mean, Jan LeCun, you know, another one of the godfathers of AI.

[00:10:27] I feel like there's a handful of them out there.

[00:10:30] Jan LeCun is definitely one of them.

[00:10:33] And I think friends with Jeffrey Hinton, if I'm not mistaken, one of the other godfathers.

[00:10:36] Yes, but they disagree about a lot of it.

[00:10:37] It becomes clear, yes.

[00:10:39] Yeah, yeah, yeah.

[00:10:40] Well, yeah, so Jan LeCun says that he considers the idea that AI is a hazard for humanity, quote, complete BS.

[00:10:48] Yay!

[00:10:48] He says that today's AI models are not as intelligent as a house cat, like you said, lacking qualities like a mental model of the physical world, persistent memory, reasoning ability.

[00:11:01] There's that word again.

[00:11:04] He says AGI is absolutely not imminent and will take decades if it happens at all.

[00:11:10] Yep, yep.

[00:11:11] Which I think is, but nonetheless, doing phenomenal things.

[00:11:15] There was another story I saw this week about how the stock value of Meta is boosted tremendously because of what they're accomplishing with AI.

[00:11:23] So for them, it is worth it, at least in that sense.

[00:11:26] And he's a leader.

[00:11:27] So I wonder if he was jealous of his friend Jeffrey getting the Nobel Prize.

[00:11:33] He doesn't say.

[00:11:34] Yeah, yeah.

[00:11:35] I doubt it.

[00:11:36] You know, one question that came up for me around this, too, if, you know, as far as AGI is concerned, if AGI is going to take, let's say, decades, you know, if at all, how does that reality impact players like Sam Altman,

[00:11:54] who keep assuring that it's right around the corner, who keep pointing at that mark on the wall and saying that's where AGI happens and we're close, we're getting closer.

[00:12:04] I mean, at that point, if we're talking decades, like he's exposed to a certain degree at that point, right?

[00:12:11] Like does that impact reputation or the reputation of open AI?

[00:12:15] And he's not the only one to be, you know, spouting off on this particular thing.

[00:12:20] But I don't know.

[00:12:20] What's your thought on that?

[00:12:21] It's really interesting, Jason, because I think one term I learned when I was in management and media was the term sandbagging.

[00:12:31] That if you're in sales, you want to sandbag your projections.

[00:12:37] So if the company says, well, I think Jason can sell 10% more next year.

[00:12:40] And Jason says, oh my God, no, it's tough out there.

[00:12:42] Didn't you hear about this?

[00:12:43] Didn't you hear about that?

[00:12:43] No, I think if I could do 3%, I'm lucky.

[00:12:46] And then when you sell 10%, you get a bigger bonus and you don't get fired, right?

[00:12:52] So I think that we get to the point here where Altman is in real danger of overselling to the extent that people are going to be inevitably disappointed with what he delivers and not necessarily trust his predictions.

[00:13:13] He'd be smarter to say, well, it's pretty cool.

[00:13:16] But then he wouldn't get a company worth billions of dollars and get all the VC money.

[00:13:21] Right?

[00:13:21] So he has to oversell for the VCs.

[00:13:25] But the market and eventual equity market, I think, could be very harsh on him.

[00:13:31] Yeah.

[00:13:32] Yeah.

[00:13:33] I'm working on a history right now of the line of type, which made type a line at a time.

[00:13:39] I won't bore you with it right now.

[00:13:40] But I was just reading stuff from when the inventor, Otmar Mergenthaler, started the machines.

[00:13:46] He got one working and he got excited.

[00:13:48] And so did the publishers who invested because they saw they could save a fortune in typesetting.

[00:13:53] So they wanted to order 100 machines.

[00:13:55] And Mergenthaler said, no, just 12.

[00:13:57] Let's get it right.

[00:13:58] No, we must have 100.

[00:13:59] So fine.

[00:14:00] They made 100.

[00:14:00] And then the next thing that happens is all the executives are carping about how it doesn't work perfectly.

[00:14:05] He tried to tell them.

[00:14:06] And I think AI is going to be a similar developmental scheme here where it's going to get oversold, over disappointed.

[00:14:14] But then at some point, this is why the reality discussions we're having, the reality checks are so important, I think.

[00:14:20] Yeah.

[00:14:20] Yeah, indeed.

[00:14:21] Well, talk about reality discussion.

[00:14:23] This next story is definitely something worth talking about.

[00:14:27] There is a debate in Silicon Valley right now around autonomous AI weaponry.

[00:14:32] TechCrunch has an article that kind of focuses on pieces that this discussion has.

[00:14:39] Palmer Luckey, former Oculus.

[00:14:41] Now, what is his company now?

[00:14:44] It is Anduril.

[00:14:44] I forget the name.

[00:14:44] He's a weapons company.

[00:14:46] Anduril.

[00:14:47] Or Andruil?

[00:14:48] Andruil.

[00:14:49] Anyways, in this article, he's making at least to a certain degree a case for them.

[00:14:56] And then, of course, there are others say they should never be allowed in the United States.

[00:15:00] Lucky is questioning, as one example, the moral high ground against autonomous weapons when things like landmines are indiscriminate and dangerous to everyone, even those not targeted.

[00:15:15] Right?

[00:15:16] That's kind of the case being made there.

[00:15:19] Yeah, I think we're always nervous about the idea that an AI could make a decision that could be fatal.

[00:15:27] Yeah.

[00:15:28] But I guess his point is—

[00:15:29] Which, I mean, by the way, happens.

[00:15:31] Tesla vehicles make mistakes.

[00:15:34] Yep.

[00:15:34] I mean, you know what I mean?

[00:15:35] Like, they've made mistakes.

[00:15:36] Self-driving AI has made mistakes and led to fatalities.

[00:15:40] Humans make a lot of mistakes, too, by the way.

[00:15:42] So my reflex is to say, right, we don't want these things making judgments.

[00:15:46] That's ridiculous.

[00:15:46] But Lucky has an interesting point that if the landmine could tell this is a nun bringing a bus full of schoolchildren over it not to explode, would we want that judgment to be there?

[00:15:59] Does it make the weapon less evil?

[00:16:02] It's an interesting ethical discussion to have, I guess.

[00:16:07] For sure it is.

[00:16:09] Yeah.

[00:16:10] Trey Stevens, also Anderl, co-founder, emphasizes the importance of human accountability in any lethal decisions.

[00:16:20] Amen.

[00:16:21] Yeah, of course.

[00:16:22] You've got to always have that human accountability, even when these autonomous systems are fired up and running.

[00:16:27] And maybe fatality or death isn't a part of the equation.

[00:16:33] But even when we're talking about copyright and all these things, really, at the end of the day, you've got to—the humans are part of this process.

[00:16:38] Yeah, I think one of the equations here is if the weapon could communicate with a human being who is responsible, then shouldn't it?

[00:16:47] So where is autonomy, in fact, necessary versus—because the landmine can't communicate back and say, should I or shouldn't I?

[00:16:58] But a communicative weapon could, and so the decision then becomes not autonomy versus nothing.

[00:17:04] It becomes autonomy versus communication and human responsibility.

[00:17:10] Yeah, yeah, super interesting.

[00:17:12] Yeah, this story kind of resonated a little bit for me because this weekend I was camping with a couple of friends of mine.

[00:17:21] We were up at their property and doing some work up there.

[00:17:25] And he was talking about how he saw this video on TikTok of, you know, within the Ukraine war that's happening right now, these videos—and I only saw one.

[00:17:38] I hate videos like this.

[00:17:39] They're really hard for me.

[00:17:41] But where there is a drone—essentially, there's someone hiding out, and these weaponized drones are just, like, burying in.

[00:17:49] You kind of hear it in the distance, and then it finds its target.

[00:17:51] And we were just kind of talking about how absolutely frightening it would be to be in that situation.

[00:18:00] And then when I read this story, I was like, oh, my goodness.

[00:18:03] Like, what a—that's just such a frightening reality for anyone to be faced with.

[00:18:09] That there is a drone, let's say, that has an autonomous assignment to go out and find anyone with.

[00:18:20] Whether they deserve it, you know, and put that in air quotes or not, you know, kind of aside from that, just from a humanity perspective, it's kind of frightening.

[00:18:29] Well, but it's the same thing, I think, Jason, with missiles.

[00:18:32] Is a dumb missile better than a smart missile?

[00:18:34] Well, that's true.

[00:18:35] That's a really good point.

[00:18:36] Yeah, right.

[00:18:37] I mean, the humans have designed these missiles to have certain targets and everything like that.

[00:18:43] But to a certain degree, the machine is acting on its directive, same as an autonomous machine would be acting on its directive.

[00:18:53] It's just using a different type of smarts.

[00:18:55] Well, we can't lose the human responsibility in the design and the use of it.

[00:19:01] Yep.

[00:19:03] Yeah.

[00:19:03] Yeah, absolutely.

[00:19:06] So last week, Elon Musk held an event, the Wii Robot event, where they, being Tesla, were holding this event at Warner Brothers Discovery Lot in Hollywood.

[00:19:23] And it was an evening time event.

[00:19:25] So, you know, it was very, you know, like, very colorful and kind of futuristic and very produced.

[00:19:33] It was a pretty short, the actual event was very short.

[00:19:36] And then it kind of was opened up to all of the attendees kind of roaming the virtual city and exploring one of three things that they announced at the event.

[00:19:48] The first thing being the cyber cab.

[00:19:50] They revealed 20 cyber cabs at the event that attendees could hop into and get a self-driving trip around the studio lot.

[00:20:01] And essentially, this is like a two-seater cab.

[00:20:04] No steering wheel.

[00:20:06] No pedals.

[00:20:07] It's like individualized mass transit.

[00:20:11] People are going to be able to buy one of these, according to Elon, for less than 30 grand.

[00:20:15] If we believe, number one, this is in a very controlled environment.

[00:20:19] And number two, it's Elon.

[00:20:21] And we never know when to believe his predictions of his own activity.

[00:20:24] Yes.

[00:20:25] Indeed.

[00:20:26] Yeah.

[00:20:26] All very important to point out.

[00:20:29] Caveats that always should be around Elon.

[00:20:31] Really controlled environment.

[00:20:34] So, you know, like, you know, part of me doesn't doubt that they were autonomous because they kind of already have autonomous vehicles to a certain degree.

[00:20:44] And it was a controlled environment.

[00:20:46] So maybe they felt safe enough.

[00:20:48] But we'll get to why you might want to doubt that in a second.

[00:20:51] The second thing that they showed off was the Robovan or the Roboven, I think, is what Elon was calling it during the announcement.

[00:21:00] It's an autonomous bus, 20-person capacity.

[00:21:04] No timeline for it.

[00:21:05] But, boy, does it look like it looks straight out of Blade Runner to me.

[00:21:09] It's just like seeing those things buzzing around.

[00:21:12] I don't know.

[00:21:12] I don't hate the design, to be honest.

[00:21:15] I'm kind of like, okay, that's a future look.

[00:21:17] I like it.

[00:21:17] I see Schwarzenegger getting out of it.

[00:21:19] Yeah.

[00:21:19] Yes.

[00:21:20] Totally.

[00:21:21] Totally.

[00:21:22] It's like straight out of Total Recall or something.

[00:21:26] And then finally, and this is the one that I think, does it have any of the robots in this article?

[00:21:32] No, it does not.

[00:21:33] But I can pull up one that does.

[00:21:34] They did an update on their humanoid robot kind of advancements, let's say.

[00:21:42] The Optimus humanoid robots, which they've been talking about for a few years now.

[00:21:47] I think the first time that Tesla talked about this was in 2021.

[00:21:51] And at the event, these robots, in air quotes, took the stage.

[00:21:56] And they were so obviously humans in robot suits dancing.

[00:22:00] Like Trump dancing this week, yeah.

[00:22:03] Oh, boy.

[00:22:05] This event, you know, things have evolved a little bit.

[00:22:08] They do look more robotic.

[00:22:10] They've got, you know, it looks like there's some servo action happening.

[00:22:14] Whether these are actually being autonomously driven is another question.

[00:22:21] Like, I don't know.

[00:22:23] Like, I don't know.

[00:22:23] Maybe that is human.

[00:22:24] Maybe it is a robot.

[00:22:25] I'm not entirely sure.

[00:22:26] Like, it's probably served up from a servo according to what many people have reported on, right?

[00:22:35] Yeah, I think it's more hype.

[00:22:41] More hype.

[00:22:41] I mean, people were talking to these robots.

[00:22:43] Like, Robert Scoble was at the event and apparently asked it, like, are you, you know, is there any human interaction?

[00:22:50] And the Optimus kind of refused to answer it directly.

[00:22:55] And then he followed up saying, like, any at all?

[00:22:58] Are you sure?

[00:22:59] And he said, well, there might be a little bit.

[00:23:00] Or I said he.

[00:23:01] The robot said, well, there might be a little bit, which who the heck knows what to believe here.

[00:23:07] But let's just set that aside for a second.

[00:23:10] Maybe this is just, you know, a Hollywood glimpse into the future, the potential future of the Jarvis household.

[00:23:18] Humanoid robots walking around, maybe walking your dog or bringing you a drink while you're watching TV.

[00:23:24] If it doesn't load the dishwasher exactly right, it'll get kicked out by my wife.

[00:23:28] That's what I come close to all the time.

[00:23:31] Right.

[00:23:32] Could you see yourself ever getting one of these things?

[00:23:35] What was he saying?

[00:23:36] $20,000 to $30,000 in the future sometime.

[00:23:40] It's the same problem I have with chat interfaces.

[00:23:44] I don't really know what to ask it.

[00:23:46] It's a solution looking for a problem.

[00:23:48] What am I going to do with you, robot?

[00:23:49] Yeah.

[00:23:49] Yeah.

[00:23:50] I had an assistant once in my career in office, you know, executive assistant once in my career for a very brief time.

[00:23:57] I had no idea what to have the person do.

[00:24:00] I'm just not used to it, right?

[00:24:02] So.

[00:24:03] Yeah.

[00:24:03] I mean, I see the price tag of $20,000 to $30,000 and I'm like, okay, well, that's not like astronomically expensive.

[00:24:11] But yet I see these things as toys for rich people.

[00:24:15] Yeah.

[00:24:16] What it should be is for people who cannot fully take care of themselves physically.

[00:24:21] Yeah.

[00:24:21] Right.

[00:24:21] But who's going to pay for them then?

[00:24:23] Because those are people who don't have huge jobs because they have difficulties and insurance won't pay for them.

[00:24:28] And so the.

[00:24:29] I was going to say insurance will take care of it.

[00:24:31] Yeah.

[00:24:31] Yeah.

[00:24:32] So the use that they could and should be put to that would demonstrate the utility and value won't happen.

[00:24:40] Mm-hmm.

[00:24:41] Yeah.

[00:24:42] Interesting stuff.

[00:24:44] I did find it interesting that that Musk alluded to these humanoid robots doing things like mowing your lawn, getting your groceries, walking your dogs, watching your kids.

[00:24:56] I'm like.

[00:24:57] No.

[00:24:58] Hell no.

[00:24:59] No.

[00:25:00] No.

[00:25:01] I don't see that in my future.

[00:25:03] That's for sure.

[00:25:05] And I should also point out that if you go to youtube.com slash at tech sploder last week, I got Sam, my friend, Sam, a bull samid on for like 20 minutes.

[00:25:15] And we talked about the event and kind of picked it apart and took a look at some of the promises and a little bit of the reality and everything.

[00:25:22] Sam's really great with that stuff.

[00:25:23] So go check out that chat.

[00:25:26] And then finally, we don't have a whole lot of time, but just real quick, Adobe Max is taking place this week.

[00:25:32] Let's just say that Adobe can't stop talking about its Firefly model.

[00:25:35] Apparently, it's everywhere at the event.

[00:25:37] Firefly video model coming to Premiere in the beta this week.

[00:25:40] I got my update for Premiere.

[00:25:42] I haven't played around with any of the new generative AI features.

[00:25:46] But essentially, what it's going to allow for is just one example of what they're doing is if you've got a timeline with some video and you need just a couple of seconds.

[00:25:57] It only goes up to two seconds, but you can do a generative extend of that clip.

[00:26:03] And it will take the context of the clip leading up to it and add a couple of seconds to it when you need, which is wild.

[00:26:12] I feel like we were just talking about this maybe last week, if not last week, the week before, how these kinds of things are coming to the tools that we're using.

[00:26:19] And that's where things are going to get really interesting.

[00:26:22] Let's see what people do with them.

[00:26:23] Yeah.

[00:26:24] Mm-hmm.

[00:26:25] Yeah.

[00:26:25] And I think a real platform and a real tool becomes interesting when people do things with it that the creators couldn't have imagined.

[00:26:32] Yeah.

[00:26:33] Yeah.

[00:26:34] I mean, like, I've gotten pretty used to using some of the generative AI features that have appeared in Photoshop in the last year.

[00:26:42] You know, not for, like, major wide-scale things, but for correcting small things, it comes in really handy.

[00:26:48] Um, and so the idea of this coming to Adobe Premiere, which is the edit suite that I actually use for all the work that I do, um, I can't, I can't envision right now when I would actually have a need for that or to use it.

[00:27:02] You know, if I'm, like, reviewing a product, like, it's very specific what I need.

[00:27:05] I need the product to do this thing, and I can't just, like, fake that, nor would I want to.

[00:27:10] Um, so I guess I'm not making, like, motion pictures or things where that generative thing might be a little bit more needed or desirable.

[00:27:19] But, but it could save certain studios from having to call everybody back in for a single second of extra footage, you know?

[00:27:27] Nope.

[00:27:28] At the same time, people are really, you know, more increasingly more and more adept at identifying with their eyes when something is generative or not.

[00:27:36] So, yeah.

[00:27:37] I don't know how effective it's going to be.

[00:27:38] We shall see.

[00:27:39] People will play with it.

[00:27:40] Find a good use for it.

[00:27:41] Yeah.

[00:27:42] Yeah.

[00:27:43] But it is, it is kind of impressive, see?

[00:27:45] And they are also embedding the kind of, the labeling as such on this content.

[00:27:53] So, when it is extended, my understanding is that it will get the appropriate labeling, the credentials, content credentials.

[00:28:02] That's what I'm looking for, which Adobe is really attuned into.

[00:28:07] And, by the way, you know, as opposed to some of the other video generation models out there, this is trained on Adobe Stock.

[00:28:14] So, it's all of their internal stuff.

[00:28:16] Anything that you create with it.

[00:28:18] Yeah.

[00:28:19] Exactly.

[00:28:20] Exactly.

[00:28:21] All right.

[00:28:22] We are going to take a super quick break.

[00:28:23] And then when we come back, we're going to speak with Sarah Bird from Microsoft.

[00:28:28] She's the global lead for responsible AI engineering.

[00:28:31] Have a great conversation with Sarah coming right up.

[00:28:36] All right, everybody.

[00:28:38] Super excited to finally make this work.

[00:28:41] We've been working behind the scenes to get this person, our next guest, on the show for quite a while.

[00:28:46] We talk a lot on this show about responsible AI, about guardrails, about all the things that companies, big major companies in the space are doing to ensure that their artificial intelligence is done safely, especially for the users.

[00:29:02] And joining us is someone who can talk exactly to all of that.

[00:29:07] Sarah Bird is Microsoft's global lead for responsible AI engineering and is here with us for the next almost 30 minutes.

[00:29:14] How are you doing, Sarah?

[00:29:15] It's really nice to meet you.

[00:29:17] I'm great.

[00:29:17] I'm glad I could finally get on the show.

[00:29:20] Yeah.

[00:29:20] Yeah.

[00:29:21] Yeah.

[00:29:22] Pulling the curtain back a little bit.

[00:29:23] We tried this like a month and a half ago, and it's the only time I've ever had the issue that we had with StreamYard where there was like a network outage or I'm not sure what was going on.

[00:29:34] So it was a little bit of a bummer.

[00:29:35] But we got you back.

[00:29:36] That's all that matters.

[00:29:37] So thank you for doing this.

[00:29:38] In some cases, you have a guest on and they don't know how to operate technology.

[00:29:41] But Sarah, we knew how to operate technology.

[00:29:43] So it had to be the technology that was broken.

[00:29:45] Yes, indeed it was.

[00:29:48] Well, it's great to get you here.

[00:29:50] So let's start off with having you kind of talk a little bit about the work you and your team are doing at Microsoft.

[00:29:59] Maybe share a little bit about what you've been working on in the past few years leading up to this moment, this kind of period of explosive growth in artificial intelligence that we're seeing around us.

[00:30:11] It's a very exciting time, I have to imagine, for you and your team.

[00:30:14] Yeah, it's been, I'll say a dream come true, but just absolutely amazing how much has changed in the last couple of years.

[00:30:23] I've been working in Responsible AI basically almost since the field started.

[00:30:28] And even a few years ago when I would meet with customers, people that are going to use AI and talk about Responsible AI, they'd say it's so great that Microsoft is doing that.

[00:30:39] But we're really early in our AI journey.

[00:30:41] We're going to need a lot more time before we even think about Responsible AI.

[00:30:45] And now it's the opposite where, you know, organizations know that they want to think about Responsible AI before they even really start thinking about AI.

[00:30:53] And so it's been really great to see the public awareness grow for this.

[00:30:58] But it's also, you know, really changed in terms of the practice of Responsible AI in the last couple of years and what we need to do.

[00:31:07] And so, you know, earlier we were focused on, you know, how do we build a model, for example, that's fair?

[00:31:14] So, you know, some of the work I did a few years ago was looking at Microsoft's facial recognition system or speech-to-text system and ensuring that that system was working well for everybody.

[00:31:27] And then, you know, being transparent of how we tested it and all of that.

[00:31:30] And so it was really focused on the model itself.

[00:31:32] But when we've transitioned into generative AI and the ChatCPT era, we found that really it's not enough just to, you know, focus on the model.

[00:31:44] We need to build AI applications that are safe and responsible.

[00:31:48] And so we now develop a defense-in-depth approach that starts with the model but builds layers of defense around the model, builds robust testing systems, monitoring systems, and puts all of that into practice with robust governance approaches around that to make sure we're doing the right thing.

[00:32:05] And so the practice of what we needed to do in the last couple of years has also exploded as well.

[00:32:10] And so it's a really exciting and rewarding time to just see how far we've come even in a few years in this space.

[00:32:18] Safety, as a word in AI, has gotten mangled, I think, in many ways, right?

[00:32:24] And in the whole long-termism view, safety is about not destroying mankind, which I think is over ridiculous as a discussion.

[00:32:33] But that distracts then from the current and present tense discussions of responsibility and safety.

[00:32:39] So when you enter into a discussion with safety, I imagine people have different minds' eyes of what the word means today.

[00:32:46] So how do you tend to begin the discussion and the definition of safety and responsibility when it comes to AI?

[00:32:53] Yeah, I love this question.

[00:32:54] It's a really important one.

[00:32:56] I completely agree with your point that the focus on this kind of science fiction existential threat to humanity really distracts from what's important right now, right?

[00:33:10] People are using these AI systems today.

[00:33:12] You're using them every day on your phone.

[00:33:14] People are using them in the education context.

[00:33:16] They're using them in the work context.

[00:33:18] And so really what we need to focus on is having the systems be trustworthy.

[00:33:23] So you know that they're going to behave in a way that's appropriate today.

[00:33:28] And so as far as terminology, I think that's something that is, you know, can be a little academic.

[00:33:35] The word AI safety has recently gained a lot of popularity.

[00:33:39] And so, you know, people like that.

[00:33:41] So, you know, we use it.

[00:33:44] But honestly, you know, you have to think about kind of many dimensions.

[00:33:48] And so Microsoft's AI principles are fairness and inclusiveness, privacy and security, safety, reliability, transparency, and accountability.

[00:34:00] And we really need to think about all of those dimensions in each of it.

[00:34:03] And so it's, you know, not enough to just say we need the AI systems to be secure.

[00:34:08] We need them to be safe.

[00:34:09] We have to look at kind of all of those.

[00:34:11] And all of those are going to be required to get to an AI system that we can trust.

[00:34:15] So I'm part of an AI governance something or other at the World Economic Forum.

[00:34:20] And I went to a meeting of this where I was way out of my pay grade with a lot of the folks like you who really know what you're talking about.

[00:34:27] But I learned a lot.

[00:34:29] And one thing that I keep coming back to is kind of a matrix of responsibility.

[00:34:33] What level should be responsible?

[00:34:35] And is it the model layer where you have efforts to say that the model maker must be responsible for what's done with it and build guardrails in?

[00:34:43] And that always strikes me as somewhat logically futile in the sense that if it's a general machine, it can be made to do anything.

[00:34:51] And you can't predict everything that anyone would ever have it do.

[00:34:54] And so it's maybe false security to think that you could build it at that level.

[00:34:57] Or it's the application level.

[00:35:00] Or it's the user level, the person who asks it to do something bad.

[00:35:04] Or the lawyer who used ChatGPT to get his case citations.

[00:35:08] I covered his hearing, by the way, which was hilarious.

[00:35:11] And I make this analogous to the early days of another technology, print.

[00:35:15] At first it was the printer, the technology that was held responsible.

[00:35:18] What came off the press, they got beheaded and behanded and burned at the stake.

[00:35:22] Then it was the middle layer of the bookseller or the publisher.

[00:35:28] And then finally it became the author was responsible.

[00:35:30] And Foucault would say that's when the author was created.

[00:35:33] So in responsible AI, the first reflex seems to be to go to the technology layer and say,

[00:35:40] you must be responsible for everything that anyone is ever going to do with this.

[00:35:44] But I see that as very difficult to do.

[00:35:46] And Microsoft is kind of up at that layer.

[00:35:48] But you're also at the application layer.

[00:35:50] And you're also worried about what users do with it.

[00:35:52] So how do you realistically apportion responsibility across the ecosystem of AI use?

[00:36:03] Yeah, I think that is actually very much one of the key questions of today.

[00:36:08] And something that we're seeing start to play out in regulations as well.

[00:36:13] Of course, in answer to your question, anytime you ask a question like that,

[00:36:16] the answer has to be, well, all of them, right?

[00:36:19] But the important thing is that each layer needs to do different things.

[00:36:24] And you're right that at Microsoft, we have all of those layers.

[00:36:29] And so a lot of maturing in our responsible AI practice has actually been understanding

[00:36:34] that we need to do different things at different layers.

[00:36:37] We have different responsibilities and accountabilities at each of those.

[00:36:41] And so, in fact, when we went to DC last year and kind of put out a template for what we thought

[00:36:49] might be some ideas for regulation, one of them was exactly that you need to have different

[00:36:54] types of regulations for each layer.

[00:36:57] But actually starting even one layer lower than you mentioned, which is the supercomputers

[00:37:03] that allow you to train these systems.

[00:37:06] Because, in fact, that is the most flexible thing.

[00:37:09] Then you can really build anything, right?

[00:37:12] And so we need to know where all that compute power is going in the world and what it's being

[00:37:18] put to work for.

[00:37:20] And so that's the layer we started at.

[00:37:23] Of course, now we spend a huge amount of our time at that model and platform layer,

[00:37:29] seeing, OK, if you're going to be offering models to people, what do you need to do?

[00:37:34] And of course, you know, the state of the art is moving really quickly here in terms of

[00:37:37] having safety built into the model, which is fantastic because it absolutely allows it

[00:37:43] in both cases to kind of get right the first time and adds an extra layer of defense.

[00:37:48] But just your point, there's many, many things we want to use these models for.

[00:37:53] And so at the lower layers, it's really important, actually, that we have the defenses, the safety

[00:38:00] systems, be flexible and be customizable.

[00:38:02] Because, for example, one of the things I like to do with foundation models or maybe

[00:38:07] don't like but need to do is actually generate all different types of harmful content that

[00:38:12] we use to distill into smaller safety models.

[00:38:16] And so that's maybe an undesirable behavior most of the time for a foundation model, but

[00:38:22] it's actually quite desirable for defense.

[00:38:25] And we see, you know, similar to like law enforcement use cases that are, you know, going to need

[00:38:29] to do things like that.

[00:38:31] And we don't want such, you know, critical use cases as like AI safety or, you know, law enforcement,

[00:38:39] cybersecurity.

[00:38:39] We don't want them to be at a disadvantage in being able to use, you know, less powerful

[00:38:43] AI than everybody else.

[00:38:45] Right.

[00:38:45] And so we absolutely at the lower layers look at how we make the system flexible.

[00:38:52] And we can make it easy for application developers to build on top and build an application that,

[00:38:59] you know, meets our principles and actually works appropriately in the context.

[00:39:04] But we don't know the context.

[00:39:05] So we shouldn't be baking that in at the bottom that it's just a one size fits all.

[00:39:09] And so the application developer absolutely has the, you know, responsibility to be setting

[00:39:15] those knobs and adding additional things to make it make sense for their application.

[00:39:19] For example, if I am, you know, one of the favorite customers that we work with, the New

[00:39:25] York City Department of Education, then, you know, I'm working with students, I'm putting

[00:39:31] a chat bot into schools.

[00:39:32] I need to be very, very cautious about what that system does, where it makes mistakes, how,

[00:39:37] you know, how much sensitive content it can produce.

[00:39:40] But if I'm a gaming application, I'm going to be much more comfortable with violent content,

[00:39:44] right?

[00:39:44] And so you don't want just a one size fits all.

[00:39:47] This is the right thing for every application, every industry, every country.

[00:39:51] And so, you know, the lower layers, we look at that.

[00:39:54] But then the upper layer, they do have the responsibility to say, okay, I know I'm an education application.

[00:39:58] I know I'm going to students.

[00:39:59] I need to set this appropriately.

[00:40:02] And then, as you said, users have a responsibility too, right?

[00:40:06] Using this without like reviewing the responses and just, you know, accepting that as correct

[00:40:11] and let's say submitting it as homework or something.

[00:40:14] You know, that's something that only the user can really make a decision about.

[00:40:18] And so they need to understand what AI can do and what AI can't do.

[00:40:21] And, you know, when it's appropriate to use and when it's not appropriate to use.

[00:40:26] Yeah.

[00:40:26] And the appropriate way of use.

[00:40:28] And I think that education is definitely an ongoing thing.

[00:40:32] It's also still very kind of a new kind of concept for a lot of people who are just beginning to consider the use cases of these systems and everything.

[00:40:41] So that's a skill that they're going to have to learn.

[00:40:43] Another customer we work with that I really love what they've done is also in education is the South Australia Department for Education.

[00:40:52] And they have a beautiful video on YouTube where what they've actually integrated in their classroom.

[00:40:57] It is a chat system called EdChat.

[00:41:00] But similarly, they have now built a stoplight system in all of their assignments to tell students,

[00:41:08] hey, this one is red.

[00:41:09] You can't use AI at all.

[00:41:10] This one's green.

[00:41:11] You can use it for whatever.

[00:41:12] This one's yellow.

[00:41:13] You can use it in these ways and not that way.

[00:41:15] So just as part of their education, they're having them get a feel for when it might be appropriate to use AI and for what reasons.

[00:41:22] Because I thought it was very much part of training people with the tools of the future.

[00:41:26] And so it's really great to see an education institution leaning in and experimenting with how are we going to teach people to think about this in their everyday work.

[00:41:35] I love that.

[00:42:05] And school matters.

[00:42:07] But you got your PhD at Berkeley, I think, right?

[00:42:09] I did, yes.

[00:42:10] So I'm curious, if you go up the education ladder to the highest one, what changes in what computer science people and engineers should be trained in today?

[00:42:22] How do you instill the kind of ethical decision making that you're working with now in the people who you're going to hire in five or ten years?

[00:42:30] Yeah.

[00:43:04] And I think that's a great idea.

[00:43:09] I think that's a great idea.

[00:43:15] I think that's a great idea.

[00:43:29] What's the relationship like in terms of how you get baked into the products and into the technology development, the work that you're doing?

[00:43:38] Yeah.

[00:43:39] You know, several years ago, I think it was challenging.

[00:43:43] People kind of thought of this work as just a trolley problem.

[00:43:49] It was just like an interesting problem you needed to think through, but maybe not the most practical every day.

[00:43:55] And this is where, you know, this is really what my work is and where I spend my time is taking, you know, what we want to achieve from a policy point of view and then taking what is possible from, you know, a research point of view and putting those together into something we can actually achieve in practice.

[00:44:15] And so, you know, we spend our time understanding what a new risk looks like, figuring how we address that risk.

[00:44:23] So, for example, you know, two new ones that really started with the chat GPT error are prompt injection attacks and jailbreaks and then, you know, hallucinations.

[00:44:32] Like those are two types of risks that we just were not really talking about very much, you know, before this most recent wave of AI.

[00:44:40] Now we talk about them a lot and we spend our time on them a lot.

[00:44:43] And so we had to sit down and say, you know, what does that risk actually look like?

[00:44:47] What does it mean?

[00:44:48] We work a lot with, you know, researchers in Microsoft Research and externally and partners like OpenAI to figure out what does this really mean?

[00:44:56] We then actually the most important next step is to start building testing for that risk because we don't really know if it's materializing in practice if we can't test for it.

[00:45:05] And so we build testing systems.

[00:45:07] And then we go and start experimenting with mitigations and have a tight loop between try this new technique and see if the measurements, you know, actually improve.

[00:45:15] And so we iterate till we kind of find solutions.

[00:45:19] But it really wouldn't work if we stopped there.

[00:45:21] We could publish a paper and say this is how we address this risk.

[00:45:23] But at that point, it's just still too hard to do in practice.

[00:45:27] And so that's where we go and build out production systems, tools, technologies as much as possible.

[00:45:33] We just integrate them in our AI platform so that people can just adopt those right away.

[00:45:38] And then the last step is that we need to make sure people actually are using them and meeting Microsoft standards or enabling our customers to use them.

[00:45:47] And so that's where we build compliance tools and processes and training and education so that people are actually adopting the newest things and they know what the best practices are.

[00:45:57] And so we go through that end to end cycle for every single risk.

[00:46:01] And that is sort of how we at scale are teaching everybody in the company how to be doing this as part of their job, but also giving them the tools and the resources so that they can absolutely be successful doing it.

[00:46:15] Well, this seems like a perfect time to bring up that just last month, Microsoft announced a bunch like a host of AI safety features as part of an initiative called Trustworthy AI.

[00:46:27] Maybe talk a little bit about some of the core concepts within that initiative.

[00:46:32] I'm sure they touch directly on exactly what you're talking about right now.

[00:46:35] Yeah, so what we have found, I mentioned there's just so much to do.

[00:46:43] You know, we look across a wide variety of risk, everything ranging from fairness and bias in the system to, you know, production of harmful content and code, production of copyright materials, hallucination, new types of adversarial attacks, and the list goes on.

[00:46:59] And we also then look across modalities.

[00:47:03] We want this to work for audio and video and text and image and all the different, you know, combinations thereof.

[00:47:10] We also want it to work across, you know, different domains.

[00:47:13] We want to work in coding.

[00:47:15] We want to work in education, as I mentioned.

[00:47:17] And so we're regularly kind of releasing the next wave of capabilities as we find new techniques that the work can practice.

[00:47:29] So some of the big themes of the recent one, the Trustworthy AI, are one of the features that we released that I think is really exciting is what we call groundedness correction.

[00:47:41] And what this is, is it's identifying when the model hallucinates, meaning specifically that it produces a result that is not in line with the data that it was given, you know, as part of the question.

[00:47:55] And if there's a mismatch between that, so it's changed what's in the data or it's added its own additional information, we can detect that in real time now and then actually rewrite it so that we can correct that answer and make sure we're getting, you know, an accurate answer according to the input data to the user the first time.

[00:48:12] And so this is an exciting new capability that, you know, we think can really change, you know, how many errors you're seeing in practice.

[00:48:20] But other themes that are also really important is, and one of the reasons we started kind of using the term Trustworthy AI, we were talking about terminology earlier, is that a huge amount of the practice is also about existing privacy and security practices and making those work for AI.

[00:48:36] And so, you know, you know, security communities, maybe they don't think about AI safety, but they think about security every day.

[00:48:43] And so if we take our safety system, which can detect a one of these prompt injection attacks or jailbreaks in real time, the system will actually just block those.

[00:48:55] But we, you know, adversaries, if they're determined enough, can keep probing and maybe eventually they'll find a, you know, a gap.

[00:49:02] And so we now connected that with Microsoft Defender for Cloud, which is our product for our security ops teams.

[00:49:11] And now they'll get an alert saying someone is attempting, you know, an adversarial attack on your AI system and they can actually go and investigate.

[00:49:19] Or we've connected in with Microsoft Perview, which is our data governance system.

[00:49:23] So you can say, oh, actually, I don't want the top security data flowing into this particular AI system because, you know, other people have access to it or something.

[00:49:32] And so we're really saying that, you know, AI is a feature of many applications and you want to govern it and monitor it the same way you monitor applications today.

[00:49:40] And so we've been extending those offerings to make them work for, you know, all for AI and make sure that you can do this end to end from, you know, development through to monitoring.

[00:49:53] And so that's some of the new pieces.

[00:49:55] But you'll keep seeing more and more releases from us because it's still early days.

[00:50:00] There's just so much to do here.

[00:50:02] And so anytime we find a new capability, we're trying to get out there in people's hands as quickly as possible because that just, you know, raises the bar in terms of the expectations and the best practices for, you know, responsible AI.

[00:50:14] I'm so heartened to hear you talk this way that I hear some companies talk about safety and responsibilities.

[00:50:21] It's kind of an add-on.

[00:50:22] It's about risk management as opposed to design.

[00:50:26] I'm curious what your view is and or the company's view about open source and AI because I hear this discussion all the time where there are some who I would say are the more radical end of safety are against open source for the fear that any guardrails that are built can then be gotten around.

[00:50:45] However, open source is also going to make it possible for poorer entities like universities to work with it and do more development.

[00:50:52] How do you see open source fitting into the structure that you're building about responsibility in the larger ecosystem past Microsoft?

[00:51:01] Yeah, we think open source is a really important predestory at Microsoft.

[00:51:06] We have open source models.

[00:51:08] We release open source models.

[00:51:10] We also obviously have closed source models.

[00:51:12] And, you know, with each new model that we're developing, we think very deeply about the benefits of open sourcing it, as you said, enabling research to build on top of that, enabling more customization, you know, things like that with also the risk.

[00:51:29] Right. And so, you know, we are very thoughtful about what it would mean if we release a sort of net new capability into the open source that changes sort of what is even possible.

[00:51:45] And so, you know, a lot of the models today, you know, they have different risk and practice.

[00:51:51] But, you know, that might be much less than what's possible in the future.

[00:51:57] And so, you know, we're watching for things like, will future models be able to generate novel malware?

[00:52:03] Will they have, you know, risk in like chemical and biological spaces and things?

[00:52:08] And I think if we, you know, end up developing a model of that level of power, then it's a serious question we have to ask ourselves about.

[00:52:16] Is it appropriate to put that out there with sort of effectively no governance and, you know, the ability for people to do just anything with it?

[00:52:24] And so, you know, as part of our responsible AI standard, we do an impact assessment where we look at the, you know, risk and rewards of the technology.

[00:52:32] And then, of course, we test it and, you know, actually see that, you know, happening in practice.

[00:52:36] And we use that to inform our decisions about, you know, what is the appropriate way to release any given technology and, you know, how flexibly should it be released?

[00:52:45] And, you know, it's something that we're going to keep experimenting with.

[00:52:49] We're going to have external conversations.

[00:52:51] But it's going to be really interesting to see where the world goes with this.

[00:52:55] Last question for me is what drew you to this part of the field personally?

[00:53:01] It's actually the what you said earlier, which is that this is about product design or quality.

[00:53:09] So I've always made my career in taking emerging new technologies and making them work in practice.

[00:53:15] I find that in an innovation journey, one of the most exciting things.

[00:53:21] And responsibly, I is just the ultimate making it work in practice, making it work every time in every way in every dimension.

[00:53:28] And so it's just an incredibly fun and rewarding space.

[00:53:32] But to add a little bit to that, I am very much a technologist.

[00:53:37] I believe that AI can, you know, change the world in amazing ways.

[00:53:42] I mentioned the education example earlier.

[00:53:45] New York City was mentioning that in early pilots, they were seeing that students who maybe didn't speak English at home

[00:53:52] and were not comfortable asking questions in class were asking a lot of questions at the chat system.

[00:53:59] So they were still getting their questions answered in a personalized way.

[00:54:03] And so they were getting support just in, you know, a different way.

[00:54:06] And I absolutely see that, you know, that's what AI can offer.

[00:54:09] It can be an incredibly empowering technology that allows people to do more powerful things with, you know,

[00:54:16] with computers to create things they could never create before.

[00:54:19] And so I really want to make that a reality, you know, in everybody's lives.

[00:54:24] But we can't do it if we don't trust AI.

[00:54:28] And so the problems that, you know, I'm trying to solve and others like me,

[00:54:32] they're essential for us to get the benefit of AI in the world.

[00:54:35] Yeah.

[00:54:36] Thank you.

[00:54:37] Excellent.

[00:54:38] Yeah.

[00:54:38] Thank you so much, Sarah.

[00:54:39] It's been a pleasure having you on the show today.

[00:54:41] Sarah Bird is, of course, Microsoft's global lead for responsible AI engineering.

[00:54:47] Really happy we could finally make this work, Sarah.

[00:54:49] Thank you for talking about what you're up to.

[00:54:52] Thanks for the very sophisticated questions.

[00:54:54] Thank you so much.

[00:54:56] Yeah, we appreciate your time.

[00:54:58] And yeah, we'll have to have you back sometime in the future as things progress.

[00:55:02] We'll be keeping an eye on things and we'll reach out again.

[00:55:05] We'd love to have you back.

[00:55:08] Thank you again, Sarah.

[00:55:09] All right.

[00:55:10] All right.

[00:55:10] Huge thank you to Sarah once again.

[00:55:12] That was excellent.

[00:55:13] That was great.

[00:55:14] And as I say, I really am heartened.

[00:55:16] I think that that's a sensible way to look at it and they're doing it.

[00:55:20] I also love her love for the educational uses.

[00:55:25] Yes.

[00:55:25] New York City public schools at the very first of ChatGPT forbade it.

[00:55:31] And obviously it's with the help of people like Sarah that got them to come back to a more sensible view

[00:55:36] to say this could be useful for students in important ways and skills they need to learn for the future

[00:55:41] and got it back into the classroom.

[00:55:43] Indeed.

[00:55:44] So I think it's because companies like Microsoft invest in that kind of effort

[00:55:48] that we can have a good and proper use of the technology.

[00:55:52] What did you think?

[00:55:52] Indeed.

[00:55:53] Totally agree.

[00:55:54] I love it.

[00:55:54] No, I think that's kind of the highlight for me too.

[00:55:57] I mean, I mentioned, you know, I don't know how many months ago about my experience being in my daughter's classroom

[00:56:03] and one of the kids raising his hand about the art assignment.

[00:56:07] Can I use AI for this?

[00:56:08] And really that really instilled in me the awareness that these kind of skills, they need to be there.

[00:56:17] They need to be expressed and taught at an early age apparently, like fifth grade.

[00:56:21] That was just fifth grade because otherwise there will be like incorrect and unethical uses of these things

[00:56:29] if they aren't taught how to use them responsibly early on.

[00:56:34] And so I think that was a real thought.

[00:56:35] Yeah, it goes all the way up to the corporation.

[00:56:37] And what I heard from Sarah's attitude here, you would think that with that kind of job title,

[00:56:44] people would be perceived as the responsibility cops.

[00:56:49] They'd be like, these are the things you can't do.

[00:56:51] But what she said at the very end I think was very illuminating is, you know, why did she do this job?

[00:56:56] It's because she's making it work.

[00:56:59] Right.

[00:56:59] And I think to make it work well and make it work safely is so critical in these companies.

[00:57:04] And to do it at the realistic, responsible, present tense level, not the BS, you know, as she said, science fiction level,

[00:57:12] I think it's great to see.

[00:57:15] Yes, indeed.

[00:57:16] Indeed. Well, a huge thank you to Sarah for taking time with us today.

[00:57:20] Thank you, Jeff, and your new website, jeffjarvis.com, with your new book, The Web We Weave.

[00:57:26] Thank you for the plug, boss.

[00:57:28] On sale now.

[00:57:31] Everybody should go there.

[00:57:33] And you don't even have to, like, consider, like, what is the URL.

[00:57:36] You know Jeff Jarvis's name.

[00:57:38] Just plug that into the browser and you'll go right there and you'll be able to buy it.

[00:57:41] And there's lots of AI in the book.

[00:57:43] Not done with AI, but about AI.

[00:57:45] Right, right.

[00:57:46] Web20 at checkout, 20% off.

[00:57:49] At least as it's still showing anyway.

[00:57:51] I think so.

[00:57:52] We'll see.

[00:57:52] Yeah.

[00:57:53] Well, you'll find out soon enough.

[00:57:56] As for this show, just go to AIinside.show.

[00:58:00] Everything you need to know about what we do can be found there, including all the ways to subscribe to all the different podcatchers of choice.

[00:58:07] If you want to jump out to the Patreon, you can do that.

[00:58:09] If you want to catch us live, there's some information as far as how you can do that both on YouTube and Twitch.

[00:58:15] See, it's like everything related to the show is all here, including some reviews that make us feel really great about what we're doing.

[00:58:22] So anyways, go there.

[00:58:24] AIinside.show is the place to subscribe to the show.

[00:58:28] I think that's the most important thing that you can do.

[00:58:30] And then finally, patreon.com slash AIinsideshow.

[00:58:34] If you want to support us on a deeper level, you can do that.

[00:58:37] You get some extra perks.

[00:58:40] Many of them, of course, add free versions of the show, Discord, regular hangouts.

[00:58:44] We've got a hangout coming up next week.

[00:58:46] If you contribute at the executive producer level, you get an AIinside t-shirt.

[00:58:53] And we have five executive producers, Dr. Du, WPVM 103.7 in Asheville, North Carolina, Paul Lang, Ryan Newell.

[00:59:02] Let's see you get called out at the end of the show.

[00:59:03] And I got to put out a special thank you to Jeffrey Maricini, who is just incredibly generous.

[00:59:09] You upped your pledge and you're an awesome dude.

[00:59:13] You obviously care about what we're doing here at the show.

[00:59:16] Thank you so much, Jeffrey.

[00:59:17] It just makes me feel so good to have you all on board.

[00:59:21] So that's patreon.com slash AIinsideshow.

[00:59:25] And I think that about wraps it up.

[00:59:27] We've reached the end of this episode of AI Inside.

[00:59:30] Make sure and check back next week.

[00:59:32] We will have another episode both live as well as published to the feeds that you're subscribed to, right?

[00:59:38] All right, everybody.

[00:59:39] Thank you so much for being here.

[00:59:41] We'll see you next time on another episode of AI Inside.

[00:59:44] Bye.

[01:00:08] Bye.

[01:00:09] Bye.