What does it mean to develop trustworthy AI?

The artificial intelligence wars are in full swing, with companies like Microsoft and Google battling it out.

Now Mozilla, the developer of the Firefox browser, is entering the fray. Last week, the company announced a new startup focused on developing what it calls “trustworthy” and independent AI, built on open-source software that’s free to the public.

As with its other products, which are more focused on transparency and privacy, Mozilla aims to distinguish itself in a crowded AI field as a brand committed to the public interest.

Marketplace’s Meghan McCarty Carino spoke to Imo Udom, senior vice president of innovation ecosystems at Mozilla, to learn more. The following is an edited transcript of their conversation.

Imo Udom: When we think about what trustworthy AI means, we want to shift the arc away from purely using this technology to profit organizations and towards more responsible and ethical best practices, like transparency, like accountability coming into the fabric of the tech. So when we think about ChatGPT or great systems like this, one of the terms we hear often coming out of society now is “hallucination,” when it comes to these large language models. When it’s applied to this space, really we’re saying that some of these tools make things up. If you’re an individual using ChatGPT for fun, this can be humorous; to some people it might be annoying. But on the other hand, as a journalist who might be using generative AI aids to research or write a story, or if a doctor were using generative, AI-powered, decision-making tools, you need to be able to validate that the sources behind these answers aren’t hallucinations or aren’t false. Better yet, wouldn’t it be great if you knew that these tools have been developed to make sure they’re only using specific data and information that really is built for that domain or area?

Meghan McCarty Carino: And so how does establishing open-source AI systems help address some of those issues?

Udom: I think some of what we want to see, and this is also part of the principles that we’re looking for is, one, choice. So when things are often done in the open, it enables more people who are taking a different lens the ability to influence and shape where this is going. At the same time, we want accountability. And if history is anything to look back on, we know that organizations often won’t hold themselves accountable. So working in the open does create that possibility for that researcher, who might have different incentives, to be able to look at the work done by an organization and test it to make sure it’s less biased or it’s more fair. And one of the things that we’ve already started to see is people applying some of the work being done by other organizations into more exciting use cases. I saw an experiment online the other day. So this person took one of the language models, combined it with an image-to-text model and computer vision so that they could use their phone to walk around and have the technology tell them what they’re seeing. And you can see that really helping people who are visually impaired or changing some of the ways that we use technology in society.

McCarty Carino: And I’m struck that when we’re thinking about these artificial intelligence tools and large language models that are built on data largely from the internet, that that ends up encoding in the outputs the disparities that exist in access to the internet.

Udom: Absolutely. And there are a number of initiatives around data quality and data access that we need to continue to improve on. There are actually teams out there who are working to ensure that as we continue to train and evolve models, we’re actually getting more humans involved in that process. Because if we purely take the web, we all know that not everything on the web is true. And we all know that the web broadly has biases. And so it’s great to have some of the research community looking into how to remove some of these sources from the core datasets, but also how to introduce premium, in some cases, private sources with the right alignment with those organizations, such that we have the right information training these models and improving the veracity or the the truth of what’s being put out there.

McCarty Carino: Just to be really clear, when we look at some of the consumer products that are already in the hands of consumers or making their way shortly into the hands of consumers, what are some of the danger zones when it comes to AI in its current forms?

Udom: Today, some of what we see around concerns have to do with a couple of areas. So one is misuse. If we have these powerful technologies out there, are people going to use them to create malware? And there have been stories out there, heartbreaking stories, of people using some of this technology to imitate distress from a loved one, right, because some of this technology can mimic your voice. And so we want to stay away from that, right? It’s always a battle between nefarious characters who are trying to scam people and those of us who are trying to make sure we enable and protect society. [Second is] understanding the limitations, right? Knowing that everything that comes out of, again, fun solutions like ChatGPT, or Google Bard, or Bing’s technology as well, the answers aren’t always correct. And then, as I’ve said before, some of what we want to be careful of are these tools are trained on texts and images of people who haven’t explicitly given consent. And we’re seeing that come up more and more. But what’s actually interesting, we’re seeing new tools or agencies who are trying to allow people to opt out of having their data being pulled into these types of places.

McCarty Carino: So much of the energy in the market right now is around chatbot-powered search engines. What does integrating trustworthy AI into a browser look like to you?

Udom: You know, when we think about that, we take it back to our core principles of what trustworthiness means. So that’s about agency, accountability, privacy and fairness. Wouldn’t it be amazing if when you’re using these types of technologies, whether it’s in a browser environment or in another product, you know that you have the ability to control how the algorithm works for you? If you want the thing to be fun, and laughable and take you to interesting areas, great! Tell it that. If you want to be safe and secure, you want to make sure your child doesn’t, by mistake, get something exposed to them, these algorithms should be able to support that work. And then privacy is one of the key things as well. It’s not an easy task. But I think we want to make sure that the information that are being used in these technologies and algorithms are being pulled from sources or places where people have said they’re willing to share their information. I think the fun thing that we’ve also been talking to people about, if we can put some of these trustworthy controls into the technology, you can see a world where everyone has their own personal assistant that helps them navigate the web, or that pulls the right type of information for them on behalf of them. And I think that’s what the future can hold if we’re able to instill the right characteristics into the work up front.

McCarty Carino: Mozilla does have its own browser, Firefox. Are there plans for AI integration there?

Udom: We’re marrying product experiments with the work that our foundation arm has done to really pull in trustworthy principles and the work that Mozilla AI is doing to build building blocks that will enable companies like us to productize this technology in a trustworthy way. So, I mean, of course, we’re looking at it, we’re playing around, but who knows what the future will hold as far as it coming out into our products?

McCarty Carino: I do have to note that the the market share of Firefox browser has dropped pretty dramatically in recent years. Given this experience in the browser space, I mean, are you worried that this kind of fast-moving AI competition could crowd out your version of trustworthy AI?

Udom: One of the things that’s great about our approach is that we’re really activating and catalyzing the community holistically. So it’s not just about Mozilla and our products, but it’s about all the products that we’re able to generate from there. So we’re doing things like our responsible AI challenge, which is bringing builders alongside us who want to partner with us to shape future products. We have a number of new offerings that are in the works, and we’re excited to share more of that to the world in the coming months and years, moving forward. I think it’s just really important that we’re able to bring to light organizations that are building products in the right way that can still solve great user problems. And so that’s really our focus — taking the technology, experimenting, both delivering value to the users through the work that we’re doing in Mozilla AI and exploring ways to incorporate these into our products where it makes sense.

Related links: More insight from Meghan McCarty Carino

I feel like we have to address the elephant in the room when we’re talking about open-source AI, because the company OpenAI, which developed ChatGPT and the models that power Microsoft’s suite of AI tools, started out — as its name suggests — as an open-source nonprofit research company.

It later moved to a “capped for profit” model so it could accept big investments like the roughly $10 billion it got from Microsoft, and it has since become decidedly less open.

When it released its latest and greatest GPT-4 language model earlier this month, the company was widely criticized for not disclosing what data the system was trained on or really any of the software’s technical details.

Co-founder Ilya Sutskever told The Verge the decision was largely informed by fear of competition but also some concerns about safety.

He said as these models become more powerful, the risk of someone using them to intentionally do harm increases. He said of the company’s previous open approach: “We were wrong. Flat out, we were wrong.”