The loose, undefined guardrails of X’s AI image generator
The social media platform X recently launched a new artificial intelligence feature for premium users: Grok-2, an AI model that can also generate images. And the outputs are a bit less censored than you might see with other similar tools.
Experimenters online have been able to generate images of Donald Trump and Kamala Harris brandishing guns, Mickey Mouse smoking a cigarette and some far more disturbing tableaus.
Grok claims to avoid images that are pornographic, excessively violent or intended to deceive, and says it’s cautious about representing content that might infringe on existing copyright. But the guardrails certainly seem to be on the looser side, in keeping with owner Elon Musk’s hands-off approach to content moderation.
Marketplace’s Meghan McCarty Carino spoke with Adi Robertson, senior tech and policy editor at the Verge about Grok and what she found while she tested the AI’s limits. The following is an edited transcript of their conversation.
Adi Robertson: Image generators tend to have a lot of hard limits of things that, if you put in a text prompt, they just will refuse to generate something. So this can be the sort of hard limits you might imagine. They tend to not generate nudity. A lot of generators also will block things that are either culturally insensitive, like, say, Nazi insignia, or they’ll block pictures of real people because that could be misinformation or deepfakes. And they also can block things that they think are just otherwise “iffy” for causing misinformation or being inflammatory. Grok tends to have, as far as I can tell, it has limitations on nudity. People have found some other limitations on things like extreme violence, but it’s really just not following most of those rules.
Meghan McCarty Carino: How have you tested its limits?
Robertson: It’s a little bit of an unpleasant process. You just think of things that could be harmful if you made them and you type them into the text prompt. That’s the unfun part of it. The fun part is just trying to think of things that are silly, that could offend companies. So think of Mario or Mickey Mouse smoking a cigarette or doing something that the copyright holders won’t like.
McCarty Carino: Yeah, I’ve seen a lot of copyrightable characters doing unsavory things involving like weaponry, you know, all the vices, all the major vices seen represented with characters that probably are copyrighted.
Robertson: Yeah, and I think that’s actually maybe a little bit of weird folk copyright, because in a lot of cases, if you drew a picture of, say, Mickey Mouse smoking a cigarette or doing something, it would be a protected parody, but the copyright law around AI is really unsettled, and so this is a thing people think is really funny, is to generate examples of this AI clearly training on copyrighted information.
McCarty Carino: So how does this compare to some of the other major image generators that people might be familiar with — MidJourney, DALL-E?
Robertson: MidJourney and DALL-E, the systems from, say, big companies like Google, they will tend to have guards on the sort of front end and back end of, there will be things that you can’t type in, and there will be things that it will maybe generate, and then it will decide, no, that’s not good. There are actually AI image generators that can make pretty much anything you want. There are systems that you can just run on your own computer. There’s an open source tool called Stable Diffusion. So X isn’t totally unique here. It’s just willing to sort of push at the edges of things that a lot of major companies don’t find acceptable.
McCarty Carino: And what seems unique about Grok and X is that the barrier to entry is pretty low, and it’s kind of hooked in automatically to a platform where information and news is disseminated.
Robertson: In some ways, the barriers to Grok are actually higher than for something like ChatGPT, because it’s only available to people with a premium subscription, so not just anybody can use this. And so I think that, yeah, as you said, even if it’s not necessarily available to everyone, it does publish to this service that for a long time was considered one of the de facto sources for news for people. It’s a really odd pipeline. At the same time, I think a lot of people don’t necessarily think of X that way anymore. The information ecosystem is fractured a bit, so I think that it’s not quite the impact that you would see if you had tried this a few years ago.
McCarty Carino: What does X say about Grox’s guardrails or lack thereof?
Robertson: An important caveat here is that X and X AI didn’t actually develop this system. They are licensing it from another company called Black Forest Labs, and so what they’ve said is this is a stopgap, that they’ve been working on something of their own, but that it’s not going to be ready for a few months, and right now, this is a way to, I believe, quote, have some fun. So I think that there’s not much indication that they’re going to change anything, but there is an indication that this might not be the final system we see on X.
McCarty Carino: Does anything concern you about this system? Or does it seem like just some fun?
Robertson: I think the thing that concerns me most is that in AI, some of the clearest real world harm we found is sexualized images of real people, almost exclusively women, that are designed not necessarily just to misinform people, but to really specifically harass and humiliate people. And I think that while there are concerns that are a little more abstract about – will people believe Donald Trump actually did this? I think that these are the cases where we have already seen real world harm. We’ve already seen legal cases around them, and for X to make that easier is a really ugly thing to do.
McCarty Carino: Is there kind of a way in which just having more of this stuff kind of inoculates people? I mean, not the specific things that you’re talking about, but say, you know, public figures or copyrighted characters in positions that are unflattering, like just the more of it that is out there, the kind of less our brain zone in on it?
Robertson: Yeah, I think that there have been, certainly periods before where we’ve had to grasp that the thing we’re seeing might not be real. Obviously, Photoshop is a very old technology, and I think that one thing this could do is make people care a bit more about the context that they’re seeing something in. If the New York Times, which is a paper that has a reputation to preserve, publishes an image and they cite the name of the photographer that’s a lot more credible than you seeing a random Twitter user posting something. So obviously, photographs have never been a really, completely hard ground truth, and I think this could encourage people to look more at the things around them.
While Grok has introduced image-generation to X, AI deepfakes have already been a problem on the platform.
Earlier this year, nonconsensual, sexually explicit images of Taylor Switft spread on X as well as other social media. Swifties mounted a defense by flooding X with real images of Swift and related posts in an attempt to bury the deepfakes. X later ended up temporarily blocking some specific search entries for Taylor Swift.
But it hasn’t stopped AI-generated images of so-called Swifties for Trump, which were recently reposted by former President Donald Trump.
Stories You Might Like
The future of this podcast starts with you.
Every day, the “Marketplace Tech” team demystifies the digital economy with stories that explore more than just Big Tech. We’re committed to covering topics that matter to you and the world around us, diving deep into how technology intersects with climate change, inequity, and disinformation.
As part of a nonprofit newsroom, we’re counting on listeners like you to keep this public service paywall-free and available to all.
Support “Marketplace Tech” in any amount today and become a partner in our mission.
