What should come next in the effort to crack down on online extremism?

The 18-year-old man accused of the racist rampage at a supermarket in Buffalo, New York, Saturday followed a pattern becoming disturbingly familiar for such attacks: online radicalization.

The suspect allegedly wrote and posted a 180-page document before the mass shooting, citing various racist and anti-Semitic memes and conspiracy theories from websites such as 4chan.

Federal officials are paying attention to this growing threat. President Joe Biden’s latest budget allocates $33 million for the FBI to investigate domestic terrorism.

Jared Holt, a resident fellow at the Atlantic Council’s Digital Forensic Research Lab, spoke with Marketplace’s Kimberly Adams about how the Justice Department has shifted efforts to address online extremism in the United States. He noted that it still has progress to make.

The following is an edited transcript of their conversation.

Jared Holt: A lot of those efforts are new, and I think what happened last weekend kind of speaks to that and speaks to the need to sort of sustain those efforts and refine them.

Kimberly Adams: Do we have any sense at this point of how extremist content or ideas possibly influenced the person responsible for the shooting in Buffalo?

Holt: Yeah, if we look at the document of the person who is believed to be responsible for this, the document left behind, kind of details out this person’s trajectory. I got the sense that they were kind of a lonely, isolated person and then found these communities online where extremism spreads. And those communities, in addition to instilling this very hateful and disgusting ideology that ultimately motivated this violence, also provided the individual a sense of community, like so many other people seek out online in various forms. So on one hand, there is the sort of meat and potatoes of it, the ideology. But then like anything else on the internet, there’s also a kind of parasocial and community aspect to extremist communities that can keep people in where, you know, they’re encouraging each other, inciting each other. And periodically, something tragic like this happens as a result.

Adams: How does online radicalization differ from in-person radicalization?

Holt: The main difference between online radicalization and in-person radicalization is speed. So the internet did not invent domestic extremism in the U.S., it did not invent racism or racial violence. But it is a very, very convenient vehicle for movements that advocate for those kind of things to operate with. You know, back in the day, if you wanted to join a militia, you would happen to come across somebody in your life that had a connection, or maybe you would mail a letter somewhere. But now, if you want to join a militia, you have a halfway decent idea of the search terms to put in. Within five minutes, you could get connected with somebody, not just generally, but maybe even locally. We should think of the internet as a vehicle that, along with all of the benefits it’s given the rest of society, has also extended those benefits to extremist movements. And it’s a big reason why modern extremist movements lean so heavily on the internet these days.

Adams: And how is the government doing in terms of tracking and monitoring the people in groups that are responsible for disseminating hate speech, or people encouraging violence or this online extremism?

Holt: You know, I’ve been doing this kind of line of work since probably about 2015, full time. And the difference between then and now is humongous. There are more researchers in this space, the government is paying more attention to this space. And particularly in this moment, after Jan. 6, there’s kind of, in the amount of political will to really kind of try to address this in a lasting way, there is still kind of a long way to go. But in addition to whatever the government may be doing, I also think if we’re going to meaningfully push back on this, it has to also come from nongovernment sources.

Adams: In line with that, how are the platforms doing in tracking and removing this kind of content?

Holt: The platform’s are also doing better. The Christchurch shooting in New Zealand, the killer there, committed his attack live on Facebook for 17 minutes before the stream came down. The killer in Buffalo was on Twitch livestreaming, but Twitch took the video down two minutes after violence began. And that kind of moderation response, it’s about as good as it gets, as far as live content goes. But ultimately, what that shooting represents and what it came from is an ideology, and that ideology is becoming more mainstream. The shooter in Buffalo was motivated to do that because of replacement theory. And I think there’s kind of a fair question for platforms here on, you know, are there ideologies that are inseparable from violence? I think replacement theory is one of those. I think it’s worth platforms reapproaching that sort of thinking and rationale.

Adams: For your research, you have to spend time in some of the darkest corners of the internet. What do you do to protect your mind and your heart?

Holt: I have a very strong sense of moral clarity in this work, and that moral clarity is what makes me want to do this work or feel like I have purpose doing this work. And having that kind of moral clarity is kind of like body armor against some of these ideologies. And then also there’s a whole lot of creative outlets I enjoy, whether it is music or photography. So even just trying to get some of the tension or the nastiness that I absorb as part of my work out in a way that has nothing to do with anything, that can be really fun and very good for mental self-preservation.

Related Links: More insight from Kimberly Adams

Twitch issued a statement on the Buffalo shooting and streaming of violence on the platform, saying white supremacy and racism have no place on Twitch. The company also encouraged users to take action to limit extremism.

Holt also noted there are unique challenges the government needs to consider when tracking online extremism, such as privacy concerns and finding the right balance of surveillance.

Congress is turning a fresh eye on the Domestic Terrorism Prevention Act, a bill that would allow the Justice Department, Homeland Security and the FBI to create domestic terrorism offices.

That bill had been shelved last month due to some concerns that it would lead to increased surveillance of civil rights groups.

And finally, Holt explained to me just how much more quickly someone can be radicalized online. Part of that is due to the speed at which this kind of hateful material can spread.

The Verge has a piece breaking down how the Buffalo shooting livestream went viral, because even though twitch had in down in minutes, the New York Times reports that was enough time for it to be shared on another site called Streamable, where it was viewed more than 3 million times before it was removed.