Algorithms for vaccine distribution have a weakness: the people behind them

To deal with the massive logistical problem of distributing COVID-19 vaccines, the federal government and some states are turning to private companies to create algorithms for prioritizing shipments. Some hospital systems, like George Washington in Washington, D.C., and Stanford in Palo Alto, California, created their own software systems to prioritize which health care workers get it first.

In Stanford’s case, we now know the process went notoriously wrong, prioritizing doctors and administrators working remotely over residents working directly with patients every day. I wondered, is vaccine allocation a problem algorithms are meant to solve, or are officials letting algorithms take the blame for built-in inequality? I spoke with Karen Hao, who reported on this for the MIT Technology Review. The following is an edited transcript of our conversation.

Karen Hao: I would say that the Stanford algorithm was not, in fact, that complex. And there’s actually been an argument within the algorithms research community about if we could even call it an algorithm because essentially, it was just a set of rules that were codified into numbers. And for many of the hospitals that I got in touch with, after this story broke at Stanford, they use just a more simple rule set. They usually just consider one factor, which is the actual department of the staff. What Stanford was trying to do was not just factor in one thing. They were trying to factor in, I guess, three things: age, department and prevalence of exposure. But the issue is they didn’t do it in a way that was correct. There are really two lessons that we can learn from the Stanford case. The first is that the way that the distribution allocation was decided, there were no residents at the decision-making table. And then, the second thing we can learn is once the algorithm was rolled out, there was no mechanism for people to provide feedback.

Molly Wood: So on top of having this thing that was somewhat flawed, they didn’t audit it, they didn’t really test it? They just sort of were like, “Here you go. Now, shots.”?

Hao: Exactly. The issue is that they didn’t have people who are impacted by this algorithm at the decision-making table. And they didn’t have ways to then quickly iterate and update the algorithm once they realized it was flawed.

Wood: Is vaccine allocation a problem that algorithms are meant to solve, or will it become something that leaders might get to hide behind when inequities inevitably happen?

Hao: A lot of times, people believe that when an algorithm is involved, it somehow is this other entity that is completely detached from human decision-making and the algorithm is suddenly unbiased, even though it was programmed by people. And what I try to hammer home to people when I talk about this is algorithmic decision-making is human decision-making. Some of the language that Stanford’s leadership started using after this happened was, “Oh, the algorithm was really complicated. It was the algorithm’s fault,” is kind of a misnomer. It doesn’t really acknowledge the fact that they were the ones literally codifying these priorities into code.

Related links: More insight from Molly Wood

VentureBeat has a nice piece about the federal vaccine distribution process and how it’s been contracted out to one of the Trump administration’s favorite private companies, the data-mining company Palantir, which has several other federal contracts under its belt. Palantir and the U.S. Department of Health and Human Services have created a software platform that prioritizes shipments of the vaccines to various states and then tracks the rollout and the vaccination process.

Now look, this is obviously how you would do something this big in the computer age, but since it’s ultimately taxpayer money funding this process, it seems like it should be easy enough to tell us all what exact data goes into making these decisions, these big public health decisions — but we don’t know. It sounds pretty cool how the Department of Defense’s site describes it, using a big database of non-personally identifiable information. The Tiberius platform will run some numbers every week, determine which states should get how much vaccine and ship it out. The DoD site notes that “Tiberius can provide a zip code-by-zip code view of priority populations, including frontline workers and nursing home residents,” which is always how computers are supposed to work — like magic, with pristine data that’s regularly updated and totally comprehensive. Except, not every ZIP code gets counted or tracked very well, especially poor or Black and brown neighborhoods or reservations where a lot of frontline workers might live and where COVID-19 is killing more people, and public health departments all over the country are totally underfunded and reporting data with fax machines, if at all.

And again, we actually don’t know what data is being used to make these determinations at the federal level. But what we do know is it’s not going great, and a lot of states are saying they’re getting less vaccines than they expected to get. And inequality is already showing up, and people are already frustrated and skeptical. And the best thing everyone could do is be as transparent as possible about how it all works. I’m trying to start off 2021 hopeful, so hey, if we keep talking about it, it’ll happen.

Fast Company had a little piece on Monday about the Centers for Disease Control’s vaccine distribution tracker — a new tab in its overall COVID-19 tracking dashboard — that at least tries to give updated information about where the vaccine is going and how many people are getting it per state or territory. It has the effect of making you feel like, hey, at least something’s happening, right?