As I read Cathy O’Neil’s Weapons of Math Destruction recently and reflected on it, I became increasingly alarmed that we haven’t been addressing the considerable ethical challenges our budding computer scientists and data scientists will face. We’re teaching them how to author algorithms and sling code, dispassionate when it comes to how their creations will be used. This can’t continue. Of all the people who shape our future, Computer Science and Data Science graduates will arguably carry the most influence. They need to understand the potential harms of the power they will wield.
As O’Neil describes it, mathematical models are opinions expressed in math. Computing power enables those opinions to scale massively. When they do, any holes in those opinions, any oversights or simplifications, magnify in influence. That can then cause serious harm to people and institutions. In fact, in many situations, the models data scientists develop and then code are causing serious damage, digitizing racism and socioeconomic prejudice, wrapping it in new virtual clothes that give it a credibility not enough people are questioning.
Admittedly, I’m no ethicist. I lack formal training in ethical decision making. But I am a pragmatist and a problem solver. As such, I see great value in the simple and direct for creating change. We need something simple, familiar, and relatable to inspire young algorithmists like my students to understand the need to consider the repercussions of their work, to note and advertise the limitations, and to push for legislation that requires transparency in the algorithms by industries that are in the business of providing opportunities to people. When those businesses are allowed to discriminate under the cover of code, we counteract so much of the progress that has been in the area of civil rights and equal opportunity. Students must recognize that as they begin their careers and as they continue to craft the technologies that support our modern way of life.
I recommend having students and current data scientists read “Data Scientists, Do No Harm“. It promotes a sort of Hippocratic oath for Data Scientists, urging them, quite simply and familiarly, to follow this creed:
I recognize that data science has material consequences for individuals and society, so no matter what project or role I pursue, I will use my skills for their well-being.
I will start requiring this of my students. Not to do so while training tomorrow’s foremost movers and shakers would be irresponsible.