Mining Without Hard Hats: Making Sense of Big Data

shovelOne of the themes we stress with our Computer Science students is that their skills uniquely qualify them to provide valuable services to people and organizations. We emphasize that they are to use their skills not just for their own advancement, but also to help others achieve their goals.

The opportunities for Computer Science majors to help others are numerous. They can set up networks, design and build databases, secure data and communications systems, configure network services, repair and refurbish computers, design websites, implement e-commerce solutions, create social networking opportunities … the list goes on and on. Of course, I’m biased, but I can’t think of any other field where the opportunity to serve is so readily available. That is why service is a key part of our department mission statement.

Perhaps the most obvious way computer scientists can help others is by writing software. Software applications are tools that help people turn data into useful information. By writing programs that help people sift through the data they’ve collected, computer scientists enable them to make connections, spot trends, and draw conclusions they would not otherwise have been able to discern.

This particular role for computer scientists is becoming increasingly important, as the amount of data we have to work with has grown tremendously over the past decade. Data is the lifeblood of any organization. Often, studying these data streams in isolation won’t suffice, as their true meaning takes shape only when considered alongside other sets of data. Identifying co-dependent data sets and then assembling them in ways that elicit meaningful knowledge is a challenging task that, if done well, can benefit an organization almost as much as a seven-figure donation can.

For example, consider the article “Hackers Called into Civic Duty”, which appeared in the Wall Street Journal on August 12. It is refreshing to see the word “hacker” used the way it was originally intended, not to describe malevolent geek-pirates who steal credit card information, but to identify people who sling code to accomplish helpful tasks. That’s the kind of activity this article describes. Municipalities across the country have solicited the help of programmers to write applications that can take input from publicly available data sources and answer questions and draw conclusions about the data. Chicago seems to be leading the way on this kind of activity, which is good news for our bustling local tech industry. However, programmers have helped cities of all sizes answer questions such as how public transportation routes should be changed to serve riders better, which bike share stops are underutilized or underserved, which intersections see the greatest amount of traffic, and which communities have the most difficult time accessing medical services. These aren’t the kinds of questions policy makers can address simply by looking at the data, because there is just too much of it, and it is so interrelated. The applications computer scientists write for them, however, give them the eyes to see what really matters.

As more and more of this kind of data becomes available, the importance of writing software to make sense of it is going to continue to grow. That’s nothing but great news for those who view computer science as an unparalleled opportunity to serve.


About Ray Klump

Associate Dean, College of Aviation, Science, and Technology at Lewis University Director, Master of Science in Information Security Lewis University,, You can find him on Google+.

Leave a Reply

Your email address will not be published. Required fields are marked *