Every day billions of people interact in some way or another with digital devices whether they know it or not. Each of these electronic interactions creates a little bit of data that is sent off and stored somewhere, forever recorded in time. If it’s a quick check of Facebook before starting the day, or having a bagel and coffee scanned and paid for by a credit card, working out at the gym with your health-tracker recording your progress, or walking past a surveillance camera on the corner of a building, or doing a load of laundry with your smart washer. This information is kept alongside all the other instances of the world in a massive web of servers and drives. There are plenty of useful applications of this data, and plenty of harmful ones. We will look at what Big Data is and how it affects us and the rest of the world.
“Big Data” means many things to many people and there isn’t a commonly agreed upon definition out there. But this happening is real and it is generating benefits in so many different areas, so it makes sense for all of us to have a working understanding of the concept.
The basic idea behind the phrase “Big Data” is that everything we do is progressively leaving a digital trace, AKA data, which can be used and analyzed. Big Data therefore refers to that data being collected and our ability to make some sort of use of it.
The term “big data” is not the greatest choice for a lot of reasons, but it seems we’re stuck with it. It’s basically a naive term for a very real phenomenon – the datafication of our world and our increasing ability to analyze data in a way that was never possible before.
Obviously data collection itself isn’t a new thing. As humans, we have been collecting and storing data for tens of thousands of years. What is new are the recent advances in chip and sensor technology, the Internet, cloud computing, and our ability to store and analyze data that have changed the quantity of data we can collect. Big, massive amounts of data.
Elements that have been a part of everyday life for decades: shopping, listening to music, taking pictures, talking on the phone, etc., now happen more and more in the digital realm, and consequently leave a long trail of data.
The other big change is in the kind of data we can analyze. It used to be that data fit neatly into tables and spreadsheets, things like sales figures and wholesale prices and the number of customers that came through the door. Now data analysts can also look at “unstructured” data like photos, tweets, emails, voice recordings and sensor data to find patterns.
How is it being used?
As with any major advances in innovation, the tool can be used for good or vicious purposes. Some people are concerned about privacy, as more and more details of our lives are being recorded and analyzed by businesses, agencies, and governments every day. Those concerns are real and not to be taken lightly. Hopefully, best practices, rules, and regulations will evolve alongside the technology to protect individuals.
Most of us have some idea that companies are using big data to better understand and target customers. With big data, retailers can predict what products will sell, telecom companies can predict if and when a customer might switch carriers, and car insurance companies understand how well their customers actually drive.
It’s also used to optimize business processes. Retailers are able to optimize their stock levels based on what’s trending on social media, what people are searching for on the web, or even weather forecasts.
But big data goes way beyond shopping and consumerism. Big data analytics enable us to find new cures and better understand and predict the spread of diseases. Police forces use big data tools to catch criminals and even predict criminal activity and credit card companies use big data analytics it to detect fraudulent transactions. A number of cities are even using big data analytics with the aim of turning themselves into Smart Cities, where a bus would know to wait for a delayed train and where traffic signals predict traffic volumes and operate to minimize jams.
Why is it so important?
The biggest reason big data is important to everyone is that it’s a trend that’s only going to grow at an accelerating rate.
As the tools to collect and analyze the data become less and less expensive and more and more accessible, we will develop more and more uses for it — everything from smart yoga mats to better healthcare tools and a more effective police force.
And, if you live in the modern world, it’s not something you can escape. Whether you’re all for the benefits big data can bring, or worried about Big Brother, it’s important to be aware of the phenomena and tuned in to how it’s affecting your daily life.
Big Data will ultimately have an impact on each and every one of us, unless you can completely go off-grid and remove all things digital form your lifestyle. The truth is that most ways this phenomenon will change our lives will have a positive effect on us – as users, consumers, creators, and business people, etc.
How can you identify moments where big data plays a role in your life? Most of us use online content services like Hulu or Netflix. Let’s say that over a recent weekend you had watched a Game of Thrones episode, then The Princess Bride, followed by How to Train Your Dragon and an episode of Downton Abbey. On Monday evening you retire on the couch to take in some more television, and you see a “Recommended Show” list, saying you should watch Vikings, The Borgias, Peaky Blinders and Spartacus. You say, “cool!” And proceed with one of those options without really thinking of how they got there. That is big data and analytics hard at work by our friends over at Hulu or Netflix. They keep all the info about what you watch, how often, what times of the day, etc. so that they can process this and start building your individual profile with your own preferences. Is this a good thing? Sure! Look how easy that made your decision making. You didn’t have to waste a bunch of time clicking and scrolling to find something, and maybe some of the options were completely off your radar, giving you the opportunity to check out something new in the defined genres of your previous interests. Is this a bad thing? Maybe – the new results kept you from making a more organic choice about your next entertainment venture. This choice could have provided you the opportunity to jump into a new genre that you may have found satisfying. It also has kept you plugged-in to that service for even a little bit longer, especially if you opted into a series that takes you several weeks to finish up, meaning one more month of subscription services with that content provider. So you can see, there are multiple reasons why these companies would bother mining the data of their consumers. Some make it look like it was purely intended for you and your viewing pleasure, the other a way to keep you and your wallet around.
This is just one example out of many. Think Amazon. They have a mountain of information about all of its users – from their profiles to their search histories to the sentences they highlight in e-books – which it uses to predict what they might want to buy next. Past patterns are used to forecast future ones. Amazon can look at the last 10 books you purchased and, with increasing accuracy, suggest what you might want to read next. However the most meaningful book you should read after those previous 10 might not be one that fits neatly into an established pattern, but rather one that surprises or challenges you to look at the world in a different way. Big data can multiply our options while filtering out things we don’t want to see, but there is something to be said for discovering that 11th book through pure serendipity.
Bernard Marr is a best-selling author who covers a lot of areas in tech, especially Big Data. He has an article on the “10 Awesome Ways Big Data is Used Today to Change Our World.”
Many people today are looking at the threats of Big Data to democracy and our political systems worldwide. This short article from The Guardian helps us understand this angle. Please read.
Should we get paid for the data we provide? Read this article from NYTimes.com on this topic.
Here is an interesting article from the New York Times about how some big data exposed something unexpected – and not an awesome thing.
The concepts and ideas around Big Dat can be very overwhelming and tough to wrap one’s head around. It’s helpful when creative and brilliant people put their energy and minds together to help us better grasp these difficult ideas. This documentary, “The Human Face of Data,” does just that. Please watch this film as a wrap for this session’s materials. Duration: 54 min.