Big Data, The New Public Utility Bill

You don't need to be a businessperson or an IT professional to realize that we're currently experiencing great growth in digital content, information and interaction, and that this fact is having a great impact on our lives —the approach of the metaverse some may say.

Just as we've done over the years with commodity services like running-water, heating, phone, lighting or cable-television, people of all cultures and backgrounds are developing a natural relationship with digital data.

In many parts of the world we often take for granted how these commodities have improved the quality of our lives. In any event, It looks like big data & analytics (BD&A) will have a similar effect on our lives, some say a multiplying one. Future generations will develop a totally natural relationship with data, and ‘older people’ will have to learn to live with it and even learn to love it.

What is Big Data?

'You may know that the commercial world is going through a transformation because of a focus on Big Data and with companies competing on the Data Analytics front. This essay is not about the commercial use of data nor about Big Data in itself, but it touches broadly on the concept of Big Data so as to explain what it is in terms that most can understand; (1) how it may affect us all as regular consumers, (2) why embracing it should let us live more creatively and squeeze more out of our lives, and (3) while readily exposing more our consciousness to the powers that be.

Plug: As a result of carrying out a research-study in 2014-15 for a client in the field Big Data, I was fortunate to learn a great deal about the world of Big Data & Analytics (BD&A). I learned about the sector and the different industries which participate in it (which eventually will be all of them), and I came to the realization that ‘data’ will simply become a commodity which everyone in the world will use and need either directly or indirectly. For this or other market research work please contact me at contact@martemartin.com .

Eventually, sometime in the next 10 years or so, BD&A will just become another utility service we get at home and at our business-place in the form of a standard or basic service, or more bluntly put, 'one more bill to pay every minute, day, month...'.

The description on 'the data bill' will indicate something along the lines of: "Monthly Data-Management service charge: capturing, storing, analysing, securing and distributing of common daily data". As you may imagine, I am not the only or even first person to project the meaning of data as a public utility, but I think it's still worth discussing the parallel.

So what's the definition of Big Data? Okay, so there's actually no standard definition of Big Data, and that's because BD is a term which is used to refer to vast amounts of real time data, which by itself can be meaningless, but which once processed it is converted into meaningful information or as input for other processes (e.g. Artificial Intelligence).

The definition of Big Data is a moving target, and unless you have studied the subject in some depth, it's difficult to describe what it is exactly. I’ll try explain it in different ways throughout the essay so as to try to help you associate with it, while I avoid becoming technical about the concept (not that I have the most technical knowledge about it). Still, I do have a clear understanding of its utility and application potential.

Essentially, the meaning of Big Data is relative to different factors which change all the time. It is perhaps best to think of Big Data as a cultural, social and technological phenomenon which sits on three pillars: computational power (the smart devices of the world), analysis (identifying patterns) and insight (which should allow us to make better and faster decisions). Think of a brain on steroids.

Yes, another utility bill —another one, just like the kind of utility you might be thinking about: gas, electric, telephone or water.

In addition to our water, electricity or gas companies, a ‘data-management company’ will provide us with a standard service to capture, store, secure, process, visualize and deliver all the data-information which is meaningful to us and our close ones – a one stop shop for all our big data needs and regardless of its origin –.

Today, BD&A is mainly regarded as an enterprise problem and a business opportunity. A problem which most people outside of the IT or Business Intelligence fields do not quite comprehend, or if simply just don't care to understand as long as their data service works.

So far, it's mostly big organizations, corporations and governments the ones thinking about the uses of Big Data (this is in 2014-15). Still, it's going to impact the lives of everyone, much more so that we can appreciate today – it will become an integral part of our daily lives and with time, it will become like a natural relationship.

The press and analysts have certainly focused more on the enterprise problem and less in how data will propel all of us into the next great phase of human evolution. Big Data will automate many human functions, but it will also complement our brain-power to help us process tasks in less time and more difficult ones.

Let's think about Big Data & Analytics in another way. In the quest for profits, businesses are exploring innovative ways of collecting and utilizing data, because they came to the realization that BD is a source of what’s referred to in the world of business as a 'competitive advantage'.

But BD today (collecting, storing, securing, managing, processing and visualizing) really is like making a new cocktail – some of this, a little bit of that, a splash of this and that, and a good shake –. We see what comes out, we taste it, and if it's any good we can serve it according to the customer's preference. And just like the famous barman in the picture above, the big data professional is also often considered as having a sexy job

SOURCE: AARON WYNIA

Due to its infant nature (again, as of 2014-15), as we are only scratching the surface of what Big Data will mean to the world, you can also think of it today as playing dice rolling – you throw the dice and see what comes out. Meaning sometimes you'll generate useful insight and sometimes you'll get nothing.

In the big leagues of data management, your franchises (companies) are also worth lots of money, they have the top players in their teams (data scientists, analysts, managers, marketers, designers, software developers, visualization experts), the greatest facilities, and pay for the best services.

Just like in Europe you have great soccer teams like Manchester United or Real Madrid, or in the US you have great baseball franchises like the Yankees or the Red Sox, in the world of BD&A you have organizations like Google or Facebook which propelled the Big Data phenomenon. These companies and others were heavy users and creators of Big Data technologies, they essentially started building the technologies they needed.

The most valuable players (MVPs) in the Big Data leagues are the big hitters known today as the Data Scientists. These stars are the beholders of what is been referred to as “the sexiest job of the 21st century”. You may not agree with referring to the Data Scientist position as being a sexy job, but the job does involve applying a great deal of creativity (which is always sexy), and today, it pays very-very well. These men and women basically spend their time looking for treasures in messy piles of digital data.

Just like in the big leagues, the biggest companies have the latest equipment to remain as competitive as possible –these tools being the latest hardware infrastructure, the safest cloud storage, or technology in the fast rise of open-software-technologies for storing and manipulating unorganized data. I predict though, that in five years’ time, the title Data Scientist will no longer exist as the sector will atomize itself, where many different specialized jobs will substitute what the data scientist does today–.

The Landscape

SOURCE: Marte Martin

The main thing to understand from a broad perspective is that the Big Data landscape is continuously evolving and evolving fast, it is like a forest that gets bigger and more dense by the minute. Where today you have participants concentrated in the spaces of Applications, Infrastructure and Technology, tomorrow you may have a consolidation of these three and the appearance of new spaces of concentration.

The 1982 movie 'The Thing' also comes to mind when I think of the Big Data landscape – it just grows in weird and non-conventional ways. As a consumer, you are most likely not familiar with companies in the different spaces, but names of organizations which operate in Big Data and which you may know of are: IBM in Infrastructure, Microsoft in Applications, and Hadoop in Technologies.

The Example - Facebook

Time Spent By Facebook Users on Facebook (Average Minutes Per Day Worldwide). SOURCE: Business Insider

As a simple example to help you understand how Big Data works in the real world, let me ask you this: 'did you know that as of 2016 Facebook users worldwide spend an average of more than 50 minutes per day on Facebook?' Imagine how much data Facebook alone collects, how much they store, how much they process, how much they sort out and analyse, how they have to secure private data, and how they transform all that data into meaningful information such a simple visual or a targeted advertisement. Facebook manages so much data and their business is so big that they have the capability to develop their own solutions in each of the landscape spaces mentioned above.

Okay, so I am not entirely sure about Facebook creating their own physical infrastructure, but you get the idea, plus I am sure they could develop it if they wanted to. [Facebook is currently experimenting with Data WIFI solar drones, and has been working on VR hardware for years]

Among the many applications developed by Facebook employees, they have developed a visualization application which shows Facebook's global reach through a world map. Looking at the maps below, the higher the concentration of users in a region the lighter the region is. You can clearly appreciate through the visuals how Facebook's user base has grown and become more dense – Facebook has essentially created its own big data forest.

Through a visualization of this sort, you can also easily appreciate how for example China pretty much wants no part in Facebook's forest as that area of the world is just dark all the way through. Sub-Saharan Africa is another area of the world where Facebook does not seem to have much of a footprint. Accordingly, as of August 31, 2016, Facebook announced plans to launch a satellite which will extend internet access to rural parts of Sub-Saharan Africa, as a first measure to provide affordable network for people and as an alternative to unaffordable infrastructure in the region.

SOURCE: Facebook

In the world of big data, It's often said that visualizing data is like taking a photograph –you can manipulate the camera settings and lenses to present the data from different points of view. You can visualize the same forest from different angles and reach different conclusions, or also to reach the same conclusion from different angles. When the forest is very dense, like it is the case of Facebook's social-graph, there are many lenses through which you can look at it.

In Facebook's visualization example, if you combine the visuals with the known fact of how much time per day each user spends on Facebook and the fact that Facebook has today (in 2016) more than 1,600 Million active monthly users, you can get a pretty good idea of the impact Facebook is having connecting people, and of how much data they capture, store, process and deliver. I mean, that's a lot of time spent on Facebook's network.

[Token: Filipinos and South Africans spend the most time on social media as of 2016.]

Facebook Number Of Users (Active Monthly Users). SOURCE: Facebook

Data, the New Public Utility

SOURCE: CB Insights

For businesses, Big Data represents the enterprise problem of the beginning of this century – a computational problem which requires a new kind of combination of human and artificial intelligence to be able to compete in a new enterprise space. Data in itself is not very valuable to us; its value comes when it is converted into something useful. Data is indeed the new oil, but it will need much more processing than oil to be valuable for everyone.

There will exist different data-management companies which will offer similar services and which will differentiate themselves by competing on price, the quality of their service, or enhanced and better applications (analytics and visuals).

Basic data-management utility companies will take care of all the basic needs around your data, such as storage, security, control, analysis, distribution and visualization – regardless of the data's origin. It seems clear to me today that the big internet companies are best positioned to become the main world operators of data-management, but it's too early to say. These companies, whomever they are, will know everything about us, and this will come with both pros and cons.

We no longer just consult through data or check out data, we are now consumers of data. We expect data and information to reach our devices, computers, TVs, smart-homes, smart-cars and smart-phones when we turn them on, and we don’t care what happens behind the scenes.

Apparently, this is just the very beginning of a new way of interacting with life. We are only now beginning to expect to have data delivered to us with the same consistency, reliability and at relative low prices as the utilities most use. We will consume data in a seamless manner, and our relationship with data will be complementing and a natural one.

Big Data as a concept has been around for many-many years, but it wasn’t until around 2011 when it gained somewhat of a popular hype; for a couple of years there after the media covered it extensively.

Even though Big Data today is as relevant as it was a few years ago, if not more, the big trend over the past year or so in Big Data & Analytics has been the increasing focus on artificial intelligence (AI) to help analyse massive amounts of data and derive predictive insight. AI is a child of Big Data and it is helping Big Data to deliver on its promise.

SOURCE: BUSINESS FINANCE NEWS

In different ways, we are still in the very early stages of the Big Data phenomenon. We are essentially watching the planet “develop a nervous system, one in which we are just another, human, type of sensor”.

The ability to collect, analyse and visualize vast amounts of data in real time is something new to us – and perhaps more importantly, it expands beyond the physical and numerical boundaries of the 'classical' utility onto the domain of consciousness, and as Big Data becomes the portal to common information and connectivity with others (and the other).

While it has taken a few years to build the infrastructure to store the vast amounts of data and to develop the first processing technologies, this was just the first phase of an industry which will be enormous. The combination of Big Data and Artificial Intelligence will drive innovation across every other industry, including those industries which provide utility services to us today. The Big Data potential is surely much bigger than what most of us can imagine.

We seem to be evolving into another great phase of human evolution based on the enhancing power of predictive analytics and data intensive computing. New generations will grow up loving the use of data – and rely on the systems that manage-control it for better or for worse, and current and older generations will have to learn to love it or risk being left behind by society.

This new basic utility service will –should– enable us to be more creative with our time, and ultimately our lives. At the end of the day, big data is not just about analytics, or about applications, it is about driving experiences to individuals and 'having/making them/us individuals' do things and act upon information in real time.

Ideally, It is about packing more experiences in the time we have. It's about allowing us to be more creative not just with our time, but also our space. At the same time –or perhaps, or alternatively, or whatever– this utility may also enable an entrance into some sort of a metaverse, which we may have entered already, where different cognitive perceptions are entertained or assigned to the peoples, with more or less awareness, and where the reality and the unknown become the same.

This essay Big Data, The New Public Utility appeared first on martemartin.com/resources.