Big data is a big concept. It’s a whole set of technologies and processes that chews up staggeringly large data sets, digests it, and produces complex decisions, often in real time.
Big data has been used by enterprises as diverse as dating websites, retailers, and insurance underwriters, and it’s capturing the attention of many other industries as well.
For big data to be worthwhile, it needs to create value, either from data you generate yourselves (internal big data) or data you can collect from elsewhere (external big data). When value is created, it opens up new opportunities for an organization.
Without big data, how would pizza places know that college students are an important demographic? OK, bad example.
But effective big data use also places big demands on IT infrastructure. Internal big data occupies a lot of server space, while external big data (like that from Data.gov) affects your network performance. Both types occupy compute cycles. Here’s how internal and external big data affect your IT ecosystem.
Internal Big Data
Say you’re GE, and you collect data on gas turbine performance. You use it for things like projecting fuel savings based on improved turbine performance. Eventually you’re going to have a massive collection of turbine data, and besides using that data for your own operations, you could theoretically choose to monetize it by developing new services around it (like offering data sets to turbine engine testing facilities, for example). But you don’t have to be GE to be dealing with huge volumes of data held in on-premise and cloud repositories.
Suppose you work in an industry that’s heavily regulated (like healthcare or accounting). To protect your organization, you need to be able to extract and review that data if necessary (say, for an impending audit), and you need a strong big data governance policy for that to happen. Fortunately, applicable technology is maturing and becoming more affordable.
To make internal big data useful and regulation-compliant (let alone to be able to monetize it in some way), your IT team has to be able to handle unstructured data across the whole organization. It also has to make it possible for relevant end-users to engage with select data sets easily so all that data is useful.
Sometimes organizations immerse themselves in data that’s piped in from elsewhere.
External Big Data
Not all big data is generated from within an organization. In fact, the importance of external big data is growing. Organizations use external big data for research, or to supplement internal data with data they can’t generate internally. Where will it come from? While the roughly 105,000 data sets on data.gov are one source, keep an eye out for emerging data as a service (DaaS). For example, startup database technology companies could provide single-purpose databases that do one or two things well, like transaction processing, and the DaaS paradigm makes it easier and cheaper for companies to try out the data before committing to using it.
Big data can come from social media, weather databases, economic databases, and countless other sources, and as organizations learn to incorporate it into their decision-making, they’re going to realize new opportunities. But while external big data doesn’t tax your on-premise storage capacity, it certainly affects your network.
Big Data and Your Network
A lot of the talk about big data is about server and storage infrastructure, and about the applications needed to crunch through all that data, but what about the networks that shuttle it around, either within your organization or to and from the cloud? Let’s just say that networking professionals will have their hands full adding capacity and integrating it into existing IT infrastructure, lest a big data initiative turn into a Big Failure. That could mean a quantum jump in the demands made on your IT asset management system and your IT service desk.
Preparing Your Organization for Big Data
The growth of big data means new challenges concerning hardware, storage, and network infrastructure. For example, you may have to choose hardware based on typical sizes and growth patterns of servers and storage, or explore your cloud storage options for big data.
If your organization has (or wants to acquire) data scientists to helm big data analytics, he or she may work on the business side of the organization rather than the IT side. But the IT team has to be aware of how data scientists do their work so they can support it with adequate IT infrastructure. What it boils down to is that organizational silos and big data don’t necessarily mix well, and IT has to be prepared to deal with big data across the organization.
Internal or external, big data can have a profound effect on your organization’s IT infrastructure. Samanage offers leading IT service management software with a powerful unified IT asset management and IT service desk capabilities. With the right tools, your organization can cope more effectively with the demands of big data, so that the opportunities big data offers won’t go unanswered.