When Big Data will Become Even Bigger: The Expert Interview

Today we are talking with Nikolay Khoroshevskiy, Big Data expert at Sigma Software. Together we are going to discuss how Big Data is changing us and how it will develop in the future.

Big Data is not a novelty for the market. The official term was introduced 10 years ago, but the idea is even older. How did it change over this decade and how did it influence the world around us?

In one way or another, Big Data entered every sphere of human activity, starting from retail to healthcare, telecom, media, aviation, automotive, and so on and so forth. It actually changes our perception of life and everyday routine. Let’s say you’re using Google or Apple music applications, and they offer you tracks based on what they know about you and your musical preferences. Another example is Facebook advertising that “follows you”: let’s say you’ve just been looking at a product on your phone, you open Facebook on your computer, it shows you an ad for that exact same product.  During the last ten years, we have used to new opportunities that Big Data provided us, without even realizing it. It may look like a nice add-on to our everyday activities, however, when it comes to business, Big Data becomes the most powerful tool for reaching new heights.

The trend of the past few years demonstrates the constant increase of life speed, so business must learn to make fast decisions. Fast, but informed still, and here Big Data plays the key role. Machine algorithms, able to analyze a huge amount of data, are business’s Magic Eight Ball that answers any questions that previously required vast and long-term researches by many people. Retailers now can manage the procurement of goods and pricing depending on the demand for them in real time. Telecom companies predict customers churn and retain them even before the final decision was made. The one could think this is a fantastic tale just a couple of years ago; today it is a common thing.

Big Data is trending for the last few years. For how long, do you think it will keep on top?

I think Big Data is on the rise, but not on the peak yet. Actually, the situation around it is quite interesting now. Many businesses, especially large ones, have been gathering various kinds of data for quite a long time. Still, they each only have their data slice and don’t have a complete picture. When the companies start exchanging data, this will be a new stage in Big Data development.

What risks do you see in the future when it comes to gathering, sharing, and processing a large amount of data? What risks businesses can face today?

Many of today’s risks can be solved on the stage of the architecture design. It often happens that as the project grows both approaches and architecture change. Therefore, it would be a good idea to look years ahead and build an architecture able to handle significant system’s growth.

As of the future risks… In my opinion, data storage shouldn’t become a problem. Most businesses have already moved to the cloud, which allows avoiding downtimes and eases the infrastructure maintenance. Amazon, Google, and other companies have created huge and reliable storages. These services will stay the main tool for data storing for many years, I think. Well, unless the DNA digital data storage will become a reality.

When it comes to processing though, we are going to face many challenges; some of them are already relevant. Since the amount of data increases rapidly, the processing should speed up as well. Many large businesses switch to data-flow computing to handle this issue.

IT companies are actively investing in Big Data development. Do you think other businesses should be interested in that?

Sure, these investments are going to pay off. And companies are actually making their contribution. For example, last December one of the European leading telecom operators, Vodafone, organized a Big Data Hackathon in Kyiv, which I was invited to join as a mentor.

I was impressed with the high level of teams and projects. Such activities are very important; it is an opportunity to try own hand in working with the data of a large company and dealing with huge amount of real requests. Such events empower with positiveness regarding the technology successful development in the future.

Nikolay Khoroshevskiy has over 5 years of experience in software development and 3 of them were dedicated to plunging deep into highly loaded systems and Big Data. At Sigma Software, Nikolay is engaged in the project for one of the world leading media conglomerates, Oath. The B2B video platform that Sigma Software team has been developing from scratch, allows connecting advertisers and owners of high-quality video content. The system processes over 2M requests per second, which is a fair proof of a platform high popularity. The new architecture is implemented on the project to cope with a serious increase of data flow since 3-4M requests per second are just around the corner.

• #BI and Big Data