Big Data Cloud Migration
Business critical Hadoop cluster migration to AWS

Download Case Study

A global player in the travel industry faced a challenge for its Market Analytics product, a platform for business intelligence and analytics for airlines. Their platform had to be migrated to the cloud to increase reliability and flexibility, while customers on the production environment should not experiencing any disruption to the service. Neos IT successfully migrated the solution to AWS for typical cloud benefits with no downtime during transition.

Initial Situation

The challenge faced by the client was clearly framed from the start. The highly critical Market Insights Analytics suite needed modernization without exposing their customers to downtimes. This meant that all analytical activities for bookings and passenger data had to continue functioning during the modernization. The analytics supporting smooth decision-making processes of their customers should not be compromised at any time.


Before modernization by Neos IT, the client’s Market Insights Analytics suite ran on physical hardware. This required intensive maintenance as well as frequent hardware upgrades and replacement, for which the client had to bear the business risk and pay in advance.

Frequent service interruptions were caused by planned maintenance, capacity increase and outages. Servers were rebooted several times, causing downtimes for customers, increased operational effort as well as costly reprocessing of data. In response, it was quickly decided that a flexible, cloud-based solution was needed. The goal was to improve availability of the platform while increasing flexibility to grow with demand, accommodate processing peaks and disconnect compute investments from ever-growing data storage.

After proposing a solution architecture, Neos IT provided a proof of concept which ultimately led to the decision to migrate the entire solution to the cloud.

Solving the problem

In the initial scope, a deep refactoring for cloud- native services was discarded in favor of a quick implementation. The cloud solution in AWS used S3 storage as well as EC2 instances to run Hadoop and Cloudera. Decoupling storage from computing allowed the customer to consume the right amount of compute power needed to process monthly, weekly or unplanned analytics jobs. Previously, an increase in compute power required the purchase of related storage, leading to costly excess capacity, lack of compute power during peaks, or both. Another benefit of S3 compared to previous storage, was high availability and durability of data. This removed the need to store data in multiple copies to meet backup and availability requirements. Compute power, on the other side, could then be consumed „in more flexible terms, like instance reservation for baseline workloads, on-demand for processing peaks and spot instances cost efficiency.

IT security and compliance procedures also had to be updated. In the field of analytics and especially when personally identifiable data is being processed, every effort must be made to prevent gaps in security. In collaboration with the client’s security team, Neos IT set up seamless, end-to-end encryption of all data with the necessary security levels.

Auch die IT-Sicherheits- und Compliance-Verfahren mussten aktualisiert werden. Im Bereich der Analytik und insbesondere bei der Verarbeitung personenbezogener Daten müssen alle Anstrengungen unternommen werden, um Sicherheitslücken zu vermeiden. In Zusammenarbeit mit dem Sicherheitsteam des Kunden richtete Neos IT eine nahtlose End-to-End-Verschlüsselung aller Daten mit den erforderlichen Sicherheitsstufen ein.

Customer value:

The technical benefits of the new solution are the ability to scale its services flexibly in the cloud, while their customers benefit from dramatically increased performance. Time-consuming routine tasks have been reduced, availability increased to 99.9% and the time required to re-scale computing capacity of the platform has been reduced from around six weeks to just four hours.

The business benefits for the client is reducing time-to-market of new features and delivering customer-specific queries much quicker. For end users this means that the Market Insight Analytics suite has higher availability and business insights can be generated quicker to react to a changing market.

The solution was successfully migrated to AWS within just four months from kick-off to full production cut- over. Moreover, users of the Market Insight Analytics suite faced no disruption during migration.

  • “Neos IT helped our company to migrate Big Data solutions to the cloud – specifically AWS and many Amazon services, like AWS S3 cloud storage, application load balancers and self-managed relational database services. There were many AWS technologies that Neos IT helped adopting. Neos IT was able to provide the infrastructure and services, so we could run our big data application as if they were running on the bare metal provider that we used to have”
    Customer Voice - Data Operations Manager
  • “Once again, Neos IT offered to work with us and aided us migrating our business from physical hardware to a cloud solution. And the results are there, what can we say, they are remarkable!”
    Customer Voice - Platform Operations Manager
  • “The technical advantages rely mainly on the ability to prototype faster – in the sense that we can easily prototype new services or new applications in the cloud, thanks to new services that are available in AWS.”
    Customer Voice - Data Scientist
  • “We were building this on AWS, and at the time this was an unknown territory for the client. There were some interesting challenges in adjusting security and compliance requirements that were important to them on a daily basis. Closely collaborating with their Security Team allowed to greatly increase security and stability, ultimately getting SOC approval.”
    Plattform Lead Engineer - Neos IT Services
  • “If you want to achieve something fast you need to find solutions that can be implemented quickly with low costs and effort. We are talking about several hundred compute nodes and their corresponding services that are serving more than a hundred customers around the world, that needed to be migrated in four months. That was the challenge Neos IT had.”
    Customer Voice - Engineering Lead

Let's talk about your project.

Get in Touch!