Building a highly scalable, distributed deployment ecosystem using Kubernetes, Docker and Mesos.

The customer is a next generation recruitment analytics platform. The platform helps their customers to get detailed insights into the recruitment process, thus, eliminating the drudgery from the hiring process while keeping candidates engaged in the process as well.

Objective

Client approached Opcito to design and develop a deployment ecosystem which could take care of the deployment, hyperscaling, and monitoring of applications across a data center having 450+ physical servers and 1700+ processing cores and 6 TB+ RAM.

The main objective was to create highly scalable deployment ecosystem which can utilize all available resources seamlessly and easily manage large deployments with minimal effort.

The initial challenge was to design an ecosystem that performed hyperscaling of applications with the isolation of a large number of server resources across the data center. This feature would be implemented with distributed architecture and different scaling patterns to maintain consistency in resource utilization and time to delivery.

Approach

Our approach to this challenge can best be explained with a metaphor: If a Docker application is a Lego brick, kubernetes would be like a kit for building the Millennium Falcon and the Mesos cluster would be like a whole Star Wars universe made of Legos. After Understanding the Client’s needs and requirements, we came up with the following strategy:

Developed a customized platform for configuring and creating Docker containers using Kubernetes.

• Customized configurations of Kubernetes for managing containerized applications across multiple hosts. This involved providing basic mechanisms for deployment, maintenance, and scaling of applications.
• Developed hyper-scale controller for Docker which scales to the demands of the deployments. Provided a customized integration bridge using Kubernetes and Docker API interfaces.
• Developed resource manager using Mesos Platform for distributed computing environments to provide resource isolation and management across a cluster of slave nodes.
• Integrated Mesos platform to join multiple physical resources into a single virtual resource.  Mesos allows scheduling of CPU and memory resources across the cluster in a way similar to how the Linux Kernel schedules local resources.
• Created/Configured deployment ecosystem with Mesos, Kubernetes, and Docker.
• Provided custom GUI for managing application deployment, scaling configuration, and container metadata updates using Kubernetes and Mesos CLI and API interfaces.