Mitra innovation has its very own Research and Development team designed to research and test emerging technologies and new areas of development. In a recent experiment, we evaluated the Google Cloud Platform as a Cloud provider that can deploy containerised WSO2 products in recommended deployment patterns.
We have previously deployed WSO2 products in the Cloud as virtual machine based deployments on Amazon Web Services (AWS) and Microsoft Azure, but – before this experiment – we had not tested a WSO2 deployment on the Google Cloud Platform. Furthermore, at the time of conducting this study, we were not aware of any similar experiments conducted by any other organisations outside of Mitra Innovation.
The R&D team uncovered some useful benefits about deploying containerised WSO2 products in production environments during the experiment, and this paper shares these insights.
Google Cloud Platform
The Google Cloud Platform (GCP) is a suite of Cloud computing services running on top of the Google infrastructure. Products and services offered by GCP include:
- Computing and hosting services
- Storage services
- Big data services
- Machine learning and artificial intelligence services.
The computing and hosting services by GCP provide the following:
- a managed application platform (App Engine)
- an unmanaged Cloud infrastructure (Compute Engine)
- a container cluster management platform (Container Engine).
Developers can choose from these solutions, depending on how they want to deploy a product on the Cloud.
WSO2 is a middleware supplier. Middleware enables different types of software to ‘communicate’ with each other even though the communicating softwares may not be of the same languages. In just the same way that a translator enables a person that speaks French to speak with a person that speaks Russian, middleware performs the same function for software. Middleware allows for information to be transferred from one system to another when needed, regardless of devices in use, operating systems, location, or time. Recently we published a blog post on WSO2 Middleware, which can be read here.
Many WSO2 middleware products are designed to be deployed on-premise as well as on the Cloud. WSO2 products are released with Docker  files which can be used to build docker images, bundling all the product configurations and dependencies required to run a WSO2 product inside a container instance..
For clustering and container deployment orchestration, WSO2 provide Kubernetes artifacts . Kubernetes is an open-source container orchestration and management tool developed by Google with features to scale and cluster containerised applications and automate deployment processes. Using the power of Docker and Kubernetes, organisations can push WSO2 middleware from the development phase to the production stage in the Cloud with greater flexibility, developer productivity and shorter time to market, whilst avoiding the product configuration complexities inherent to distributed middleware server setups. For example, organisations can deploy WSO2 API Manager in a fully distributed containerised setup, using available Docker images and Kubernetes configurations provided by WSO2, within a very short time.
In the R&D team experiment, we evaluated deploying WSO2 Docker images on the Google Container Engine (GKE as GCE is abbreviated for Google Compute Engine) which is the managed Cloud environment to deploy containerised applications on the Google Cloud Platform. GKE inherently supports Kubernetes as the container orchestration and management framework, as it was developed by Google. GKE provides many intuitive and developer friendly tools to deploy, scale and manage containerised applications.
The Google Cloud SDK  is a set of tools that can be used to manage the application and resources hosted on the Google Cloud Platform.
- The gcloud command line tool is a very useful tool to manage applications and resources in the Google Cloud Platform.
- Kubectl  which is a command line tool to control Kubernetes artifacts is also inherently supported by GKE.
With the features provided by kubectl and gCloud, developers can work very efficiently on GKE with their docker and Kubernetes Cloud artifacts.
For example when developers build and deploy containerised applications with GKE tools, they can:
- Create and manage pods which are logical hosts of containers.
- Create and manage GKE Kubernetes pod clusters.
- Create and manage replication controllers, which can create and manage pods based on a template. (Replication controllers ensures the applications have sufficient resources and manages scaling and the reliability of the pods.)
- Create and manage Kubernetes services. Services provide an abstract layer that decouples clients from pods that provide backend functions. So, clients can call the service endpoints to get their work done without depending on how pods are being managed at the backend. Services also provide a load balancing mechanism for pod
- Define requirements for Docker containers in a simple JSON configuration file.
- Push Docker images to Google Container Registry and pull images to any GKE instance and deploy
Create load balancers.
Before this paper moves onto the deployment experiment and details, it is useful for readers to understand the different container artifact types offered by WSO2 and related concepts.
WSO2 Dockerfiles provide the configurations and instructions to build WSO2 product docker images. There are two methods to provision WSO2 products in a docker image. They are ‘default’ and ‘puppet’ provisioning. The default provisioning method allows the user to install the JDK and other dependencies and the relevant WSO2 product pack with the necessary product configuration files. The ‘puppet’ provisioning method uses the WSO2 puppet modules to configure a product in the docker image at build time. To conduct the experiment, we followed the default provisioning method, due to it’s simplicity.
WSO2 Kubernetes artifacts
As mentioned above, WSO2 provides a set of Kubernetes artifacts including Kubernetes services, deployments and data storage related artifacts such as persistence volumes and volume claims for different products. These artifacts can be deployed on multiple container runtimes including Docker, Rocket (Rkt) or AppC.
There are two main artifact types for a WSO2 product Kubernetes deployment.
- Service: A Kubernetes service provides a mechanism for load balancing pods. Each service exposes an internal DNS entry to call the service within the Kubernetes internal network. The containerised applications can call the service endpoints within the Kubernetes cluster to consume the backend functionality provided by pods. There are two service types:
- ClusterIP which makes the service only visible to internal network
- The NodePort service type exposes the service via the node’s ports to the external network by mapping the service ports to a port in the node and the load balancer type which is a dynamic load balancer enabled by the cloud provider (eg : GKE).
- Deployment: The Deployment defines how the containerised application can be deployed on a Kubernetes cluster. It creates and updates the instances (pods and replica-sets) of the application. The deployment controller monitors the health of the instances and provides self-healing to the Kubernetes cluster
WSO2 API Manager on Kubernetes
As part of the experiment, the Mitra Innovation R&D team explored the features of GKE to deploy a containerised and distributed middleware server named WSO2 API Manager .
WSO2 API Manager is an open source enterprise-grade API management solution that supports API publishing, lifecycle management, application development, access control, rate limiting and analytics in one integrated system. WSO2 API Manager comes with a set of production grade deployment patterns  to deploy a fully distributed API Manager deployment enabling high availability and higher efficiency.
The R&D team used the pattern-1 single node deployment with an external MySQL database for API-Manager. The rest of the distributed deployment patterns will be explored with GKE later, as they were in the process of being updated by WSO2 at the time of the study.
(Figure 1 : API Manager Container Deployment)
This graphic depicts the high level deployment architecture of API Manager pattern-1 container deployment in GKE. The deployment has two containers. The mysql database container which provides the API-Manager mysql databases service to the internal network via a Kubernetes service. And the API-Manager container which consumes the mysql database service to access the user-management, registry and API-Manager databases. The API-Manager service is exposed to the external network as a NodePort.
The deployment process of the WSO2 artifacts is shown in the figure below.
Figure 2 : Deployment process
These are the steps involved in the WSO2 container artifact deployment:
- Create a container cluster in GKE with a small number of nodes
- Pull the latest dockerfiles and the Kubernetes artifacts from the WSO2 repositories
- Build the required docker images from the relevant dockerfiles
- Push the docker images to a private repository hosted on the Google Container Registry.
- Deploy the Kubernetes artifacts relevant to the particular deployment pattern (pattern 1 in our case) to the GKE. The Kubernetes deployment artifacts should refer to the previously pushed docker images in Google Container Registry to spin up the container instances
Some points to consider during the deployment process are as follows:
- The WSO2 APIM container deployments currently run on a single VM instance, therefore there is no use in creating Kubernetes clusters with many VM machines.
- The built images should be tagged with the below form
- gcr.io/project_id/image_name:version (gcr.io is the GCR host name)
- The above image tag name should be used as the image name in the Kubernetes deployment artifacts
Open the exposed node ports in the nodes section since all ports in Google cloud nodes are set to be closed by default.
Comparison with AWS ECS
In the experiment, the R&D team tried to deploy WSO2 containers on both Kubernetes on GKE, and the AWS Elastic Container Service (ECS). In our opinion both have their own strengths and weaknesses, but overall the R&D team thought Kubernetes with GKE provides a more cohesive user experience.
Kubernetes has been implemented as an open source container orchestration framework with well abstracted architecture layers so that it can be run on any Cloud provider including Google Cloud, Microsoft Azure, OpenStack and also AWS. The Kubernetes pluggable architecture allows it to be mapped to the provider’s different network models, storages and container instances. AWS ECS is a proprietary service with dependencies to many other AWS specific services such as AWS EC2, VPCs, IAMs and AWS ELB. Its architecture is more vendor locked-in to AWS, whereas the Kubernetes architecture is vendor agnostic.
Both the Google Cloud Platform and AWS have command line tools for main Cloud functions (gcloud and aws cli). GKE and ECS also have CLI tools for specific container service commands (kubectl and ecs-cli). Kubectl which is the default CLI tool for Kubernetes is seamlessly integrated with gcloud tool so that you can call kubernetes commands via the gcloud tool.
A notable difference between the two tools, is the way they handle authentication. GCloud uses oauth to authenticate the client, and subsequent commands (e.g. docker push) are all wrapped with authentication. On the contrary, when developers want to push a docker image to the AWS registry, they need to call a docker login command prior to each docker push with the ssh key pair provided. Therefore from a developers’ point of view, the gcloud-cli from Google is more cohesive to use across different container commands such as docker and kubectl.
From a dev-ops point of view, the R&D team at Mitra Innovation think GKE provides an easy to setup container service, where container clusters can be launched by executing just one command (gcloud container cluster create). Everything from cluster instances and cluster orchestration management, networking and routing are all setup with default configurations.
In AWS however, creating a container cluster requires a step-by-step process going through different AWS service configurations such as: IAM roles; creating container instances; assigning the instances to ECS; adding a load balancer; setting up VPC and routing etc. This setup takes a few minutes to configure a functional container cluster. The R&D team noticed the container cluster startup time is much better with GKE compared to ECS in the experiment. The team experienced the same, when deleting clusters where GKE cluster deletion only takes one command, and 1-2 seconds to clean up the cluster setup. Whereas in ECS the team experienced delays in deleting the cluster resources.
Another difference between the two container management systems is the management dashboard for containers. GKE provides a Kubernetes dashboard UI which developers can connect from their localhost by simply issuing a command (kubectl proxy). This provides insightful details about the container cluster they are connected to. In a single dashboard developers can view the status of all Kubernetes artifacts such as deployments, replication controllers, services and pods etc. Any errors in the Kubernetes resources are very easy to detect and container logs are easy to navigate within the dashboard. With such intuitive features in the dashboard, the R&D team found it very useful to debug the deployments.
The following two screenshots show the GKE Kubernetes dashboard.
The following two screenshots show the GKE Kubernetes dashboard.
Figure 3 : Kubernetes Dashboards
When managing container clusters, the R&D team did not receive such a seamless experience with the AWS ECS management console. In AWS ECS developers need to navigate through different service consoles to view the status of different container artifacts. For example, developers will need to navigate to the EC2 console to view the status (CPU and memory consumption) of the running EC2 instances, which can be accessed from the GKE Kubernetes dashboard more easily.
The R&D team also believe that for a beginner it is much easier and intuitive to navigate through the Google Cloud Platform console to different services, when compared to the AWS service consoles.
Another handy feature the R&D team discovered in GCE for quick debugging, is the ability to ssh to your cluster VM instances via the browser itself. Developers do not need to install any new software or generate key pairs to ssh to the instances. Instead they can simply open a ssh session from the browser itself via the GCE console. This feature really helped the R&D team for quick debugging in the cluster instances.
In the container deployment exercise, the R&D overall prefer GKE due to its intuitive features in the Kubernetes dashboard and GCE management console, CLI tool kit and better performance.
Performance Test with GKE
As part of the experiment the R&D team also executed a performance test to validate the GKE WSO2 API Manager deployment, by configuring a backend REST service http://jsonplaceholder.typicode.com/ with API-Manager, and executing a JMeter load test with different numbers of concurrent users. The team executed the same load test with the direct backend service as well as with the configured API proxy in API-Manager to compare the performance results.
They used a single node for our deployment and the VM specification for the node is:
- N1-standard-1 : Standard machine type with 1 virtual CPU and 3.75 GB of memory.
The performance metrics used to evaluate the system are latency and throughput.
- Latency: the time taken to handle a request
- Throughput: the number of requests handled by the server for a specific time interval (e.g. per second)
The following table shows the test results.
Direct Backend REST service endpoint
API-Manager REST service endpoint
(Figure 4 : Latency chart for different concurrency levels)
When analysing the test results above, the R&D team saw that the latency increases at a higher rate with increasing numbers of concurrent users. The lower performance at higher concurrency levels could probably be due to the basic specification of the VM being used for the Kubernetes deployment. Further, WSO2 Kubernetes deployments are not yet supporting multiple node deployments, and the distributed API-Manager container setup is configured in a single node. With a higher grade VM instance and support for a load balanced multiple node setup of APIM-Manager container cluster, the R&D team feel they should be able to achieve a much better performance.
Higher performance of the deployment is achievable with GKE on Kubernetes because with the 1.2 release onwards, Kubernetes support 1000 node clusters with a massive scalability. Kubernetes claim their API responsiveness with a 99% percentile of less than one second, which means 99% of all Kubernetes API calls are handled in less than one second. In terms of Pod startup time, Kubernetes claim a 99% of pods and containers with pre-pulled images start within five seconds . Kubernetes provide these performance statistics based on a deployment on GKE.
Currently GKE runs on Kubernetes 1.7 so the R&D team expects a better Kubernetes performance. The team also expects the WSO2 API-Manager container deployment can be improved upon to perform with much higher performance and scalability when multiple node based deployment artifacts are supported from WSO2.
Security and Configuration Management in GKE
For security and user access control, Kubernetes has unique features such as Kubernetes Secretes to store sensitive information such as passwords, OAuth tokens and ssh keys. Kubernetes also has a feature named Config Maps to manage container configurations.
These Kubernetes artifacts help to decouple the security and configuration management from container images which would otherwise have to be specified and managed at the container images level. In ECS there is no direct replacements to Kubernetes secrets and config maps, which gives an added advantage to GKE users.
Furthermore, with Kubernetes 1.7, more security enhancements have been added with features like encrypted secrets, network policy for pod-to-pod communication, node authoriser to limit kubelet access and client / server TLS certificate rotation .
Based on the experience of Mitra Innovation’s R&D team, executing a WSO2 container deployment on the Google Container Engine is very efficient, provided the WSO2 docker and Kubernetes artifacts are configured correctly.
The production grade distributed container cluster deployments of WSO2 products can also be carried out on GKE with minimal effort, with the powerful tooling support provided by GKE to control and monitor Kubernetes clusters.
With improved performance, scalability and advanced security and cluster management features built into GKE, the Mitra Innovation R&D team believe the Google Cloud Platform will be a great choice for the deployment of enterprise container clusters for Internet scale containerised applications.
Dileepa Jayakody is a Technical Lead at Mitra Innovation. He is passionate about research and development and currently doing research in machine learning, natural language processing and artificial intelligence domains. He is also an open source software enthusiast and a committer in the Apache Software Foundation.
Abdullah Muhsin Specialises in Digital Marketing and Content Writing at Mitra Innovation. He holds a Bachelor’s Degree in Business Administration from Staffordshire University and is passionate about new and disruptive technologies. He has worked in areas such as Micro Finance, Product Marketing, e-Commerce, Content Marketing, Email Marketing and Social Media Marketing.