Draft: A Command Line Utility for the Auto-Production & Orchestration of Docker Containers
In April of 2017, Microsoft acquired Deis, a small start-up company specializing in container services for data center management, in order to integrate the Workflow, Helm, and Steward utilities for Kubernetes into the Azure Cloud platform. Workflow is a continuous integration suite for DevOps teams that enables Agile/Scrum workstation collaboration through distributed network teams with Docker, Kubernetes, and custom web server stack extensions for common programming languages like PHP, Python, Java, ASP.NET, Perl, Go, & Ruby on Rails. Helm charts allow for the easy management of build snapshots for Docker and Kubernetes containers at scale over millions of server nodes in the cloud. Steward is part of the Kubernetes incubator for Service Catalog development with Red Hat, IBM, & the Microsoft Deis team working together on the project. This month Microsoft announced the availability of Draft on the Azure Container Service, a new command line utility that speeds up the deployment and orchestration of Docker containers in Kubernetes clusters through the auto-installation of required stack software. Kubernetes can also be used with Apache Mesos, Rancher, and Docker Swarm on the Azure Cloud Platform to compete with similar product offerings from AWS, Google Cloud, Rackspace, & IBM.
Containers – Custom Web Server Stack Deployments with Docker, Kubernetes, & Microsoft Azure
The Azure Service Fabric represents the cloud model of a multi-layered Software Designed Data Center (SDDC) developed by Microsoft to meet the needs of enterprise in advanced database, website, and mobile application support at scale in production environments. The Azure Software Stack can be customized to meet the needs of any size business, government, or non-profit organization on the web. While VMware still leads in private cloud orchestration for enterprise, AWS, Google, Red Hat, IBM, Oracle, Mirantis, Rancher Labs, and other new start-up companies compete for market share in the data center operating system sector with Apache Mesos, OpenStack, & CloudStack the most popular open source platforms with similar functionality of managing elastic cloud networks of web servers at scale.
A good example of a DevOps deployment for programming web and mobile software applications with continual security updates, version upgrades, and bug fixes pushed to live production using Microsoft Azure Cloud as a web publishing solution with containers on this model would include:
- Azure Container Service – scalable cloud deployment & container workload management
- Docker – the world’s leading software container platform for Linux & Windows
- Kubernetes – automated deployment, elastic scaling, & management of containerized apps
- Workflow – DevOps publishing, version control, online collaboration, & continual integration
- Helm – Kubernetes package manager that uses Charts as buildmaps for Docker containers
- Draft – A tool for developers to create cloud-native applications on Kubernetes
One main advantage of container-based web publishing is that there is increased data portability between cloud providers that can be used over time to procure hosting services on the most cost efficient basis. Many web hosting and data center companies are competing in this sector as large businesses turn to cloud infrastructure in order to build long-term IT solutions with different vendor companies, as well as mixing available PaaS/SaaS/IaaS products with custom-coded or originally developed software applications that maintain support for a company’s business operations on the web. Docker containers are already used widely across the Fortune 500 and the continual development of Kubernetes is governed by the Cloud Native Computing Foundation with broad industry support across competing platform standards. The Azure Cloud expansion with Deis, Draft, Helm, & Workflow tools are all part of a larger industry trend towards greater container adoption in enterprise that is leading Docker & Kubernetes to be valued as top among the fastest growing companies in the world.
Introduction to Containers – Sheng Liang, CEO of Rancher Labs
19th Cloud Expo (2016) – “In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker containers gain prominence. He explored these challenges and how to address them, while considering how containers will influence the direction of cloud computing.” Learn More About Rancher Labs.
Draft Buildpacks with Docker & Kubernetes – ‘When developers run “draft create” the tool detects the application language and writes out a simple Dockerfile and a Kubernetes Helm chart into the source tree. Language detection uses configurable Draft “packs” that can support any language, framework, or runtime environment. By default, Draft ships with support for languages including Node.js, Go, Java, Python, PHP, and Ruby. You can customize Draft to streamline the development of any application or service that can run on Kubernetes. Draft packs are just a simple detection script, a Dockerfile, and a Helm chart. This simple yet flexible user experience for developers is inspired by PaaS systems like Deis and Cloud Foundry which support the concept of buildpacks.’ Learn More about Draft.
Docker & Rancher OS: Overview of the Container Industry & Data Center Management Software
For more background on the development and use of containers in data center operations in contrast to Virtual Machines (VMs), Sheng Liang’s keynote at 19th Cloud Expo in 2016 provides an excellent overview of the sector. Liang is CEO of Rancher Labs and discusses how he founded the company in 2008 when AWS was only 2 years old in order to emulate the Amazon service by providing similar tools to enterprise that enabled corporations to build their own private clouds with elastic scaling capabilities. Originally CloudStack & OpenStack were used as the main solutions to build private, public, & hybrid cloud networks with elastic scalability features. At this time, AWS did not have the 50+ different SaaS/ DaaS/XaaS products available now, where the major telecoms, banks, corporate IT, & finance groups all quickly sought to built their own private clouds to compete with Amazon independently. Cloud hosting has subsequently become a highly concentrated, “winner take all” market with AWS, Google Cloud, Microsoft Azure, IBM, Oracle, Rackspace, & VMware all struggling to maintain major enterprise market share, according to Liang. Containers have emerged with Docker as a major alternative to VMs in Fortune 500 IT teams, primarily due to their reduced overhead size in web server resource consumption.
Containers and VMs were first popularized by VMware, AWS, Docker, Parallels, SWsoft, and other companies to the extent that Docker first surpassed 100 million downloads in 2015. Docker’s growth has subsequently led the expansion of the OS market for containers: Core OS, Photon OS, Windows for Containers, Rancher OS, etc. all integrate with Docker & Kubernetes to build PaaS/IaaS options for enterprise corporations, webhosts, & SMEs. Container operating systems are designed to run natively on cloud infrastructure anywhere, which can include installs on AWS, private networks, Google Cloud, Digital Ocean, CloudStack servers, OpenStack deployments, Bare Metal hardware, VMware clusters, etc. Docker creates individual containers whereas Kubernetes is used to deploy and manage elastic web server clusters using Docker Swarm, Rancher, or Apache Mesos for container orchestration in advanced data center management & cloud web hosting services. In comparing containers to VMs, the overhead resource usage of containers is much less as VMs must include the complete guest OS plus hypervisor in every installation, whereas containers leverage the Linux kernel to isolate processes with near bare-metal performance. However, VMs usually offer better process isolation in untrusted environments over containers where unknown code may be run on the platform in production.
Core OS & Rancher Labs led the container movement that radically shrunk the size of Linux web server distros when compared to Red Hat & Ubuntu. Core OS & Rancher OS are only around 42 MB to install, where the Linux kernel in total only requires 4.2 MB and the system drivers are around 3x the size. The Docker Engine required to run containers including all basic file systems & tools is only 8.4 MB, with the smaller file size leading to better security, fewer upgrade patches, & more platform stability over other web server software solutions. Other OS choices require multiple GBs of disk space to install, and take much longer to provision with additional web server stack software layers. Draft not only automates this process with Helm using the command line to auto-provision Kubernetes clusters at scale, it does so in a manner that can be scripted by other software applications to further data center automation and advanced network administration. Neutron, Cinder, CNI, Flex Volume, Lib Network, Docker Volume Plugin, Rocket, etc. all operate similarly across the data center operating system software utilities available with Docker Swarm, Apache Mesos, vSphere, & OpenStack. Whereas OpenStack & CloudStack have both primarily centered on data center virtualization through VM network management at scale, container systems like Docker provide a different level of abstraction through shared kernel virtualization that can enable common problem solving in new ways for web/mobile app programmers and systems administrators.
Containers as infrastructure: Getting started with Azure Container Instances
AzureCon 2015 – “Learn the core concepts of containers on Azure. Find out what containers are and what Docker is. We’ll look at both Linux containers and Windows Server containers (including Hyper-V containers). Finally, we’ll take a brief look at Azure Container Service.” Learn more about Azure Containers.
Azure Service Fabric – “One could say that ASF is based on the concept of architectures based on microservices because it takes all the amount of features and services available and segregates them in completely individual features and services. We can find this new way of deploying everywhere, whether in the Azure itself, as in a private cloud or even in other clouds…” Learn more about Microsoft Azure.
Data Center Management Software Platforms – Building Public, Private, & Hybrid Cloud Solutions
The leading open source alternatives to AWS, VMware, Google Cloud, IBM Bluemix, & Microsoft Azure software for data center management and private/public/hybrid cloud orchestration currently are: OpenStack, CloudStack, Mesosphere, & Cloud Foundry. The most popular container-specific Linux operating system distributions for elastic cloud web server network administration currently are: Photon OS, Rancher OS, & Core OS. Cloud hosting providers offering VPS plans and PaaS/SaaS products often use Mirantis Cloud Platform, SolusVM, Parallels (GoDaddy/MediaTemple), & Virtuozzo/OpenVZ for VM management, or use containers to provide managed WordPress hosting with custom stack software installed on AWS, Google Cloud, Azure, etc. hardware at commodity rates. The combination of these data center management tools, network software utilities, programming languages, & web development standards in hybrid cloud deployments is used in many new SaaS/PaaS products for web hosting.
Ross Kukulinski addressed the state of the container ecosystems in a recent seminar at the 2017 DataLayer Conference where he discussed the use of “Containers, Storage, & Databases” in DevOps in terms of three core proficiencies: packaging, performance, & efficiency. Packaging refers to the use of application binaries only in container systems to allow for multiple versions of programming language extensions to be installed and active in stack configurations for continuous deployment, staging, production, & easy debugging of web/mobile apps. Performance in a proprietary data center can mean near bare metal speeds with the “in-cloud” dynamic web servers providing the ability for companies to scale faster in development with SaaS/PaaS products. Through this it is possible for administrators to scale web/mobile applications independently of database formats with different levels of abstractions to increase the efficient use of hardware resources at scale.
It is important to consider the effects of imperative vs. declarative runtime processes and monitoring in Kubernetes/Mesos to appreciate the power of elastic networks & the need for data synchronization through container orchestration in distributed systems at each higher level of abstraction. Kubernetes originally developed as an attempt to open source the Google “Borg” which was a container framework developed over 15 years by Google internally for data center management requirements and cloud app support (Gmail, Search, Maps, etc.). The Kubernetes open source software for managing the deployment, development, & operation of containerized systems is currently being governed by the Cloud Native Computing Foundation. Higher level abstraction layers enable portability for business data across the competing vendor platforms of public/private clouds with extensibility by building on the Kubernetes primitives to solve more complex problems. In the Kubernetes Master, everything is an API-driven process where the Scheduler even understands the system infrastructure resources available to a cluster and the web server nodes run code that is further abstracted in order to deploy applications at scale with load balancing on traffic for the most optimal overall hardware performance in web operations. This also enables rolling software version updates in continual delivery systems for live production websites & mobile apps with large traffic bases or active social communities.
Kubernetes stateful sets allow for more easy moving of data across server hardware resources in a data center as well as applying backup & restore maps. There is a further need for custom programmed integration of new utilities in the ecosystem that is accomplished by third-party developers and start-up companies internationally. Kubernetes operators (observe –> analyze —> act loop for systems at scale // applied to databases) integrate with ElasticSearch, Prometheus, ETCD., and other database frameworks as high level abstractions, but MySQL & PostgreSQL are not overly recommended for Kubernetes according to Kukulinski. Redis is difficult to manage in Kubernetes clusters with a lot of custom coding required for the integration, whereas influxdb, ElasticSearch, MongoDB, CockroachDB, Memcached, rethinkDB, & Kafka are the most recommended database options for new apps building on the advantages of Docker containers and Kubernetes, due primarily to the better ability of these database frameworks to cache, render, and scale under heavy user traffic. Ansible, Puppet, & Bash scripts can be easily replaced with more simple YAML/JSON scripting in Kubernetes. In order to synchronize remote cloud storage versions or between multiple copies of data located in server nodes with local storage requirements, advanced data backup facilities using multiple redundancies to protect user data are required. Rook is another new app currently being developed to deploy Ceph on Kubernetes clusters with synchronization under traffic, duress, and at elastic scale. (See also: Istio)
Video: “The State Of State In Containers” – Ross Kukulinski (Compose, Inc.)
DataLayer Conference 2017 – “Application container technologies like Docker and Kubernetes have revolutionized the way in which developers can deploy and manage stateless applications. Containers are quick to launch and make efficient use of underlying compute resources. Orchestration engines like Kubernetes simplify the deployment, lifecycle, and scalability of applications.” Learn more about Container Systems.
Rancher Container Management Platform – “Rancher Labs develops open source software that makes it easy to deploy and manage Docker containers and Kubernetes in production on any infrastructure.“ Learn more about Rancher.
Container Orchestration Resources & Utilities – Docker Hub, Rancher Sandbox, & Azure Marketplace
Additional resources recommended for those looking to learn more about Docker containers, Kubernetes clusters, and the Rancher OS are:
- Docker Hub – Dev-test pipeline automation with over 100,000 free cloud applications
- Rancher Sandbox – install Rancher to AWS or another cloud host for container management
- Azure Marketplace – deploy Docker container apps & custom stack software from snapshots
- Linux Foundation – how to run Kubernetes for testing on a local machine using Minikube
Developers looking for new articles, software reviews, and free books with in depth overviews of the installation and deployment process for Docker, Kubernetes, OpenStack, & Apache Mesos can view:
- Mirantis – Introduction to Kubernetes (2017)
- Kinsta – Google Cloud Hosting (2017)
- Linux.com – The Mobi Project Pivot (2017)
- StorageReview – OpenStack Summit (2017)
- Citrix – App Layering & CloudHouse (2017)
- Rancher Labs – Deploying Scaling Kubernetes (2017)
- VMware – Introducing Photon OS (2015)
- Google – Cloud Launcher Container Support (2017)
- Core OS – Rocket Documentation (2017)
- Microsoft – Kubernetes on Azure (2017)
- VMware – Data Center Virtualization (2017)
- Mirantis – Introduction to Salt and SaltStack (2017)
Another option to consider is AWS Certification – “(to) validate technical knowledge with an industry-recognized credential.” – Learn More about AWS Certifications.
Docker with Azure Container Service – “Marathon is a cluster-wide init and control system for services in cgroups–or, in the case of Azure Container Service, Docker-formatted containers. Marathon provides a web UI from which you can deploy your applications.” PDF Download: Microsoft Azure Infographic.