Lightweight Linux Distro for Native Kubernetes Web Server Clusters & SaaS Apps
CoreOS, the cloud software development company that manages the Container Linux distribution, recently released Tectonic version 1.64 (5/31/2017) which provides a quick and easy way to deploy a Kubernetes cluster for web/mobile app support on AWS or a self-hosted cloud server network. Core OS has over 100 open source software projects under company development on GitHub including etcd, flannel, rkt, Ignition, Clair, Matchbox, dex, and prometheus-operators. Tectonic is free to use for Kubernetes clusters of up to 10 nodes, after which a number of different enterprise, data center, & business licenses are available with professional technical support. Quay.io is another cloud SaaS product offered by CoreOS under proprietary license which helps DevOps teams manage container snapshots with Docker or rkt, Kubernetes clusters, Container Linux, & CoreOS Tectonic in production at scale. Quay.io includes container snapshot management features with integrated security, analytics, billing, monitoring, repository builds, automated backups, and version control for web server stack software with advanced GitHub, Bitbucket, or Gitlab support (Dockerfile Build Triggers).
CoreOS is one of the leading software companies developing new open source cloud solutions on Linux and the company was started in a Palo Alto, California garage by Alex Polvi (CEO), Brandon Philips (CTO), and Michael Marineau in 2013. Polvi had sold his company Cloudkick to Rackspace in 2010 and then formed Core OS with Philips (former SUSE Linux kernel developer & Rackspace cloud programmer) & Marineau (from Google). The original aim of the company was essentially to build a lightweight Linux distribution for containers that would replicate what Google had been operating in their data centers with “Borg“. At that time, Kubernetes had not yet been released publicly or open-sourced, but after 2014/15 when governance of the Kubernetes platform was taken over by the Linux Foundation, CoreOS has played a major role in the Cloud Native Computing Foundation (CNCF) in order to “help facilitate collaboration among developers and operators on common technologies.” Google has also subsequently launched the Google Container Engine for Kubernetes and Docker (GKE) with use of both standards becoming adopted across the competing cloud platforms at AWS, Azure, OCP, VMware, RHEL, etc. Almost every major cloud software company has a Kubernetes cluster web server solution in 2017 with widespread use of the etcd & rkt software by CoreOS (CloudFoundry, Fedora, ArchLinux). CoreOS Tectonic largely competes with GKE, RancherOS, Docker Swarm, or using Kubespray as self-hosted hybrid cloud option for bare-metal web server cluster management on independent (“vendor-agnostic”) data center hardware.
CoreOS Container Linux was developed to fit the needs outlined in the famous research paper published by Google in 2013 known as “The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines.” (Barroso, Clidaras, & Hölzle) This paper outlined the principles behind conceiving of data center infrastructure on the cloud model as opposed to the single server design. With commodity hardware and millions of servers in enterprise deployments, cloud network administrators need to plan for inevitable CPU, RAM, & other system hardware failure. Multiple backup copies and the veracity of each in instances of unit failure is a major part of RAID storage where the same principles apply to cloud servers in elastic clusters via the CAP Theorem & Paxos algorithm. CoreOS solved these problems with etcd allowing for upgrades of Kubernetes clusters through self-updating & repairing processes that push coordinated asynchronous updates to servers using reboot locks. This allows for the easier application of mass updates of security patches in a data center across huge numbers of rackmount servers at a time. etcd has better storage applications for saving the configuration, private keys, password changes, and other environment variables for web servers in case of hardware failure that has made it a major standard in the industry across platforms on open source licensing terms. etcd is used by Google (CFS, GFS, Big Table, Chubby, Spanner), Amazon (E2 Replicated Logs), Microsoft (Boxwood), Hadoop (ZooKeeper), and many other cloud software companies to solve the problem of Paxos in leader election, web server sharding, loss of private encryption keys, & other configuration variables with data center hardware failure. Docker & rkt containers isolate the web/mobile application code including user run input from the hardware operating system for better overall isolated data security in the cloud.
Demo Video: Brandon Philips (Linux Conference 2015 – Auckland, NZ)
CoreOS: An Introduction – “The architectural patterns of a large scale platform are changing. Dedicated VMs and configuration management tools are being replaced by containerization and new service management technologies like systemd. This presentation will be giving an overview of their key technologies, including etcd, fleet, and docker. Come and learn how to use these new technologies to build performant, reliable, large distributed systems.” Learn More About CoreOS, Kubernetes, & Container Linux
It is important to understand the differences between application containers and system containers that function as a VM, such as VPS operating under OpenVZ, KVM, Xen, Parallels, VMware, etc. platforms. Containers provide an alternative virtualization hypervisor that can be used to create object-oriented solutions at various layers of a data center through the use of isolated partitions. VPS/VMs can be installed and scaled in elastic clusters within containers and vice versa. The combination of container configurations with the wide variety of other virtualization platforms available on the market can be used to create complex solutions for enterprise IT, SaaS/PaaS/IaaS vendors, mobile software applications, and web hosting companies. Most container solutions do not include the dedicated SSH & networking tools characteristic of VPS or VM instances, but this is part of what leads the application containers used in production to be based upon the most lightweight Linux distros available. The current practice in DevOps is to include the full Linux stack for the top layer of code in production bundled with all of the specific server extensions required by web & mobile apps through build packages or binaries. The unique way that Docker and rkt containers utilize the Unix kernel and Unix API leads to a combination of isolated & global namespaces which can be used to map configuration variables inside/outside of containers for better isolated resource consumption on web server hardware in data centers.
rkt with CoreOS, Fedora, ArchLinux, & NixOS - “The core execution unit of rkt is the pod, a collection of one or more applications executing in a shared context (rkt’s pods are synonymous with the concept in the Kubernetes orchestration system). rkt allows users to apply different configurations (like isolation parameters) at both pod-level and at the more granular per-application level. rkt’s architecture means that each pod executes directly in the classic Unix process model (i.e. there is no central daemon), in a self-contained, isolated environment. rkt implements a modern, open, standard container format, the App Container (appc) spec, but can also execute other container images, like those created with Docker.” Learn More about the rkt Container Engine
Build & Deploy Containers at Scale: “Use Quay.io to automate your container builds, with integration to GitHub, Bitbucket, and more… Quay continually scans your containers for vulnerabilities, giving you complete visibility into known issues and how to fix them.” Learn More About Quay.io
Demo Video: Kelsey Hightower (CoreOS Workshop 2015 – Geekdom, SF)
Container Orchestration using CoreOS and Kubernetes – “This hands-on workshop… will teach modern practices for container orchestration, and show examples of how components work together to manage a cluster of Linux containers. With its ability to power infrastructure in the cloud or on bare-metal, the session will use Kubernetes with CoreOS as an example showing attendees how to deploy and manage a multi-tier web application.” Learn More About CoreOS & Kubernetes
According to Kelsey Hightower, a well-known Kubernetes evangelist who has worked at Google & CoreOS, Kubernetes does not compete with CloudFoundry, Heroku, or OpenDeis as a “complete path” for DevOps and cloud hosting solutions in production. Rather, it is a virtualization framework primarily specializing in container management, scheduling, and service discovery that many competing PaaS/SaaS companies can build new open source solutions around with greater cross-platform interoperability and data portability between cloud hosting providers. Kubernetes service discovery primarily relates to monitoring and regulating the number of server nodes in an elastic cluster and using etcd for synchronization between copies with state change updates. Unlike Docker, which uses shared IP addresses for containers, Kubernetes provides a unique IP address to every pod while maintaining communication between all server nodes in a network. The unique IP addresses decouple the web/mobile application code from the underlying container operating system, allowing any programming language extensions, web server platform, or customized stack to be run in production that is integrated with continuous publication or version control tools in DevOps. Other alternatives to Docker & rkt containers are:
Job scheduling, service discovery, & cluster management are the main aspects of data center architecture that containers improve for web server networks. Before advances in CoreOS, RancherOS, and other Container Linux solutions, programmers in the first stage of cloud hosting were using Chef/Puppet scripts to share build snapshots for automated container installs at scale. Containers make it easier to support multi-tenant Python, Java, Nginx, MySQL, OpenSSL, etc. apps in runtime. Containers also open up greater flexibility for developers in maintaining active code between extension platform updates to keep web servers patched while insuring that rolling security updates don’t break production code. Because this is proverbially difficult in practice, the principle of Container Linux is to keep the OS simple. By only maintaining kernel security and never breaking the kernel API in Linux, containers can run in user space with all extensions & tools in isolation with greater overall security in multi-tenant network management.
CoreOS Container Linux – “Kubernetes is powerful container management software inspired by Google’s operational experience with containers. Essential features like service discovery, automatic load-balancing, container replication and more are built in. Plus, it’s all powered via an HTTP API… Linux containers provide numerous benefits for both application developers and operations teams. Container Linux only allows for software to be installed as containers, which is an important abstraction layer between the operating system and the applications (and dependencies) running on top.” Learn More About Container Linux
Concept Video: etcd v 3 – Brandon Philips (CTO / Co-founder CoreOS)
Raft: Distributed Consensus Protocol – “etcd is written in Go which has excellent cross-platform support, small binaries and a great community behind it. Communication between etcd machines is handled via the Raft consensus algorithm.” Learn More About the Raft DCP
The Kubernetes system allows for different components to be used in auto-scaling web server node cluster levels, for example using Terraform, Fleet (similar to System D), Mesos, CoreOS Tectonic, Docker Swarm, etc. where etcd is the vital component (similar to Locksmith) which coordinates the state changes from user variable input changes across elastic clusters with many simultaneous run processes in isolation. Puppet & Chef scripting can also be implemented. In this manner, systems administrators can establish parameters such as running 100 instances of a SaaS application (website or mobile app) with 1 GB of RAM and the Kubernetes scheduler will maintain the levels across system resources like a thermometer scaling up servers and shutting down nodes as required by the network state. Because of this it is not recommended to use large builds with many stored static files in the replicated container-based web servers, leading to more API-driven cloud storage functionality all of the elastic node web servers can draw upon as required by https traffic requests. This also enables service discovery mechanisms (SkyDNS, discoverd, Confd, magic proxies, etc.) to relate user geolocation from browser requests on public networks for better load balancing, as well as building new solutions for internal reverse proxy file caching in data centers for better web hosting performance.
Dockerfile Build in the Cloud – “Like to use Dockerfiles to build your images? Simply upload your Dockerfile (and any additional files it needs) and we’ll build your Dockerfile into an image and push it to your repository. If you store your Dockerfile in GitHub, Bitbucket, or Gitlab, add a Build Trigger to your repository and we’ll start a Dockerfile build for every change you make… Share any repository with as many (or as few) users as you choose. Need a repository only for your team? Easily share with your team members… Want to share with the world? Make your repository fully public.” Learn More About Quay.io
etcd & Kubernetes Cluster States – “etcd is a distributed key value store that provides a reliable way to store data across a cluster of machines. It’s open-source and available on GitHub. etcd gracefully handles leader elections during network partitions and will tolerate machine failure, including the leader.” Learn More About CoreOS & Kubernetes (etcd)
If you want to get the best cloud hosting, click here and get the suitable one.
