Cloud computing is the cornerstone of the digital economy. Some of the most successful companies are built on cloud-centered applications. Paypal, Google, Netflix, Facebook, Amazon, Twitter, LinkedIn have all taken advantage of open source components to deploy products and services in a fast way while doing so in a cost-effective way. With cloud deployment comes aspects of distributed, micro-services architecture that leverage on DevOps and container technology that significantly increases the overall agility and maintainability of applications to gain competitive advantage.

Companies across industries use the cloud, either as private, public or hybrid in their operations. These range from banking, healthcare, governments, finance and insurance. 41 per cent of all enterprise workloads are currently channeled through the cloud, public or private. This number is expected to rise to 60 per cent by mid 2018. Currently, 95 per cent of companies are experimenting on the cloud. As such, cloud strategies can define the success or failure of a company’s competitive edge. Inadvertently, cloud strategies are built on Linux and open source software.

Adoption of the cloud is more prevalent among small and medium businesses (SMB), especially with public cloud. The share of enterprise workloads moved to the public cloud is expected to triple over the next 5 years. This shift will be facilitated by the reducing costs of cloud services and an increase in security awareness and technologies in the cloud platform. In any case, security is a top concern for companies that wish to migrate to the cloud platform. As companies gain more experience in deployment and use of cloud, security concerns become less of a hindrance.

With the continued growth of cloud, open source systems and technologies will continue to lead the innovation and the foundation for new frontiers and markets. A case in point is the evolution of data center infrastructure. Previously, finite hosted resources defined the data center. This has since shifted to the more flexible elastic compute fabric.

Cloud by Numbers

As of December 2016, the public cloud revenues jumped 172 percent to $208.6 Billion up from $178 Billion the previous year. This has largely been driven by cloud application services popularly known as Software as a Service (SaaS). Infrastructure as a Service (IaaS) came second with a growth projection of 42.8 percent in 2017. What is driving this immense growth? By the open nature of these technologies, they provide open APIs (Application Programming Interface) and framework that is consistent enough to allow users to duplicate their infrastructure from one cloud to another without a significant amount of customization. This, I bet, is largely attributed to the fact that the base of most of these technologies run a flavor of Linux operating systems.

Emerging Trends

Initially, applications would be written on premises then pushed to the public cloud. This has effectively seen changes in that applications are now written and deployed directly to the cloud. These are loosely referred to as cloud native applications.

Cloud Native Applications

Refers to applications that have been designed to run on modern distributed systems with the ability to scale to thousands of nodes.

Containers

Containers enables applications to move from development environment to production without the need for reconfiguration. The deployment is either from registries or through continuous development and deployment (usually referred to as DevOps) using tools like Ansible, Chef and Puppet. They also rely on schedulers (popularly referred to as orchestration tools) like Docker Swarm, Kubernetes, Mesos to offer coordination of containers across machines and nodes.

Unikernels

These are stripped down operating systems that hosts a single application to run in a virtual machine. This technology bears similarity to containers with a little variation.

Open source and cloud technologies enable companies to harness the power of shared R&D. in this case, the collaboration enables companies build systems to the scale that would be either too expensive or technologically not feasible for them to accomplish on their own. By collaborating on open source projects, companies benefit from;

  • Improved code quality
  • Increased security with the ability to find and fix vulnerabilities
  • Visibility into inner layers of the infrastructure
  • Code access to customize features
  • Lower costs through shared development

The profile stack of open cloud may be categorized as;

  • Infrastructure as a Service
    • Apache Cloudstack: software designed to deploy and manage large networks of virtual machines, as a highly available, highly scalable Infrastructure as a Service (IaaS) cloud computing platform
    • HPE Helion Eucalyptus: an open solution for building private clouds that are compatible with Amazon Web Services (AWS)
    • OpenNebula: software to manage virtualized data centers for private, public, and hybrid IaaS clouds
    • Openstack: open source software for creating private and public clouds
  • Platform as a Service
    • Apache Stratos: is a highly-extensible PaaS framework that helps run Apache Tomcat, PHP, and MySQL applications and can be extended to support many more environments on all major cloud infrastructures
    • Cloud Foundry: open source cloud application platform that provides a choice of clouds, developer frameworks, and application services
    • Deis Workflow: source Deis PaaS that makes it easy to deploy and manage applications on Kubernetes
    • Flynn: open source PaaS for running applications in production
    • Heroku: cloud platform that lets companies build, deliver, monitor and scale applications
    • OpenShift: Red Hat’s PaaS that allows developers to quickly develop, host, and scale applications in a cloud environment
  • Virtualization, Containers and Cloud Operating Systems
    • Micro or Minimalist OSs
      • Project Atomic: Red Hat’s umbrella for many open source infrastructure projects to deploy and scale containerized applications
      • CoreOS: lightweight Linux operating system designed for clustered deployments providing automation, security, and scalability for containerized applications
      • Photon OS: minimal Linux operating system for cloud-native apps optimized for VMware’s platforms
      • RancherOS: minimalist Linux distribution for running Docker containers
    • Virtualization
      • KVM: (Kernel-based Virtual Machine) is a full virtualization solution for Linux on x86 hardware containing virtualization extensions (Intel VT or AMD-V)
      • LXC: Linux Containers (LXC) are lightweight virtual machines enabled by functions within the Linux kernel, including cgroups, namespaces and security modules
      • LXD: Canonical’s container hypervisor and a new user experience for LXC
      • Xen Project: a Linux Foundation project, develops virtualization technologies for a number of different commercial and open source applications including server virtualization, Infrastructure as a Service (IaaS), desktop virtualization, security applications, embedded and hardware appliances on x86 and ARM CPU architectures, and supports a wide range of guest operating systems
    • Management and Automation
      • Apache Aurora: Mesos framework that runs applications and services across a shared pool of machines, and is responsible for keeping them running, forever
      • Apache Mesos: an Apache Software Foundation project, abstracts CPU, memory, storage, and other compute resources away from machines (physical or virtual), to build and run fault tolerant and elastic distributed systems
      • Cloud Foundry Diego: next-generation container management runtime system
      • Docker Engine: Docker’s open source containerization technology combined with a workflow for building and containerizing applications
      • Docker Swarm (Machine and Compose): native clustering for Docker. It turns a pool of Docker hosts into a single, virtual Docker host
      • Kontena: platform to deploy, manage, scale, and monitor containerized applications and microservices
      • Kubernetes: a Cloud-Native Computing Foundation project at The Linux Foundation, is a container cluster manager for automating deployment, scaling, and management of containerized applications, developed and open sourced by Google
      • ManageIQ: open source project that powers Red Hat CloudForms
      • Ovirt: virtualization management application. It can manage hardware nodes, storage and network resources, and deploy and monitor virtual machines running in a data center
      • Skippbox (Kmachine and Kompose): a Kubernetes tool from Skippbox that helps developers create a standalone, single-node Kubernetes instance in public clouds, similar to Docker machine
      • Sysdig: distributed system monitoring and troubleshooting tool for Linux with native container support
      • Weaveworks: a set of tools to cluster, view, and deploy micro-services and cloud-native applications across local and external networks
      • Wercker: an automation platform to build and deploy containers for multi-tiered, cloud-native applications
    • Unikernels
      • ClickOS: NEC’s high-performance, virtualized software middlebox platform for network function virtualization (NFV) built on top of MiniOS/ MirageOS
      • Clive: an operating system written in Go and designed to work in distributed and cloud computing environments
      • HalVM: Haskell Lightweight Virtual Machine (HaLVM) is a port of the Glasgow Haskell Compiler toolsuite that enables developers to write high-level, lightweight virtual machines that can run directly on the Xen hypervisor
      • IncludeOS: a Unikernel operating system for C++ services running in the cloud
      • Ling: an Erlang platform for building super-scalable clouds that runs directly on top of the Xen hypervisor
      • MirageOS: a library operating system incubating under the Xen Project at The Linux Foundation
      • OSV: open source operating system from Cloudius Systems designed for the cloud
      • RumpRun: a production-ready unikernel that uses the drivers offered by rump kernels, adds a libc and an application environment on top, and provides a toolchain with which to build existing POSIX-y applications as Rumprun Unikernels
      • Runtime.js: open source library operating system (unikernel) for the cloud that runs JavaScript, can be bundled up with an application and deployed as a lightweight and immutable VM image
      • Unik: EMC’s tool for compiling application sources into unikernels (lightweight bootable disk images) rather than binaries
    • DevOps CI/CD
      • Complete CI/CD Cycle
        • Concourse: a scalable continuous integration pipeline tool sponsored by Pivotal. It was built for the needs of Cloud Foundry projects to test and deploy on different architectures and platforms and remain compatible with previous versions
        • Drone: open source continuous integration platform built on container technology for GitHub and Bitbucket
        • HygieIA: a configurable DevOps dashboard to visualize near real-time status of the entire software delivery pipeline, built and released as open source by Capital One
        • Jenkins: open source automation server. It provides hundreds of plugins to support building, deploying, and automating any project across multiple platforms. It can be used as a simple CI server or turned into a continuous delivery hub
        • Shippable: a continuous deployment platform built on Docker that provides end-to-end deployment pipelines for automation and integration
        • Travis-CI: a hosted continuous integration and deployment system that integrates with GitHub
      • Configuration Management
        • Ansible: Red Hat’s open source IT automation engine for cloud provisioning, configuration management, application deployment, intra-service orchestration, and other IT needs on multi-tier architectures
        • Chef: a configuration management tool to automate infrastructure. It manages servers in the cloud, on-premises, or in a hybrid environment
        • Puppet: open source server automation tool for configuration and management. It works on Linux, Unix, and Windows systems
        • Salt: orchestration and configuration management software to manage infrastructure and applications at scale
      • Logging and Monitoring
        • FluentD: open source data collector for unified logging layer, sponsored by Treasure Data
        • Heapster: a container cluster monitoring and performance analysis tool in Kubernetes. It supports Kubernetes and CoreOS natively and can be adapted to run on OpenShift
        • LogTash: Elastic’s open source data pipeline to help process logs and other event data from a variety of systems
        • Prometheus: open source systems monitoring and alerting toolkit, originally built at SoundCloud and now a Cloud-Native Computing Foundation project at The Linux Foundation
        • Weave Scope: Weaveworks’ open source tool to monitor distributed applications and their containers in real time. It integrates with Kubernetes and AWS ECS
      • Software defined Networking
        • ONOS: Open Network Operating System (ONOS), a Linux Foundation project, is a software-defined networking OS for service providers that has scalability, high availability, high performance and abstractions to create apps and services
        • OpenConTrail: Juniper Networks’ open source network virtualization platform for the cloud. It provides all the necessary components for network virtualization: SDN controller, virtual router, analytics engine, and published northbound APIs
        • OpenDayLight: OpenDaylight Foundation project at The Linux Foundation, is a programmable, software-defined networking platform for service providers and enterprises
        • Open Vswitch: a Linux Foundation project, is a production-quality, multilayer virtual switch. It’s designed for massive network automation through programmatic extension, while still supporting standard management interfaces and protocols including NetFlow, sFlow, IPFIX, RSPAN, CLI, LACP, and 802.1ag
        • OPNFV: Open Platform for Network Functions Virtualization (OPNFV), a Linux Foundation project, is a reference NFV platform for enterprise and service provider networks
        • Networking for Containers
          • Flannel: an etcd-backed overlay network for containers from CoreOS. It was designed for Kubernetes, but is a generic overlay network that can be used as an alternative to existing software-defined networking solutions
          • Libnetwork: Docker’s project to create a multi-platform library for networking containers
          • Project Calico: Tigera’s data center networking software that uses a pure Layer 3 approach. It integrates with cloud orchestration systems (such as OpenStack) to enable secure IP communication between virtual machines, containers, or bare metal workloads
          • Weave Net: Weaveworks’ networking software for container applications across data centers and public clouds. It supports peer-to-peer encryption, traffic isolation, and segmentation for building secure control plane networks, running in production environments, or for joining container deployments in public clouds over the internet
        • Software Defined Storage
          • Apache Cassandra: an Apache Software Foundation project, is a scalable, high-availability database for mission-critical applications
          • Ceph: Red Hat’s distributed, highly scalable block, object, and file storage platform for enterprises deploying public or private clouds. It’s commonly used with OpenStack
          • CouchDB: Apache CouchDB, an Apache Software Foundation project, is a single-node or clustered database management system. It provides a RESTful HTTP API for reading and updating database documents
          • Docker Volume Plugins: Docker Engine volume plugins enable Engine deployments to be integrated with external storage systems and enable data volumes to persist beyond the lifetime of a single Engine host
          • GlusterFS: Red Hat’s scalable network filesystem and data management platform. It can deploy on-premise, in private, public, or hybrid clouds, and in Linux containers for media streaming, data analysis, and other data- and bandwidth-intensive tasks
          • MongoDB: MongoDB is a high performance document database designed for ease of development and scaling
          • Nexenta: a scalable, unified software-defined file and block storage service that includes data management functionality
          • Redis: an in-memory data structure store, used as database, cache and message broker. It supports multiple data structures and has built-in replication, Lua scripting, LRU eviction, transactions and different levels of on-disk persistence
          • Riak CS: object storage software built on top of Riak KV, Basho’s distributed database. It provides distributed cloud storage at any scale, and can be used to build public or private cloud architectures or as storage infrastructure for heavy-duty applications and services
          • Swift: OpenStack’s object storage system designed to store and retrieve unstructured data with a simple API. It’s built for scale and optimized for durability, availability, and concurrency across the entire data set

Image courtesy of http://mastervoice.it