Kubernetes Operators and Controllers

• Introduction
• operatorhub.io
• Red Hat Container Community of Practice Operators
• Operator Capability Levels
• Cluster Addons
• K8Spin Operator. Kubernetes multi-tenant operator
• K8s KPIs with Kuberhealthy Operator
• Writing Kubernetes Operators and Controllers
• Tweets
• Videos

Introduction

• kruschecompany.com: What is a Kubernetes Operator and Where it Can be Used?
• kruschecompany.com: Prometheus Operator – Installing Prometheus Monitoring Within The Kubernetes Environment
• redhat.com: Kubernetes operators - Embedding operational expertise side by side with containerized applications
• hashicorp.com: Creating Workspaces with the HashiCorp Terraform Operator for Kubernetes
• banzaicloud.com: Kafka rolling upgrade made easy with Supertubes
• devops.com: Day 2 for the Operator Ecosystem 🌟
  • KUDO: The Kubernetes Universal Declarative Operator 🌟 KUDO is a toolkit that makes it easy to build Kubernetes Operators, in most cases just using YAML.
• itnext.io: Operator Lifecycle Manager (OLM) 🌟
• kube-fluentd-operator 🌟 is a sane, no-brainer Kubernetes+Helm distribution of Fluentd with batteries included, config validation, no needs to restart, with sensible defaults and best practices built-in. You can use Kubernetes labels to filter/route logs!
• Domain-harvester is an operator that collects domains from all Ingress resources in a Kubernetes cluster and provides its expiry information
• Cass Operator The DataStax Kubernetes Operator for Apache Cassandra®
• Kotal operator is cloud agnostic blockchain deployer that make it easy to deploy highly available, self-managing, self-healing blockchain infrastructure (networks, nodes, storage clusters …) on any cloud.
• Speculator: Redis Operator A Golang based redis operator that will make/oversee Redis standalone/cluster mode setup on top of the Kubernetes. It can create a redis cluster setup with best practices on Cloud as well as the Bare metal environment. Also, it provides an in-built monitoring capability using redis-exporter.
• github.com/carlosedp/lbconfig-operator: External Load Balancer Operator 🌟 a Kubernetes/openshift Operator to dynamically configure external load-balancers distributing the traffic to the cluster nodes. It’s not 100% (will it ever be?) but already configures the F5 BigIP. The idea is to have multiple LB backends soon.
• Sentry Operator A Kubernetes operator for automating the provisioning and management of Sentry resources via Kubernetes CRDs.
• thenewstack.io: When to Use, and When to Avoid, the Operator Pattern 🌟
• infoq.com: Kubernetes Operators in Depth
• DB Operator 🌟 is a Kubernetes Operator for the management of cloud databases, primarily Google Cloud SQL(GCSQL). It is designed to support the on demand creation of test environments in CI/CD pipelines.
• cncf.io: Kubernetes Operators 101
• container-solutions.com: Kubernetes Operators Explained
• kubeload - load testing is a Kubernetes operator that lets you configure your load-test initial load, max load, interval and hatch-rate. You can use CRD to define all the parameters and repeat your load testing experiments.
• contentful.com: Open-sourcing kube-secret-syncer: A Kubernetes operator to sync secrets from AWS Secrets Manager
  • contentful-labs/kube-secret-syncer 🌟
• registry-creds is a Kubernetes operator that can be used to propagate a single ImagePullSecret to all namespaces within your cluster. The primary reason for creating this operator is to make it easier to consume images from Docker Hub.
• gemini is a Kubernetes CRD and operator for managing VolumeSnapshots. This allows you to back up your PersistentVolumes on a regular schedule, retire old backups, and restore backups with minimal downtime.
• Kdo: deployless development on Kubernetes 🌟 Kdo is a command line tool that enables developers to run, develop and test code changes in a realistic deployed setting without having to deal with the complexity of Kubernetes deployment and configuration.
• HostPort Operator is a Kubernetes Operator to allocate host ports
• iximiuz.com: Exploring Kubernetes Operator Pattern 🌟
• isaaguilar/terraform-operator: Terraform Operator A Kubernetes CRD and Controller to handle Terraform operations by generating k8s jobs catered to perform Terraform workflows
• hashicorp/terraform-k8s: Terraform Cloud Operator for Kubernetes The Terraform Cloud Operator for Kubernetes provides first-class integration between Kubernetes and Terraform Cloud by extending the Kubernetes control plane to enable lifecycle management of cloud and on-prem infrastructure.
• didil/autobucket-operator The autobucket operator is a Kubernetes operator that automatically creates and manages Cloud Buckets (Object Storage) for k8s Deployments.
• openshift.com: Is your Operator Air-Gap Friendly?
• kuberhealthy 🌟 An operator for synthetic monitoring on Kubernetes. Write your own tests in your own container and Kuberhealthy will manage everything else. Automatically creates and sends metrics to Prometheus and InfluxDB. Included simple JSON status page. Supplements other solutions like Prometheus very nicely!
• Bare Metal Operator The Bare Metal Operator implements a Kubernetes API for managing bare metal hosts. It maintains an inventory of available hosts as Custom Resource Definitions.
• Meerkat Meerkat is a Kubernetes Operator that facilitates the deployment of OpenVPN in a Kubernetes cluster. By leveraging Hashicorp Vault, Meerkat securely manages the underlying PKI.
• Logging Operator A golang based CRD operator to setup and manage logging stack (Elasticsearch, Fluentd, and Kibana) in the Kubernetes cluster. It helps to setup each component of the EFK stack separately.
• gst-pipeline-operator: A Kubernetes operator for running audio/video processing pipelines
• uptimerobot-operator A Kubernetes operator that creates UptimeRobot monitors for your ingresses
• medium.com: Getting Started With Kubernetes Operators (Helm Based) - Part 1
  • medium.com: Getting Started With Kubernetes Operators (Ansible Based) — Part 2
  • velotio.com: Getting Started With Kubernetes Operators (Golang Based) - Part 3
• IngressMonitorController (Deprecated) A Kubernetes controller to watch ingresses and create liveness alerts for your apps/microservices in UptimeRobot, StatusCake, Pingdom, etc.
• FairwindsOps/rbac-manager: RBAC Manager 🌟 A Kubernetes operator that simplifies the management of Role Bindings and Service Accounts. RBAC Manager is designed to simplify authorization in Kubernetes. This is an operator that supports declarative configuration for RBAC with new custom resources. Instead of managing role bindings or service accounts directly, you can specify a desired state and RBAC Manager will make the necessary changes to achieve that state.
• KubePlus - Kubernetes Operator to deliver Helm charts as-a-service 🌟
• kubernetes.io: Writing a Controller for Pod Labels
• kubermatic.com: Why Implementing Kubernetes Operators Is a Good Idea! 🌟
• thenewstack.io: We Pushed Helm to the Limit, then Built a Kubernetes Operator 🌟
• cncf.io: CNCF Operator White Paper (PDF) 🌟 This white paper defines Operators in a wider context than Kubernetes. It describes their characteristics and components, gives an overview of common patterns currently in use and explains how they differ from Kubernetes controllers.
• itnext.io: Kubexpose: A Kubernetes Operator, for fun and profit! Access your Kubernetes Deployment over the Internet - abhirockzz/kubexpose-operator Access your Kubernetes Deployment over the Internet
• redhat.com: Kubernetes Operators on Red Hat Marketplace
• itnext.io: Kubernetes Operators: Cruise Control for Managing Cloud-Native Apps
• digitalis-io/vals-operator Kubernetes Operator to sync secrets between different secret backends and Kubernetes
• banzaicloud/thanos-operator 🌟 Thanos Operator is a Kubernetes operator to manage Thanos stack deployment on Kubernetes.
• cloud-bulldozer/benchmark-operator: The Chuck Norris of cloud benchmarks The intent of this Operator is to deploy common workloads to establish a performance baseline of Kubernetes cluster on your provider.
• pravega/pravega-operator Pravega Kubernetes Operator. Pravega is an open source distributed storage service implementing Streams. It offers Stream as the main primitive for the foundation of reliable storage systems: a high-performance, durable, elastic, and unlimited append-only byte stream with strict ordering and consistency. The Pravega Operator manages Pravega clusters deployed to Kubernetes and automates tasks related to operating a Pravega cluster.The operator itself is built with the Operator framework.
• Quentin-M/etcd-cloud-operator Deploying and managing production-grade etcd clusters on cloud providers: failure recovery, disaster recovery, backups and resizing.
• spring.io: Get to Know a Kubernetes Operator!
• levelup.gitconnected.com: Operators : Extending Kubernetes Capabilities Operators are software extensions to Kubernetes that make use of custom resources to manage applications and their components. So what it means is that there are some applications whose deployment and management might require manual intervention and operators is the solution to automate it. Let’s say you need to deploy a database cluster for which each pod needs to be brought in sync after deployment, or say you need to perform a security scan whenever a new component is deployed, or maybe some configuration needs to be populated based on some event. Such a functionalities are not available in Kubernetes out of the box but can be implemented using operators.
• developer.redis.com: Kubernetes Operator: What It Is and Why You Should Really Care About It
• VictoriaMetrics/operator Kubernetes operator for Victoria Metrics
• blog.px.dev/k8s-operator: 3 Reasons to Use Kubernetes Operators (and 2 Reasons Not To)
• medium.com/@adnn.selimovic: Creating Kubernetes operator using Kubebuilder
• dzone.com: What Is a Kubernetes Operator? A Kubernetes Operator fills in the gaps between the capabilities and automation provided by Kubernetes and how your software uses Kubernetes for task automation.
• practicalkubernetes.blogspot.com: Making the case for Kubernetes Operators
• reactive-tech/kubegres Kubegres is a Kubernetes operator allowing to deploy one or many clusters of PostgreSql instances and manage databases replication, failover and backup.
• Capsule Operator is a Kubernetes multi-tenant Operator. It aggregates multiple namespaces in a Tenant. Within each tenant, users are free to create their namespaces and share all the assigned resources between the namespaces of the tenant.
• redhat-cop/keepalived-operator: Keepalived operator An operator to manage VIPs backed by keepalived. The objective of the keepalived operator is to allow for a way to create self-hosted load balancers in an automated way. From a user experience point of view the behavior is the same as of when creating LoadBalancer services with a cloud provider able to manage them.
• medium.com/@samng1991216: Building Kubernetes Operator Application from Scratch (Part 1)
• redhat-cop/dynamic-rbac-operator: Dynamic RBAC Operator Flexible definitions of Kubernetes RBAC rules. Writing Kubernetes RBAC definitions by hand can be a pain. This operator allows you to define “Dynamic” RBAC rules that change based on the state of your cluster, so you can spend your time writing the RBAC patterns that you’d like to deploy, rather than traditional, fully enumerated RBAC rules.
• spotify/flink-on-k8s-operator: Kubernetes Operator for Apache Flink Kubernetes operator for that acts as control plane to manage the complete deployment lifecycle of Apache Flink applications. This is an open source fork of GoogleCloudPlatform/flink-on-k8s-operator with several new features and bug fixes.
• kube-green An operator to reduce CO2 footprint of your clusters.
  • Sleep your pods: Suspend your pods when no-one’s using them, scale down your cluster and save energy
  • Reduce CO2 emissions: See how much you save in the Green Dashboard (coming soon)
• krestomatio/keydb-operator A KeyDB (Drop-In Alternative to Redis) Operator for Kubernetes
• Keel 🌟 Kubernetes Operator to automate Helm, DaemonSet, StatefulSet & Deployment updates:
  • You can use policies to define when to update an application
  • Users can specify how many approvals do they need before a resource is updated.
  • https://keel.sh
• medium.com/@mjkool: Kubernetes Operator — Simplified!
• medium.com/geekculture: A New Pattern that Simplifies Operator Building Build Kubernetes Operator with Kubebuilder and declarative pattern. kubebuilder-declarative-pattern provides a set of tools for building cluster operators with kubebuilder. Declarative operators provide a fast path to orchestrating deployments instead of reinventing the wheel i.e. “how do I get/update this YAML?”
• medium.com/@timebertt: Kubernetes Controllers at Scale: Clients, Caches, Conflicts, Patches Explained A developer guideline to Kubernetes clients in go. As most development in the Kubernetes space is done in Go, available client libraries for interacting with the Kubernetes API have evolved over time to make controllers more scalable.
• openshift/machine-api-operator The Machine API Operator manages the lifecycle of specific purpose CRDs, controllers and RBAC objects that extend the Kubernetes API. This allows to convey desired state of machines in a cluster in a declarative fashion
• rancher/system-upgrade-controller: System Upgrade Controller This project aims to provide a general-purpose, Kubernetes-native upgrade controller (for nodes). It introduces a new CRD, the Plan, for defining any and all of your upgrade policies/requirements. A Plan is an outstanding intent to mutate nodes in your cluster.
• alenkacz.medium.com: Kubernetes operator best practices: Implementing observedGeneration There’s a lot of hidden knowledge in core controllers and api conventions doc that is not followed by many controllers in the wild. One of these patterns is observedGeneration. In this article, you will learn what problems it can help solve.
• ckotzbauer/vulnerability-operator Scans SBOMs for vulnerabilities. This operator scans all SBOMs from a git-repository for vulnerabilities using Grype. The result-list can be emitted as JSON-file served via an endpoint and/or as Prometheus metrics. There may be more targets in the future. The scans are done periodically.
• Michaelpalacce/SimpleSecrets K8S Secrets Manager Operator. SimpleSecrets is a secure operator that allows you to create secrets on demand. You can commit the SimpleSecrets, which are references to a database secret, and the operator will create Kubernetes Secrets automatically for you.

operatorhub.io

• operatorhub.io OperatorHub.io is a new home for the Kubernetes community to share Operators. Find an existing Operator or list your own today.

Red Hat Container Community of Practice Operators

• cloud.redhat.com: Red Hat Container Community of Practice Operators

Operator Capability Levels

• Operator Capability Levels Operators come in different maturity levels in regards to their lifecycle management capabilities for the application or workload they deliver. The capability models aims to provide guidance in terminology to express what features users can expect from an Operator.

Cluster Addons

• Cluster Addons 🌟 With cluster addon operators, we are exploring a kubernetes-native way of managing addons using CRDs(Custom Resource Definitions) and controllers where the controllers encode how best to manage the addon. Installing and managing an addon could be as simple as creating a custom resource.

K8Spin Operator. Kubernetes multi-tenant operator

• K8Spin Operator 🌟 Kubernetes multi-tenant operator. Enables multi-tenant capabilities in your Kubernetes Cluster. We defined some small features to implement. If you know python & Kubernetes and want to contribute to this project, ping us!
• thenewstack.io: K8Spin Provides Multitenant Isolation for Kubernetes
• Discover K8Spin open source software

K8s KPIs with Kuberhealthy Operator

• K8s KPIs with Kuberhealthy 🌟 transforming Kuberhealthy into a Kubernetes operator for synthetic monitoring. This new ability granted developers the means to create their own Kuberhealthy check containers to synthetically monitor their applications and clusters. Additionally, we created a guide on how to easily install and use Kuberhealthy in order to capture some helpful synthetic KPIs.

Writing Kubernetes Operators and Controllers

• Kubernetes.io: Operator pattern
• opensource.com: Build a Kubernetes Operator in 10 minutes with Operator SDK
• itnext.io: Testing the Operator SDK and making a prefetch mechanism for Kubernetes
• magalix.com: Creating Custom Kubernetes Operators
• medium.com: Writing Your First Kubernetes Operator
• bmc.com: What Is a Kubernetes Operator?
• Writing a Kubernetes Operator in Java Cheat Sheet
• linuxera.org: Writing Operators using the Operator Framework SDK
• openshift.com: 7 Best Practices for Writing Kubernetes Operators: An SRE Perspective
• medium: From Zero to Kubernetes Operator In this post you will learn how to build a simple Kubernetes Operator. The article starts with the main concepts and then continues with hands-on labs where you will create a Kubernetes Operator from the ground up.
• openshift.com: Build Your Kubernetes Operator With the Right Tool 🌟 Go-based operators are by far the most popular. That is why Go is probably the first option to consider. The other good choice is Helm, especially if you already have a Helm chart for your software or you want to build your operator quickly and you don’t need any complex capability levels. I’d leave Operator Frameworks or Bare Programming Language implementations only for the cases when keeping a single programming language in your organization is a priority.
• codilime.com: How to create a custom resource with Kubernetes Operator Implementing DaemonJob from scratch learn how to create a custom resource with the Kubernetes Operator Framework.
• rookout.com: Lessons Learned When Building A Kubernetes Operator
• pavel.cool: Oxidizing the Kubernetes operator
• brennerm.github.io: Kubernetes operators with Python #1: Creating CRDs
• vivilearns2code.github.io: Writing Controllers For Kubernetes Resources
• cloudark.medium.com: Writing Kubernetes Custom Controllers
• developers.redhat.com: Managing stateful applications with Kubernetes Operators in Golang 🌟 Explore this pattern by creating a Kubernetes Operator in Golang to keep a WordPress site up to date.
• medium: Kubernetes Dummy Operator in Java - youtube: Creating a Kubernetes Operator in Java by Rudy De Busscher
• betterprogramming.pub: Build a Highly Available Kubernetes Operator Using Golang Develop a simple Kubernetes operator from scratch. In this article, you will build a “hello world” operator using the client-go library, make adaptations to it to achieve high availability, and deploy it to a Kubernetes cluster using Helm.
• kubernetes/sample-controller Repository for sample controller. Complements sample-apiserver
• betterprogramming.pub: Writing Custom Kubernetes Controller and Webhooks Create a Kubernetes API, controller, validate webhooks, and test.

Tweets

Click to expand!

Generic automation tools like Helm are limited by the interfaces exposed to them, and often lack enough context to make the right state machine transitions.

Ideally, software evolves to expose better automation hooks, then custom tools, aka operators, can leverage them. https://t.co/v38aj4ukn4

— Kelsey Hightower (@kelseyhightower) September 8, 2021

Videos

Click to expand!