Patterns for Composite Containers

Building an application from modular containers means thinking about symbiotic groups of containers that cooperate to provide a service, not one container per service. In Kubernetes, the embodiment of this modular container service is a Pod. A Pod is a group of containers that share resources like file systems, kernel namespaces and an IP address. The Pod is the atomic unit of scheduling in a Kubernetes cluster, precisely because the symbiotic nature of the containers in the Pod require that they be co-scheduled onto the same machine, and the only way to reliably achieve this is by making container groups atomic scheduling units.

Kafka Connect and Streams APIs

Here is a typical architecture having Sources, Sinks, Connect Cluster, Kafka Cluster and Kafka Streams Applications.

Kafka Basics

Apache Kafka is a distributed publish-subscribe messaging system that is designed to be fast, scalable, and durable. Kafka stores streams of records (messages) in topics. Each record consists of a key, a value, and a timestamp. Producers write data to topics and consumers read from topics.

CGO - How to use C shared library code

I was working with CGO code and realized the need of keeping the C code in a shared library and using it from Go code. Here is a brief run down of how to go about it.

CGO - Bridge between C and Go

cgo allows Go and C programs to interoperate. This is very useful feature which lets C existing libraries to be called by Go. On the other hand Go libraries can also be called by C code.

Extended Berkeley Packet Filter (eBPF)

eBPF stands for extended Berkeley Packet Filter but officially referred to as BPF. BPF was intially developed for high performance packet capture in 1992. In recent years (2012 - 2014), BPF is rewritten to a general purpose virtual machine that can be used for many things like networking, monitoring, security and performance analysis.

Netfilter and IPTables

iptables is a management tool for the firewall software netfilter in the Linux kernel. iptables is located in the user space while netfilter is located in the kernel space, where functionalities of firewalling, network address translation (NAT), packet content modification and packet filtering is implemented. Both together are commonly referred to just iptables.

Service Mesh with Istio

As per description on Istio website, Service mesh is used to describe the network of microservices that make up applications and the interactions between them. As a service mesh grows in size and complexity, it can become harder to understand and manage. Its requirements can include discovery, load balancing, failure recovery, metrics, and monitoring. A service mesh also often has more complex operational requirements, like A/B testing, canary rollouts, rate limiting, access control, and end-to-end authentication.

Sockets

Sockets are the preferred way of network communication. They are provided by Operating Systems as socket API, which are based on the principles of reading and writing files.

Running Memcached on Ubuntu

Memcached is an in-memory key-value, distributed memory object caching system, generic in nature, but originally intended for use in speeding up dynamic web applications by alleviating database load.