Golang Notes

Golang is an open-source, compiled, and statically typed programming language designed by Google. It is built to be simple, high-performing, readable, and efficient.

Redis Client In Golang

Remote dictionary service (redis) is an in-memory key-value data structure store used as a cache/database. Generally, it used as a cache in front of another persistent database to speed up application performance.

Kafka Producer and Consumer In Golang

Apache Kafka is a distributed publish-subscribe messaging system that is designed to be fast, scalable, and durable. Kafka stores streams of records (messages) in topics. Each record consists of a key, a value, and a timestamp. Producers write data to topics and consumers read from topics.

Cloud Setup on Macbook

Setting up cloud development machine can be challenging, especially for new comers. Here is what I did to setup various components on Macbook.

AWS EC2

EC2 is one of the most popular AWS offering. It provides the capabilities of:

  • Renting Virtual Machines (EC2)
  • Storing data on the virtual drives (EBS)
  • Distributing load across machine (ELB)
  • Scaling a service using auto-scaling group (ASG)

Apache Storm

Apache Storm is a free and open source distributed realtime computation system. Apache Storm makes it easy to reliably process unbounded streams of data, doing for realtime processing what Hadoop did for batch processing.

Apache Spark - Stream Processing

Initially big data started with collecting huge volume of data and processing it in smaller and regular batches using distributed computing frameworks such as Apache Spark. Changing business requirements needed to produce results within minutes or even in seconds.

Apache Spark

Initially software applications were small which can be deployed on a single computer. Over a period of time, data volume processed by these application grew in size. Hence, the requirement for storage and computing power grew. These requirements were fulfilled by rapid advancements in storage and compute hardware by having larger disks and faster CPUs. This way of scaling is termed as vertical scaling, which soon became costlier when applications started being consumed over Internet.

Linux Virtualization

Virtualization solutions allow multiple operating systems and applications to run in independent partitions on a single computer. Using virtualization capabilities, one physical computer system can function as multiple “virtual” systems.