V I J A Y

Lead SRE/DevOps- Kafka Consultant

About

SRE/DevOps Kafka Engineer with 13+ years of Exposure on Production, Non-production Installation, managing and implementation Kafka on Kubernetes ,On-premises & on Cloud Expertly designed, installed, and managed Kafka, RabbitMQ, Redis, MySQL, MariaDB, MongoDB, and Cassandra on Kubernetes, on-premises, and cloud environments to meet diverse business needs. Deployed Kafka Connectors, including JDBC SQL source/sink, ADLS Gen-2, and HTTP sink connectors, to streamline real-time data ingestion and transformation. Led Kafka multi-site data center deployments, optimizing producer/consumer performance and managing Kafka ACLs for secure operations. Implemented fraud detection solutions and Data transformations using Flink stream API. Integrated Apache Flink with Kafka for high-throughput event processing, optimizing streaming pipelines for data-intensive applications. Migrated on-premises Confluent clusters to Confluent Cloud, transitioning schema-less topics to schema topics for improved data structure and governance. Integrated on-premises Confluent connectors with Confluent Cloud, facilitating seamless hybrid cloud data movement and reducing downtime. Installed and configured Zookeeper, Apache Kafka, Confluent Kafka, and Confluent Control Center on VMs, cloud, and Kubernetes environments. Developed and maintained infrastructure pipelines using Terraform to automate the provisioning of cloud and on-premises resources, ensuring consistent and repeatable deployments. Configured Grafana monitoring with Prometheus for Kafka and Kafka Connect, providing real-time metrics and alerting for efficient operational monitoring. Deployed and managed Kafka components using Confluent for Kubernetes (CFK), utilizing Confluent Operator for automated cluster scaling and maintenance. Proficient in Python scripting, implementing automation with Flask API and Swagger OpenAPI to streamline system operations and custom workflows. Led RabbitMQ installations, cluster setups, user management, and vHost policies, optimizing messaging solutions for enterprise-grade applications. Expertly handled Kafka monitoring using JMX, Prometheus, and Grafana, ensuring visibility into Kafka cluster health and performance. Deployed Kafka HA clusters on AWS EC2, Azure VMs and GCP VMs and leveraging Kubernetes (AKS, EKS, GKE) for Kafka cluster setup.Proficient in Python scripting, automating tasks with Flask API and Swagger Open API, as well as troubleshooting Kafka-related issues.

Country

India

City

Hyderabad

Industry

Information Technology & Services

Skill

Terraform, Microsoft Azure, Site Reliability Engineering, Oracle Database, Cassandra, Apache Kafka, Apache Spark, Apache ZooKeeper, Amazon Web Services (AWS), Google Cloud Platform (GCP), OpenStack, Grafana, Apache Ignite, Docker Products, Ansible, Database Administration, Database Engine Tuning Advisor

Experience

Lead SRE/DevOps- Kafka Consultant

TechBirds Consultancy Services

2023-10 - Present · 2 yrs 11 mos

Hyderabad, Telangana, India

Excellent in Confluent Kafka and Apache Kafka Installation, cluster setup Upgradation & Monitoring implementation and troubleshooting production errors. Expertise in CFK implementation deployed confluent cluster by using confluent operator on Kubernetes. Implemented Kafka monitoring with JMX, Prometheus , Grafana Confluent Kafka Setup & upgraded confluent Kafka from 6.0.2 to 7.0.1 Implemented various Confluent Kafka connectors Very good hands on Kafka schema registry ,KSQL & K-streams implementation Kafka topic creation, RBAC security implementation with SASL/SSL MySql, Maria and Mongo DB installation , user management backup & restoration and troubleshooting RabbitMQ installation, cluster setup, user management, exchange and queue creation , vHost , policies and troubleshooting Redis installation and High availability cluster setup by using sentinel Configured Grafana monitoring with Prometheus for Redis, Cassandra and MongoDB. Daily day 2day tasks automation by using ansible and ansible runner Proficient in Python programming and implemented many scripts and Implemented few automations using Flask API and Swagger Open API. Very good in ApiGateway installation, SSL implementation and policy creations using DB less method. Cloud SQL , DNS and Load balancer setup in GCP Good Knowledge and hands on experience with AWS, Azure and GCP cloud platform Good Hands-on experience with Docker file creation, Helm charts creation, deploying micro services on Kubernetes cluster including AKS, EKS and GKE Good knowledge in Terraforms and worked on few PoC projects using Terraform.

Senior SRE Kafka Engineer

TechBirds

2022-4 - 2023-10 · 1 yr 7 mos

Hyderabad, Telangana, India

SRE Kafka Adminstrator

Netcracker Technology

2017-4 - 2022-3 · 5 yrs

Hyderabad, Telangana, India

Designed, Implemented &Supported deployment of Zookeeper, Kafka, Spark, Nifi &Cassandra in cloud, K8’s and non-virtualized deployments. Designed & implemented Multi-Datacenter Cassandra cluster. Experience in performance tuning a Cassandra a cluster to optimize writes and reads. Involved in the process of data modeling Cassandra Schema. Excellent hands-on experience with confluent platform. Expert in configuring Kafka Multi site data center environments. Expertise in confluent replicator. Configured & implement Kafka ACL & multi cluster control center & Confluent kafka connector, schema registry Strong hands on experience with kafka producer & consumer optimization. Expertise in consumer groups management. Exposure to securing the Kafka cluster with SASL/PLAINTEXT, SASL/SCRAM and SASL/SSL. Designed and implemented by configuring Topics in new Kafka cluster in all environment. Efficiently worked with the following tools/Instances but not limited to including Kafka, Zookeeper, Console Producer, Console Consumer, Kafka Tool etc. Proficient with container systems like Docker and container orchestration like AKS, EKS,GKE , Kubernetes. Managed Docker orchestration and Docker containerization using Kubernetes. Excellent & strong hands-on experience from confluent platform to CFK migration Used Kubernetes to orchestrate the deployment, scaling and management of Docker Containers. Implement & administrated the Kubernetes cluster. Set up backup and recovery procedure using OpsCentre for Cassandra and Installed and Implemented Oracle Backup & Recovery procedures. Installing & managing NiFi, Strong hands on NiF-Toolkit , Zk-migrator And good knowledge on NiFi Registry. Create VMs, assign floating IPs, snapshot volume & image creation on AWS,GCP and AZURE Strong experience with Confluent Hub, Confluent Cloud and On-Prem Infrastructure, Cluster Link Migrated kafka clusters from confluent platform to CFK cluster.