Vivek C. Email & Phone Number

With over a decade of experience as a Data Engineer, I specialize in BI, Data Warehousing, ETL, Cloud, and Big Data technologies across the Finance, Retail, E-commerce, and Healthcare domains. I am proficient in a wide array of Big Data technologies, including the Hadoop ecosystem, Spark, Impala, Flume, Kafka, Hive, Sqoop, and Google Cloud Storage, as well as programming and visualization tools such as Python, SQL, and Tableau.My expertise encompasses Hadoop architecture and components, including YARN, HDFS, Name Node, Data Node, Job Tracker, Application Master, Resource Manager, Task Tracker, and the MapReduce programming paradigm. I am well-versed in Google Cloud Platform (GCP) services like Compute Engine, Cloud Load Balancing, Cloud Storage, Cloud Dataproc, Cloud Pub/Sub, Cloud SQL, BigQuery, and Stackdriver Monitoring. Additionally, I have extensive experience with AWS services such as S3, EMR, Redshift, RDS, Athena, Lambda, Step Functions, and SQL, as well as a deep understanding of AWS cloud services, including Compute, Network, Storage, and Identity & Access Management.I have a proven track record in building and optimizing AWS data pipelines, architectures, and datasets, utilizing Athena, Glue, Data Catalog, and QuickSight. My experience extends to AWS databases, including RDS (Aurora), Redshift, DynamoDB, and ElastiCache. I am adept at implementing monitoring solutions using Docker and Jenkins, and have substantial experience in data analysis, database programming (stored procedures, triggers, views), and table partitioning.Translated business metrics into technical specifications by developing detailed technical requirements. Furthermore, I have expertise with Apache Airflow, including installation, configuration, and monitoring of Airflow clusters. I possess extensive experience in normalization and denormalization techniques for both OLTP and OLAP systems, and I am proficient in Apache Spark job execution components such as DAG, Executors, Scheduler, Stages, and Spark Streaming. I have utilized CI/CD pipelines with Jenkins to deploy code into production, and I am skilled in using automation tools like Git, Terraform, and Ansible.My skills in visualization tools are robust, including Power BI and Excel (formulas, Pivot Tables, Charts), as well as DAX commands. I have orchestrated and managed ETL ingestion flows using Azure Data Factory and Azure Stream Analytics and have extensive experience with Azure cloud platforms such as Data Lake, Data Bricks, Blob Storage, Data Factory, and Synapse.