Vineeth C Email & Phone Number

Bengaluru, Ka, In

Arizona, United States

● Involved in analyzing business requirements and prepared detailed specifications that follow project guidelines required for project development.● Extensively used Apache Kafka, Apache Spark, HDFS and Apache Impala to build a near real time data pipelines that get, transform, store and analyze click stream data to provide a better personalized user experience.● Primarily involved in Data Migration using SQL, SQL Azure, Azure Storage, and Azure Data Factory, SSIS, PowerShell.● Designed and implemented configurable data delivery pipeline for scheduled updates to customer facing data stores built with Python● Proficient in Machine Learning techniques (Decision Trees, Linear/Logistic Regressors) and Statistical Modeling ● Implement medium to large scale BI solutions on Azure using Azure Data Platform services (Azure Data Lake, Data Factory, Data Lake Analytics, Stream Analytics, Azure SQL DW, HDInsight/Databricks, NoSQL DB).● Performed data extraction, transformation, loading, and integration in data warehouse, operational data stores and master data management● Experienced in ETL concepts, building ETL solutions and Data modeling

Bengaluru, Karnataka, India

● Developing Spark scripts, UDFS using both Spark DSL and Spark SQL query for data aggregation, querying, and writing data back into RDBMS through Sqoop.● Written multiple MapReduce Jobs using Java API, Pig and Hive for data extraction, transformation and aggregation from multiple file formats including Parquet, Avro, XML, JSON, CSV, ORCFILE and other compressed file formats Codecs like gZip, Snappy, Lzo.● Strong understanding of Partitioning, bucketing concepts in Hive and designed both Managed and External tables in Hive to optimize performance.● Interacted with business partners, Business Analysts and product owners to understand requirements and build scalable distributed data solutions using the Hadoop ecosystem.● Developed Spark Streaming programs to process near real time data from Kafka, and process data with both stateless and stateful transformations.● Worked on implementation of a log producer in Scala that watches for application logs, transforms incremental logs and sends them to a Kafka and Zookeeper based log collection platform.● Used Hive to analyze data ingested into HBase by using Hive-HBase integration and compute various metrics for reporting on the dashboard.

Bengaluru, Karnataka, India

● Used Spark-Streaming APIs to perform necessary transformations and actions on the data got from Kafka.● Worked on Apache NIFI to decompress and move JSON files from local to HDFS. ● Sustaining the Big Query, PySpark and Hive code by fixing the bugs and providing the enhancements required by the Business User.● Authoring Python (PySpark) Scripts for custom UDF’s for Row/ Column manipulations, mergers, aggregations, stacking, data labeling and for all Cleaning and conforming tasks.● Performing ETL testing activities like running the Jobs, Extracting the data using necessary queries from database transform, and upload into the Data warehouse servers.● Performed data extraction, transformation, loading, and integration in data warehouse, operational data stores and master data management.● Strong understanding of AWS components such as EC2 and S3● Designed and implemented configurable data delivery pipeline for scheduled updates to customer facing data stores built with Python.● Worked on Ingesting data by going through cleansing and transformations and leveraging AWS Lambda, AWS Glue and Step Functions.● Developed Hive UDFs to incorporate external business logic into Hive script and developed join data set scripts using Hive join operations.● Migrated Map reduce jobs to Spark jobs to achieve better performance.● Built Data pipelines using Apache Beam framework in GCP for ETL related jobs for different Airflow operators.

Bengaluru, Karnataka, India

• Analyzed the system for new enhancements/functionalities and perform Impact analysis of the application for implementing ETL changes• Managed security groups on AWS, focusing on high-availability, fault-tolerance, and auto scaling using Terraform templates. Along with Continuous Integration and Continuous Deployment with AWS Lambda and AWS code pipeline.• Wrote various data normalization jobs for new data ingested into Redshift.• Defined facts, dimensions and designed the data marts using the Ralph Kimball's Dimensional Data Mart modeling methodology using Erwin.• Developed SSRS reports, SSIS packages to Extract, Transform and Load data from various source systems• Having experience in developing a data pipeline using Kafka to store data into HDFS.• Worked on Big data on AWS cloud services i.e. EC2, S3, EMR and DynamoDB• Created Entity Relationship Diagrams (ERD), Functional diagrams, Data flow diagrams and enforced referential integrity constraints and created logical and physical models using Erwin.• Created ad hoc queries and reports to support business decisions SQL Server Reporting Services (SSRS).• Strong understanding of AWS components such as EC2 and S3• Used Hive SQL, Presto SQL and Spark SQL for ETL jobs and using the right technology for the job to get done.• Measured Efficiency of Hadoop/Hive environment ensuring SLA is met• Optimized the Tensor Flow Model for efficiency