Jayanth R Email & Phone Number

Chicago, Illinois, United States

• Collaborated with project managers to define data synchronization requirements, including data scope and mapping.• Worked closely with Data Scientists and Business Analysts to understand specific requirements and extract relevant business stories.• Developed and maintained data pipelines using Spark, Kafka, Redshift, S3, Java, and Python for streaming and transactional data ingestion.• Created Spark Streaming jobs in Python for Kafka message processing and AWS S3 data retrieval.• Implemented Spark applications for data validation, cleansing, transformation, custom aggregation, and analysis.• Designed Column families in Cassandra and ingested data from RDBMS, performing necessary transformations.• Utilized Spark SQL in PySpark for data joins and storage in S3.• Employed AWS Glue for ETL pipeline development and AWS EMR for data transformation and movement.• Explored Spark to enhance Hadoop algorithms using Spark context, Spark-SQL, PostgreSQL, Data Frame, OpenShift, and Talend.• Integrated Apache Airflow with AWS for monitoring machine learning processes with Amazon Sage Maker jobs.• Created AWS CloudFormation and Terraform templates for infrastructure provisioning and versioning.• Provisioned and configured infrastructure using AWS CloudFormation and Terraform (JSON/YAML).• Implemented data quality scripts using SQL and Hive for data validation and created data visualizations with Python and Tableau.• Analyzed data lineage processes to identify vulnerabilities, control gaps, and data quality issues.• Resolved data governance issues to ensure enterprise-level data consistency.• Worked on Teradata, SQL Server, and Oracle for data loading and migration.• Developed T-SQL queries, user-defined functions, and dynamic SQL scripts.• Designed data warehousing solutions, dimensional/cube data models, and DAX scripting for reporting.• Developed ETL pipelines using PySpark, SSIS, Informatica, and Python.• Created reporting data marts on AWS Redshift.

Blue Bell, Pa

• Gathered requirements and established a framework to migrate customer data from Excel to SQL.• Proficient in Database Design, Business Intelligence using SQL Server 2014/2016 (SSIS, SSAS, DAX), and various data modeling techniques (Star Schema, Snowflake Schema).• Managed backend databases supporting web applications.• Implemented Python-based exception handling mechanisms.• Introduced case sensitivity for data quality checks using Python and SSIS.• Created Linked Services for multiple data sources (e.g., Oracle, SQL Server, Teradata, ADLS, BLOB, File Storage).• Designed and managed data pipelines to extract data from on-premises systems to Azure Data Lake Storage.• Implemented T-SQL error handling techniques in SQL Server.• Developed end-to-end ETL strategies using SSIS for loading data from various sources to OLAP systems.• Worked with ACL to set data permissions and access controls.• Experience with cloud-based technologies like AWS (Redshift, S3, EC2, etc.).• Designed data models and visualized data using Power BI (bar charts, pie charts, maps).• Created data integration processes for ETL from internal and external data sources.• Utilized SQL for data cleaning and aggregation, generating status reports and dashboards.• Designed and implemented ETL processes, including test script and SSIS package reviews.• Developed retry framework for restarting SSIS jobs after configurable timeouts.• Automated email notifications for data load completion.• Migrated legacy applications to AWS cloud.• Implemented security measures and centralized logging with AWS CloudWatch.• Developed AWS account security and integrated endpoint protection.• Participated in project status and daily scrum meetings.• Designed SSRS reports with various chart types for trend analysis.• Conducted JRD meetings for data mart planning.• Documented business rules and use cases based on requirements analysis.

Austin, Tx

• Developed real-time data processing apps in Scala and Python using Apache Spark Streaming (Kafka, Flume, JMS).• Converted Hive/SQL queries to Spark transformations with Spark RDDs and Scala.• Created APIs and data models adhering to FHIR specifications for secure data exchange.• Conducted performance comparisons of Spark, Hive, and SQL/Teradata through Scala-based POCs on Yarn cluster.• Analyzed SQL scripts and devised Scala-based solutions.• Developed analytical components with Scala, Spark, and Spark Stream.• Wrote Java UDFs for Hive and Pig, handling various data formats in HDFS with Scala.• Troubleshot and debugged Hadoop ecosystem runtime issues.• Provisioned Kubernetes clusters on AWS using Docker, Ansible, and Terraform.• Automated pipeline workflows with Oozie Scheduler for map reduces jobs.• Integrated data systems and applications with .NET code for accurate and secure data processing.• Utilized Hue GUI for job scheduling, file browsing, and megastore management.• Collaborated with the BI team to implement Big Data Hadoop clusters and data integration.• Installed/configured/maintained Apache Hadoop clusters and related tools.• Managed cluster nodes, monitored, and reviewed backups and logs.• Continuously monitored and managed the cluster via Cloudera Manager.• Installed and configured Cloudera Hadoop components.• Created a POC for storing server log data in MongoDB.• Monitored Hadoop cluster performance and performed capacity planning.• Worked with AWS cloud services like EMR, S3, Redshift, and EMR cloud watch.• Loaded data into the cluster from dynamically generated files and relational databases.• Developed PySpark and Spark-Scala applications for various data processing tasks.• Analyzed unused user navigation data with MapReduce jobs.• Wrote MapReduce jobs using Java API and Pig Latin.• Integrated and migrated data from external sources using ETL tools like Talend, Informatica, and Alteryx.