. Vignesh Chandra Email & Phone Number

I am Vignesh Vajja, a Senior Data Engineer with over 10 years of experience in developing, deploying, and managing data solutions in cloud and big data ecosystems. I am passionate about crafting robust data pipelines and architectures that drive business insights and efficiency.My Work at Prudential Financial: At Prudential Financial, I designed and implemented a robust ETL pipeline using PySpark on Amazon EMR, processing over 500GB of daily data and loading it into Amazon Redshift for analytics. I developed automated data quality checks using Python and Apache Airflow, and created interactive dashboards with Tableau and Power BI, enabling real-time insights for stakeholders. My role also included building CI/CD pipelines using Jenkins and Git, ensuring faster deployment cycles.My Work at Kaiser Permanente: At Kaiser Permanente, I developed and optimized PySpark jobs in Azure Databricks and built comprehensive data pipelines using Azure Data Factory. I configured and managed Azure Monitor for real-time tracking and optimized Apache Spark configurations to boost performance. I also leveraged Azure Synapse Analytics and Snowflake for building efficient data solutions and integrated advanced data quality checks within the ETL pipelines.Responsibilities Include:Designing and implementing scalable ETL pipelines and data workflows in cloud environments.Developing real-time data streaming solutions using Apache Kafka, Spark Streaming, and Azure Event Hubs.Building and maintaining data warehouses and data lake architectures for seamless data management.Creating interactive dashboards and visualizations using Tableau and Power BI.Ensuring data security and compliance with regulations such as GDPR and PCI-DSS through encryption and access controls.Skills:Python, PySpark, SQL, Apache Spark.Cloud technologies: AWS (EMR, S3, Redshift), Azure (Data Factory, Databricks, Synapse Analytics).Data visualization: Tableau, Power BI.CI/CD tools: Jenkins, GitHub Actions.Containerization and orchestration: Docker, Kubernetes.Real-time data streaming: Apache Kafka, AWS Kinesis.Infrastructure as code: Terraform.Data modeling, data governance, and compliance.What I Bring to the Table:Extensive experience in architecting end-to-end data solutions tailored to meet business needs.Proven track record of optimizing data pipelines and improving system performance.Expertise in ensuring data integrity, security, and compliance with industry standards.Strong problem-solving skills with a focus on innovation and process improvement.