Ramya D Email & Phone Number

St Louis, Missouri, United States

I specialize in ensuring the reliability and accuracy of data processing within Azure Data Lake by incorporating automated testing frameworks. I've developed incident response plans for quick issue resolution, minimizing downtime in Azure Data Lake and Azure Databricks.For efficient collaboration, I implemented version control for data artifacts and code within Azure Blob Storage. I showcased the capabilities of Azure Stream Analytics by integrating Azure Event Hub and Stream Analytics with Power BI and Azure ML.My expertise extends to developing scalable data processing pipelines using Apache Spark in Scala for efficient ETL operations. I've conducted Proof of Concepts (POCs) comparing Spark and Scala performance with Hive and SQL, deploying them on the Yarn Cluster.Comprehensive testing of Scala applications in big data environments ensures reliability and performance. Performance analysis and optimization strategies are applied to enhance the efficiency of Hadoop clusters.I maintain clear documentation for Python scripts, facilitating collaboration. Python code is developed for workflow management and automation using the Airflow tool. SQL scripts are created for data migration, transformation, and integration tasks within PostgreSQL.I've ensured regulatory compliance for MongoDB databases and managed Relational Database Management Systems (RDBMS) like MySQL and PostgreSQL, optimizing queries and ensuring data integrity.I follow GitLab CI/CD best practices to automate testing, building, and deployment of data engineering solutions, resulting in accelerated development cycles and reduced errors.

Minnesota, United States

I've optimized data transfer speeds with AWS S3 Transfer Acceleration, enhancing performance for large datasets across geographical locations. Event-driven architecture using AWS S3 event notifications and Lambda functions automates responses to S3 changes, enabling real-time data processing.In database management, I've handled AWS RDS, Oracle, and NoSQL databases like MongoDB. AWS Glue and EMR are used for ETL processes with security measures like IAM roles.I've focused on performance monitoring and tuning in Informatica Power Centre, ensuring efficient resource utilization. Optimizing data processing workflows on Hadoop and Spark clusters, I've enhanced data ingestion, storage, and retrieval on the Hadoop platform.Python libraries like Pandas and Plotly are utilized for data manipulation and interactive dashboards. SQL code and database structures are documented for knowledge sharing.Optimization techniques in PostgreSQL and sharding strategies in MongoDB improve database performance. Documentation for Scala APIs aids developers, and tools like Map Reduce and Spark with Scala are used for operations like Clickstream Analysis.Kafka topics are configured and optimized for efficient data flow, and Oozie Workflows are developed for daily incremental loads using Kafka. Version control practices are implemented for Snowflake objects, and metadata management processes enhance data governance.In Tableau, I've designed various visualizations, dashboards, and charts, addressing specific business requirements. Power Query Editor is used for data transformations and cleansing in Power BI reports.Experience in creating and maintaining documentation for Power BI solutions, along with Git repository management for data-centric applications, ensures effective versioning and rollback strategies for data pipelines.

Delhi, India

I specialize in maintaining high data integrity standards within Azure Data Lake by implementing data quality checks and validation processes. This ensures reliable analytical insights. Additionally, I've developed disaster recovery plans for Azure Data Lake and Azure Databricks to ensure data resilience and business continuity in unexpected events.Within Azure Data Factory, I've implemented data quality checks to ensure accuracy, completeness, and reliability of data throughout the ETL workflow. I've utilized Azure Synapse Analytics for data enrichment, working with Azure BLOB and Data Lake storage, and loading data into Azure SQL Synapse Analytics.I've integrated data governance policies in Informatica workflows to promote data quality, compliance, and accountability. Real-time data processing pipelines using Hadoop ecosystem tools have been developed to ensure timely insights from streaming data sources.Security best practices, including encryption and access controls, have been implemented in Python code to protect sensitive data. I've devised SQL scripts for checking and validating Dataflow in various applications. Seamless data migrations between NoSQL and PostgreSQL databases, while maintaining data integrity, demonstrate my expertise in ETL processes.Managing MongoDB security features and developing Spark applications using Scala for data ingestion highlight my skills. I've worked with the Spark ecosystem, integrating Spark with data warehousing solutions like Hive or HBase.In Snowflake, I've integrated with other cloud services, maintained a data catalog, and ensured interoperability within the broader technology landscape. Creating scorecards, dashboards, and visualizations using Tableau and Power BI, including automated alerts, completes my skill set in simplifying complex data processes.

Chennai, Tamil Nadu, India

I design and develop ETL processes in AWS Glue to move campaign data from sources like AWS S3 and text files into AWS Redshift. I use AWS Redshift, S3, Spectrum, and Athena to query large data on AWS S3, creating a virtual data lake without the need for ETL.For unique data transformation needs, I develop custom transformations in Informatica. I optimize queries using Hive and Pig for data analysis on Hadoop. I design Hadoop applications to analyze and extract data clusters.Using Python libraries like Matplotlib and Seaborn, I create interactive visualizations for data exploration. In code reviews, I maintain quality and conduct thorough testing of Python scripts. I design and maintain NoSQL data models for data analysis.In Spark, I develop Scala scripts for data aggregation, queries, and writing data back into OLTP systems through Sqoop. I automate loading Excel files into SQL Server and PostgreSQL databases using Spark scripts.Within Spark and Kafka pipelines, I implement data quality checks for integrity and reliability, following defined quality standards. Snowflake roles, users, and security configurations are managed for access control. I conduct capacity planning for Tableau Server and use scripting in Tableau for custom calculations.In Power BI, I handle advanced topics like complex calculations, table calculations, geographic mapping, and performance optimization. I implement security measures at Power BI Service through authentication and authorization methods.