Subha B Email & Phone Number

United States

Texas, United States

As an Azure Data Engineer, I led the development and maintenance of end-to-end ETL data pipeline operations in Azure Data Factory, managing large datasets seamlessly. Leveraging Azure Functions, Azure Databricks, and PowerShell scripts, I crafted custom activities for data transformations, cleaning, and validation. My expertise extended to utilizing Azure Data Lake for scalable storage and analytics, enabling efficient ingestion, storage, and processing of structured and unstructured data. I harnessed Azure Synapse analytics for advanced data analytics, including complex transformations and statistical analysis, facilitating data-driven decision-making. Additionally, I designed Azure Data Factory pipelines for data extraction, transformation, and loading from diverse sources like Azure Data Lake, Azure SQL, and Blob storage, integrating real-time data ingestion via Azure EventHub. I optimized Azure Integration Runtime settings, enhanced throughput with Data Integration Units (DIUs), and implemented data archival strategies in Azure Synapse Analytics. My role involved designing solutions for NRT data processing using Azure Stream Analytics, Azure Event Hub, and Service Bus queue, as well as deploying Azure Cosmos DB for efficient data storage and event sourcing. I also automated workflows using Apache Airflow, Logic Apps, and Power Automate, along with continuous integration and deployment using Azure DevOps. Overall, I applied Agile methodologies to deliver documented solutions and ensure seamless data management and integration.

Texas, United States

In this project, I played a pivotal role in implementing robust data solutions using Azure services. I extensively utilized Azure Data Factory to ingest data from various sources, including on-premises and cloud-based systems, and optimized Azure Functions for efficient data processing and integration. Leveraging Azure Synapse Analytics, I facilitated big data processing and advanced analytics, while also implementing Key Vault integration for enhanced security measures. My expertise extended to designing event-driven data pipelines with Azure Event Hubs, deploying data storage solutions using Azure services, and optimizing data models for efficient storage and retrieval. I also leveraged tools like Azure Databricks and Synapse Pipelines for comprehensive data transformations, managed ETL/ELT workflows, and implemented monitoring systems for workflow execution. Additionally, I contributed to Agile methodologies and established CI/CD frameworks for data pipelines using Azure DevOps, ensuring streamlined project management and deployment processes.

Los Angeles, California, United States

In my capacity as a Big Data Developer, I have been instrumental in designing and implementing data pipeline jobs for extracting data from diverse source systems. I have a strong command over Hadoop infrastructure components like Pig, Hive, and HBase, contributing to their upgrading, configuration, and maintenance. My expertise lies in orchestrating data pipelines using Sqoop, Flume, MapReduce, and Kafka for ingesting, transforming, and processing customer behavioral data to enable advanced analytics. I've performed extensive data aggregation and analysis on large datasets using Apache Spark, Scala, and Hive, leading to valuable business insights and enhanced data processing capabilities. Additionally, I've leveraged Python scripting and PySpark for efficient data transfer and distributed data processing, ensuring efficient and scalable operations. I've designed and implemented automated workflows with Oozie, integrated HBase with Hive for optimized data querying, and developed Spark applications using Scala and Spark SQL for streamlined data operations. My experience includes end-to-end data processing pipelines, real-time data processing with Spark Streaming, and successful migrations using Sqoop. I've also utilized Terraform for infrastructure management, created visually compelling visualizations with Tableau and Power BI, and collaborated with teams using JIRA and GIT for project management and version control.

Los Angeles, California, United States

Working as a Big Data Developer, I have implemented efficient data imports from MySQL to HDFS using Sqoop and conducted aggregations on large datasets using Apache Spark and Scala, storing results in the Hive data warehouse. I have extensive experience working with Data Lakes and big data ecosystems like Hadoop, Spark, Hortonworks, and Cloudera, developing multiple MapReduce programs for data extraction and aggregation from diverse sources with various file formats. I have administered and transformed data encompassing diverse file formats such as Avro, Parquet, Sequence files, ORC, and Text/CSV, leveraging Hive queries for data analysis and addressing specific business requirements. Additionally, I have utilized Kafka and Spark Streaming for processing streaming data, developed data pipelines using Flume and Sqoop for ingesting customer behavioral data, and conducted data cleansing using Pig. My experience also includes migrating data from RDBMS to Hadoop using Sqoop, optimizing Hive queries, analyzing data using HiveQL and Pig Scripts, and developing custom scripts and tools for data validation and transformation. I have a deep understanding of schedulers, workload management, availability, scalability, and distributed data platforms, and have employed Spark APIs in Scala, PySpark, Spark SQL, and Spark Streaming for real-time analysis and processing. I have also implemented CI/CD pipelines, conducted performance tuning, managed issue and project workflows using JIRA, and utilized Git for version control.

India

In my role as a Data Warehouse Developer, I have been responsible for ETL development using SSIS, extracting data from various sources like SQL Server, MS Access, and Excel sheets, and executing data migration and transformation processes. I have built OLAP cubes using SSAS, optimized SQL Server performance, and applied tuning methods to enhance query execution efficiency. My experience includes dimensional data modeling, designing Data Mart structures, developing logical and physical data models using concepts like Star Schema and Snowflake Schema, and implementing complex Mappings and Transformations for data loading. I have also developed PL/SQL Stored Programs for data transformations, utilized Informatica ETL tool, and created reports using SSRS from OLTP and OLAP data sources. Additionally, I have experience with DAX functions in SSAS Tabular Models, Power BI, and Excel, and have worked with software development methodologies like Agile/SCRUM AGILE.