Over 9+ years of IT industry knowledge with hands on working experience in Data Engineering & Data Analysis.Good knowledge in Data Quality & Data Governance practices & processes.Well versed with Agile with Scrum, Waterfall Model and Test-driven Development (TDD) methodologies.Over 4+ years of experience in Golang . Excellent coding and problem - solving skills with ability to work as Developer.Strong working knowledge in developing Restful webservices and Micro Services using Golang .Implemented REST services by Golang with microservices architecture.Experience on Migrating SQL database to Azure data Lake, Azure data lake Analytics, Azure SQL Database, DataBricks and Azure SQL Data warehouse and Controlling and granting database access and Migrating On premise databases to Azure Data lake store using Azure Data factory.Proficient with complex workflow orchestration tools namely Oozie, Airflow, Data pipelines and Azure Data Factory, CloudFormation & Terraforms. Proficient in SQLite, MySQL and SQL databases with Python.Practical understanding of the Data modeling (Dimensional & Relational) concepts like Star - Schema modeling, Snowflake Schema Modeling, Fact and Dimension tables.Experience in handling python and spark context when writing PySpark programs for ETL.Strong knowledge in data visualization using Power BI and Tableau.Hands in experience on NoSQL database like Snowflake, HBase, Cassandra and MongoDB.Experience in building and architecting multiple Data pipelines, end to end ETL and ELT process for Data ingestion and transformation in GCP and coordinate task among the team.Experience in GCP Dataproc, GCS, Cloud functions, Big Table and Big Query.Experience with Apache Spark ecosystem using Spark-Core, SQL, Data Frames and RDD's.Experienced in data manipulation using python.Hands on experience working Azure using Elastic Map Reduce (EMR), Redshift, and EC2 for data processing.Experience on Palantir Foundry and Data warehouses (SQL Azure and Confidential Redshift.orchestration and familiar with building custom Airflow operators and orchestration of workflows with dependencies involving multi-cloudsStrong understanding of Data Modelling (Relational, dimensional, Star and Snowflake Schema), Data analysis, Palantir Foundry, implementations of Data warehousing using Windows and UNIX.Proficient in installing, configuring and using Apache Hadoop ecosystems such as MapReduce, Hive, Pig, Flume, Yarn, HBase, Sqoop, Spark, Storm, Kafka, Oozie, and Zookeeper.
-
Sr. Azure Data EngineerHumana Oct 2021 - Aug 2023Texas, United StatesInstalled and developed with Apache Bigdata Hadoop components like HDFS, MapReduce, YARN, Hive, HBase, Sqoop, Pig, Ambari and Nifi.Design and implement database solutions in Azure SQL Data Warehouse, Azure SQL.Design & implement migration strategies for traditional systems on Azure (Lift and shift/Azure Migrate, other third-party tools.Migrated from JMS solace to Apache Kafka and used Zookeeper to manage synchronization, serialization, and coordination throughout the cluster.Improved execution by employing distributed caching for small datasets, partitioning, bucketing in Hive, and Map Side Joins.Developed new RESTful API services that work as a middleware between our application and third-party APIs that we will used using Golang.Using GO, developed a microservice for reading large volume of data(millions) from PostgreSQL database.Experience writing data APIs and multi-server applications to meet product needs using Golang.Experience in writing the HTTP RESTful Web services and SOAP API's in GolangCreate and maintain the data pipelines using Matillion ETL, FivetranMaintained data engineering solutions by monitoring, automating, refining them on a regular basis.Created a few Databricks Spark jobs with Pyspark to execute numerous table-to-table operations.Experienced with agile and waterfall methodologies in a fast-paced environment.Stored the log files in Azure S3. Used versioning in S3 buckets where the highly sensitive information is stored.Evaluated Fivetran and Matillion for streaming and batch data ingestion into Snowflake.Integrated Azure Dynamo DB using Azure lambda to store the values of items and backup the DynamoDB streams. -
Sr. Azure Data EngineerDsv - Global Transport And Logistics Feb 2020 - Sep 2021North Carolina, United StatesWorked Azure SQL Database Environment.Lead the estimation, review the estimates, identify the complexities and communicate to all the stakeholders.Involved in complete SDLC life cycle of big data project that includes requirement analysis, design, coding, testing and production.Defined the business objectives comprehensively through discussions with business stakeholders, functional analysts and participating in requirement collection sessions. Implemented end-to-end systems for Data Analytics, Data Automation and integrated with custom visualization tools.Design Setup maintain Administrator the Azure SQL Database, Azure Analysis Service, Azure SQL Data warehouse, Azure Data Factory, Azure SQL Data warehouse.Mesa code in helping data transformation and data integration required for Palantir Analytic use cases.Created airflow DAG’s to sync files from box, analyze data quality, and alert for missing files.Designed the business requirement collection approach based on the project scope and SDLC (Agile) methodology.Built an ETL framework for Data Migration from on premise data sources such as Hadoop, Oracle to Azure using Apache Airflow, Apache Sqoop and Apache Spark (PySpark).Created external tables in ADW with 4 compute nodes and scheduled.Extensively used Agile Method for daily scrum to discuss the project related information.Configured Spark streaming to receive real time data from the Kafka and store the stream data to HDFS. Data Engineering - Snowflake, Python, FiveTran, dbt.Developed a data pipeline using Kafka to store data into HDFS. Implemented Kafka producers create custom partitions, configured brokers and implemented High level consumers to implement data platform. Created Airflow Scheduling scripts in Python. -
Sr. Azure Data EngineerSuhan Infotech Private Limited Mar 2017 - Nov 2019IndiaAs a Sr. Data Engineer designed and deployed scalable, highly available, and fault tolerant systems on Azure.Migrated on-primes environment on Cloud using MS Azure. Moved data to Azure Data Lake to Azure data warehouse using PolybasicWorked with data ingestions from multiple sources into the Azure SQL data warehouseDevelopment and maintenance of data pipeline on Azure Analytics platform using Azure Databricks.PerformedData Ingestionusing Azure Services-(Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in Azure Databricks.Integrated Python and SQL to process data frames by integrating millions of raw records on Azure DataBricks.Working on Azure Data Pipeline to setup pipeline to ingest data from Spark and migrate to Snowflake DatabaseImplemented report generation onto Power BI using Synapse databases.Used Azure Data Factory as an orchestration tool for integrating data from up to downstream systems.Implemented database solutions in Azure SQL Data Warehouse, Azure SQL.Created job schedules with python notebooks and libraries in Azure DataBricks platform.Used Azure Data Factory extensively for ingesting data from distinct source systems.Migrated SQL and Oracle DB's to Azure Cloud using Azure database Migration Service (AMS).Created Pipelines in ADF using Linked Services to Extract, Transform and load data from different sources like Azure SQL, Azure SQL Data warehouse.Created data pipelines for handling Streaming data from web server console logs.Developed Pyspark process to perform enrichment of clickstream data merged with user profile data.Transformed and loading data into Azure SQL Database -
Aws Data EngineerCubic Corporation Oct 2015 - Feb 2017IndiaExperience in end-to-end design and deploy rich Graphic visualizations with Drill Down and Dropdown menu option and Parameterized using Tableau. Proficient in SQL databases MSSQL Server, MySQL (RDBMS), Oracle DB, Postgres, and MongoDB. Expert in various Azure services like Compute (Web Roles, Worker Roles), Caching, Azure SQL, NoSQL, Storage, and Network services, Azure Active Directory (AD), API Management, Scheduling, Azure Auto Scaling, and Azure shell, ARM, PowerShell Automation. Developed strategy for cloud migration and implementation of best practices using AWS services like database migration service, AWS server migration service from On-Premises to cloud. Responsible for Setup and build AWS infrastructure using resources VPC, EC2, S3, Dynamo DB, IAM, EBS, Route53, SNS, SES, SQS, Cloud Watch, Cloud Trail, Security Group, Auto scaling and RDS using Cloud Formation templates. Backing up AWS Post GREtoS3on daily job run on EM Rising Data Frames. Implementation of new tools such as Kubernetes with Docker to assist with auto-scaling and continuous integration (CI) and upload a Docker image to the registry so the service is deployable through Kubernetes. Use the Kubernetes dashboard to monitor and manage the services. Experience in implementing AWS lambda to run servers without managing them and to trigger run code by S3 & SNS.Created program in python to handle PL/SQL functions like cursors and loops which are not supported by snowflake. -
Gcp Data EngineerPike Solutions, Inc. Jun 2012 - Sep 2015Worked on analyzing Hadoop cluster and different big data analytic tools including Hive and Sqoop.Develop data pipeline using Sqoop and MapReduce to ingest current data and historical data in data staging area.Experience in GCP Dataproc, GCS, Cloud functions, BigQuery.Have Extensive Experience in IT data analytics projects, Hands on experience in migrating on premise ETLs to Google Cloud Platform (GCP) using cloud native tools such as BIG query, Cloud Data Proc, Google Cloud Storage, Composer.Responsible for defining Azure data flow in Hadoop ecosystem to different teams. Wrote Pig scripts for data cleansing and data transformation as ETL tool before loading in HDFS.Create Managed tables and External tables in Hive and load data from HDFS.Performed query optimization for HiveQL and denormalized Hive tables to increase speed of data retrieval.
Frequently Asked Questions about Huda R
What is Huda R's role at the current company?
Huda R's current role is Actively looking for Azure, AWS Data engineer C2C roles |Azure certified | Python | PySpark | SQL | Databricks | Spark | Azure | AWS | ETL | GCP | Kafka | Airflow | Snowflake | Big Data.
Not the Huda R you were looking for?
-
1gmail.com
Free Chrome Extension
Find emails, phones & company data instantly
Aero Online
Your AI prospecting assistant
Select data to include:
0 records × $0.02 per record
Download 750 million emails and 100 million phone numbers
Access emails and phone numbers of over 750 million business users. Instantly download verified profiles using 20+ filters, including location, job title, company, function, and industry.
Start your free trial