Rafi Shaik Email and Phone Number
➡I am a Senior Data Engineer with over 12+ years of diverse experience as a Data Engineer and Analytics Engineer.➡ Skilled in Cloudera, Hadoop, Azure Databricks, PySpark, Spark (DF, SQL, Streaming), Kafka, Python, Hive, Airflow, ETL, ELT, SQL, NoSQL, Multi-Cloud Analytics (AWS, GCP & AZURE)➡Passionate about designing and implementing large and complex enterprise data environments for structured, semi-structured, and unstructured data, using the latest technologies and best practices.➡Enhancing data pipelines, adopting the robust Medallion Lakehouse Architecture, and leveraging the capabilities of Azure Databricks, GCP, and AWS. ➡Successfully migrated big data projects to the cloud, and built near real-time dashboards.➡I enjoy working with diverse teams and clients and delivering high-quality data solutions that enable data-driven decision-making and business growth.🎯SKILLS & EXPERTISE• Programming skills-----------------------: Python, PySpark, Java 1.8, Scala 2.11.• Hadoop ecosystem tools-----------------: HDFS, Sqoop, Hive, HBase, Apache Spark (DF, SQL, Streaming), NiFi.• Data Orchestration-----------------------: Apache Airflow.• Database/Data Warehouse---------------: Oracle, Microsoft SQL Server, Postgres SQL.• SQL----------------------------------------: Complex SQL Queries, Window Functions, Performance Tuning, etc. • PL/SQL------------------------------------: Stored Procedure, Function, Collections, Views, Triggers, etc.• ETL-Tool-----------------------------------: Talend Data Fabric 7. x 🎯Cloud Skills• Azure--------------------------------------: ADLS2, Databricks, Synapse Analytics, Azure SQL DWH• Google------------------------------------: GCS, Pub/Sub, Dataproc, Dataflow, Cloud SQL, Big Query.• AWS---------------------------------------: S3, Glue, Athena, RDS, RedShift, EMR, lambda, Step Functions, kinesis, Cloud Formations •
Persistent Systems
View- Website:
- persistent.com
- Employees:
- 11515
-
Senior Data EngineerPersistent Systems Nov 2023 - PresentHyderabad, Telangana, India
-
Senior Data EngineerElm Company Jan 2021 - Mar 2023Riyadh, Saudi Arabia▪ Enhanced data pipelines to efficiently process both structured and semi-structured data, channeling vast volumes of information from diverse sources into a centralized lakehouse.▪ Involved in Data Ingestion, Data Processing, and Data Analysis, adopting the robust Medallion Lakehouse Architecture with distinct (Bronze, Silver & Gold) layers, leveraging the capabilities of Azure Databricks.▪ Developed Spark jobs using DF API and Spark SQL for data cleansing, transformations, and aggregations.▪ Built near real-time dashboards by leveraging Apache Spark streaming, Kafka topics, and NoSQL DB.▪ Design optimization and performance improvement to meet end-to-end throughput requirements.▪ Successful cloud migration initiative, from on-premises to Azure Databricks cloud. -
Big Data EngineerElm Company Apr 2018 - Jan 2021Al-Riyadh Governorate, Saudi Arabiao Involved in Big Data projects on GCP using GCS, Big Query, data proc, and PySpark.o Involved in Hive and HBase design, partitioning & bucketing, and query Optimizations.o Worked on AWS Bigdata pilot project using S3, EMR, Athena, and Redshift
-
Big Data EngineerCapgemini Jun 2016 - Mar 2018Bangaloreo Interfacing with the client’s Business Analyst to understand data requirements.o Created Talend DI jobs, Joblets, Master jobs, and execution plans in TAC.o Developing the ETL detail design and unit test cases for each target table (Fact and dimension tables).o Involved in Talend BigData project for Coca-Cola on GCP -
Etl DeveloperCognizant Technology Solutions Nov 2014 - Jun 2016BangalorePart of the Philips team for the Product Data Hub MDM project responsible for ETL development using Informatica power center and Data Integration Hub (DIH)o Created Informatica PowerCenter mappings and workflows for product data hub solution.o Designed and developed complex mapping for varied transformation logic like Expression, Filter, Aggregator, Router, Joiner Update Strategy, and Unconnected & Connected lookupso Data analysis to ensure accuracy and integrity of data in the context of business functionalityo Involved in solution design for DIH and PowerCenter Components.o Comprehensive code reviews for all deliverable components.o Release documentation – deployment steps, run book creation in coordination with other teams involved in the end-to-end deployment.o Technical documentation which includes a manual for operational support teams. -
Talend Etl DeveloperL&T Infotech Aug 2011 - Nov 2014BangaloreEnterprise risk management (ERM) is the process of planning, organizing, leading, and controlling the activities of an organization in order to minimize the effects of risk on an organization's capital and earnings. This project has been developed with AbInitio implementation and as we are moving to the Talend tool the same requirement has been redeveloped using Talend data integration, Data quality, and publishing analysis report to the dashboard on UI.o Involved in requirements gathering, analysis, and design of this whole Project.o Reading messages from JMS Queues and passing them to Talend jobs.o Writing shell scripts to read data from JMS Queues and pass them to Talend flow.o Data Cleansing, Aggregating, Scrubbing, and validating source data using various Talend data integration, and quality components as per requirements.o Streamline the Deployment and Scheduling of Data Integration jobs using the TAC.o Handling administrative tasks using Talend Admin Center 5.3.1.o Involved in creating technical specification documents.o Prepared technical documentation to configure the TAC to create projects, Users, Roles, Project Authorizations, Reference and sub-projects, Branch Management, backup of repositories, Project Audit capabilities etc.
Rafi Shaik Education Details
Frequently Asked Questions about Rafi Shaik
What company does Rafi Shaik work for?
Rafi Shaik works for Persistent Systems
What is Rafi Shaik's role at the current company?
Rafi Shaik's current role is Senior Data Engineer | Databricks | Cloudera | Hadoop | PySpark | Python | Airflow | Batch & Streaming | SQL & NoSQL | AZURE | AWS | GCP | ETL | ELT | Talend.
What schools did Rafi Shaik attend?
Rafi Shaik attended Andhra University.
Who are Rafi Shaik's colleagues?
Rafi Shaik's colleagues are Samrudhi Dant, Aditya Sontakke, Ankita Burele, Gaurav Chaudhari, Anjali Wadhwa, Chetan Pandey, Tushar Roy.
Not the Rafi Shaik you were looking for?
-
Rafi Shaik
It Doesn'T Matter How Much Knowledge Do You Have Until You Share And Put Into Action ✌️Hyderabad -
rafi Shaik
Meet The Visionary Technical Lead Revolutionizing The Industry|To Be Aspired As A Agile Coach|Intgration SpecialistHyderabad1yahoo.co.in -
-
Rafi shaik
Bangalore Urban
Free Chrome Extension
Find emails, phones & company data instantly
Aero Online
Your AI prospecting assistant
Select data to include:
0 records × $0.02 per record
Download 750 million emails and 100 million phone numbers
Access emails and phone numbers of over 750 million business users. Instantly download verified profiles using 20+ filters, including location, job title, company, function, and industry.
Start your free trial