Associate Specialist
CurrentData Engineer, Data Analyst, Data Integrator, and Big Data Specialist, demonstrating expertise in Data Modeling (Logical and Physical) as well as implementing Business Applications using the Oracle Relational Database Management System (RDBMS).Extensive hands-on experience in a wide range of technologies such as Hadoop, MapReduce, Hive, Spark (PySpark), Sqoop, Airflow, Autosys, Snowflake, Teradata, Oracle, RDMS, Python, Scala, ETL, and Data Visualization.Proven track record of delivering successful data and analytics solutions using cloud data lakes, with a focus on AWS, Azure, and similar Cloud platforms.Proficient in utilizing various AWS services including Amazon EC2, Amazon S3, Amazon RDS, VPC, IAM, Amazon Elastic Load Balancing, Auto Scaling, CloudFront, CloudWatch, SNS, SES, SQS, and others.A notable achievement in developing PySpark Streaming applications capable of processing both static and streaming data from diverse sources.Adept at adapting to different industries and demonstrating flexibility in utilizing cutting-edge technologies in the field of data engineering and analytics.Proficient in SQL script analysis and designing PySpark solutions for efficient data processing.Leveraged Spark Streaming to receive real-time data from Apache, storing it in HDFS using Python, and seamlessly integrating with NoSQL databases such as HBase and Cassandra.Developed Spark/Scala and Python projects in the Hadoop/Hive environment, utilizing regular expressions (regex) to handle large-scale data processing on both Linux and Windows platforms.Implemented data transformations using AWS Glue dynamic frames in conjunction with PySpark to ensure streamlined data processing.Utilized Scala components to design and implement credit line policies, applying them to Spark data frames for efficient credit evaluation.Designed and executed real-time stream processing applications using Spark, Kafka, Scala, and Hive, enabling