Possess 9 years of Data Engineering experience, specialized in designing and implementing data models for enterprise-level applications and BI solutions, spanning Storage Analysis, Integration, Data Processing, and Warehousing. Firm knowledge in programming languages like Python (its various libraries like NumPy, Pandas for data Wrangling, manipulation, and Visualization), SQL, Spark integrated with Scala, (for Restful APIs) with a good flexibility on variable platforms and IDEs. Worked with both NoSQL like DynamoDB, MongoDB and SQL like MySQL, PostgreSQL databases. Great Knowledge in managing the Data Management Lifecycle, covering Ingestion using tools like Apache Kafka, Azure Data Factory, Flume, Integration SQL-based tools, Spark, Hadoop, and Consumption (Tableau, Power BI). Proficient in monitoring data pipelines for performance and reliability using Azure Monitor and CloudWatch. My skills extend to Hadoop's architecture, including components like HDFS, Job Tracker, Task Tracker, and MapReduce, enhancing ETL methods and Data engineering pipelines. Highly skilled at utilizing Hive for data warehousing including table creation, bucketing for data distributions and Apache Kafka for real-time data processing. I excel in orchestrating workflows with Airflow and have proficiency in version control Git, GitLab, SVN, Azure Repos and deployment tools like Jenkins and AWS Code Deploy. My documentation and knowledge management skills using tools like Confluence and SharePoint facilitate team collaboration and maintain comprehensive documentation. I'm enthusiastic about bringing my expertise to the table, actively contributing to innovation, and driving towards continuous improvement within the organization.
