Senior Data Engineer
CurrentDeveloping and design ELT/ETL processes and data pipelines and analyzing data at scale on a big data ecosystem, Integrating 3rd party API’s, and Designing data lake and Lakehouse modeling.● Languages: Python, Pyspark, SQL.● Orchestration: Airflow.● Processing: Spark, Kafka.● Cloud provider: AWS services – Databricks, S3, Athena, Lambda, RDS, Glue, Secret manager etc.● CI/CD: CircleCI.● Version control: GitHub● Infrastructure as code: Terraform and serverless.● DB: MongoDB, Redis, Postgres, MySQL, SQL Server