Senior Data Engineer
CurrentData engineer/cloud engineer consultant:- Data ingestion pipeline design and implementation (AWS) - Batch (event-driven: s3, eventBridge, lambda, glue, rds) - Streaming (event-driven: event mapping, eventBridge, kafka, lambda, kinesis data streams)- Docker image creation for glue and lambda unit tests- CICD integration (Docker image and SonarQube as part of the CI section)- Operational monitoring via Opensearch (event-driven, eventBridge, Firehose, lambda, Opensearch Domain)- Orchestration (airflow)- Data analysis (Snowflake, Microsoft server)- ETL via AB Initio on EC2 instances- Data pipeline generation- Data processing via databricks (SparkSQL)