Data Engineer
Current• AWS Data Pipeline Expertise: Designing and implementing AWS architecture to establish end-to-end data pipelines for different data formats encompassing ingestion, storage, transformation, visualization and optimizing performance.• AWS Services Proficiency: I work extensively with Amazon Redshift for data warehousing, AWS Glue for data ETL (Extract, Transform, Load), API Gateway for building and managing APIs, Amazon S3 for object storage, ECS Fargate for containerized applications, DynamoDB for NoSQL database needs, AWS Lambda for serverless computing, AWS CodeBuild for continuous integration, and RDS (Relational Database Service) for managing relational databases.• Cross-functional Collaboration: Collaborating closely with cross-functional teams to identify business requirements and develop data-driven solutions.• Client Communication: Regularly engaging with clients to gather, resolving issues and refine project requirements.• Deployment Skills: Deploying data applications on AWS using EC2, Fargate and ECS services, ensuring scalability and reliability.• Data Quality Assurance: Conducting data cleaning and pre-processing to uphold data quality and consistency standards.• Data Visualization: Utilizing tools like Python Dash, Power BI, and QuickSight for data visualization to uncover trends, generate reports, identify patterns, and reveal data relationships.• Map Visualization: Using tools and libraries such as Mapbox, Leaflet, or geospatial extensions for spatial data visualization tools like Power BI and QuickSight to create interactive maps and visualizations. • Version Control: Managing code and project assets using Bitbucket as a code repository.• Agile Methodology: Operating within an Agile software methodology framework, utilizing JIRA for project management.• Documentation: I use Confluence for documentation and collaboration, seamlessly integrating it with JIRA for Agile project management, to ensure efficient project documentation.