Data Engineer
Current- Minimized computation costs and optimized resource usage by implementing a Lambda service to match unique identifiers from potential repeated EventBridge messages against a DynamoDB table to eliminate all redundant step function executions.- Streamlined the development of unit tests for functions incorporating frequently recurring logic within the Hamilton framework by introducing a library of standardized pytest parameters, resulting in a 70% reduction in development time of the unit tests.- Led an initiative to standardize coding and documentation practices by introducing comprehensive coding and docstring guidelines, reinforced through the integration of the Ruff linter into the codebase to ensure widespread adherence to established rules.- Introduced a systematic approach of assigning story numbers to code docstrings to enhance code traceability and improve project transparency, enabling team members to easily pinpoint corresponding story details on the Sprint board.- Refactored and streamlined a Step Function by transitioning it to an event-driven model and incorporating a response service, enhancing the logging mechanism to capture crucial details and responses for improved tracking and analysis.