Andy Chew Email & Phone Number

Sydney, New South Wales, Australia

• Data Lake: implemented a data lake architecture on Amazon S3 to store large-scale e-commerce user behavior and order data. Managed the ingestion pipeline using Databricks, ensuring efficient data storage and accessibility for downstream analysis and modeling.• ETL Pipelines and OLAP Reporting System: Built and maintained automated ETL pipelines in Databricks, extracting data from S3 and transforming it into structured formats for analysis in Amazon Redshift. Developed an OLAP system to… Show more • Data Lake: implemented a data lake architecture on Amazon S3 to store large-scale e-commerce user behavior and order data. Managed the ingestion pipeline using Databricks, ensuring efficient data storage and accessibility for downstream analysis and modeling.• ETL Pipelines and OLAP Reporting System: Built and maintained automated ETL pipelines in Databricks, extracting data from S3 and transforming it into structured formats for analysis in Amazon Redshift. Developed an OLAP system to support Power BI dashboards, providing insights into user activity, purchase trends, and operational metrics.• User Profiling and Feature Engineering: Developed comprehensive user profiles by aggregating behavior and transaction data. Applied advanced feature engineering in Databricks using PySpark, creating features such as session duration, category preferences, search patterns, and purchase behaviors, which were leveraged for personalized recommendations and search ranking models. Show less

New South Wales, Australia

• Data Integration and Migration: Utilized AWS Glue to extract data from MongoDB and PostgreSQL, including event tracking and order data, and performed data cleaning and transformation before loading it into Amazon Redshift. Automated the data processing workflows by setting up Glue Jobs.• Data Modeling and Metrics Calculation: Designed a star schema centered around user behavior and order data, creating fact and dimension tables in Redshift. Wrote SQL queries to calculate key business… Show more • Data Integration and Migration: Utilized AWS Glue to extract data from MongoDB and PostgreSQL, including event tracking and order data, and performed data cleaning and transformation before loading it into Amazon Redshift. Automated the data processing workflows by setting up Glue Jobs.• Data Modeling and Metrics Calculation: Designed a star schema centered around user behavior and order data, creating fact and dimension tables in Redshift. Wrote SQL queries to calculate key business metrics such as user conversion rates and order repeat rates, generating multiple essential operational reports.• Data Visualization and Reporting: Integrated Quicksight to build data visualization dashboards, showcasing key metrics like user conversion rates, order success rates, and customer retention rates, to support data-driven business decisions. Show less

Sydney, New South Wales, Australia

Research & Develop Chatbot(GenAI)• Development: Choose a Base Model: Use the 13B LLaMA2 model；Data Collection: Gather data from Wikipedia and ChatGPT responses.；QA Dataset: Convert the data into a Question-Answer format.Fine-Tuning: Use LoRA (Low-Rank Adaptation) to adapt the model specifically for financial and medical topics.• Optimization: Merge the adapted model with the base model；Test and adjust it to make sure it responds accurately in financial and medical… Show more Research & Develop Chatbot(GenAI)• Development: Choose a Base Model: Use the 13B LLaMA2 model；Data Collection: Gather data from Wikipedia and ChatGPT responses.；QA Dataset: Convert the data into a Question-Answer format.Fine-Tuning: Use LoRA (Low-Rank Adaptation) to adapt the model specifically for financial and medical topics.• Optimization: Merge the adapted model with the base model；Test and adjust it to make sure it responds accurately in financial and medical contexts.• Deployment & Track: Set Up Infrastructure: Use cloud services to run the model；API Access: Make the model accessible via API for easy integration；Containerize: Use Docker for portability, with Kubernetes for scaling； Monitor Performance: Track the chatbot’s speed and accuracy, and improve based on user feedback. Show less

• Data Warehouse: Assisted in the development of a Metrics Monitoring System (V1.0) and incrementally built a data warehouse, calculating and verifying corresponding metrics to provide market business intelligence.• User Profiling: Integrated user data from various sources to construct comprehensive user profiles, employing machine learning algorithms to handle missing values effectively.• User Journey Analysis: Conducted pattern matching and scenario reconstruction for user behaviour… Show more • Data Warehouse: Assisted in the development of a Metrics Monitoring System (V1.0) and incrementally built a data warehouse, calculating and verifying corresponding metrics to provide market business intelligence.• User Profiling: Integrated user data from various sources to construct comprehensive user profiles, employing machine learning algorithms to handle missing values effectively.• User Journey Analysis: Conducted pattern matching and scenario reconstruction for user behaviour logs across different user segments, categorizing user journey types and identifying attrition points. Leveraged churn user profile characteristics to pinpoint business entry points for further analysis, ultimately improving key metrics such as user watch duration.• Cross-Platform Modeling: Performed data mining on active users from both the mobile and TV. Developed predictive models based on user profiles using LightGBM, generating key features and their distributions to support data-driven decision-making. Show less

1. Data Warehouse + Governance• Key Responsibilities: Integrated 9 data domains (user behavior, business activities, transactions, and sales) using Snowflake for data warehousing and governance. Employed dbt (data build tool) for 4-layer dimensional modeling and constructed user profiles. Analyzed sales operation history and order dimensions, and maintained data indicators. Managed ETL workflows with Airflow and executed high-performance queries with SnowSQL. Ensured data storage… Show more 1. Data Warehouse + Governance• Key Responsibilities: Integrated 9 data domains (user behavior, business activities, transactions, and sales) using Snowflake for data warehousing and governance. Employed dbt (data build tool) for 4-layer dimensional modeling and constructed user profiles. Analyzed sales operation history and order dimensions, and maintained data indicators. Managed ETL workflows with Airflow and executed high-performance queries with SnowSQL. Ensured data storage efficiency with Snowflake's compression capabilities and improved data accuracy by managing table lifecycles and monitoring data quality.2. Real-time and Offline BI Dashboard• Designed fact and dimension tables using dimensional modeling techniques to ensure efficient data querying and analysis. Built an efficient data pipeline leveraging Kafka for data streaming and Flink for real-time processing, with processed data directly ingested into Redshift. Leveraged Tableau to create interactive dashboards that allow users to visualize and analyze both real-time and historical data. Implemented dynamic filtering options within Tableau to enable users to select and view data across different time periods, providing comprehensive insights for business decision-making.3. Data services for CRM and AI• Key Responsibilities: Designed efficient ETL processes, leveraging Hive tables with timestamped data to populate detailed HBase tables. Developed daily user profiles in HBase and conducted feature engineering based on these profiles, expanding features across product lines and time dimensions. The processed features were then stored in S3 for further analysis and machine learning applications. Managed ETL workflows using Airflow, automating the pipeline to ensure the timely availability of high-quality feature data for AI models. Show less

Master Of Computer Science

Master Of Information Technology, Information Technology

Bachelor Of Engineering - Be, Electronics Engineering