Yi-Jia Su Email & Phone Number

Taiwan

台灣 新北市 新店

-Big Data Processing: Hands-on experience with open-source big data tools like Apache Spark for data processing and analysis in cloud environments.-Data Warehousing & Data Lakes: Extensive experience in designing and implementing data warehouses, data lakes, and enterprise-level big data platforms.-Data Pipeline Optimization: Continuously improving data pipelines, related processes, and architectures with a focus on cost-effectiveness to enhance overall operational efficiency.-Data Governance & Security: Implementing data governance based on best practices to ensure the security and compliance of company and customer data.-Infrastructure as Code (IaC): Practical experience managing cloud resources using IaC tools like AWS CloudFormation and AWS CDK and CDKTF, enhancing automation and consistency in resource management.

台灣 臺中市

Custom Monitoring & Visualization:Developed custom monitoring scripts using Azure service APIs (e.g., Databricks Job API, Azure Bill API) to enhance insights beyond standard Azure Monitor capabilities.Analyzed job execution times, identifying anomalies to optimize performance.Segmented service costs by region/department using Azure Bill API and custom tags, enabling real-time cost surge detection.Designed an automated ETL pipeline using Azure Databricks and Synapse Serverless, with data visualized in Tableau dashboards to support decision-making.Reduced unnecessary expenses by 5-10% monthly through continuous monitoring and optimization.Azure Databricks Workspace Management:Managed Databricks Workspace, including cluster configuration, ETL scheduling, SQL Warehouse, and Delta Live Tables.Provided solutions and adjustments tailored to internal user needs, improving data flow and resolving issues.Promoted new technologies and maintained effective communication with vendors.Monitored and managed multiple Databricks Workspaces using digital dashboards.Databricks UC Data Governance Implementation:Led the transition of multiple Databricks Workspaces from non-UC to UC environments, using REST APIs to streamline migration.Implemented Databricks UC tools for permissions, data lineage, and secure sharing, enhancing data governance.Provided training and technical support, improving internal data management practices.Azure Synapse Management & Backup Mechanism:Secured Azure Synapse environment with role-based access control and executed complex queries using Synapse SQL/Spark.Optimized performance and cost-efficiency with serverless capabilities.Developed automated backup ETL processes in Databricks, and integrated Azure CLI for rapid workspace creation.Led database migration and ensured smooth operations as Database Administrator.

台灣 臺北市 內湖區