I am a Principal data solutions architect and accomplished data leader with 20+ years experience in architecting, designing and implementing big data, data lake and data warehouse solutions in the cloud, for leading healthcare and financial services organizations. I am highly passionate about analyzing information systems and data ecosystems, evaluating business data requirements and translating them into data architecture and management strategies, creating the overall technical blueprint/framework, that defines, how the data will be ingested, modeled, integrated, organized and stored in the cloud data lake, by configuring ETL pipelines managing complex data structures, for both batch & streaming data, to enable data-driven decision-making.My keen focus while developing applications is on code efficiency, maintainability and scalability by following industry standard data quality, data governance & performance optimization best practices.My Core Competencies:Cloud engineering | Big Data, Data Lake | Data integration - ETL solutions | Data strategy-quality-governance | Data Architecture | Data modeling | IT Leadership | Agile Methodology | Data analytics | Pipeline AutomationMy Technical skills:Cloud AWS:- Databricks (AWS), Redshift, Glue, EMR, Lambda, Athena, Terraform
Big Data:- Apache Hadoop, Spark, Hive, HDFS, Impala, Hue
Data streaming:- Spark streaming, DLT (Databricks) using Auto Loader, Apache KafkaData workflow:- Apache Airflow, Databricks WorkflowsLanguages:- Python/PySpark, Advanced SQL, Unix shell scripting