Swetha L Email & Phone Number

Frisco, Tx, Us

San Diego, California, Us

 Led migration of SSIS packages to Azure Databricks platform for SunRx MadImpact healthcare project. Proficient in SSIS packages, SSRS reports, and SQL Server Management Studio (SSMS). Expertise in SQL Server, including stored procedures, queries, and database administration tasks. Implemented ETL processes using Azure Data Factory for data ingestion and orchestration. Leveraged Databricks for data transformation tasks using Python and PySpark. Utilized Unity catalog features such as Metastore for metadata management and data workflows. Developed data processing pipelines for big data using Azure Databricks. Experience in designing and optimizing data models for efficient data storage and retrieval. Familiarity with Power BI for creating interactive and insightful reports. Collaborated with cross-functional teams to ensure smooth migration and integration of SSRS reports to Power BI. Proficiency in Python programming language for scripting and automation tasks. Knowledge of big data processing frameworks and distributed computing concepts. Strong understanding of healthcare data domains including patient information, claims processing, and revenue tracking. Demonstrated ability to analyze complex business requirements and translate them into technical solutions. Excellent communication skills and ability to articulate technical concepts to non-technical stakeholders.

Foster City, California, Us

 Implemented SSIS IR to run SSIS packages from ADF. Developed mapping document to map columns from source to target. Created azure data factory (ADF pipelines) using Azure blob. Performed ETL using Azure Data Bricks. Migrated on-premises Oracle ETL process to Azure Synapse Analytics. Worked on python scripting to automate generation of scripts. Data curation done using azure data bricks. Created a framework to do data profiling, cleansing, automatic restart ability of batch pipeline and handling rollback strategy. Design and implement database solutions in Azure SQL Data Warehouse, Azure SQL Lead a team of six developers to migrate the application. Implemented masking and encryption techniques to protect sensitive data. Worked on azure data bricks, PySpark, HDInsight, Azure ADW and hive used to load and transform data. Implemented and Developing Hive Bucketing and Partitioning. Creating Spark clusters and configuring high concurrency clusters using Azure Databricks to speed up the preparation of high-quality data. Extract Transform and Load data from Sources Systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, Spark SQL, and U-SQL Azure Data Lake Analytics.  Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in in Azure Databricks. Architect and implement ETL and data movement solutions using Azure Data Factory, SSIS Create and maintain optimal data pipeline architecture in cloud Microsoft Azure using Data Factory and Azure Databricks

Memphis, Tn, Us

 Designing and implementing scalable ETL pipelines to process variety of data types (structured, unstructured), file formats (Json, csv, text delimited…) Databricks job configuration, Refactoring of ETL Databricks notebooks Created pipelines to load data from Lake to Databricks and Databricks to Azure SQL DB. Utilized Spark SQL API in PySpark to extract and load data and perform SQL queries. Developed data pipeline using Spark, Hive, python to ingest customer. Created/modified the existing SQL views as needed. Created Power BI reports and dashboards as per the business requirement using different data sources. Extensively used DAX to create complex calculated measures and columns in power bi and cubes. worked on an Azure copy to load data from an on-premises SQL server to an Azure SQL Data warehouse. Worked on redesigning the existing architecture and implementing it on Azure SQL. Worked on migrating MapReduce programs into Spark transformations using Python. Created pipelines using ADF to run SQL scripts. Prepared ETL design document which consists of the database structure, change data capture, Error handling, restart and refresh strategies. Data warehouse solutions using polybase/external table on Azure Synapse/Azure SQL Data warehouse (Azure DW), Using Azure Data Lake as source. Rewriting exiting SSAS cubes to Azure Synapse/Azure SQL Data warehouse (Azure DW). Implement ad-hoc analysis solutions using Azure Data Lake Analytics/Store, HDInsight. Used TES Scheduler engine to manage interdependent Hadoop jobs and to automate several types of Hadoop jobs such as Java map-reduce Hive, Spark, Kafka and Sqoop.

New York, Ny, Us

 Contributed to the development of PySpark Data Frames in Azure Data Bricks to read data from Data Lake or Blob storage and utilize Spark SQL context for transformation. Experience in Creating, developing, and deploying high-performance ETL pipelines with PySpark and Azure Data Factory. Experience with Azure SQL database configuration and tuning automation, vulnerability assessment, auditing, and threat detection. Integration of data storage solutions in spark – especially with Azure Data Lake storage and Blob snowflake storage. Developed ETL pipelines in and out of data warehouse using a combination of Python and Snowflakes Snow SQL Writing SQL queries against Snowflake. Involved in Migrating Objects from Teradata to Snowflake and created Snow pipe for continuous data load. Improving the performance of Hive and Spark tasks. Created Hive scripts from Teradata SQL scripts for data processing on Hadoop. Developed Hive tables to hold processed findings, as well as Hive scripts to convert and aggregate heterogeneous data.

 Developed data pipeline using Flume, Sqoop, Pig and Java MapReduce to ingest customer behavioral data into HDFS for analysis.  Used Python & SAS to extract, transform & load source data from transaction systems, generated reports, insights, and key conclusions. Installed/Configured/Maintained Apache Hadoop clusters for application development and Hadoop tools like Hive, Zookeeper and Sqoop.  Used Hive and created Hive tables and involved in data loading and writing Hive UDFs and worked with Linux server admin team in administering the server hardware and operating system.  Designed, developed and did maintenance of data integration programs in a Hadoop and RDBMS environment with both traditional and non-traditional source systems as well as RDBMS and NoSQL data stores for data access and analysis.  Effectively Communicated plans, project status, project risks and project metrics to the project team planned test strategies in accordance with project scope. Close monitoring and analysis of the MapReduce job executions on cluster at task level and optimized Hadoop clusters components to achieve high performance.