Smriti Das Email and Phone Number

Project : Databricks Product DevelopmentDescription: Implemented, configured, and maintained various Microsoft Azure Databricks solutions, focusing on access control, cost management, performance optimization, backups (using Infrastructure as Code), networking, storage (DBFS, ADLS), Databricks SQL, Delta Live Tables, and disaster recovery strategies.Environment : Databricks, Terraform, Overwatch.Key Responsibilities:• Developed and managed Databricks infrastructure with… Show more Project : Databricks Product DevelopmentDescription: Implemented, configured, and maintained various Microsoft Azure Databricks solutions, focusing on access control, cost management, performance optimization, backups (using Infrastructure as Code), networking, storage (DBFS, ADLS), Databricks SQL, Delta Live Tables, and disaster recovery strategies.Environment : Databricks, Terraform, Overwatch.Key Responsibilities:• Developed and managed Databricks infrastructure with Terraform modules, ensuring scalability, reliability, and cost-effectiveness.• Automated provisioning tasks, including Cluster Policies, Unity Catalog, Secrets Management, and Role-Based Access Control (RBAC).• Monitored resource utilization and identified cost-saving opportunities by optimizing instance types and utilizing spot instances, along with implementing cost control strategies.• Deployed and managed Overwatch on single and multiple Databricks workspaces to gain insights through data analysis and performance tracking.• Designed Overwatch dashboards in Databricks using Plotly and Pandas for better visualization and reporting.• Automated resource creation and management through Jenkins pipelines for streamlined operations.• Deployment of views and tables on databricks using Databricks Asset Bundle(DAB). Show less

Project: Data ingestion from SAP HANA database to postgresql.Description: This project is data ingestion from SAP HANA database and storing the data in s3 bucket and loading the data to postgresql, redshift data warehouse.Key responsibilities:•Design, implement and enhance end-to-end data workflows using Apache nifi.•Creating nifi flows to extract data from HANA and loading it to landing or warehousing s3 bucket after required transformation.•Creating and executing emr jobs… Show more Project: Data ingestion from SAP HANA database to postgresql.Description: This project is data ingestion from SAP HANA database and storing the data in s3 bucket and loading the data to postgresql, redshift data warehouse.Key responsibilities:•Design, implement and enhance end-to-end data workflows using Apache nifi.•Creating nifi flows to extract data from HANA and loading it to landing or warehousing s3 bucket after required transformation.•Creating and executing emr jobs to load data from warehousing bucket to enrichment bucket in s3.•Executing glue crawler to discover and catalog metadata from s3 data source and store the information to glue data catalog.•Writing pyspark script to read files from s3 bucket and performing necessary transformations and writing the file into different s3 bucket location.•Copying files from s3 bucket from source to destination environment using AWS CLI on cloud shell terminal.•Validating files in s3 bucket.•Creating tables in Athena to query data from s3.•Executing line by line pyspark and python code in linux putty. Show less

 Project: Source to Redshift ingestionDescription: This project is data ingestion from web url’s and storing the data in s3 bucket and loading the data to redshift and snowflake data warehouse.Key responsibilities:•Analyse business requirement and build end-to-end data model in python, pyspark, data bricks and snowflake.•Extract, analyse and ingest data from web url’s and uploading the data into s3 bucket using pythom API and loading the transformed data to redshift… Show more  Project: Source to Redshift ingestionDescription: This project is data ingestion from web url’s and storing the data in s3 bucket and loading the data to redshift and snowflake data warehouse.Key responsibilities:•Analyse business requirement and build end-to-end data model in python, pyspark, data bricks and snowflake.•Extract, analyse and ingest data from web url’s and uploading the data into s3 bucket using pythom API and loading the transformed data to redshift, snowflake.• Writing pyspark script on databricks to retrieve data from weburl using ‘requests’ or ‘urllib’ library.•Data cleansing, data profiling, data transformation using pypark script on databricks and load data to stage, base views and data mart using pyspark.•Creating and scheduling tidal jobs to execute databrick notebook and monitoring the jobs. Analyze and resolve job errors.•Creating tables in databricks to load data. Show less

Project: JnJKey responsibilities:•Design, implement and enhance end to end data workflows using Alteryx.•Creating complex sql queries and creating excel reports using power queries.•Validating backend data as per business requirement.•Validating tableau dashboards as per requirement.•Documenting end to end data workflows, tableau dashboard functionalities and test documents.•Analysing and debugging sql queries and making necessary changes if required.•Writing… Show more Project: JnJKey responsibilities:•Design, implement and enhance end to end data workflows using Alteryx.•Creating complex sql queries and creating excel reports using power queries.•Validating backend data as per business requirement.•Validating tableau dashboards as per requirement.•Documenting end to end data workflows, tableau dashboard functionalities and test documents.•Analysing and debugging sql queries and making necessary changes if required.•Writing python script to establish connection and downloading or reading files from sharepoint.•Analytics automation platform like alteryx that help to collect, prepare and blend data. Show less

Project : Oracle to Snowflake acquisition Description: This project is data acquisition or data ingestion from oracle server to snowflake data warehouse for data analytics as well as complete etl migration from informatica to matillion.Key responsibilities:•Creating simple as well as complex mappings in matillion.•Handling cdc implementation in the mappings in matillion.•Writing sql queries required for transformation in the mappings.•Executing jobs to load data to… Show more Project : Oracle to Snowflake acquisition Description: This project is data acquisition or data ingestion from oracle server to snowflake data warehouse for data analytics as well as complete etl migration from informatica to matillion.Key responsibilities:•Creating simple as well as complex mappings in matillion.•Handling cdc implementation in the mappings in matillion.•Writing sql queries required for transformation in the mappings.•Executing jobs to load data to snowflake.•Matillion to git integration.•Unit testing of mappings in matillion.•Working on data warehouse and etl development.•Enhancing and working with snowflake concepts like snowpipe, zero copy cloning, unloading, scd, time travel, etc. Show less

-Proficient in frontend development using Angular 12 and WordPress-Skilled in version control with Git and GitHub-Experienced in designing and implementing WordPress websites-Knowledgeable in the Domain Name System (DNS) and DNS server configurations-Proficient in Cascading Style Sheets (CSS) for styling web pages-Strong command of HTML and JavaScript for web development-Familiar with website hosting and domain management

Web Study Portal Development:Utilize Python and Flask for backend development.Employ HTML, Jinja, and CSS for frontend design.Utilize PostgreSQL as the backend database.User Authentication:Implement a seamless login system for users.Store user details in a PostgreSQL database, utilizing two separate tables for usernames and passwords.User Interface:Create a user-friendly interface for easy navigation.Display video materials on the left side of the… Show more Web Study Portal Development:Utilize Python and Flask for backend development.Employ HTML, Jinja, and CSS for frontend design.Utilize PostgreSQL as the backend database.User Authentication:Implement a seamless login system for users.Store user details in a PostgreSQL database, utilizing two separate tables for usernames and passwords.User Interface:Create a user-friendly interface for easy navigation.Display video materials on the left side of the frame.Study Features:Enable students to access study materials easily.Provide a toggle button for students to practice in the UNIX terminal, allowing them to switch it on or off.Interactive Terminal:Implement a toggle button at the top of the frame to control the visibility of the UNIX terminal.Allow students to practice within the terminal for hands-on learning.Efficient Learning Environment:Design the portal to be a comprehensive study platform.Ensure that students can watch related videos and engage in practical exercises seamlessly.Database Management:Implement robust database management using PostgreSQL.Store user details securely in separate tables for enhanced data organization.User Experience:Focus on creating a smooth and intuitive user experience.Prioritize easy login processes and straightforward access to study materials.Scalability:Design the portal with scalability in mind to accommodate future features or content expansions.Overall Integration:Ensure seamless integration of all technologies (Python, Flask, HTML, Jinja, CSS, PostgreSQL) for a cohesive and functional Study Portal. Show less