 Built data APIs and delivery services to support critical operational process, analytical models, and machine learning applications Skilled in Python programming for automation, scripting, and data manipulation tasks. Implemented and maintained PostgreSQL databases, including performance tuning and data replication. Assisted in the development and maintenance of CI/CD pipelines using Bamboo. Experience in building efficient pipelines for moving data between GCP and Azure using Azure Data Factory Worked on migrating data from Teradata to AWS using Python and BI tools like Alteryx Automate the data flow process in the Alteryx from data sources(flat files, Postgress database) to S3 bucket using Python, SQL and Alteryx tool inbuild capabilities. Created BigQuery authorized views for row level security or exposing the data to other teams Built data pipelines in airflow in GCP for ETL related jobs using different airflow operators both old and new operators Writing and optimizing complex SQL queries for data extraction, transformation, and loading, ensuring high performance and data accuracy

 Involved in building data pipelines that extract, classify, merge and deliver new insights on the data.  Worked on python and shell scripting to automate and schedule the workflows to run on Azure.  Experienced in working with UNIX/LINUX environments, writing UNIX shell scripts, Python.  Designed and build Modern data solutions using Azure PaaS service to support visualization of data.  Designed and built ETL pipelines to automate ingestion of structured and unstructured data. Developed Spark jobs using Pyspark and Spark-SQL for data extraction, transformation & aggregation from multiple file formats.  Optimization of Hive queries using best practices and right parameters and using technologies like Hadoop, Python, PySpark.  Developed framework for converting existing PowerCenter mappings and to PySpark(Python and Spark) Jobs.  Worked with Enterprise data support teams to install Hadoop updates, patches, version upgrades as required and fixed problems, which raised after the upgrades.

 Deployed the HBase cluster in cloud (AWS) environment with scalable nodes as per the incremental business requirement.  Implemented AWS IAM for managing the user permissions of applications that runs on EC2 instances.  Deployed applications onto AWS lambda with HTTP triggers and integrated them with API Gateway  Developed multiple ETL Hive scripts for data cleansing and transformations for data.  Developed spark applications in python (PySpark) on distributed environment to load huge number of CSV files with different schema in to Hive ORC tables.

 Perform data validation and reconciliation on raw data in the Data Lake using Spark.  Maintain data quality and completeness as well as check user credentials so that right people are accessing the right data.  Responsible for validation of transactional and profile data from RDBMS which are transformed and loaded to Data Lake using Hadoop Bigdata technologies.  Perform tasks such as writing scripts, calling APIs, write SQL queries, etc.  Schedule the Spark jobs in cluster using Airflow.  Involved in loading data from UNIX file system to HDFS.  Responsible for creating Jenkins pipeline for deployment using Ansible.  Identified opportunities to improve the quality of data, through process and system improvements.

 Created Source to Target Mapping documents, documented business and transformation rules and participated in working sessions and ensured full business participation throughout the process. Created DFD Data Functional Design artifacts that incorporated the process flow visios, S-T Mapping document and all the specifications for proper ETL implementation. Involved in creating the test strategy. Prepared Test plan, test case according to the Source to target mapping document Tested the FSDs and ETL mapping which were developed to load from different source systems in the Teradata Staging /Target areas. Validated the data between the Source to Staging and Staging to Target using the Source to target mapping document as reference. Performing own Data profiling on source to compare with list of valid values while mapping source to target and writing exception cases if necessary. Helped the Infrastructure team in loading the XML's generated by the Informatica into the ALGO Limit management system.

 Prepared interview questionnaires for the users and gathered requirements. Performed Functional and GUI testing to ensure that the user acceptance criteria are met. Examined, analyzed and modeled Test plans Use Case Model cases Test procedures based on RUP methodology. Identified Use cases from the Functional requirements and wrote Use Case Specifications and created business process workflow diagrams Activity diagrams, Sequence diagrams, Collaboration diagrams in VISIO Actively participated in performing testing for thoroughly understanding of the engineering process Worked with SQL and Data Warehousing for arrangement of customer data. Created Use Case scenarios and work flow diagrams after requirements gathering.