Working as a Data Engineer, Assisted in leading the plan, building, and running states within the Enterprise Analytics Team.Worked in Agiledevelopment environment in sprint cycles of two weeks by dividing and organizing tasks.Highly Involved into Data Architecture and Application Design using Cloudand Big Data solutions on AWS.Work independently to develop and testHive based blocks to load and register data into Data Lake and DataWarehouse.Migrated application from internal data center to AWS cloud platform.Used and developed python scripts to migrate data to AWS.Involved in Dataset Registration in Exchange for S3, One Lake, and Snowflake.Created ETL jobs using matillion to load server data into the Snowflake Data Warehouse.Worked on Data management and Data Integration.Implemented solutions for ingesting data from various sources and processing the Data-at-Rest utilizing Big Data technologies such as Hadoop, Map Reduce and Hive.Provided guidance to development team working on PySpark as ETL platform.Developed data pipeline using Sqoop to ingest cargo data and customer histories into HDFS for analysis.Maintain existing ETL workflows, data management and data query components.•Extensively worked using AWS services along with wide and in depth understanding of each one of them.Involved in creating Hive tables and loading and analyzing data using hive queries.Built REST APIs to easily add new analytics or issuers into the model.Developed Map Reduce program for parsing and loading into HDFS.Used Hive to do transformations, event joins and some pre-aggregations before store and register data into snowflake.Used Hive SQL to do analysis on the data and identify different correlations.Used Hive script and Amazon Elastic MapReduce for unit testing.Imported data from Dynamo DB to Redshift in Batches using Amazon Batch using TWS scheduler.Performed Data analysis and Data validation on Snowflake and Hue Editor.

Worked independently in the development, testing, implementation, and maintenance of systems of moderate-to-large size and complexity.Used the Agile methodology to build the different phases of Software development life cycle (SDLC). Extensively worked using AWS services along with wide and in depth understanding of each one of them.Handled Business logics by backend Python programming to achieve optimal results.Responsible for ETL (Extract, Transform, and Load) processes to bring data from multiple sources into a single warehouse environment.Provide production support for existing products that include SSIS, SQL Server, stored Procedures, interim data marts, Matillion, AWS, Snowflake. Troubleshoot and maintain ETL/ELT jobs running using Matillion.Worked on standard python packages like boto and boto3 for AWS.Investigated data sources to identify new data elements needed for dataintegration. Worked with delivery of Data & Analytics applications involving structured and un-structured data on Hadoop based platforms.

Worked as a Data Engineer worked with the analysis teams and management teams and supported them based on their requirements.Worked on Agile, testing methodologies, resource management and scheduling of tasks.Worked on all phases of data warehouse development lifecycle, from gathering requirements to testing, implementation, and support.Worked withAmazon Web Services (AWS) for improved efficiency of storage and fast access.Worked on Data migration project from Teradata to Snowflake.Developed data pipeline using Sqoop to ingest cargo data and customer histories into HDFS for analysis.Worked on AWS Redshift and RDS for implementing models and data on RDS and Redshift. Implemented Data pipelines for big data processing using Spark transformations and PythonAPI and clusters in AWS.Specified nodes and performed the data analysis queries on Amazon redshift clusters on AWS.Involved in developing ETL pipelines in and out of data warehouse using combination of Python and Snowflakes SnowSQL.Monitoring Produced and Consumed Data Setsof ADF.Transforming data in Azure Data Factory with theADF Transformations.Developed MapReduce programs to parse the raw data, populate staging tables and store the refined data in partitioned tables in the EDW.Added support for Amazon AWS S3 to host static/media files and the database into Amazon Cloud.

Worked as a Data Analyst/Engineer to generate Data Models using Erwin and developed relational database system.Analyzed data where it lives by Mounting Azure Data Lake and Blob to Databricks.Involved in extracting and mining data for analysis to aid in solving business problems.Used Azure Data Lake as Source and pulled data using Azure Polybase.Formulated SQL queries, Aggregate Functions, and database schema to automate information retrieval.Involved in manipulating data to fulfill on analytical and segmentation requests.Managed data privacy and security in Power BI.Written complex SQL queries for data analysis to meet business requirements.Using Data Visualization tools and techniques to best share data with business partners.Designed and implemented a Data Lake to consolidate data from multiple sources, using Hadoopstackstechnologies like Sqoop, hive.Created HBase tables to load large sets of structured, semi-structured and unstructured data coming from UNIX, NoSQL and a variety of portfolios. Reviewed code and system interfaces and extracts to handle the migration of data between systems/databases.Developed a Conceptualmodel using Erwin based on requirements analysis.Involved in ETL mapping documents in data warehouse projects. Involved in loading of data into Teradata from legacy systems and flat files using complexMultiLoad scripts and Fast Load scripts.

Performed Data Analysis using SQL queries on source systems to identify data discrepancies and determine data quality. Performed extensive Data Validation, Data Verification against Data Warehouse and performed debugging of the SQL-Statements and stored procedures for business scenarios. Designed and developed Tableaudashboards using stack bars, bar graphs, scattered plots, and Gantt charts.Familiar with DBMS table design, loading, Data Modeling, and experience in SQL.Worked on ER/Studio for Conceptual, logical, physical data modeling and for generation of DDL scripts.Handled performance requirements for databases in OLTP and OLAP models.Analyzed the data which is using the maximum number of resources and made changes in the back-end code using PL/SQL stored procedures and triggers.Performed data completeness, correctness, data transformation and data quality testing using SQL.Involved in designing Business Objects universes and creating reports.Conducted design walk through sessions with Business Intelligence team to ensure that reporting requirements are met for the business.Prepared complex T-SQL queries, views and stored procedures to load data into staging area.Wrote UNIX shell scripts to invoke all the stored procedures, parse the data and load into flat files.Created reports analyzing large-scale database utilizing MicrosoftExcelAnalytics within legacy system.