Data Integration Specialist And Sql Database Administrator: Integration, Hardware, Hadoop, Hive
Current• Wrote SQL server stored procedures.• Applied information security and assurance policies, principles and practices for data management services.• Used Talend-Spark and Hadoop to manage, compile and process big data as the monthly Comprehensive Capital Analysis and Review (CCAR) with a 25+ TB size. • Loaded data into partitioned parquet-format Hive tables.• Developed SAS codes and SQL server stored procedures.• Manipulated and loaded the daily Federal Financial Institutions Examination Council data (FFIC) to transaction or transposed tables.• Integrated data stored in Hadoop with data from other sources using Talend and SAS access to Hive.• Captured data requirements, developed ETL jobs and scripts using Talend, SAS Data Integration Studio tools.• Extract and manipulate millions of records from XML, SAS, SQL and Text files.• Transformed and loaded data as SEC FormPF and FormADV into SQL Server databases and Hive databases.• Developed SAS codes to add a User Written Transformation to SAS DI Studio applications. • Added pre-code OPTIONS to manipulate and transfer data between platforms(Linux,windows) or overwrite input files statements.• Created documentation step by step on loading data.• Conducted training and presentations on SAS DI to superiors, co-workers and clients and provided recommendations.• Co-developed a procedures manual with a team.