Kiran N Email and Phone Number
I've been working in IT for over 10 years, as a Data Analytics Engineering and as a Programmer Analyst. I'm familiar with cloud platforms like Azure, ETL, and Amazon Web Services. I've worked with Azure Cloud and its services like Azure Data Factory, Azure Databricks, Azure DevOps services, Azure functions, and Azure storage. I've also used "Big data" tools like Hadoop, Spark, PySpark, Hive, and others.My experience includes moving data between different parts of the Azure stack, like from Data Lake to Azure Blob Storage, and I'm skilled in Data Load/Integration using ADF. I've built ETL data pipelines using Azure Databricks and PySpark, as well as working on data processing tasks using tools like Apache Flume, Kafka, PowerBI, and Microsoft SSIS.I'm familiar with data visualization tools like Power BI and Tableau, and I'm proficient in developing pipelines using Spark with Scala and PySpark. I've also worked with the Azure Logic App Integration tool and have strong knowledge of Azure Cloud Services.Additionally, I have experience with AWS, including transferring data using the Informatica tool from AWS S3 to AWS Redshift. I've used Python Boto 3 to configure services like AWS Glue, EC2, and S3, and I've worked with the AWS Glue catalog for data operations.I'm experience in working with the phases of development, including Data Cleansing, Data Conversion, Data Profiling, Data Mapping, Performance Tuning, and System Testing. I'm skilled in SQL queries and database management, as well as Python programming and Bash scripting.I'm also experienced in conducting risk assessments for data engineering pipelines and systems, establishing data governance frameworks, and deploying SaaS applications securely. I've built RESTful APIs and implemented web services, fostering a culture of creativity and experimentation for process optimization.I've handled Hive tables using Spark SQL, optimized algorithms in Hadoop using Spark, and integrated diverse data sources into architecture frameworks. I've developed enterprise-wide applications using PySpark and Cassandra NoSQL Database and have experience in Agile and Waterfall project methodologies.I've worked with big data ecosystems, developed Map Reduce Programs, and provided technical consulting and end-to-end delivery for various projects. I'm proficient in APM tools, NoSQL databases like MongoDB and Cassandra, and data analysis using MS Excel and Access. I've designed and implemented data pipelines using Google Dataflow and Cloud Data Fusion, optimizing data architecture and workflows.
Bankunited
View- Website:
- bankunited.com
- Employees:
- 1597
-
Senior Data EngineerBankunited May 2022 - PresentMiami Lakes, Florida, United StatesAs Sr Data Engineer in Cummins to drive projects using Spark, SQL, and Azure cloud environment.Worked on data governance to provide operational structure to previously ungoverned data environments.Participated in the requirement gathering sessions to understand the expectations and worked with system analysts to understand the format and patterns of the upstream source data.Done data migration from an RDBMS to a NoSQL database and gives the whole picture for data deployed in various data systems.Designed and implemented end-to-end data solutions (storage, integration, processing, and visualization) in Azure.Analyze, design, and build Modern data solutions using Azure PaaS service to support visualization of data.Designed and Configured Azure Cloud relational servers and databases analyzing current and future business requirements.Worked on data migration from On-prem SQL server to Cloud databases (Azure Synapse Analytics (DW) & Azure SQL DB). Designed and developed data pipeline in Azure cloud which gets customer data from API and processes it to Azure SQL DB.Created External tables in Azure SQL Database for data visualization and reporting purposes.Orchestrated all Data pipelines using Azure Data Factory and built a custom alerts platform for monitoring.Extract Transform and Load data from Sources Systems to Azure Data Storage services using a combination of Azure Data Factory.Done Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in Azure Databricks.Performed SAP Data Migration by using Business Objects Data Services as the ETL tool.Working with Azure BLOB and Data Lake storage and loading data into Azure SQL Synapse analytics (DW).Spark with Hive and SQL/Oracle. Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs and Scala.Hands-on experience in handling Hive tables using Spark SQL.
-
Sr Data EngineerThe Hartford Jul 2020 - Apr 2022Hartford, Connecticut, United StatesAs a Sr. Data Engineer designed and deployed scalable, highly available, and fault tolerant systems on Azure.Lead the estimation, review the estimates, identify the complexities, and communicate to all the stakeholders.Involved in the complete SDLC life cycle of big data project that includes requirement analysis, design, coding, testing, and production.Defined the business objectives comprehensively through discussions with business stakeholders, and functional analysts and participating in requirement collection sessions. Implemented end-to-end systems for Data Analytics and Data Automation and integrated with custom visualization tools.Migrated on-primes environment on Cloud using MS Azure. Designed the business requirement collection approach based on the project scope and SDLC (Agile) methodology.Moved data from Azure Data Lake to Azure Data Warehouse using PolyBase. Created external tables in ADW with 4 compute nodes and scheduled.Extensively used Agile Method for a daily scrum to discuss the project-related information.Worked with data ingestions from multiple sources into the Azure SQL data warehouseTransformed and loading data into Azure SQL Database.Build data pipelines in airflow in CP for ETL-related jobs using different airflow operators.Configured Spark streaming to receive real-time data from Kafka and store the stream data to HDFS. Development and maintenance of data pipeline on Azure Analytics platform using Azure Databricks.Developed a data pipeline using Kafka to store data in HDFS. Implemented Kafka producers to create custom partitions, configured brokers, and implemented High-level consumers to implement the data platform. Implemented stringent data governance and security measures within the Medallion architecture framework, ensuring compliance with regulatory requirements and safeguarding sensitive data assets.Created Airflow Scheduling scripts in Python. -
Data EngineerHumana Mar 2018 - Jun 2020Louisville, Kentucky, United StatesAs a Data Engineer I was responsible for building a data lake as a cloud-based solution in AWS using Apache Spark and Hadoop.Involved in Agile methodologies, daily Scrum meetings, and Sprint planning.The objective of this project was to install and configure Hadoop and was responsible for maintaining the cluster and managing and reviewing Hadoop log files.Used AWS Cloud and On-Premises environments with Infrastructure Provisioning/ Configuration. Contributed to the development of key data integration and advanced analytics solutions leveraging Apache Hadoop.Wrote complex Hive queries to extract data from heterogeneous sources (Data Lake) and persist the data into HDFS.Developed Big Data solutions focused on pattern matching and predictive modeling.Developed the code for Importing and exporting data into HDFS and Hive using SqoopDeveloped a data pipeline using Kafka, HBase, Spark, and Hive to ingest, transform, and analyze customer behavioral data.Developed Spark jobs and Hive Jobs to summarize and transform data.Developed Spark code using Scala and Spark-SQL/Streaming for faster processing of data. Implemented Sqoop to transform the data from Oracle to Hadoop and load back in parquet formatDeveloped incremental and complete load Python processes to ingest data into Elastic Search from Oracle database Used Hive to analyze data ingested into HBase by using Hive-HBase integration and compute various metrics for reporting on the dashboard Created Hive External tables to stage data and then move the data from Staging to main tables Pulled the data from the data lake (HDFS) and massaged the data with various RDD transformations. Load the data through HBase into Spark RDD and implement in-memory data computation to generate the output response.Continuously tuned Hive UDFs for faster queries by employing partitioning and bucketing. -
Data EngineerSolugenix Oct 2013 - Dec 2017Hyderabad, Telangana, IndiaWorked as a Data Engineer to collaborate with other Product Engineering team members to develop, test, and support data-related initiatives. Developed understanding of key business, product, and user questions.Followed Agile methodology for the entire project.Defined the business objectives comprehensively through discussions with business stakeholders, and functional analysts and participating in requirement collection sessions. Provided a summary of the Project's goals, the specific expectations of business users from BI, and how it aligns with the project goals. Lead the estimation, review the estimates, identify the complexities, and communicate to all the stakeholders.Responsible for data governance rules and standards to maintain the consistency of the business element names in the different data layers.Migrated on-primes environment on Cloud using MS Azure. Worked on migration of data from On-prem SQL server to Cloud databases (Azure Synapse Analytics (DW) & Azure SQL DB).Performed data flow transformation using the data flow activity.Performed ongoing monitoring, automation, and refinement of data engineering solutions.Creating pipelines, data flows, and complex data transformations and manipulations using ADF and PySpark with Databricks.Developed a mapping document to map columns from source to target.Created Azure data factory (ADF pipelines) using Azure PolyBase and Azure blob.Performed ETL using Azure Data Bricks. Wrote UNIX shell scripts to support and automate the ETL process.Worked on Python scripting to automate the generation of scripts. Data curation is done using Azure data bricks.Used stored procedure, lookup, execute pipeline, data flow, copy data, and Azure function features in ADF.Worked on Kafka to bring the data from data sources and keep it in HDFS systems for filtering.Created several Databricks Spark jobs with PySpark to perform several tables-to-table operations.
Kiran N Education Details
-
Computer Science
Frequently Asked Questions about Kiran N
What company does Kiran N work for?
Kiran N works for Bankunited
What is Kiran N's role at the current company?
Kiran N's current role is AZURE CERTIFIED DATA ENGINEER | AZURE | AWS | HADOOP | DATA ANALYSIS | SQL | DATAWAREHOUSE | CLOUD PLATFORM |.
What schools did Kiran N attend?
Kiran N attended Jawaharlal Nehru Technological University Hyderabad (Jntuh).
Who are Kiran N's colleagues?
Kiran N's colleagues are Ileana Rodriguez, Dawn Catalano, Hruthik Reddy M, Rafael Weisson, Gabriel Couto, Jose Cortes, Nichole Jenkins.
Not the Kiran N you were looking for?
Free Chrome Extension
Find emails, phones & company data instantly
Download 750 million emails and 100 million phone numbers
Access emails and phone numbers of over 750 million business users. Instantly download verified profiles using 20+ filters, including location, job title, company, function, and industry.
Start your free trial