Pradeep . Email & Phone Number

Canada

 Created the Node.js EXPRESS Server combined with Socket.io to build MVC framework from front-end side AngularJS to back-end MongoDB, to provide broadcast service as well as chatting service. Resolved update, merge and password authentication issues in Jira. Optimization of Hive queries using best practices and right parameters and using technologies like Hadoop, YARN, Python, PySpark. Experience in deploying Hadoop 2.0(YARN). Configured Spark Streaming to receive real time data from the Apache Kafka and store the stream data to HDFS using Scala.  Installing and configuring Hadoop eco system like HBase, hive. Monitored multiple Hadoop clusters environments using Ganglia, Kibana, Nagios. Experience in Big Data processing using Apache Hadoop. Worked closely with the ETL SSIS Developers to explain the complex Data Transformation using Logic. Develop framework for converting existing PowerCenter mappings and to PySpark(Python and Spark) Jobs. Create Pyspark frame to bring data from DB2 to Amazon S3. Extensively worked on the naming standards which incorporated the enterprise data modeling. Building and maintaining libraries to support analytics using NumPy, pandas and SciPy. Translated high level requirements into functional and technical user stories in JIRA. Involved in designing and deploying multitude applications utilizing almost all of the AWS stack (Including EC2, Route53, S3, RDS, Dynamo DB, SNS, SQS, IAM) focusing on high-availability, fault tolerance, and auto-scaling in AWS cloud Formation.

San Jose, California, United States

 Designed and developed middleware, using RESTful web services based on a centralized schema Wrote Python scripts to parse JSON documents. Developing micro services using Node.js and Go and used AWS Lambda functions to build Services. Analysed the sql scripts and designed it by using PySpark SQL for faster performance. Created, managed and supported all administrative functions associated with JIRA. Experience in using Bug tracking tools like JIRA, Remedy, Confidential Quality Center. Effectively communicated and assisted other team members in the ongoing assignments as part of an agile environment. Dealt with data ambiguity and performed lazy evaluation in pyspark for code optimization. Worked on data cleaning and reshaping, generated segmented subsets using NumPy and Pandas in Python. Used JavaScript, Angular JS in developing single page applications. Developing front end screens using Angular2, Twitter Bootstrap, JavaScript, Backbone.js for creating grid layout, responsive screen sizes and devices.  Did server-side scripting using Node.JS to develop the applications, which are data intensive real time applications that run on distributed devices. Configured an AWS Virtual Private Cloud (VPC) and Data Base subnet group for isolation of resources within AWS RDS. Used AngularJS Services $http for consuming REST API’s and utilized Postman tool to debug Web API. Used Angular Reactive forms to implement complex forms and performed the validations using Form Builder. Utilized the Chart.js to provide visualization and utilized AWS lambda for email notification for the item in stock running below alert quantity and worked on building RESTful service API to allow users to see Block chain transactions, orders, and info. Developed views and templates with Python and Django view controller and templating language to created user-friendly website interface using angular and Developed REST Microservices which are like API’s used for Home Automation.

• Created APIs to send data to another application.• Created ETL jobs to transform and mapping data to required field in target table.• Used glue jobs and crawler to migrate data.• Used Celery and RabbitMQ for queueing emails.• Used Docker for containerization of applications which was easier for deployment.• Used Jenkins to test and deploy project on EC2 instance.• Created service to send report mail after the completion of ETL job.Designed, created, executed, and kept up answers for utilizing Docker, Jenkins, Git, and Puppet for microservices and ceaseless arrangement.• Created a Python/Elasticsearch based web application using Python scripting for data processing, MySQL for the database, and HTML/CSS/Ruby and High Charts for data visualization of the served pages.• Managed and reviewed Hadoop log file and worked in analyzing SQL scripts and designed the solution for the process using PySpark.• Developed SAS code for data analysis and report generation using Macro Processing, Proc Report to generate Excel spreadsheets.• Develop python code to automate the ingestion of common formats such as JSON, CSV by using Logstash from elastic search to Kibana dashboard to be viewed by clients.• Responsible for designing and deploying new ELK clusters (Elasticsearch, Logstash, Graphite Kibana, beats, Kafka, zookeeper etc.• Responsible for the Automation of the deployment of the Conductor application on AWS lambda using high-end AWS architectural components.

 Developed RESTful Web Services using Flask Framework.  Writing Complex queries using SQLAlchemy library as an ORM.  Writing test cases using units test and mock. Created Swagger UI using Flassger for API Documentation.  Create Splunk Search Processing Language (SPL) queries, Reports and Dashboards. Create Splunk applications and deploy using Splunk Deployment Server. Creating docker files, building images and pushing images to registry.  Created Decorator for maintaining logs of applications.  Worked on Celery for running async tasks. Write comprehensive tests, including unit tests, service-level, HTTP-level, UI component, and browser-level integration tests that provide confidence in the stability and function of Kibana Used Gitlab CI and Jenkins for CI implementation  Deployed web Services on AWS Elastic Beanstalk and configured EC2 Instance with Active Directory so that multiple users can use that instance.