Dhaval K Email & Phone Number

Baltimore, Maryland, Us

As a Data Engineer, I’m applying pre-existing machine learning and data mining techniques to create predictive models. Developed high speed data ingestion pipeline using Scala, Apache Spark, Akka Streams, HDFS, Hive, and Cassandra. Developed python scripts for data cleaning, analysis and automating day to day activities. Hydrate the data lake by creating data ingestion pipelines for using various technologies likeAttunity Replicate, Kafka, Spark, HDFS, Hive, Java, Bash Scripting HBase, Teradata, Query Grid, CA7.The data is then used by stakeholders for machine learning and reporting. Used pandas, NumPy, Seaborn, matplotlib, Scikit-learn, SciPy, NLTK in Python for developing various machine learning algorithms. Setup storage and data analysis tools in Amazon Web Services (AWS) cloud computing infrastructure.

Wayzata, Minnesota, Us

Developed ETL components to load log files into Elastic Search and send the JSON files to thecustomer. Used shell script to load the incremental data to hadoop.Designed data flow architecture for ingestion projects according to the requirement.Developed spark application to invoke the spark property file and load data into Elastic Search.Developed Map Reduce/Spark Python modules for machine learning & predictive analytics inHadoop on AWS. Implemented a Python-based distributed random forest via Python streaming.Data migration from on premise database to cloud based data store over AWSAnalyzed data which need to be loaded into Hadoop MapR and contacted with respective sourceteams to get the table information and connection detailsUsed Sqoop to import data from different RDBMS systems like Oracle, DB2 and MySQL and loadedinto HDFS.Used Pandas, NumPy, Seaborn, SciPy, Matplotlib, Scikit-learn, NLTK in Python for developing various machine learning algorithms and utilized machine learning algorithms such as linear regression, multivariate regression, naive Bayes, Random Forests, K-means, &KNN for data analysis.

Atlanta, Georgia, Us

Hands-on experience with Informatica power center and power exchange in integrating withdifferent applications and relational databases. AWS CI/CD Data pipeline and AWS Data Lake using EC2, AWS Glue, AWS Lambda. Hands on with different API Endpoints like Edge Optimized, Regional, Private in AWS API Gateway. Configured control connections different levels like API Key, Method level, Account Level. AWS API Gateway protection strategies like Resource Policies, IAM, Lambda, Cognito Authentications. Building ETL data pipeline on Hadoop/Teradata using Hadoop/Pig/Hive/UDFs. Utilized Kubernetes and Docker for the runtime environment for the CI/CD system to build, test, and deploy. Expertise in implementing DevOps culture through CI/CD tools like Repos, Code Deploy, Code Pipeline, GitHub.

Dublin 2, Ie