Marcelo Alves Email & Phone Number

Miami, Fl, Us

Midland, Michigan, Us

Designed, maintained, and implemented data pipeline using REST API, AWS S3, AWS Firehose, AWS Lamb- da, AWS Kinesis AnalyBcs, and RedshiD➢ Designed and implemented S3 data lake for storage of data➢ Designed and implemented data lake, data warehouse, and data mart design on AWS plaHorm➢ Migrated custom models wriJen by data scienBsts into S3-based producBon workflow➢ Monitored daily producBon runs running on AWS cloud➢ Tested and validated REST API end points➢ Implemented custom ETL/ELT pipelines at request of other departments➢ Held KT sessions with junior developers and other stakeholders as requested

Bloomington, Illinois, Us

➢ Implemented AWS EMR Spark using PySpark/Scala, DataFrames and SparkSQL API for faster processing of data➢ Queried with Athena on data stored in AWS S3 bucket➢ TesBng using PyCharm and PyTest funcBons upgraded from Python2 to Python3➢ Inject Data into AWS Snowflake as Data warehouse.➢ Monitored and managed services with AWS CloudWatch➢ Performing transformaBons using Apache SparkSQL➢ Wrote and upgraded lambda funcBon wriJen in Python2 to Python3➢ Having in-depth knowledge in ETL PlaHorms, Hadoop, Spark/PySpark and AWS Stack➢ Worked with ETL PlaHorms like InformaBca PowerCenter, InformaBca Cloud(IICS)➢ Experience in AWS Cloud: AWS Glue, AWS S3, AWS Athena and other services➢ Skills in Python, ETL operaBons and Bigdata analyBcs, Hadoop, Google Big Query➢ Extensive working experience in Hadoop eco-system components like MapReduce, Hive, MongoDB, Sqoop,Spark-SQL➢ ExperBse knowledge on the parsing of JSON, XML, AVRO file formats in HadoopEnvironment➢ Good experience with data migraBon from on premises data lake to AWS➢ Experience on Spark-SQL, Spark-Streaming, PySpark for faster processing of data➢ Experience on imporBng and exporBng data from different databases like Oracle, MySQLand Sql Server into HDFS using Sqoop➢ Knowledge on developing Terraform scripts for infrastructure and services set up for AWS➢ Involved in wriBng data transformaBons using Spark SQL (Joins, Sub Queries, ParBBonsand Clusters)➢ Good Knowledge with Data MigraBon from On Premise to AWS Cloud

Bentonville, Arkansas, Us

➢ Designed, developed, and maintained pipeline using Apache Ka6a, Apache Spark, and Apache Hive.➢ Developed a data pipeline for extrac>ng social media informa>on from online sources.➢ Used Python for web scraping of relevant ar>cles in order to gauge consumer sen>ment.➢ Used Python to make requests from news sources Web API’s as well as social media API’s such as Facebookand TwiJer➢ Worked with Apache Spark for large data processing integrated with func>onal programming languagePython for fast, distributed processing➢ Worked with both batch and real->me processing with Spark frameworks.➢ Created a Ka6a producer to connect to different external sources and push data to a Ka6a broker.➢ Stored unprocessed JSON and HTML files in HDFS data lake➢ Worked with analy>cs team to provide querying insights and helped develop methods to map informa>vetext more efficiently➢ Op>mized HIVE scripts where relevant by replacing it with Spark queries.➢ Supported analy>cs team with HDFS data lake using HIVE combined with Spark for data query.➢ Retrieved structured and unstructured data from HDFS using HIVE and Spark in addi>on to MapReduce jobs➢ Implemented advanced procedures such as text analy>cs and processing using in memory compu>ngcapability methods through Python scripts running on Apache Spark➢ Used Spark Context and Spark Session to process text files by flat mapping, mapping to RDD, and reducingRDD’s by key to iden>fy text containing key metrics

Minnetonka, Mn, Us

➢ Designed, maintained, and implemented healthcare data pipeline using Teradata, Airflow, HIVE, Spark, and bash scripBng combined with Python ML (machine learning) library.➢ Coordinated with SCRUM target deadlines as well as Git repository.➢ Designed, maintained, and debugged Airflow scripts.➢ Debugged data issues stemming from primary source data residing in Teradata.➢ Deployed custom models wriJen by data scienBsts into producBon pipeline.➢ Monitored periodic producBon model data flow.➢ Resolved code, data, and combined code/data issues stemming from upward source table issues.➢ Experienced with both ETL and ELT pipeline design and implementaBon.