Data Engineer
CurrentKey responsibilities:-Pre-processing data, transferring files to and from S3 buckets, file manipulation and building logic using Python-Building Directed Acyclic Graphs(DAGs) to automate jobs on Airflow-Developing new dashboards and making enhancements on existing dashboards on tableau Desktop and publishing the same on Tableau serverKey Achievements:-Worked on a POC for multiprocessing for use cases on an EC2 instance which resulted in faster run times for jobs. This… Show more Key responsibilities:-Pre-processing data, transferring files to and from S3 buckets, file manipulation and building logic using Python-Building Directed Acyclic Graphs(DAGs) to automate jobs on Airflow-Developing new dashboards and making enhancements on existing dashboards on tableau Desktop and publishing the same on Tableau serverKey Achievements:-Worked on a POC for multiprocessing for use cases on an EC2 instance which resulted in faster run times for jobs. This helped in reducing the run time of production jobs by 3 hrs-Optimized a data source of a critical dashboard on Tableau server which helped reduce the time taken for refresh by 1 hr Show less