Vidya K

Vidya K Email and Phone Number

Data Engineer and SWE @ Shubh Solutions LLC
McKinney, TX, US
Vidya K's Location
San Diego Metropolitan Area, United States
About Vidya K

Highly skilled Data Engineer with a passion for transforming raw data into actionable insights. Proficient in Palantir, SQL, Python, and Snowflake, adept at designing robust data pipelines and optimizing database performance. Leveraging expertise to drive innovation and solve complex challenges in data-driven environments.Specialties: • Comprehensive knowledge & experience in analysis of extensive, complex data involving data interpretation, manipulation & optimization. • Data warehousing expert with strong experience managing and building enterprise wide data architecture.• Liaison with product/marketing team to gather, analyze and evaluate the business requirements and work towards developing, enhancing and delivering the solution.• Design and automate ETL jobs to enhance the overall quality of data while maintaining consistency between the data sources and reducing redundancy.Technical Skills:• Languages: JAVA, Python, SQL, Shell script• Data Gadgets: Snowflake, Databricks, Tableau, Palantir, Slate, Quiver• DBMS: Oracle ODS, MySQL, Teradata, Big Data• Big Data Tools: Apache Hadoop (HDFS, HIVE, Yarn, SQOOP, Cloudera Impala), AWS (S3, Athena)• Tools: JIRA, Putty, UC4, GIT, Jenkins

Vidya K's Current Company Details
Shubh Solutions LLC

Shubh Solutions Llc

View
Data Engineer and SWE
McKinney, TX, US
Website:
shubsolutions.com
Employees:
27
Vidya K Work Experience Details
  • Shubh Solutions Llc
    Data Engineer And Swe
    Shubh Solutions Llc
    Mckinney, Tx, Us
    View
  • Shubh Solutions Llc
    Data Engineer/Swe
    Shubh Solutions Llc Oct 2023 - Present
    • Customized Palantir Slate to create user interfaces and workflows for specific business requirements.• Proficient in Palantir Foundry, Slate, Workshop, Contour and AIP for data integration, analytics, and platform customization.• Explored the option of creating the transforms using AIP wherein we input what we want in a text format and it is building the logic block.• Designed and implemented a customized workflow in Palantir Slate to streamline the data review process.• Contributed to the development of internal tools and utilities to streamline data engineering workflows.• Designed and implemented scalable data pipelines using Databricks and Apache Spark, enabling efficient data processing and analysis.• Assisted in data mapping and data modeling activities to support data integration and migration projects.• Conducted data profiling and quality assessments to identify inconsistencies and discrepancies.
    View
  • Amazon
    Sde
    Amazon Jan 2023 - Oct 2023
    Dallas, Texas, United States
    • Designed and developed complex data pipelines using Python and PySpark, enabling efficient data extraction, transformation, and loading from various sources (Salesforce, FTP, SFTP, API’s) into data warehouses like Amazon Redshift.• Participated in on-call support and incident resolution for critical data issues, ensuring data system availability and reliability.• Developed and maintained data documentation, data dictionaries, and process documentation.• Participated in regular code reviews and implemented best practices to ensure code quality, maintainability, and adherence to data engineering standards• Developed an automated start up process to install all the required packages and deploy Hadoop table in Amazon EMR.• Contributed to the moving of the entire data infrastructure from on-Prem Infrastructure to AWS Cloud Service.
    View
  • Cloud Infra It Solutions
    Data Engineer
    Cloud Infra It Solutions May 2022 - Dec 2022
    Dallas, Texas, United States
    • Worked in the Neo ingestion team which focuses on ingestion of data from different sources and on-prem databases to Azure cloud.• Worked on ingestion of data from Azure to snowflake and then to palantir.• Designed a stored procedure to automate ingestion process for the incremental data.• Effectively managed JIRA ticketing system, ensuring seamless tracking, prioritization, and conducted regular JIRA ticket grooming sessions to ensure tickets were well-defined, actionable, and aligned with the team's development goals• Created health checks and data quality checks to ensure data accuracy
    View
  • Wichita State University
    Graduate Data Engineer
    Wichita State University Jan 2021 - May 2022
    United States
    • Worked in Maintaining Library database for the books inventory.• Created and maintained order management system for the students to book study rooms in library.• Created a Data Catalog for the books in library and managed international library loans data.• Installed and configured Hadoop MapReduce, HDFS, developed multiple MapReduce jobs in Java for data cleaning and preprocessing.• Prepared comprehensive reports and presentations outlining risk analyses and coverage recommendations.• Gathered and analyzed data to provide insights into market trends and customer behavior.• Leverage technologies such as Tableau to create data driven insights and reporting.
    View
  • Wipro
    Data Engineer
    Wipro Apr 2019 - Jan 2021
    Hyderabad, Telangana, India
    • Managing and analyzing of the data and reporting generating from SQL Server to Microsoft Excel for creating Pivot Tables and V-Lookups• Worked on data migration projects, transferring on-premises data to GCP using Google Transfer Service and ensuring data integrity and security throughout the migration process• Ingested Claim data into Hive tables using partitioned tables. Denormalized Diagnosis tables using the complex Array and Map Datatypes. • Cleaned and loaded data in append mode using spark data frames• Developed spark streaming pipelines for CMS response file processing like 999, HRR, MAOs etc.• Developed Metadata-driven framework for best performance in processing the claim life cycle.• Developed ETL workflows utilizing GCP Google Cloud Dataprep and Apache Airflow to extract, transform, and load data from various sources into GCP Google BigQuery and Google Cloud Storage for analysis and reporting purposes.• Implemented Google Cloud SQL and Google Cloud Bigtable in GCP to manage structured and unstructured data, ensuring high availability and optimal performance for critical business applications
    View

Frequently Asked Questions about Vidya K

What company does Vidya K work for?

Vidya K works for Shubh Solutions Llc

What is Vidya K's role at the current company?

Vidya K's current role is Data Engineer and SWE.

Who are Vidya K's colleagues?

Vidya K's colleagues are Moizullah Khan, Sree Devi Godala, Mirza Sabeha Samrin, Mukesh Goel, Rishabh Kataria, Shubham Paliwal, Prem Kumar.

Not the Vidya K you were looking for?

View similar profiles

Free Chrome Extension

Find emails, phones & company data instantly

Find verified emails from LinkedIn profiles
Get direct phone numbers & mobile contacts
Access company data & employee information
Works directly on LinkedIn - no copy/paste needed
Get Chrome Extension - Free

Aero Online

Your AI prospecting assistant

Download 750 million emails and 100 million phone numbers

Access emails and phone numbers of over 750 million business users. Instantly download verified profiles using 20+ filters, including location, job title, company, function, and industry.