Senoir Bigdata Engineer
CurrentResponsibilities:Design and Implementation of Tracking Data Systems: Spearheaded the design of comprehensive tracking systems across multiple platforms including iOS, Android, Android Smart TV, web, and next-gen Smart TVs. Defined key metrics and standardized event structures to ensure consistent data collection across all platforms. Ensured compliance with data privacy regulations, implementing user consent mechanisms and anonymization processes.Log Data Processing: Implemented robust log processing systems, managing both real-time streaming data and batch file-based data ingestion. Utilized Apache Spark for data transformation, filtering, and aggregation to prepare log data for storage. Managed data storage solutions using PostgreSQL for structured data and Redis for caching, optimizing system performance.Data Reporting and Analytics: Developed and deployed analytics data models, providing insights via Power BI.Designed and implemented Power BI reports to visualize key performance metrics, facilitating data-driven decision-making. Implemented real-time data processing pipelines to provide up-to-date reports on Power BI.Google Analytics Integration: Integrated Google Analytics for tracking traffic and user experiences. Configured custom events and page view tracking to capture user interactions and behavior across platforms.Tools & Technologies:Apache Spark, Apache Kafka, Redis, PostgreSQL, Power BI, Google Analytics