Developer

Responsibilities

• Design efficient PySpark based data pipelines that ingest transform and prepare large scale media and entertainment data sets to support analytics and reporting for content performance and audience engagement.
• Develop reusable PySpark frameworks and modules that standardize data processing for streaming metrics viewership trends and campaign attribution across multiple media platforms.
• Optimize PySpark jobs by refining partitioning strategies caching resource configurations and query logic to improve reliability processing speed and cost efficiency in large data environments.
• Collaborate with product data science and business stakeholders in media and entertainment to translate analytical needs into scalable PySpark solutions that support audience insights and content valuation.
• Implement robust data quality checks validation rules and monitoring mechanisms in PySpark workflows to ensure accuracy of key performance indicators for ratings engagement and revenue metrics.
• Integrate data from diverse media sources including streaming logs ad servers customer profiles and content catalogs into unified PySpark based datasets that enable comprehensive analysis and reporting.
• Maintain clear technical documentation for PySpark pipelines data models and operational procedures to support knowledge sharing and smooth onboarding for other developers and analysts.
• Troubleshoot production issues in PySpark jobs and related data workflows by performing root cause analysis applying fixes and implementing preventive enhancements to improve system stability.
• Collaborate in a hybrid work model with onsite and remote team members by using effective communication practices regular checkpoints and shared tools to ensure alignment on deliverables.
• Ensure that PySpark solutions comply with data governance privacy and security standards relevant to media and entertainment data including viewer behavior and content metadata.
• Contribute to continuous improvement by evaluating new PySpark features big data tools and best practices that can enhance media analytics capabilities and operational efficiency.
• Provide guidance to peers on PySpark usage patterns coding standards and performance tuning techniques to promote consistency and maintain high quality data engineering outputs.