Senior Data Engineer - Apache Spark, PySpark, Azure Databricks

About the Role

As a Senior Data Engineer at Ocean Finance, you will be a key player in our Data Platform and Services team. Your main responsibilities will include maintaining, optimizing, and evolving our data infrastructure, which is primarily built on Databricks and leverages PySpark and Delta Lake. This role involves designing and implementing data integrations with various advertising platforms, enabling data-driven decisions and providing meaningful insights to our teams.

What You’ll Do

• Design and Develop: Participate in the design, development, and maintenance of our Databricks-based data infrastructure using PySpark and Delta Lake to ensure scalability, reliability, and efficiency.
• CI/CD Pipelines: Implement continuous integration/continuous delivery pipelines to automate deployments and monitor system health.
• Data Integration: Design and implement strategies for integrating with advertising platforms like Google Analytics and Facebook Ads, employing best practices for data extraction, transformation, and loading (ETL).
• Collaboration: Work closely with internal teams such as paid media, CRM, and business intelligence to understand their data needs and tailor solutions accordingly.
• Data Management: Develop and maintain robust data pipelines and ELT processes, ensuring data integrity and accuracy.
• Troubleshooting: Proactively troubleshoot data platform performance issues and contribute to the continuous improvement of system stability and performance.
• Industry Trends: Stay current with the latest trends and advancements in data platform technologies, analytics, data security, and industry best practices.
• Agile Environment: Work as part of a cross-functional Scrum team, actively participating in Scrum ceremonies and contributing to a collaborative, agile development environment.
• Data Governance: Support the development and enforcement of data governance policies, adhering to best practices for data privacy and security.
• Data-Driven Culture: Promote a data-driven culture within your role, advocating for the use of data and analytics in decision-making and process optimization.

Skills You Need

• Apache Spark: Proven experience with Apache Spark is essential.
• Programming: Proficiency in programming languages like Python and PySpark.
• Databricks: Hands-on experience with Databricks is crucial.

What We Offer

• Competitive Compensation: We pay you competitively and review your salary every year.
• Team Integration: Welcome you into our awesome team, providing all the necessary tools and support for an amazing job performance.
• Benefits: Comprehensive benefits package including health plans, eye test vouchers, and a pension scheme.
• Professional Growth: Support for training, mentoring, and career development.
• Work-Life Balance: Generous holiday allowance, including special days off and a half day on the last Friday of every month.
• Social and Community Engagement: Regular team socials, retreats, and community volunteer days.