Data Engineer with Apache Spark Experience
MapiqJoin Mapiq as a Data Engineer
Are you passionate about data engineering and eager to work with cutting-edge technology? Mapiq is looking for a Data Engineer to join our dynamic Data Team. In this role, you will have the opportunity to make a significant impact on our product by building and maintaining scalable data pipelines that leverage large datasets from sensors and business applications.
Key Responsibilities
- Design and Develop Data Pipelines: Create and maintain scalable data pipelines and systems to process large datasets efficiently.
- Optimize ETL Processes: Work with large datasets to optimize data extraction, transformation, and loading (ETL) processes.
- Implement Spark Solutions: Develop and optimize Spark-based solutions for data processing and analysis.
- Collaborate Across Teams: Work with cross-functional teams to define, implement, and monitor data quality metrics.
- Ensure Data Security: Maintain data security, compliance, and governance across all pipelines.
- Troubleshoot Issues: Resolve issues related to data systems, processes, and infrastructure.
Who We Are Looking For
We are seeking a candidate with at least 2 years of experience in a similar role. The ideal candidate will have:
- A Bachelor's degree in Computer Science or a related field.
- Proficiency in Python for data processing and automation.
- Proficiency in SQL for database management and query optimization.
- A solid understanding of Spark concepts, including Spark DataFrame, Spark SQL, and Spark Streaming.
Nice-to-Have Qualifications
- Experience with Databricks and Databricks certifications.
- Knowledge of DBT (Data Build Tool) for data transformation.
- Experience with Infrastructure as Code (IaC) tools like Terraform or Bicep.
- Familiarity with Azure DevOps Pipeline for CI/CD processes.
- Experience with the Azure Ecosystem, including Azure SQL Server, Azure CosmosDB, Azure EventHub, Azure ServiceBus, and Azure FunctionApp.
What We Offer
- Hybrid Working Environment: Enjoy the flexibility of working both on-site and off-site.
- Global Workstation: Work from anywhere in the world for one month each year.
- Professional Development: Access to Databricks Academy for continuous learning and certification opportunities.
- Competitive Compensation: We offer a fair salary based on your experience and skills.
- Comprehensive Benefits: Pension plan, 25 vacation days, travel reimbursement, and more.
- Supportive Community: Join a team of board game enthusiasts, sports fanatics, coffee lovers, and more.
At Mapiq, we value diversity and strive to create an inclusive environment where everyone can bring their unique ideas and backgrounds to the table. If you are ready to take on this exciting challenge, we would love to hear from you!
Benefits Extracted with AI
- Pension plan
- Hybrid working environment
- One month workstation from anywhere
- Free mental health support
- Competitive pay
- €300 home office reimbursement
- Databricks Academy access
- 25 vacation days
- Travel reimbursement
Similar jobs
Last update: 23 minutes ago
Data Engineer with ETL and PySpark Experience
Join Sanoma Learning as a Data Engineer, focusing on ETL, PySpark, and data warehousing in a dynamic educational environment.
Data Platform Engineer (Kafka, Databricks, Python, Azure)
Join Albert Heijn as a Data Platform Engineer to enhance our data platform using Kafka, Databricks, Python, and Azure.
Senior Back End Developer (C#, .NET, Azure)
Join Mapiq as a Senior Back End Developer. Work with C#, .NET, and Azure to enhance our workplace experience platform.
Lead DevOps Engineer – Data Platform
Lead DevOps Engineer for Data Platform in Bangkok, expertise in Kubernetes, Spark, and cloud technologies, relocation provided.
Data Engineer with AWS, Java, and Python
Join Poppi Technologies as a Data Engineer in Valenzano, Italy. Work with AWS, Java, and Python to drive AI in finance.
Senior Data Engineer
Join us as a Senior Data Engineer in Lisbon to design and maintain data infrastructure. Hybrid role with flexible benefits.
Senior Data Engineer
Join Roche as a Senior Data Engineer in Sant Cugat del Vallès, Spain. Work on data pipelines, automation, and cloud services.
Senior Data Engineer (Fintech)
Join Agoda's fintech team as a Senior Data Engineer in Bangkok. Work with cutting-edge technologies in a dynamic environment.
Senior Data Engineer
Join Expleo Group as a Senior Data Engineer in Madrid. Work on international projects with flexible hours and hybrid work model.
Senior Data Engineer with Azure Expertise
Join Eliq as a Senior Data Engineer to enhance our Azure-based data platform and drive the energy transition.
Observability Platform Data Engineer
Join Swift as an Observability Platform Data Engineer in Leiden, enhancing our Observability Platform with ELK stack expertise.
Data Engineering Intern
Join Metyis as a Data Engineering Intern in Amsterdam. Gain hands-on experience in data pipelines, warehousing, and modeling.
Data Engineer with Blockchain and Cloud Experience
Join Messari as a Data Engineer to design blockchain data models, build dashboards, and derive insights. Remote role with competitive benefits.
Senior Data Engineer - Fintech Team
Join Agoda's Fintech team as a Senior Data Engineer in Bangkok. Enhance data systems with Java, Scala, and Spark. Relocation provided.
Lead DevOps Engineer – Data Platform
Lead DevOps Engineer for Data Platform in Milan. Work with Kubernetes, Spark, and more. Relocation provided.
Data Engineer II
Join Accolade as a Data Engineer II in Prague. Design and maintain cloud-native data infrastructure using AWS and modern technologies.
Senior Data Engineer - Fintech Team
Join Agoda's Fintech team as a Senior Data Engineer in Bangkok. Lead data system improvements using Java, Scala, and Spark.
Cloud Data Engineer
Join NPO as a Cloud Data Engineer to enhance data platforms using GCP, Python, and more. Flexible hours and growth opportunities.
Senior Python Engineer
Join Topicus as a Senior Python Engineer to innovate in healthcare data solutions. Work with APIs, ETL, and Kubernetes in a hybrid role.
Senior Data Engineer - Fintech Team
Join Agoda's fintech team as a Senior Data Engineer in Bangkok. Work with cutting-edge technology in a diverse and inclusive environment.
AI Specialist with Azure Expertise
Join Summ.link as an AI Specialist to develop and integrate AI solutions using Azure tools. Boost your career in a dynamic environment.
Senior Data Engineer - Fintech
Join Agoda's Fintech team as a Senior Data Engineer in Bangkok. Work with Scala, Spark, Java, and more. Relocation provided.
Senior Data Engineer - Fintech
Join Agoda's Fintech team as a Senior Data Engineer in Bangkok. Work with Scala, Spark, Java, and more. Relocation provided.
FullStack Software Developer
Join SPREAD AI as a FullStack Software Developer to innovate in data management and engineering intelligence.