Senior Data Engineer

About the Role

As a Senior Data Engineer at Etraveli Group, you will be a key player in our innovative IT DSR team (Data Science and Research), driving the transformation of our data platform. This role involves replacing our existing infrastructure and integrating with broader data warehouse initiatives. You will develop and optimize data pipelines, champion data quality and governance, and contribute to a culture of technical excellence utilizing our growing dynamic data infrastructure.

Responsibilities

• Interpret and analyze data from various source systems to support machine learning, data integration, and data reporting needs.
• Design, develop, and maintain ETL and streaming pipelines according to business and ML solution needs for pricing, conversion, and revenue optimization.
• Define and ensure data quality standards and assist in implementing relevant monitoring solutions.
• Assist in selecting, evaluating, and integrating industry-leading solutions and tooling required within our DE platform, ensuring scalability, performance, maintainability, and associated documentation.
• Optimize and create new data pipelines together with Machine Learning Engineers and core engine backend developers.
• Work with internal and external stakeholders to prioritize business and information needs.
• Opportunity to assist in ML Ops engineering and machine learning engineering.
• Plan, communicate, and share status updates together with your fellow colleagues and IT Dev developers creating synergies and using best practice.

Learning Opportunities

In this role, you will be using the best selection of tools and processes in the DE domain. The role enables you to learn even more about creating, developing, and implementing standards for DE applications, and get deep knowledge about DE concepts and frameworks. You will also have the possibility to learn more about (and get hands-on experience of) adjacent fields like Data Science, ML Pipeline Architecture, and Platform Engineering.

Requirements

• 5+ years of experience in data engineering or comparable relevant work including:
  • Creating, maintaining, and testing data pipelines both using batch processing and streaming (Scala, Spark, SQL)
  • Data Modelling and mapping business requirements towards data solutions
  • Ingestion (NifiKops, Nifi-Registry)
  • Transformation (Spark, PySpark, Spark operator, Dremio)
  • Metadata Store (Nessie)
  • Data formats (parquet, avro, json, csv)
  • Table formats (Iceberg, Delta)
  • Data lakes and storages (S3 compatible like Ceph)
  • Distributor messaging systems (Kafka)
  • Orchestration (Airflow, ArgoWorkflows, Zookeeper)
  • CI/CD (Jenkins, Artifactory)
  • Service monitoring and alarms (Grafana, Kibana)
  • Data pipeline performance tuning
  • Data quality management
  • Kubernetes, Linux and secrets management (Vault) is a merit

Benefits

At Etraveli Technology, you will be part of a gang of +180 people, all with different cutting-edge skills and personalities. We believe that it is precisely the differences that make us a good team because we share the same ambition - we strive for the same goals, are passionate about what we do and work hard. Our brand new office is located in Merkurhuset in Gothenburg, a stone's throw from public transport and lunch restaurants. We love gaming and after work activities. We know the value of personal development and continuous training. Hackathons and Dev Weeks - We believe in promoting creativity and trying new things. Wellness allowance - Each employee receives a maximum allowable amount each year according to the Swedish Tax Agency to spend on health-promoting activities. Through partners, we offer comprehensive pension and health insurance so that you can quickly get help if the accident occurs. We have a hybrid working model so once you are onboarded you have the flexibility to work from home two days a week. And we serve breakfast every morning.