Software Engineer - Data Platform

Job Description

Company Introduction

At Bot Auto, we are revolutionizing the transportation of goods with our cutting-edge autonomous trucks, enhancing the quality of life for communities around the globe. With the agility of a start-up and the wisdom of seasoned experts, Bot Auto boasts a team that has achieved numerous world-firsts and unparalleled innovations. United by a shared vision, we create miracles and propel the future of transportation. Join us and transform your dreams into reality.

Role Overview

We are seeking a highly skilled and motivated Software Engineer to design, build, and evolve our hybrid-Cloud data plane. The ideal candidate will be a strong hands-on coder and has foundational skills in large scale data storage systems as well as data extraction & transformation pipelines and core & analytical data management.

Key Responsibilities

Data Infrastructure

Design and implement data infrastructure including data lakehouse systems such as S3, Datalake, Data Catalog, as well as common data formats such as Parquet, Avro, JSON, and more.
Manage core Data storage, including Relational Databases, NoSQL databases, and Realtime DB.
Architect container jobs and establish end-to-end workflows using Kubernetes (K8s) and distributed computing for efficient data processing and transformation.

Data Engineering

Create a robust data collection pipeline to seamlessly transfer substantial data from autonomous systems to both on-premises data centers and cloud environments.
Build Data Lakehouse and Dashboard to support executive/algorithm/operational decision making.

Platform

Work with the infrastructure team to deploy and manage the data platform across hybrid cloud solutions using cloud native compute services as well as on premise data centers.
Engage in full-stack projects such as data platforms and Human-Machine Interfaces (HMIs) using a variety of technologies like Python, Node.js, React, and TypeScript.

Data Mining/Science

Build Robust Metadata system.
Scenario/Case Mining from large scale Autonomous Driving Dataset and Operational dataset.

Embedded Engineering

Develop a versatile data serving Software Development Kit (SDK) and API server (using either C++ or Python) to support training and simulation jobs.
High performance onboard data recording (C++).

Qualifications

Required

Bachelor's degree in Computer Science, Engineering, or related field.
Strong problem-solving skills and attention to detail.
Excellent communication and collaboration abilities.
Ability to work in a fast-paced, dynamic environment.
Proven track record of python production-level development (at least 3+ years).
Strong background in building modern data infrastructure including data lake (such as S3, delta lake & databricks, AWS Lake formation, Glue Catalog and etc), distributed computing framework (Spark, Presto & Athena and etc), and Business Intelligence Infrastructure (Tableau, Looker, Superset and etc).
Strong background in data engineering & data science including data modeling, data warehouse, pipeline, reporting tools, and analytics.
Experienced in Full-stack development, e.g. React, Flask.
Experienced in managing core data store (such as relational database and document store).
Familiarity with data privacy and security.
Familiarity with data retention and cost control.
Familiarity with cloud infrastructure, e.g. AWS, GCP, K8s, Iac.