Software Engineer - Data Platform

Job Description

Company Introduction

At Bot Auto, we are revolutionizing the transportation of goods with our cutting-edge autonomous trucks, enhancing the quality of life for communities around the globe. With the agility of a start-up and the wisdom of seasoned experts, Bot Auto boasts a team that has achieved numerous world-firsts and unparalleled innovations. United by a shared vision, we create miracles and propel the future of transportation. Join us and transform your dreams into reality.

Role Overview

We are seeking a highly skilled and motivated Software Engineer to design, build, and evolve our hybrid-Cloud data plane. The ideal candidate will be a strong hands-on coder and has foundational skills in large scale data storage systems as well as data extraction & transformation pipelines and core & analytical data management.

Key Responsibilities

Data Infrastructure

• Design and implement data infrastructure including data lakehouse systems such as S3, Datalake, Data Catalog, as well as common data formats such as Parquet, Avro, JSON, and more.
• Manage core Data storage, including Relational Databases, NoSQL databases, and Realtime DB.
• Architect container jobs and establish end-to-end workflows using Kubernetes (K8s) and distributed computing for efficient data processing and transformation.

Data Engineering

• Create a robust data collection pipeline to seamlessly transfer substantial data from autonomous systems to both on-premises data centers and cloud environments.
• Build Data Lakehouse and Dashboard to support executive/algorithm/operational decision making.

Platform

• Work with the infrastructure team to deploy and manage the data platform across hybrid cloud solutions using cloud native compute services as well as on premise data centers.
• Engage in full-stack projects such as data platforms and Human-Machine Interfaces (HMIs) using a variety of technologies like Python, Node.js, React, and TypeScript.

Data Mining/Science

• Build Robust Metadata system.
• Scenario/Case Mining from large scale Autonomous Driving Dataset and Operational dataset.

Embedded Engineering

• Develop a versatile data serving Software Development Kit (SDK) and API server (using either C++ or Python) to support training and simulation jobs.
• High performance onboard data recording (C++).

Qualifications

Required

• Bachelor's degree in Computer Science, Engineering, or related field.
• Strong problem-solving skills and attention to detail.
• Excellent communication and collaboration abilities.
• Ability to work in a fast-paced, dynamic environment.
• Proven track record of python production-level development (at least 3+ years).
• Strong background in building modern data infrastructure including data lake (such as S3, delta lake & databricks, AWS Lake formation, Glue Catalog and etc), distributed computing framework (Spark, Presto & Athena and etc), and Business Intelligence Infrastructure (Tableau, Looker, Superset and etc).
• Strong background in data engineering & data science including data modeling, data warehouse, pipeline, reporting tools, and analytics.
• Experienced in Full-stack development, e.g. React, Flask.
• Experienced in managing core data store (such as relational database and document store).
• Familiarity with data privacy and security.
• Familiarity with data retention and cost control.
• Familiarity with cloud infrastructure, e.g. AWS, GCP, K8s, Iac.

Preferred

• Experience with Large Scale Data/ML System Design.
• Experience with Python and C++.
• Experience with the Autonomous Driving Industry.

Benefits

• Remote work.
• Dynamic work environment.
• Opportunity to work with cutting-edge technology.

Join us at Bot Auto and be part of a team that is transforming the future of transportation with autonomous technology.