Senior Software Engineer (Elasticsearch)

About The Role

We seek a highly skilled and motivated Senior Software Engineer, specializing in Elasticsearch (ELK Stack), to join our Core Infrastructure team at Uber. Our sophisticated infrastructure spans both on-premise data centers and the cloud, supporting a diverse variety of workloads (stateless, batch, streaming), each different in its characteristics and requirements, that support our internal corporate use cases.

What You Will Do

• Work with the team to design, document, build, secure, and maintain Elasticsearch, Logstash, Kibana (ELK Stack) Enterprise solutions deployed with the ECK K8s operator to support non-production and production deployments.
• Collaborate closely with architects, engineers, and integrators to assess customer requirements and design and support an ELK Stack solution to ensure needs are met.
• Follow life-cycle processes to move solutions from Staging to Production.
• Maintain and develop Logstash pipelines to process numerous log sources.
• Install and configure an Elastic Cloud Kubernetes (ECK) solution in non-production and production environments, ensuring communication and integration among the Elasticsearch products and data sources.
• Document the installation and configuration for deployment into production.
• Secure the solution by being familiar with TLS, certificates, SSO/PIV authentication, and encryption technologies.
• Develop and apply quantitative and qualitative analytic methods to identify, collect, process, and analyze large data sets for specified purposes.
• Work with data lifecycle management.
• Test data flows, troubleshoot issues, and monitor the health of the solution and servers to maximize performance and minimize downtime.
• Serve as a trusted advisor, providing subject matter expertise, guidance, and best-practice recommendations.

Basic Qualifications

• Bachelor's/Master's degree in Computer Science or equivalent experience.
• Proficient in one of the following programming languages: Java, Go, C/C++, or similar languages. Good scripting skills and ability to pick up new ones.
• Systematic problem-solving approach and knowledge of algorithms, data structures, and complexity analysis.
• 5+ years of Elasticsearch stack (Elasticsearch, Logstash, Beats, Kibana) experience in production.
• Experience with data ingest, Extract, Transform, and Load (ETL) techniques, preferably with Logstash and Beats.
• Experience with multi-node clustering, architecture, development, implementation, and health monitoring of the indexes.
• Knowledge of hot/warm/cold data/index lifecycle management implementations.
• Knowledge of Elasticsearch Management, reindexing, and data mappings, JSON and KQL searching syntax, and Kibana dashboard options.
• Expertise in containerization with Docker/Kubernetes and orchestration tools Ansible/Puppet.

Bonus Qualifications

• Experience in architectures involving multiple microservices and their interaction with web services.
• Experience designing and implementing highly scalable, robust, and fault-tolerant services.
• Familiarity with observability tools such as Prometheus, Grafana, and the Elastic stack.
• GCP, AWS, and/or Kubernetes certifications are strongly preferred.