AI Search Engineer
Overview
We are seeking an AI Search Engineer with a deep understanding of large-scale information retrieval systems, deep learning, databases, and Retrieval-Augmented Generation (RAG) architectures. The ideal candidate will have expertise in developing and optimizing search algorithms, implementing efficient indexing techniques, and leveraging RAG to enhance AI-powered search and question-answering systems.
Key Responsibilities
- RAG System Research and Implementation: Lead the design and implementation of advanced retrieval systems like Deep Memory by Activeloop, delivering optimized RAG systems across the entire value chain - from embedding or model fine-tuning to retrieval optimization with custom algorithms, to enhance knowledge retrieval accuracy.
- Search Algorithm Optimization: Develop and refine search algorithms, including semantic search, hybrid search, and multi-modal search techniques, to improve retrieval performance and relevance ranking.
- Vector Database Integration: Implement and optimize vector storage and indexing solutions within Deep Lake, ensuring efficient similarity search capabilities for high-dimensional embeddings used in RAG systems.
- Query Understanding and Processing: Design and implement advanced query processing pipelines, including query expansion, intent recognition, and contextual interpretation to enhance search precision.
- Information Retrieval Model Development: Create and fine-tune machine learning models specifically for information retrieval tasks, such as document ranking, query-document relevance scoring, and zero-shot retrieval.
- Performance Evaluation and Metrics: Establish comprehensive evaluation frameworks for search and RAG systems, including relevance assessments, A/B testing, and user satisfaction metrics to continually improve system performance.
- Scalability and Efficiency: Optimize RAG and search systems for high throughput and low latency, ensuring they can handle large-scale datasets and real-time query processing demands.
- Data Ingestion and Indexing: Develop efficient data ingestion pipelines and indexing strategies to support rapid updates and real-time search capabilities across diverse data types and sources.
Qualifications
- Master's or PhD degree in Computer Science, Machine Learning, Statistics, or a related field.
- Strong programming skills in one or more programming languages, such as Python or C++.
- Extensive experience with machine learning libraries, such as TensorFlow, PyTorch, Llama Index, LangChain, etc.
- Proven experience in developing and deploying complex machine learning models in production environments, including experience with cloud-based platforms, edge devices, or embedded systems.
- Strong understanding of advanced machine learning algorithms, such as deep learning, reinforcement learning, RAGs, and ensemble methods.
- Experience with model optimization techniques, such as hyper-parameter tuning, model compression, and quantization.
- Solid understanding of data pre-processing, feature engineering, and data quality assurance techniques.
- Experience with large-scale, complex data sets.
- Excellent problem-solving skills and ability to analyze and interpret complex data sets to extract meaningful insights and drive decision-making.
Preferred Qualifications
- Experience in training deep learning models in a distributed manner.
- Publications in top-tier machine learning and AI conferences such as ICML, NeurIPS, and CVPR.
- A builder attitude and a passion for developing hyper-scalable software for ML.
- Ability to proactively identify and anticipate problems and provide tangible solutions.
- Enthusiasm for the startup journey of building an endurable, scalable business.
About Us
Activeloop is a cutting-edge company focused on developing Deep Lake, a database for AI powered by a unique storage format optimized for deep-learning and Large Language Model (LLM) based applications. We simplify the deployment of enterprise-grade LLM-based products by offering storage for all data types, querying and vector search, data streaming while training models at scale, data versioning and lineage for all workloads, and integrations with popular tools such as LangChain, LlamaIndex, Weights & Biases, and many more. Join us in our mission to revolutionize AI data management.
Location
This is a remote position, allowing you to work from anywhere.
Salary
The salary range for this position is €160,000 to €220,000 per year.
If you are passionate about AI and search technologies and want to work in a dynamic and innovative environment, we would love to hear from you!
Similar jobs
Last update: 23 minutes ago
AI Engineer
Join Alibaba Group as an AI Engineer to develop groundbreaking AI search products using LLM technology.
AI Engineer with LLM Expertise
Join Resolve AI as an AI Engineer in San Francisco to build AI-powered workflows with LLM expertise.
Generative AI Engineer
Join Alibaba Group as a Generative AI Engineer to develop cutting-edge AI search products using LLM technology.
Full Stack AI Engineer
Join Nexla as a Full Stack AI Engineer to develop AI-powered applications, enhance platform capabilities, and work with cutting-edge technologies.
Senior Machine Learning Engineer
Join Bloomreach as a Senior Machine Learning Engineer to design and implement AI-driven components for personalized digital experiences.
Machine Learning/AI Engineer
Join Keysight Technologies as a Machine Learning/AI Engineer to develop and optimize AI/ML models for EDA applications.
Senior Machine Learning Engineer
Join Adobe as a Senior Machine Learning Engineer to lead Gen AI projects, focusing on document cloud technologies.
AI Solutions Software Engineer
Join DwellFi as an AI Solutions Software Engineer to develop innovative AI solutions using LangChain or Llama.
Senior Machine Learning Engineer, Marketing Technology
Join Airbnb as a Senior Machine Learning Engineer to enhance personalized customer experiences using AI/ML.
AI Solutions Software Engineer
Join DwellFi as an AI Solutions Software Engineer to develop innovative AI solutions using LangChain or Llama.
Senior Full Stack Engineer - AI Applications
Join Coactive AI as a Senior Full Stack Engineer to develop AI-driven solutions using React, TypeScript, and FastAPI.
Software Engineer, Applied Engineering
Join OpenAI as a Software Engineer in Applied Engineering to develop innovative AI products using JavaScript, React, and Python.
Principal AI/ML Engineer
Join Activision as a Principal AI/ML Engineer to develop tools for game content creation using ML and DL technologies.
AI Engineer with Computer Vision and NLP Expertise
Join Algorhythm as an AI Engineer to develop cutting-edge AI solutions with a focus on Computer Vision and NLP.
AI Researcher / Engineer
Join Moonwalk Biosciences as an AI Researcher / Engineer to develop AI models for epigenetic control in therapeutics.
Senior Generative AI/Machine Learning Engineer and Data Scientist
Join Adobe as a Senior Generative AI/Machine Learning Engineer and Data Scientist to innovate with AI models.
Software Engineer - AI Training Data
Join Voltai as a Software Engineer to build and optimize AI training data systems, focusing on semiconductor datasets.
Backend Engineer with Machine Learning Focus
Join Accrete AI as a Backend Engineer with a focus on machine learning, building scalable AI solutions.
Software Engineer, Generative AI
Join Meta as a Software Engineer in Generative AI, focusing on NLP and large language models. Work with a global team to innovate AI products.
Senior Applied AI Model Researcher
Join Ampere as a Senior Applied AI Model Researcher to lead AI model development and optimization in a remote role.
Machine Learning Engineer with AI/ML Experience
Join us as a Machine Learning Engineer to develop AI/ML models and applications. Work remotely with top-tier companies.
Senior Machine Learning/Generative AI Engineer
Join GlobalLogic as a Senior ML/GenAI Engineer to develop and optimize AI chatbots using LLMs. Remote work available.
Senior Software Engineer, Personalization and Contextualization Infrastructure
Join Airbnb as a Senior Software Engineer to enhance search infrastructure, focusing on personalization and contextualization.
Principal Software Engineer - AI Platform
Join Accrete AI as a Principal Software Engineer to lead AI platform development, leveraging AI/ML frameworks and cloud technologies.