We build massive-scale infrastructure to crawl the web, train state-of-the-art embedding models to index it, and develop super high performant vector databases in Rust to search over it.
We are building a search engine from scratch and need a Data Engineer to architect and build the data infrastructure that powers everything we do. You will have enormous autonomy in designing systems that scale to hundreds of petabytes.
We build massive-scale infrastructure to crawl the web, train state-of-the-art embedding models to index it, and develop super high performant vector databases in Rust to search over it.