Inference Software Engineer - Collectives
EtchedSan Jose, California, United States
Posted October 28, 2025
175000 - 275000 USD year
Full Time
About the Company
Etched specializes in developing cutting-edge servers by embedding transformer architecture directly onto chips. This innovative approach delivers unparalleled performance for transformer inference tasks, setting a new standard in the industry.
Job Description
Etched is building AI inference systems for transformers, aiming for higher performance, lower cost, and latency than B200. The team is seeking a highly skilled Inference SW engineer to formalize and optimize collectives (e.g., Send/Recieve, AllReduce, Broadcast) to serve Sohu’s dataflow architecture. This role focuses on scaling out Sohu's system with MoE architectures.
Requirements
- Strong proficiency in Rust and/or C++; familiarity with PyTorch and/or JAX.
- Experience designing/optimizing collectives (e.g. NCCL, MPI collectives, XLA collectives, etc.).
- Strong systems knowledge, including Linux internals, accelerator architectures (e.g., GPUs, TPUs), high-speed interconnects (e.g., NVLink, InfiniBand) and RDMA.
- Solid understanding of distributed systems concepts, algorithms, and challenges.
Benefits
- Full medical, dental, and vision packages
- Housing subsidy
- Daily lunch and dinner
- Relocation support
- Compensation range
