Remote Jobs Now
D Matrix

Senior Staff LLM Inference Engineer

Who can apply: US-based role. The employer has not said which states are eligible — confirm before applying.
Location as listed by the employer: “Santa Clara”
Apply on D Matrix's site ↗

Opens the employer's official application page.

Generate a CV for this job

A CV tailored to this job's requirements in one click — boosted with testimonials from people who know your work.

Generate my CV

About this role

D Matrix is hiring for a Senior Staff LLM Inference Engineer role (full-time). See the official posting for full details.

Requirements

  • Bachelor’s degree in Computer Science, Electrical Engineering, or a related field, and 10+ years of relevant engineering experience or equivalent demonstrated experience.
  • Master’s or PhD in Computer Science, Electrical Engineering, or a related field preferred, with 6+ years of relevant industry experience.
  • Strong proficiency in Python and C/C++.
  • Hands-on experience optimizing LLM inference — attention kernels, KV cache, batching strategies, quantization (INT8/FP8/INT4).
  • Experience with at least one major inference framework (vLLM, SGLang, TensorRT-LLM, ONNX Runtime, or similar) at a contributor level.
  • Familiarity with GPU kernel programming (CUDA/Triton) and performance profiling tools.
  • Experience with heterogeneous compute deployments — scheduling inference workloads across dissimilar hardware (accelerators, CPUs, GPUs).
  • Familiarity with custom silicon or ASIC-based inference (beyond GPU-only environments).

Summarized from the employer's posting — check the full posting for every requirement.