Engineering Manager, Inference Benchmarking — AI Perf

免费注册后解锁"你的匹配分 vs 高分画像"与个人技能缺口。

NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It’s an outstanding legacy of innovation that’s fueled by great technology—and amazing people. Today, we’re tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world. Doing what’s never been done before takes vision, innovation, and the world’s best talent. As an NVIDIA N, you’ll be immersed in a diverse, supportive environment where everyone is inspired to do their best work. Come join the team and see how you can make a lasting impact on the world. NVIDIA ’s open-source benchmarking platform, AIPerf, is the growing standard for assessing LLM serving performance across various inference frameworks. Hyperscalers, cloud providers, and enterprises use AIPerf to inform decisions on production inference. This includes choosing GPUs, optimizing costs, reducing latency, improving efficiency, and scaling. As Technical Lead Manager, you will lead the engineering team within NVIDIA ’s Dynamo organization. Your responsibility is to build and advance the platform so AIPerf becomes the leading benchmarking tool for datacenter, local, and edge use cases. This span LLM, multimodal, diffusion, and computer vision inference. This position combines hands-on leadership with expertise in systems engineering, inference infrastructure, and open-source communities. It has a direct effect on how AI performance is measured and pushed forward. What you'll be doing: Driving the technical roadmap for AIPerf's core infrastructure: load generation, ZMQ-based microservices, GPU telemetry (DCGM/PyNVML, Prometheus metrics, statistical confidence intervals, and Kubernetes-native deployment. Taking ownership for the accuracy and statistical soundness of benchmark results that engineering groups throughout the industry depend on to inform production infrastructure decisions. Advising upstream engine integrations involving vLLM, TRT-LLM, and SGLang in partnership with NVIDIA 's Dynamo and NIM teams to maintain AIPerf's relevance across emerging hardware, workload categories, and inference configurations. Hiring, mentoring, and growing a team of senior engineers operating in a high-velocity open-source environment with active external contributors worldwide. What we need to see: Bachelor's degree in Computer Science, Electrical Engineering, or related field, or equivalent experience. 8+ overall years of software engineering experience building performance-critical infrastructure, ML tooling, or distributed systems. 3+ years of engineering leadership experience as a tech lead, TLM, or engineering manager. Deep understanding of LLM inference mechanics — TTFT, ITL, KV caching, Prefill/Decode, speculative decoding — and the ability to reason about measurement correctness and reproducibility. Proven track record of collaborating across multi-functional groups and delivering production-quality output in high-velocity, high-external-visibility environments. Ways to stand out from the crowd: Extensive experience with vLLM, TRT-LLM or SGLang internals along with contributions to their upstream projects. Experience building Kubernetes-native infrastructure including operators, Helm charts, and GPU observability tooling (DCGM, dcgm-exporter, PyNVML). Background in competitive benchmarking frameworks such as MLPerf or equivalent industry-standard evaluation systems. History leading or making meaningful contributions to active open-source projects with external communities. Widely considered to be one of the technology world’s most desirable employers, NVIDIA offers highly competitive salaries and a comprehensive benefits package. As you plan your future, see what we can offer to you and your family Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary

NVIDIA 致力于变革计算机图形、PC 游戏和加速计算领域已超过 25 年。这是一段卓越的创新传承，由伟大的技术和杰出的人才共同推动。如今，我们正在挖掘 AI 的无限潜力，以定义下一个计算时代。在这个时代，我们的 GPU 将成为计算机、机器人和自动驾驶汽车的大脑，使它们能够理解世界。做前所未有的事情需要远见、创新和世界上最优秀的人才。作为 NVIDIA 的一员，你将沉浸在一个多元、支持性的环境中，每个人都被激励着发挥最佳水平。加入团队，看看你如何能为世界带来持久的影响。NVIDIA 的开源基准测试平台 AIPerf 正在成为评估各种推理框架下 LLM 服务性能的日益通行的标准。超大规模云服务商、云提供商和企业使用 AIPerf 来为生产推理决策提供依据，包括选择 GPU、优化成本、降低延迟、提高效率和扩展规模。作为技术主管经理，你将领导 NVIDIA Dynamo 组织内的工程团队。你的职责是构建和推进该平台，使 AIPerf 成为面向数据中心、本地和边缘用例的领先基准测试工具。这涵盖 LLM、多模态、扩散模型和计算机视觉推理。该职位结合了亲力亲为的领导力与系统工程师、推理基础设施和开源社区方面的专业知识。它对 AI 性能的衡量和推进方式有着直接影响。你将要做的事情：推动 AIPerf 核心基础设施的技术路线图：负载生成、基于 ZMQ 的微服务、GPU 遥测（DCGM/PyNVML、Prometheus 指标、统计置信区间）以及 Kubernetes 原生部署。对基准测试结果的准确性和统计严谨性负责，全行业的工程团队都依赖这些结果来为生产基础设施决策提供依据。与 NVIDIA 的 Dynamo 和 NIM 团队合作，指导涉及 vLLM、TRT-LLM 和 SGLang 的上游引擎集成，以保持 AIPerf 在新兴硬件、工作负载类别和推理配置中的相关性。招聘、指导和培养一支由资深工程师组成的团队，在一个高速运转的开源环境中工作，拥有遍布全球的活跃外部贡献者。我们需要看到的：计算机科学、电气工程或相关领域的学士学位，或同等经验。8 年以上构建性能关键型基础设施、ML 工具或分布式系统的软件工程经验。3 年以上作为技术负责人、TLM 或工程经理的工程领导经验。深入理解 LLM 推理机制——TTFT、ITL、KV 缓存、Prefill/Decode、投机解码——以及推理测量正确性和可复现性的能力。在跨多功能团队协作方面有可靠的成功记录，并在高速、高外部可见度的环境中交付生产级成果。脱颖而出的方式：在 vLLM、TRT-LLM 或 SGLang 内部机制方面有丰富经验，并对其上游项目有贡献。构建 Kubernetes 原生基础设施的经验，包括 operator、Helm chart 和 GPU 可观测性工具（DCGM、dcgm-exporter、PyNVML）。在竞争性基准测试框架（如 MLPerf 或同等行业标准评估系统）方面的背景。领导或对活跃的开源项目及外部社区做出重要贡献的经历。作为技术界最受青睐的雇主之一，NVIDIA 提供极具竞争力的薪资和全面的福利待遇。在规划你的未来时，看看我们能为你和你的家人提供什么。你的基本工资将根据你的地点、经验以及类似职位员工的薪酬来确定。基本工资

⚑ 岗位信息收录自公开招聘页（Himalayas），以雇主官网为准