A Chip-Agnostic Inference Engine Just Cut Robot Reaction Time by 90 Percent

Infinigence AI, working with Tsinghua University and Shanghai Jiao Tong University, open-sourced an inference engine on Monday that cuts the time a robot takes to turn a vision-language-action model's decision into a physical movement from 278 milliseconds down to under 26 milliseconds on NVIDIA's Jetson Thor hardware, a tenfold reduction that the three institutions say removes one of the last unsolved bottlenecks standing between embodied AI demonstrations and robots that actually work a full shift. The engine, called APXInf, is free to download and modify from GitHub, and it launches supporting two vision-language-action models already in wide industry use, Physical Intelligence's Pi 0.5 and X Square Robot's WALL-OSS, across three hardware targets: NVIDIA's RTX 4090, Jetson Orin, and Jetson Thor chips. At 38.46 hertz inference frequency in FP8 precision, the engine now runs fast enough to support the kind of continuous, real-time control loop a robot needs to walk, grasp, and correct course without the visible stutter that has made many embodied-AI demos look impressive on stage and unreliable on a factory floor.

Infinigence AI, founded in Beijing in 2023 by a team spun out of Tsinghua University's electronic engineering department and led by professor Wang Yu, has raised more than RMB 2.2 billion, approximately US$323 million, from investors including Tencent, Baidu, Lenovo, and Xiaomi, positioning itself as an infrastructure company that optimizes how large AI models actually run on hardware rather than building the models themselves. That positioning matters for understanding why APXInf exists: Infinigence is not trying to out-train Physical Intelligence or X Square Robot on model capability. It is trying to solve the narrower, less glamorous problem of what happens after a capable model gets shipped onto a robot with a fraction of a data center's compute budget, a limited power envelope, and no tolerance for the kind of latency a cloud chatbot user would never notice.

Why a Few Hundred Milliseconds Breaks a Robot and Not a Chatbot

The specific failure mode APXInf targets is one that buyers evaluating any embodied-AI vendor's real-world readiness should understand before signing a deployment contract. A cloud-based language model that takes an extra 300 milliseconds to respond produces an imperceptible delay for a person typing into a chat window. A robot's control loop that takes an extra 300 milliseconds to translate a decision into a joint movement produces exactly the kind of jerky, discontinuous motion that makes a demo look unfinished and, more seriously, can cause a multi-step manipulation task to fail outright when a gripper closes a fraction of a second late on a moving object. Infinigence's own framing of the problem, that a model running well in the cloud faces an entirely different set of questions once it lands on a physical robot, whether it can boot instantly, whether it can extract every available unit of compute from limited onboard silicon, whether latency stays low enough not to become the bottleneck, whether a model that worked flawlessly in a development environment stays stable in continuous operation, describes exactly the gap between a laboratory result and a deployable product that has stalled multiple humanoid and industrial robotics programs over the past two years.

APXInf's answer to that gap rests on three engineering choices rather than a single algorithmic trick. The speed improvement comes from end-to-end optimization across the model's pipeline, computation graph, and individual kernel operations, combined with a custom fused-operator technique the team calls Agent4Kernel, rather than from optimizing any single model operation in isolation, which is why the resulting 10.7-times latency reduction holds up as a full-loop figure rather than a benchmark cherry-picked from one processing stage. Stability comes from a minimal runtime built in Rust, a systems programming language chosen specifically for its memory-safety guarantees, wrapped in a Python interface that keeps the tool accessible to robotics developers who are not systems programmers by training, a combination meant to prevent the memory leaks, race conditions, and unpredictable crashes that can bring down a robot's control system hours into a shift rather than during initial testing. The third property, which the team calls agility, uses what it describes as an agentic engineering approach to let new vision-language-action models get integrated into the engine faster, an explicit bet that the pace of new embodied-model releases will keep accelerating and that an inference engine locked to today's model architectures will be obsolete within a year.

The Ecosystem Play Underneath the Benchmark Numbers

APXInf did not appear as a standalone project. It extends an existing open-source framework called RLinf, which Infinigence and Tsinghua released in September 2025 as, by the team's description, the first large-scale reinforcement-learning framework built specifically for post-training embodied AI models. That earlier release handled the training and evaluation side of building an embodied model; APXInf now handles the deployment side, giving developers a single, continuous open-source pipeline from training a model, to validating its behavior, to running it on a physical robot's onboard chip. For a robotics startup or industrial automation buyer evaluating build-versus-buy decisions on their embodied-AI stack, that end-to-end continuity is arguably a more significant signal than either individual benchmark number, because it lowers the switching and integration costs that currently force many robotics teams to stitch together separate, incompatible tools for training and deployment.

The roadmap Infinigence has published alongside the launch reinforces that this is meant to be infrastructure, not a one-off research release. The team says it is already adapting APXInf to support additional model families, including Alibaba's Qwen and NVIDIA's Groot, alongside broader categories of vision-language and world models, and it has committed to extending hardware support beyond NVIDIA's chip lineup to domestic Chinese silicon, domestic robot operating systems, and AMD's processors. That multi-chip ambition is a direct hedge against the same export-control and supply-chain uncertainty that has pushed a wide range of Chinese AI infrastructure companies to build hardware-agnostic tooling rather than betting a company's technology stack on a single chip vendor's continued availability. A robotics buyer building a deployment plan around Pi 0.5 or WALL-OSS today gets a tool that already runs on mainstream Nvidia hardware, with an explicit commitment to portability if geopolitical or supply constraints eventually force a hardware substitution.

Why Infinigence Is Recruiting Contributors Rather Than Selling a License

Infinigence has explicitly framed this first release as a starting point rather than a finished product, and the company is soliciting outside contributors, robotics teams already integrating Pi 0.5 on Jetson-class hardware, developers working on RTX 4090 or Orin projects, and anyone building a deployment pipeline for a new embodied model, to file issues and submit code changes directly to the GitHub repository. That open-contribution posture is a deliberate strategic choice rather than a courtesy gesture. An inference engine's long-term value comes from how many hardware configurations, model architectures, and edge cases it has actually been tested against, and Infinigence cannot internally validate every combination of chip, model, and task a robotics industry moving as fast as this one will produce over the next year. Crowdsourcing that validation through open contribution is the same strategy that made frameworks like PyTorch and Hugging Face's Transformers into industry standards well beyond what their original creators could have tested alone, and it is a more credible long-term bet on becoming default infrastructure than a closed, vendor-locked alternative would be.

What This Changes for Teams Evaluating Embodied-AI Vendors

The practical test for any buyer or integrator is not whether APXInf's benchmark numbers hold up in Infinigence's own testing, but whether they hold up on a specific robot running a specific task in a specific facility, since inference-engine performance claims have historically been sensitive to exactly which model variant, quantization level, and workload pattern a team measures. FP8 precision, the setting behind the headline 26-millisecond figure, trades some numerical accuracy for speed, and a procurement team should ask any vendor pitching an APXInf-based deployment to show the same latency and stability numbers at the precision level the actual application requires, rather than accepting a best-case FP8 figure as a guarantee across every configuration. That caveat does not diminish the underlying achievement so much as it defines the specific diligence question a serious buyer needs answered before committing a production timeline to it.

What the open-source release does change immediately is the cost of experimentation. Before APXInf, a robotics team wanting to run Pi 0.5 or WALL-OSS on Jetson Thor with production-grade latency needed either in-house systems engineering talent capable of building a comparable optimization stack from scratch, or a paid vendor relationship with whoever had already solved that problem internally. A free, open-source engine that Infinigence, Tsinghua, and Shanghai Jiao Tong University have already validated against two widely used models removes that barrier for smaller robotics startups and university labs that could not previously justify the systems-engineering investment required to move a capable model from a research demo into a robot that runs the same task reliably, hour after hour, without an engineer standing by to restart it.

Picture the difference in practice: a warehouse-automation startup with three engineers and a single Jetson Thor development kit can now clone a GitHub repository, load Pi 0.5, and watch a robotic arm's pick-and-place cycle tighten from a visibly hesitant quarter-second lag to a fluid, continuous motion inside an afternoon, a gap that used to separate a promising research prototype from a fundable product.

This analysis draws on the public open-source release and technical documentation published by Infinigence AI, Tsinghua University, and Shanghai Jiao Tong University describing the APXInf engine. It is for general information purposes only and does not constitute investment, financial, or legal advice.

Hero image credit: NVIDIA.