C++ Controller API¶

High-performance C++ controller for real-time policy execution with ONNX Runtime.

Overview¶

The C++ controller provides a real-time inference system for deploying RL policies exported to ONNX format. It features:

ONNX Runtime Integration - Efficient model execution
Extensible Interface Design - Easy integration with different robot platforms
Component-Based Architecture - Modular input/output handling
Automatic Component Matching - Maps ONNX model I/O to robot interfaces

Core Classes¶

Controller¶

class OnnxRLController¶

Controller wrapping around an ONNX policy.

This class can be used to control a robot with an ONNX policy.

Public Functions

explicit OnnxRLController(RobotStateInterface &state, CommandInterface &command, DataCollectionInterface &data_collection)¶

Constructor of the OnnxRLController.

Parameters:

state – A RobotStateInterface to communicate with the robot.
command – A CommandInterface to send commands to the controller.
data_collection – A DataCollection interface for data collection.

bool create(const std::string &onnx_model_path, bool register_default_matchers = true, const OnnxRuntimeOptions &options = {})¶

Create the ONNX model and context.

This function only parses the configuration needed to interface with the ONNX model. Call init() to fully initialize the controller.

If called more than once (e.g. to reload a model), default matchers are registered only on the first call where register_default_matchers is true; custom matchers added via context() are preserved across calls.

Parameters:

onnx_model_path – Path to the ONNX model file.
register_default_matchers – If true (default), all built-in matchers (including StepCountMatcher) are registered the first time create() is called with this parameter set to true. Passing false disables all built-in matchers for that call; only matchers added via context() will be used, and a later call with true can still register the built-in matchers.
options – Runtime options (e.g. execution provider) forwarded to the ONNX runtime.

Returns:

True if parsing succeeds, false otherwise.

bool init(bool enable_data_collection, WorkerMode mode = WorkerMode::SYNC)¶

Initialize the controller.

Parameters:

enable_data_collection – Whether to enable data collection.
mode – Whether to run the ONNX inference synchronously or asynchronously.

Returns:

True if initialization succeeds, false otherwise.

void reset()¶: Reset the controller.

bool update(uint64_t time_us)¶

Update the controller.

Read from state, evaluate ONNX model and write to state. Should be run at updateRate()

Parameters:: time_us – Timestamp in microseconds.
Returns:: True if update succeeds, false otherwise.

inline OnnxContext &context()¶

Context Management¶

class OnnxContext¶

Manages the context for ONNX model input/output components.

OnnxContext is responsible for parsing ONNX model metadata, matching tensor names to appropriate input/output components using registered matchers, and creating the component instances needed to interface between the robot and the ONNX model.

Public Functions

void registerMatcher(std::unique_ptr<Matcher> matcher)¶

Matchers are used to identify ONNX tensor names and create corresponding input/output components.

Parameters:: matcher – Unique pointer to a Matcher instance.

void registerGroupMatcher(std::unique_ptr<GroupMatcher> matcher)¶

Group matchers handle patterns where multiple related tensors need to be processed together (e.g., joint.pos and joint.vel).

Parameters:: matcher – Unique pointer to a GroupMatcher instance.

bool createContext(OnnxRuntime &onnx_model, bool strict = true)¶

Create the context by parsing ONNX model and generating components.

Analyzes the ONNX model’s input/output tensors and metadata, matches them against registered matchers, and creates the appropriate input/output components.

Parameters:

onnx_model – Reference to an initialized OnnxRuntime instance.
strict – If true, fails if any tensor cannot be matched; if false, continues with partial matches.

Returns:

true if context creation succeeded, false otherwise.

inline const std::vector<std::unique_ptr<Input>> &getInputs() const¶

Get all created input components.

Returns:: Const reference to vector of input component unique pointers.

inline const std::vector<std::unique_ptr<Output>> &getOutputs() const¶

Get all created output components.

Returns:: Const reference to vector of output component unique pointers.

inline int updateRate() const¶

Get the control loop update rate from ONNX model metadata.

Returns:: Update rate in Hz, or 0 if not specified in model metadata.

ONNX Runtime¶

class OnnxRuntime¶

A class for evaluating ONNX models using ONNX Runtime.

The OnnxRuntime class manages the initialization, configuration, and inference execution of an ONNX model.

Public Functions

bool initialize(const std::string &model_path, const OnnxRuntimeOptions &options = {})¶

Initializes the ONNX model and configures input/output tensors.

Loads the ONNX model from the specified file path, sets up session options, and prepares memory information. If the model file does not exist, the initialization will fail.

Parameters:

model_path – The file path to the ONNX model.
options – The runtime options.

Returns:

True if initialization is successful, false otherwise.

bool evaluate()¶

Runs inference on the ONNX model with the buffered input data.

This method runs inference and updates the output buffers.

Returns:: True evaluation succeeds, false if validation fails.

std::optional<std::string> getCustomMetadata(const std::string &key) const¶

Getter for custom metadata from ONNX model.

Parameters:: key – The key of the custom metadata stored in the ONNX model.
Returns:: A string with the corresponding custom metadata, if the key exists, nullopt otherwise.

template<typename T> inline std::optional<std::span<T>> inputBuffer(const std::string &name)¶

Retrieves a mutable span to the input tensor buffer of the specified name.

Template Parameters:: T – The data type of the tensor elements (e.g., float, int32_t, bool).
Parameters:: name – The name of the input tensor.
Returns:: An optional span to the tensor buffer if it exists and matches the requested type, nullopt otherwise.

template<typename T> inline std::optional<std::span<T>> outputBuffer(const std::string &name)¶

Retrieves a mutable span to the output tensor buffer of the specified name.

Template Parameters:: T – The data type of the tensor elements (e.g., float, int32_t, bool).
Parameters:: name – The name of the output tensor.
Returns:: An optional span to the tensor buffer if it exists and matches the requested type, nullopt otherwise.

void resetBuffers()¶: Resets all input and output buffers to zero.

std::unordered_set<std::string> inputNames() const¶

Retrieves the set of input tensor names.

Returns:: An unordered set containing the names of all input tensors.

std::unordered_set<std::string> outputNames() const¶

Retrieves the set of output tensor names.

Returns:: An unordered set containing the names of all output tensors.

inline bool isInitialized() const¶

Checks if the ONNX runtime is properly initialized.

Returns:: True if the runtime has been initialized with a valid model, false otherwise.

bool copyOutputToInput(const std::string &output_name, const std::string &input_name)¶

Copies the output tensor data to the input tensor.

Parameters:

output_name – The name of the output tensor.
input_name – The name of the input tensor.

Returns:

True if the copy was successful, false otherwise.

Interfaces¶

These abstract interfaces define the contract for robot integration.

State Interface¶

class RobotStateInterface¶

Interface which provides methods to communicate with the robot.

This class describes the interface for the communication with the robot state and commands.

Public Functions

virtual ~RobotStateInterface() = default¶

inline virtual bool initBasePosW(const BasePosWInfo &info)¶

Initialize data source of base position in world frame (x,y,z).

Called once during initialization (usually non real-time).

Parameters:: info – Arguments including the name of the articulation or rigid object the base belongs to.

inline virtual bool initBaseQuatW(const BaseQuatWInfo &info)¶

Initialize data source of base orientation in world frame (w,x,y,z).

Called once during initialization (usually non real-time).

Parameters:: info – Arguments including the name of the articulation or rigid object the base belongs to.

inline virtual std::optional<Position> basePosW(const BasePosWInfo &info) const¶

Get base position in world frame.

Parameters:: info – Arguments including the name of the articulation or rigid object the base belongs to.
Returns:: The base position in world frame (x,y,z).

inline virtual std::optional<Quaternion> baseQuatW(const BaseQuatWInfo &info) const¶

Get base orientation in world frame.

Parameters:: info – Arguments including the name of the articulation or rigid object the base belongs to.
Returns:: The base orientation in world frame (w,x,y,z).

inline virtual bool initBaseLinVelB(const BaseLinVelBInfo &info)¶

Initialize data source of linear base velocity in base frame.

Called once during initialization (usually non real-time).

Parameters:: info – Arguments including the name of the articulation or rigid object the base belongs to.

inline virtual bool initBaseAngVelB(const BaseAngVelBInfo &info)¶

Initialize data source of angular base velocity in base frame.

Called once during initialization (usually non real-time).

Parameters:: info – Arguments including the name of the articulation or rigid object the base belongs to.

inline virtual std::optional<LinearVelocity> baseLinVelB(const BaseLinVelBInfo &info) const¶

Get linear base velocity rotated in base frame.

Parameters:: info – Arguments including the name of the articulation or rigid object the base belongs to.
Returns:: The linear base velocity in base frame (vx, vy, vz).

inline virtual std::optional<AngularVelocity> baseAngVelB(const BaseAngVelBInfo &info) const¶

Get angular base velocity rotated in base frame.

Parameters:: info – Arguments including the name of the articulation or rigid object the base belongs to.
Returns:: The angular base velocity in base frame (ωx, ωy, ωz).

inline virtual bool initJointPosition(const JointPositionInfo &info)¶

Initialize data sources for joint position data.

Called once during initialization (non real-time).

Parameters:: info – Arguments including the articulation name and the joint name.

inline virtual bool initJointVelocity(const JointVelocityInfo &info)¶

Initialize data sources for joint velocity data.

Called once during initialization (non real-time).

Parameters:: info – Arguments including the articulation name and the joint name.

inline virtual bool initJointEffort(const JointEffortInfo &info)¶

Initialize data sources for joint effort data.

Called once during initialization (non real-time).

Parameters:: info – Arguments including the articulation name and the joint name.

inline virtual bool initJointOutput(const JointOutputInfo &info)¶

Initialize data sinks for joint output data.

Called once during initialization (non real-time).

Parameters:: info – Arguments including the articulation name and the joint name.

inline virtual std::optional<double> jointPosition(const JointPositionInfo &info) const¶

Get joint position.

Parameters:: info – Arguments including the articulation name and the joint name.
Returns:: The position of the joint.

inline virtual std::optional<double> jointVelocity(const JointVelocityInfo &info) const¶

Get joint velocity.

Parameters:: info – Arguments including the articulation name and the joint name.
Returns:: The velocity of the joint.

inline virtual std::optional<double> jointEffort(const JointEffortInfo &info) const¶

Get joint effort.

Parameters:: info – Arguments including the articulation name and the joint name.
Returns:: The effort of the joint.

inline virtual bool setJointPosition(const SetJointPositionInfo &info)¶

Set joint position.

The following control law is assumed: u = joint_effort + p_gain * (joint_position - joint_position_measured) + d_gain * (joint_velocity - joint_velocity_measured)

Parameters:: info – Arguments including the articulation name, joint name, and the joint position to be set.

inline virtual bool setJointVelocity(const SetJointVelocityInfo &info)¶

Set joint velocity.

The following control law is assumed: u = joint_effort + p_gain * (joint_position - joint_position_measured) + d_gain * (joint_velocity - joint_velocity_measured)

Parameters:: info – Arguments including the articulation name, joint name, and the joint velocity to be set.

inline virtual bool setJointEffort(const SetJointEffortInfo &info)¶

Set joint effort.

The following control law is assumed: u = joint_effort + p_gain * (joint_position - joint_position_measured) + d_gain * (joint_velocity - joint_velocity_measured)

Parameters:: info – Arguments including the articulation name, joint name, and the joint effort to be set.

inline virtual bool setJointPGain(const SetJointPGainInfo &info)¶

Set joint p-gain.

The following control law is assumed: u = joint_effort + p_gain * (joint_position - joint_position_measured) + d_gain * (joint_velocity - joint_velocity_measured)

Parameters:: info – Arguments including the articulation name, joint name, and the p-gain to be set.

inline virtual bool setJointDGain(const SetJointDGainInfo &info)¶

Set joint d-gain.

The following control law is assumed: u = joint_effort + p_gain * (joint_position - joint_position_measured) + d_gain * (joint_velocity - joint_velocity_measured)

Parameters:: info – Arguments including the articulation name, joint name, and the d-gain to be set.

inline virtual bool initSe2Velocity(const Se2VelocityInfo &info)¶

Initialize data sources of se(2) frame velocity.

Called once during initialization (non real-time).

Parameters:: info – Arguments including the name of the considered frame.

inline virtual bool setSe2Velocity(const Se2VelocityInfo &info)¶

Set se(2) velocity reference of a specific frame.

Parameters:: info – Arguments including the considered frame name and the se(2) velocity to be set.

inline virtual bool initImuLinearVelocityImu(const ImuLinearVelocityImuInfo &info)¶

Initialize data source of IMU linear velocity.

Called once during initialization (usually non real-time).

Parameters:: info – Arguments including the name of the IMU.

inline virtual std::optional<LinearVelocity> imuLinearVelocityImu(const ImuLinearVelocityImuInfo &info) const¶

Get IMU linear velocity in IMU frame.

Parameters:: info – Arguments including the name of the IMU.
Returns:: The linear velocity of the IMU in IMU frame (vx, vy, vz).

inline virtual bool initImuAngularVelocityImu(const ImuAngularVelocityImuInfo &info)¶

Initialize data source of IMU angular velocity.

Called once during initialization (usually non real-time).

Parameters:: info – Arguments including the name of the IMU.

inline virtual bool initImuOrientationW(const ImuOrientationWInfo &info)¶

Initialize data source of IMU orientation.

Called once during initialization (usually non real-time).

Parameters:: info – Arguments including the name of the IMU.

inline virtual std::optional<AngularVelocity> imuAngularVelocityImu(const ImuAngularVelocityImuInfo &info) const¶

Get IMU angular velocity in IMU frame.

Parameters:: info – Arguments including the name of the IMU.
Returns:: The angular velocity of the IMU in world frame (x,y,z) .

inline virtual std::optional<Quaternion> imuOrientationW(const ImuOrientationWInfo &info) const¶

Get IMU orientation in world frame (w, x, y, z).

Parameters:: info – Arguments including the name of the IMU.
Returns:: The orientation of the IMU in world frame represented as a unit quaternion (w,x,y,z).

inline virtual bool initBodyPositionW(const BodyPositionWInfo &info)¶

Initialize data source of body position in world frame.

Called once during initialization (usually non real-time).

Parameters:: info – Arguments including the articulation name and the body name.

inline virtual bool initBodyOrientationW(const BodyOrientationWInfo &info)¶

Initialize data source of body orientation in base frame (w, x, y, z).

Called once during initialization (usually non real-time).

Parameters:: info – Arguments including the articulation name and the body name.

inline virtual bool initBodyLinearVelocityB(const BodyLinearVelocityBInfo &info)¶

Initialize data source of body linear velocity in body frame.

Called once during initialization (usually non real-time).

Parameters:: info – Arguments including the articulation name and the body name.

inline virtual bool initBodyAngularVelocityB(const BodyAngularVelocityBInfo &info)¶

Initialize data source of body angular velocity in body frame.

Called once during initialization (usually non real-time).

Parameters:: info – Arguments including the articulation name and the body name.

inline virtual std::optional<Position> bodyPositionW(const BodyPositionWInfo &info) const¶

Get body position in world frame.

Parameters:: info – Arguments including the articulation name and the body name.
Returns:: The position of the body in world frame.

inline virtual std::optional<Quaternion> bodyOrientationW(const BodyOrientationWInfo &info) const¶

Get body orientation in world frame (w, x, y, z).

Parameters:: info – Arguments including the articulation name and the body name.
Returns:: The orientation of the body in world frame.

inline virtual std::optional<LinearVelocity> bodyLinearVelocityB(const BodyLinearVelocityBInfo &info) const¶

Get body linear velocity in body frame.

Parameters:: info – Arguments including the articulation name and the body name.
Returns:: The linear velocity of the body in body frame.

inline virtual std::optional<AngularVelocity> bodyAngularVelocityB(const BodyAngularVelocityBInfo &info) const¶

Get body angular velocity in body frame.

Parameters:: info – Arguments including the articulation name and the body name.
Returns:: The angular velocity of the body in body frame.

inline virtual bool initHeightScan(const HeightScanInfo&)¶

Initialize data source of the heightscan.

Called once during initialization (usually non real-time).

Parameters:: info – The sensor info, including the sensor name, grid pattern, and layer names.

inline virtual std::optional<const HeightScan*> heightScan(const HeightScanInfo&, const Position&, const Quaternion&)¶

Get subsampled heightscan in base frame.

This function returns the flattened (column-major), subsampled heightscan with the pattern specified in the init function. The pattern is centered around the base.

Parameters:

info – The sensor info, including the sensor name and layer names.
base_pos_w – The current base position in world frame.
base_quat_w – The current base orientation in world frame.

Returns:

A vector of heightscans.

inline virtual bool initSphericalImage(const SphericalImageInfo&)¶

Initialize the spherical image data source.

Called once during initialization (usually non real-time).

Parameters:: info – The sensor info, including the sensor name, resolution, field of view, and channel names.

inline virtual std::optional<const MultiChannelImage*> sphericalImage(const SphericalImageInfo&)¶

Get the spherical image with multiple channels.

This function returns the multi-channel spherical image data, with each channel flattened in row-major storage order.

Parameters:: info – The sensor info, including the sensor name and the channel names to retrieve.
Returns:: Pointer to the spherical image data.

inline virtual bool initPinholeImage(const PinholeImageInfo&)¶

Initialize the pinhole image data source.

Called once during initialization (usually non real-time).

Parameters:: info – The sensor info, including the sensor name, image dimensions, intrinsics, and channel names.

inline virtual std::optional<const MultiChannelImage*> pinholeImage(const PinholeImageInfo&)¶

Get the pinhole image with multiple channels.

This function returns the multi-channel pinhole image data, with each channel flattened in row-major storage order.

Parameters:: info – The sensor info, including the sensor name and the channel names to retrieve.
Returns:: Pointer to the pinhole image data.

Command Interface¶

class CommandInterface¶

Interface which provides methods to send commands to the controllers.

Public Functions

virtual ~CommandInterface() = default¶

inline virtual bool initSe2Velocity(const Se2VelocityCommandInfo &info)¶

Initialize data source of commanded se2 velocity.

Called once during initialization (usually non real-time).

Parameters:: info – The command info, including the command name and optional ranges.
Returns:: True if initialization succeeded, false otherwise.

inline virtual std::optional<SE2Velocity> se2Velocity(const Se2VelocityCommandInfo &info)¶

Get commanded se2 velocity.

Parameters:: info – The command info, including the command name.
Returns:: The commanded se2 velocity (lin x, lin y, ang z).

inline virtual bool initSe3Pose(const Se3PoseCommandInfo &info)¶

Initialize data source of commanded SE3 pose.

Called once during initialization (usually non real-time).

Parameters:: info – The command info, including the command name.
Returns:: True if initialization succeeded, false otherwise.

inline virtual std::optional<SE3Pose> se3Pose(const Se3PoseCommandInfo &info) const¶

Get commanded SE3 some pose.

Parameters:: info – The command info, including the command name.
Returns:: The commanded SE3 pose.

inline virtual bool initBooleanSelector(const BooleanSelectorCommandInfo &info)¶

Initialize data source of a boolean.

Called once during initialization (usually non real-time).

Parameters:: info – The command info, including the command name.
Returns:: True if initialization succeeded, false otherwise.

inline virtual std::optional<bool> booleanSelector(const BooleanSelectorCommandInfo &info) const¶

Get commanded boolean selector.

Parameters:: info – The command info, including the command name.
Returns:: The commanded bool.

inline virtual bool initFloatValue(const FloatValueCommandInfo &info)¶

Initialize data source of a float.

Called once during initialization (usually non real-time).

Parameters:: info – The command info, including the command name and optional range.
Returns:: True if initialization succeeded, false otherwise.

inline virtual std::optional<float> floatValue(const FloatValueCommandInfo &info) const¶

Get commanded float value.

Parameters:: info – The command info, including the command name.
Returns:: The commanded float.

inline virtual bool initJointPosition(const JointPositionCommandInfo &info)¶

Initialize data source of a commanded joint position.

Called once per joint during initialization (usually non real-time).

Parameters:: info – The command info, including the command name and joint name.
Returns:: True if initialization succeeded, false otherwise.

inline virtual std::optional<float> jointPosition(const JointPositionCommandInfo &info) const¶

Get the commanded position for a single joint.

Parameters:: info – The command info, including the command name and joint name.
Returns:: The commanded joint position, or std::nullopt if unavailable.

Data Collection Interface¶

class DataCollectionInterface¶

Interface which provides methods to collect data.

Public Functions

virtual ~DataCollectionInterface() = default¶

inline virtual bool registerDataSource(const std::string&, std::span<const double>)¶

Parameters:

prefix – A prefix to identify the data.
data – The data to be logged.

inline virtual bool registerDataSource(const std::string&, std::span<const float>)¶

Parameters:

prefix – A prefix to identify the data.
data – The data to be logged.

inline virtual bool registerDataSource(const std::string&, const double&)¶

Parameters:

prefix – A prefix to identify the data.
data – The data to be logged.

virtual bool collectData(uint64_t time_us) = 0¶

Collect registered data.

Parameters:: time_us – Timestamp in microseconds.
Returns:: true if data collection succeeded, false otherwise.

Interface Info Structs¶

Every public CommandInterface and RobotStateInterface method takes a dedicated info struct instead of loose parameters. This keeps call sites and overrides stable as new fields are added over time.

Command Interface¶

struct Se2VelocityCommandInfo¶

Arguments for CommandInterface::initSe2Velocity and CommandInterface::se2Velocity.

Public Members

std::string command_name¶: The name of the command.

std::optional<SE2VelocityRanges> ranges = {}¶: Optional ranges for the commanded se(2) velocity.

struct Se3PoseCommandInfo¶

Arguments for CommandInterface::initSe3Pose and CommandInterface::se3Pose.

Public Members

std::string command_name¶: The name of the command.

struct BooleanSelectorCommandInfo¶

Arguments for CommandInterface::initBooleanSelector and CommandInterface::booleanSelector.

Public Members

std::string command_name¶: The name of the command.

struct FloatValueCommandInfo¶

Arguments for CommandInterface::initFloatValue and CommandInterface::floatValue.

Public Members

std::string command_name¶: The name of the command.

std::optional<Range> range = {}¶: Optional range for the commanded float scalar.

struct JointPositionCommandInfo¶

Arguments for CommandInterface::initJointPosition and CommandInterface::jointPosition.

Public Members

std::string command_name¶: The name of the command.

std::string joint_name¶: The name of the joint.

State Interface¶

struct BasePosWInfo¶

Arguments for RobotStateInterface::initBasePosW and RobotStateInterface::basePosW.

Public Members

std::string articulation_name¶: Articulation or rigid object the base belongs to.

struct BaseQuatWInfo¶

Arguments for RobotStateInterface::initBaseQuatW and RobotStateInterface::baseQuatW.

Public Members

std::string articulation_name¶: Articulation or rigid object the base belongs to.

struct BaseLinVelBInfo¶

Arguments for RobotStateInterface::initBaseLinVelB and RobotStateInterface::baseLinVelB.

Public Members

std::string articulation_name¶: Articulation or rigid object the base belongs to.

struct BaseAngVelBInfo¶

Arguments for RobotStateInterface::initBaseAngVelB and RobotStateInterface::baseAngVelB.

Public Members

std::string articulation_name¶: Articulation or rigid object the base belongs to.

struct JointPositionInfo¶

Arguments for RobotStateInterface::initJointPosition and RobotStateInterface::jointPosition.

Public Members

std::string articulation_name¶: Articulation the joint belongs to.

std::string joint_name¶: Name of the joint.

struct JointVelocityInfo¶

Arguments for RobotStateInterface::initJointVelocity and RobotStateInterface::jointVelocity.

Public Members

std::string articulation_name¶: Articulation the joint belongs to.

std::string joint_name¶: Name of the joint.

struct JointEffortInfo¶

Arguments for RobotStateInterface::initJointEffort and RobotStateInterface::jointEffort.

Public Members

std::string articulation_name¶: Articulation the joint belongs to.

std::string joint_name¶: Name of the joint.

struct JointOutputInfo¶

Arguments for RobotStateInterface::initJointOutput.

Public Members

std::string articulation_name¶: Articulation the joint belongs to.

std::string joint_name¶: Name of the joint.

struct SetJointPositionInfo¶

Arguments for RobotStateInterface::setJointPosition.

Public Members

std::string articulation_name¶: Articulation the joint belongs to.

std::string joint_name¶: Name of the joint.

double position = {}¶: Joint position to be set.

struct SetJointVelocityInfo¶

Arguments for RobotStateInterface::setJointVelocity.

Public Members

std::string articulation_name¶: Articulation the joint belongs to.

std::string joint_name¶: Name of the joint.

double velocity = {}¶: Joint velocity to be set.

struct SetJointEffortInfo¶

Arguments for RobotStateInterface::setJointEffort.

Public Members

std::string articulation_name¶: Articulation the joint belongs to.

std::string joint_name¶: Name of the joint.

double effort = {}¶: Joint effort to be set.

struct SetJointPGainInfo¶

Arguments for RobotStateInterface::setJointPGain.

Public Members

std::string articulation_name¶: Articulation the joint belongs to.

std::string joint_name¶: Name of the joint.

double p_gain = {}¶: Proportional gain to be set.

struct SetJointDGainInfo¶

Arguments for RobotStateInterface::setJointDGain.

Public Members

std::string articulation_name¶: Articulation the joint belongs to.

std::string joint_name¶: Name of the joint.

double d_gain = {}¶: Derivative gain to be set.

struct Se2VelocityInfo¶

Arguments for RobotStateInterface::initSe2Velocity and RobotStateInterface::setSe2Velocity.

Public Members

std::string frame_name¶: Name of the considered frame.

SE2Velocity velocity = {}¶: The se(2) velocity to be set (used by setSe2Velocity).

struct ImuLinearVelocityImuInfo¶

Arguments for RobotStateInterface::initImuLinearVelocityImu and RobotStateInterface::imuLinearVelocityImu.

Public Members

std::string imu_name¶: Name of the IMU.

struct ImuAngularVelocityImuInfo¶

Arguments for RobotStateInterface::initImuAngularVelocityImu and RobotStateInterface::imuAngularVelocityImu.

Public Members

std::string imu_name¶: Name of the IMU.

struct ImuOrientationWInfo¶

Arguments for RobotStateInterface::initImuOrientationW and RobotStateInterface::imuOrientationW.

Public Members

std::string imu_name¶: Name of the IMU.

struct BodyPositionWInfo¶

Arguments for RobotStateInterface::initBodyPositionW and RobotStateInterface::bodyPositionW.

Public Members

std::string articulation_name¶: Articulation or rigid object the body belongs to.

std::string body_name¶: Name of the body.

struct BodyOrientationWInfo¶

Arguments for RobotStateInterface::initBodyOrientationW and RobotStateInterface::bodyOrientationW.

Public Members

std::string articulation_name¶: Articulation or rigid object the body belongs to.

std::string body_name¶: Name of the body.

struct BodyLinearVelocityBInfo¶

Arguments for RobotStateInterface::initBodyLinearVelocityB and RobotStateInterface::bodyLinearVelocityB.

Public Members

std::string articulation_name¶: Articulation or rigid object the body belongs to.

std::string body_name¶: Name of the body.

struct BodyAngularVelocityBInfo¶

Arguments for RobotStateInterface::initBodyAngularVelocityB and RobotStateInterface::bodyAngularVelocityB.

Public Members

std::string articulation_name¶: Articulation or rigid object the body belongs to.

std::string body_name¶: Name of the body.

struct HeightScanInfo¶

Arguments for RobotStateInterface::initHeightScan and RobotStateInterface::heightScan.

Public Members

std::string sensor_name¶: Name of the height scan sensor.

HeightScanPattern pattern = {}¶: Grid pattern specification (used by initHeightScan).

std::unordered_set<std::string> layer_names = {}¶: Layer names to include / retrieve.

struct SphericalImageInfo¶

Arguments for RobotStateInterface::initSphericalImage and RobotStateInterface::sphericalImage.

Public Members

std::string sensor_name¶: Name of the spherical image sensor.

int v_res = {}¶: Number of pixels in vertical direction (used by init).

int h_res = {}¶: Number of pixels in horizontal direction (used by init).

double v_fov_min_deg = {}¶: Minimum vertical field of view angle in degrees (used by init).

double v_fov_max_deg = {}¶: Maximum vertical field of view angle in degrees (used by init).

double unobserved_value = {}¶: Sentinel value for pixels with no sensor return (used by init).

std::unordered_set<std::string> channel_names = {}¶: Channel names to include / retrieve.

struct PinholeImageInfo¶

Arguments for RobotStateInterface::initPinholeImage and RobotStateInterface::pinholeImage.

Public Members

std::string sensor_name¶: Name of the pinhole image sensor.

int width = {}¶: Image width in pixels (used by init).

int height = {}¶: Image height in pixels (used by init).

double fx = {}¶: Focal length in x direction in pixels (used by init).

double fy = {}¶: Focal length in y direction in pixels (used by init).

double cx = {}¶: Principal point x-coordinate in pixels (used by init).

double cy = {}¶: Principal point y-coordinate in pixels (used by init).

std::unordered_set<std::string> channel_names = {}¶: Channel names to include / retrieve.

Components¶

Input and output components for data flow management.

struct Input¶

Abstract base class for input components that read data into ONNX model inputs.

Input components read robot state, sensor data, or commands and populate ONNX runtime input tensors. Subclasses implement specific data sources (joints, IMU, cameras, etc.).

Subclassed by exploy::control::BaseAngularVelocityInput, exploy::control::BaseLinearVelocityInput, exploy::control::BaseOrientationInput, exploy::control::BasePositionInput, exploy::control::BodyAngularVelocityInput, exploy::control::BodyLinearVelocityInput, exploy::control::BodyOrientationInput, exploy::control::BodyPositionInput, exploy::control::CommandBooleanInput, exploy::control::CommandFloatInput, exploy::control::CommandJointPositionInput, exploy::control::CommandSE2VelocityInput, exploy::control::CommandSE3PoseInput, exploy::control::HeightScanInput, exploy::control::IMUAngularVelocityInput, exploy::control::IMULinearVelocityInput, exploy::control::IMUOrientationInput, exploy::control::JointPositionInput, exploy::control::JointVelocityInput, exploy::control::PinholeImageInput, exploy::control::SphericalImageInput, exploy::control::StepCountInput

Public Functions

inline explicit Input(const std::string &name)¶

virtual ~Input() = default¶

inline virtual bool init(RobotStateInterface&, CommandInterface&)¶

Initialize the input component (non-real-time).

Called once during controller setup to register data sources and configure parameters.

Parameters:

state – Robot state interface for reading sensor and state data.
command – Command interface for reading external commands.

Returns:

true if initialization succeeded, false otherwise.

virtual bool read(OnnxRuntime &runtime, RobotStateInterface &state, CommandInterface &command) = 0¶

Read data from robot/command interface into ONNX input buffer (real-time).

Called every control loop to update the ONNX model inputs with current data.

Parameters:

runtime – ONNX runtime containing input/output buffers.
state – Robot state interface for reading sensor and state data.
command – Command interface for reading external commands.

Returns:

true if read succeeded, false if data unavailable.

inline virtual const std::string &getName() const¶

Get the name of this component.

Returns:: Name of this component.

struct Output¶

Abstract base class for output components that write ONNX model outputs.

Output components read ONNX runtime output tensors and write commands to robot control interfaces. Subclasses implement specific output targets (joint targets, velocity commands, memory buffers, etc.).

Subclassed by exploy::control::JointTargetOutput, exploy::control::MemoryOutput, exploy::control::SE2VelocityOutput

Public Functions

inline explicit Output(const std::string &name)¶

virtual ~Output() = default¶

inline virtual bool init(RobotStateInterface&, CommandInterface&)¶

Initialize the output component (non-real-time).

Called once during controller setup to configure output targets.

Parameters:

state – Robot state interface for accessing robot configuration.
command – Command interface for configuring output targets.

Returns:

true if initialization succeeded, false otherwise.

virtual bool write(OnnxRuntime &runtime, RobotStateInterface &state, CommandInterface &command) = 0¶

Write ONNX output buffer to robot/command interface (real-time).

Called every control loop after ONNX inference to apply model outputs.

Parameters:

runtime – ONNX runtime containing input/output buffers.
state – Robot state interface for writing state commands.
command – Command interface for writing control commands.

Returns:

true if write succeeded, false otherwise.

inline virtual const std::string &getName() const¶

Get the name of this component.

Returns:: Name of this component.

Data Types¶

Core data structures and type aliases used throughout the controller.

using exploy::control::Position = Eigen::Vector3d¶

3D position vector.

Represents a position in 3D Cartesian space using Eigen’s Vector3d.

using exploy::control::Quaternion = Eigen::Quaterniond¶

Unit quaternion for 3D orientation.

Represents a rotation in 3D space using Eigen’s quaternion representation. The quaternion should be normalized (unit quaternion).

using exploy::control::LinearVelocity = Eigen::Vector3d¶

3D linear velocity vector.

Represents linear velocity in 3D Cartesian space.

using exploy::control::AngularVelocity = Eigen::Vector3d¶

3D angular velocity vector.

Represents angular velocity in 3D space (rotation rates around x, y, z axes).

using exploy::control::SE2Velocity = Eigen::Vector3d¶

SE(2) velocity vector [v_x, v_y, omega_z].

Represents planar velocity with linear components in x and y directions and angular velocity around the z-axis.

struct SE3Pose¶

SE(3) pose representation.

Represents a 6-DOF pose in 3D space combining position and orientation.

Public Members

Position position¶: Position in 3D space.

Quaternion orientation¶: Orientation as unit quaternion.

struct HeightScan¶

A flattened height scan with multiple layers.

The height scan was flattened from a grid pattern according to IsaacLab conventions. Each layer represents different terrain properties (e.g., height, color, normals). All layers come from the same grid pattern and must have the same length.

Note

As soon as the ONNX wrapper supports multi-dimensional inputs, this should be changed to a 2D array representation for better performance and clarity.

Public Members

std::unordered_map<std::string, std::span<const float>> float_layers¶

Map of layer names to their data spans.

Each entry maps a layer name (e.g., “height”, “color”) to a span of float values representing the flattened grid data for that layer.

Configuration¶

struct OnnxRuntimeOptions¶

Configuration options for initializing the ONNX Runtime.

This struct encapsulates all configuration parameters needed to initialize an ONNX model.

Public Types

enum class ExecutionProvider¶

Values:

enumerator CPU¶

enumerator CUDA¶

Public Members

ExecutionProvider provider = ExecutionProvider::CPU ¶

std::optional<std::string> profiling_path = std::nullopt¶

struct SE2VelocityRanges¶

SE(2) velocity constraints.

Defines the allowable ranges for planar velocity commands including linear velocities in x and y directions and angular velocity around z-axis.

Public Members

Range lin_vel_x¶: Linear velocity range in x direction.

Range lin_vel_y¶: Linear velocity range in y direction.

Range ang_vel_z¶: Angular velocity range around z-axis.

Metadata¶

Metadata structures for component configuration.

struct JointMetadata¶

Metadata for joint configurations.

Specifies the list of joint names for joint-related inputs.

Public Members

std::vector<std::string> names = {}¶: Joint names.

struct JointOutputMetadata¶

Metadata for joint output commands.

Specifies joint names and PD controller gains (stiffness and damping) for position-controlled joints.

Public Members

std::vector<std::string> names = {}¶: Joint names in order.

std::vector<double> stiffness = {}¶: Proportional gains (stiffness) for each joint.

std::vector<double> damping = {}¶: Derivative gains (damping) for each joint.

struct SE2VelocityCommandMetadata¶

Metadata for SE(2) velocity commands.

Specifies optional constraints on planar velocity commands including linear and angular velocity ranges.

Public Members

std::optional<SE2VelocityRanges> ranges = {}¶: Optional velocity range constraints.

struct HeightScanMetadata¶

Metadata for height scan sensors.

Specifies the grid pattern configuration for terrain height scanning including resolution, size, and offset relative to the base frame.

Public Members

std::string pattern_type = {}¶: Grid pattern type (e.g., “grid”, “radial”).

std::string articulation_name = {}¶: Name of the articulation providing the base frame for the scan.

double resolution = {}¶: Grid resolution (spacing between points).

double size_x = {}¶: Grid size in x direction.

double size_y = {}¶: Grid size in y direction.

double offset_x = {}¶: Grid offset in x direction from base.

double offset_y = {}¶: Grid offset in y direction from base.

struct SphericalImageMetadata¶

Metadata for spherical image sensors (e.g., LiDAR).

Specifies the configuration for spherical images including resolution, field of view, and sentinel value for unobserved points.

Public Members

std::string pattern_type = {}¶: Spherical image pattern type.

int v_res = {}¶: Vertical resolution (number of vertical scan lines).

int h_res = {}¶: Horizontal resolution (points per scan line).

double v_fov_min_deg = {}¶: Minimum vertical field of view in degrees.

double v_fov_max_deg = {}¶: Maximum vertical field of view in degrees.

double unobserved_value = {}¶: Sentinel value for unobserved/invalid points.

struct PinholeImageMetadata¶

Metadata for pinhole camera image sensors.

Specifies camera configuration including image dimensions and intrinsic parameters.

Public Members

std::string pattern_type = {}¶: Pinhole image pattern type.

int width = {}¶: Image width in pixels.

int height = {}¶: Image height in pixels.

double fx = {}¶: Focal length in x direction (pixels).

double fy = {}¶: Focal length in y direction (pixels).

double cx = {}¶: Principal point x-coordinate (pixels).

double cy = {}¶: Principal point y-coordinate (pixels).

Workers¶

Execution strategies for the read → inference → write pipeline. Select a strategy via exploy::control::WorkerMode when calling OnnxRLController::init().

enum class exploy::control::WorkerMode¶

Selects the ONNX inference execution strategy.

Pass this enum to OnnxRLController::init() to choose between:

SYNC — inference blocks the calling thread
ASYNC — inference runs on a background thread

Values:

enumerator SYNC¶: Run inference synchronously on the calling thread.

enumerator ASYNC¶: Run inference on a background thread (see AsyncWorker).

class Worker¶

Abstract base class for controller execution strategies.

A Worker owns the read → work → write pipeline. Concrete subclasses decide when and how the three callbacks are invoked:

read_fn — called on the main thread to snapshot robot state.
work_fn — runs the ONNX inference.
write_fn — called on the main thread to dispatch joint targets.

Use setCallbacks() to register all three before calling update().

Subclassed by exploy::control::AsyncWorker, exploy::control::SyncWorker

Public Functions

virtual ~Worker() = default¶

inline virtual void reset()¶: Reset the worker to its initial state.

virtual bool update(uint64_t time_us) = 0¶

Advance the control pipeline by one tick.

Parameters:: time_us – Current timestamp in microseconds.
Returns:: true if the update succeeded or was safely skipped, false on error.

bool setCallbacks(std::function<bool()> read_fn, std::function<bool()> work_fn, std::function<bool()> write_fn)¶

Must be called exactly once before the first update(). All three arguments must be non-null; the function returns false otherwise.

Parameters:

read_fn – Reads observations from the robot state (main thread).
work_fn – Runs ONNX inference (may execute on a background thread).
write_fn – Writes joint targets back to the robot (main thread).

Returns:

true on success.

class SyncWorker : public exploy::control::Worker ¶

Synchronous worker — runs read → work → write on the calling thread.

All three callbacks execute inline inside update(). The call blocks until inference is complete, so the caller’s thread must afford the full inference latency every control cycle.

Phase is maintained across updates: the first call initialises the phase reference and subsequent calls fire whenever the elapsed time exceeds the configured period.

Public Functions

explicit SyncWorker(double update_rate_hz)¶

Parameters:: update_rate_hz – Desired control frequency in Hz.

virtual bool update(uint64_t time_us) override¶

Advance the control pipeline by one tick.

Parameters:: time_us – Current timestamp in microseconds.
Returns:: true if the update succeeded or was safely skipped, false on error.

virtual void reset() override¶: Reset the worker to its initial state.

class AsyncWorker : public exploy::control::Worker ¶

Asynchronous worker — offloads ONNX inference to a background thread.

The pipeline is split across two consecutive update() calls:

First call at cycle boundary — read_fn executes on the main thread, then work_fn is dispatched to a dedicated background thread.
Subsequent calls — if inference has finished, write_fn runs on the main thread to commit joint targets. If the worker is still busy (overrun), the cycle is skipped and update() returns true.

This decouples the main control loop from the inference latency, allowing the robot to keep receiving state updates while the GPU or CPU is busy.

Thread safety: the internal mutex guards all shared state between the main thread and the worker thread.

reset() stops the background thread and clears all state. The thread is re-started on the next update() call.

Error handling: when work_fn returns false the worker latches into a faulted state. All subsequent update() calls return false immediately without dispatching new work. Call reset() to clear the fault and resume.

Public Functions

explicit AsyncWorker(double update_rate_hz)¶

Parameters:: update_rate_hz – Desired control frequency in Hz.

~AsyncWorker() override¶

virtual void reset() override¶: Reset the worker to its initial state.

virtual bool update(uint64_t time_us) override¶

Advance the control pipeline by one tick.

Parameters:: time_us – Current timestamp in microseconds.
Returns:: true if the update succeeded or was safely skipped, false on error.

Matchers¶

Component matchers for automatic ONNX I/O mapping.

class Matcher¶

Abstract base class for single-tensor pattern matching.

Matcher implementations recognize specific ONNX tensor name patterns and create corresponding input/output components. Each matcher handles one type of component.

Subclassed by exploy::control::BaseMatcher, exploy::control::BodyMatcher, exploy::control::CommandBooleanMatcher, exploy::control::CommandFloatMatcher, exploy::control::CommandJointPositionMatcher, exploy::control::CommandSE2VelocityMatcher, exploy::control::CommandSE3PoseMatcher, exploy::control::IMUAngularVelocityMatcher, exploy::control::IMULinearVelocityMatcher, exploy::control::IMUOrientationMatcher, exploy::control::SE2VelocityMatcher, exploy::control::StepCountMatcher

Public Functions

inline explicit Matcher(const std::string &name)¶

virtual ~Matcher() = default¶

virtual bool matches(const Match &maybe_match) = 0¶

Check if a tensor matches this matcher’s pattern.

Parameters:: maybe_match – Potential match containing tensor name and metadata.
Returns:: true if the tensor matches and was stored, false otherwise.

inline virtual std::vector<std::unique_ptr<Input>> createInputs() const¶

Create input components from all matched tensors.

Returns:: Vector of input component unique pointers.

inline virtual std::vector<std::unique_ptr<Output>> createOutputs() const¶

Create output components from all matched tensors.

Returns:: Vector of output component unique pointers.

inline void resetMatcher()¶

Reset the matcher by clearing all found matches and any subclass state.

Non-virtual: clears the base found_matches_ storage and then dispatches to the virtual reset() hook.

inline const std::string &getName() const¶

Get the name of this matcher.

Returns:: Name of the matcher.

struct Match¶

Represents a single matched ONNX tensor.

Contains the tensor name and optional metadata associated with the tensor from the ONNX model.

Public Members

std::string name = {}¶: ONNX tensor name.

std::optional<std::string> metadata = {}¶: Optional JSON metadata string.

std::unordered_map<std::string, std::string> base_names¶: Map of base names.