Skip to content

About

Prometheus exporter for InfiniBand optical module metrics (temperature, voltage, Tx/Rx power, bias, BER, link counters) via mlxreg/mlxlink, designed for minimal subnet-manager load

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

SuperLinear Infiniband Optical Module Exporter

CI Release Go

This is a prometheus exporter to expose metrics related to Infiniband optical modules.

This project is powered by SuperLinear Team.

Development Requirements

Above golang 1.24

Usage

slibom_exporter [--config|-c CONFIG_FILE] [--verbose|-v]

The format of CONFIG_FILE is like:

# Backend to fetch optical module info: `mlxlink` or `mlxreg`.
#   mlxlink - one heavy mlxlink dump per port (legacy)
#   mlxreg  - read only the needed access registers (recommended; ~15x fewer
#             in-band MADs per port -> far less subnet-manager / VL15 pressure)
backend: mlxreg

# mlxlink backend: binary path + argument template
bin: mlxlink
args: -d {{ .LID }},mlx5_0 -p {{ .PortNumber }} -m -c --json

# mlxreg backend options (bin / device template / module_index) — see
# config.example.yaml for the full, documented reference of every option.

# Inventory of optical modules
optical_modules:

  # Support `csv` and `sqlite3` (default: sqlite3)
  backend: sqlite3

  # For `csv` backend, it is the path to the csv file;
  # for `sqlite3` backend, it is the path to the sqlite database file.
  # (default: sqlite.db)
  path: sqlite.db

  # Only for `sqlite3` backend.
  table: OPTICAL_MODULES

#========================================
# Other common prometheus exporter config
#========================================

# The IP and port to listen (default: 0.0.0.0:9190)
listen: 0.0.0.0:9190

# The prefix of the path of the web URI to get the metrics (default: /metrics)
path_prefix: /metrics

# The prefix of the metrics exposed by the exporter
metrics_prefix: slibom

# Interval (unit: seconds) to fetch the new optical module information (default: 60)
interval: 120

How It Works

Every interval, the exporter will:

  1. Read the optical module inventory (CSV or sqlite3) with columns hostname, NodeGUID, DeviceName, LID, PortNumber. (For the mlxreg backend, PortNumber must be a plain integer, e.g. 30, not 30/1.)

    For each row, query the module in parallel (up to max_threads): the mlxlink backend runs the templated command; the mlxreg backend reads the needed access registers (MCIA / PPCNT) directly. Results go into a cached in-memory staging map.

  2. After all the rows in the inventory are proceed, move the pointer of the optical module info map to the staging map in a concurrency-safe way.

When a new request comes, the exporter returns metrics using the current cached optical module info map.

Appendix

There is an example of the mlxlink command output:

{
  "result": {
    "output": {
      "Module Info": {
        "Active Set Host Compliance Code": "IB NDR",
        "Active Set Media Compliance Code": "400GBASE-DR4",
        "Attenuation (5g,7g,12g,25g)[dB]": "N/A",
        "Bias Current [mA]": "83.642,69.908,76.354,72.912 [1..130]",
        "CDR RX": {
          "values": [
            "ON",
            "ON",
            "ON",
            "ON"
          ]
        },
        "CDR TX": {
          "values": [
            "ON",
            "ON",
            "ON",
            "ON"
          ]
        },
        "Cable Breakout": "Unspecified",
        "Cable Rx AMP": "1",
        "Cable Rx Emphasis (Pre)": "0",
        "Cable Rx Post Emphasis": "0",
        "Cable Technology": "1310 nm EML",
        "Cable Tx Equalization": "0",
        "Cable Type": "Optical Module (separated)",
        "Compliance": "IB NDR,400GBASE-DR4",
        "DataPath FW Fault": "0",
        "DataPath state [per lane]": {
          "values": [
            "DPActivated",
            "DPActivated",
            "DPActivated",
            "DPActivated"
          ]
        },
        "Digital Diagnostic Monitoring": "Yes",
        "Error Code Response": "ConfigUndefined",
        "FW Version": "0.16.1",
        "IB Cable Width": "1x,2x,4x",
        "Identifier": "OSFP",
        "LOS Alarm": "N/A",
        "Linear Direct Drive": "0",
        "MAX Power": "36",
        "Manufacturing Date": "09_06_25",
        "Memory Map Revision": "80",
        "Module FW Fault": "0",
        "Module State": "Ready state",
        "Nominal bit rate": "N/A",
        "OUI": "Other",
        "Power Class": "9.0 W max",
        "Rev": "01",
        "Rx CDR LOL [per lane]": {
          "values": [
            "0",
            "0",
            "0",
            "0"
          ]
        },
        "Rx LOS [per lane]": {
          "values": [
            "0",
            "0",
            "0",
            "0"
          ]
        },
        "Rx Output Valid [per lane]": {
          "values": [
            "0",
            "0",
            "0",
            "0"
          ]
        },
        "Rx Power Current [dBm]": "2.765,2.751,3.450,3.081 [-12.007..6.999]",
        "Rx Power Type": "Average power",
        "SMF Length": "500m",
        "SNR Host Lanes [dB]": {
          "values": [
            "N/A"
          ]
        },
        "SNR Media Lanes [dB]": {
          "values": [
            "N/A"
          ]
        },
        "Temperature [C]": "31 [-10..80]",
        "Transfer Distance [m]": "0.0",
        "Tx Adaptive EQ Fault [per lane]": {
          "values": [
            "0",
            "0",
            "0",
            "0"
          ]
        },
        "Tx CDR LOL [per lane]": {
          "values": [
            "0",
            "0",
            "0",
            "0"
          ]
        },
        "Tx Fault [per lane]": {
          "values": [
            "0",
            "0",
            "0",
            "0"
          ]
        },
        "Tx LOS [per lane]": {
          "values": [
            "0",
            "0",
            "0",
            "0"
          ]
        },
        "Tx Power Current [dBm]": "2.445,2.448,2.448,2.433 [-5.901..6.999]",
        "Vendor Name": "OEM",
        "Vendor Part Number": "OSFP-400G-DR4",
        "Vendor Serial Number": "P7A2010767",
        "Voltage [mV]": "3269.1 [3000..3600]",
        "Wavelength Tolerance": "6.5nm",
        "Wavelength [nm]": "1311"
      },
      "Operational Info": {
        "Auto Negotiation": "ON",
        "FEC": "Ethernet_Consortium_LL_50G_RS_FEC_PLR -(272,257+1)",
        "Loopback Mode": "No Loopback",
        "Physical state": "N/A",
        "Speed": "IB-NDR",
        "State": "Active",
        "Width": "4x"
      },
      "Physical Counters and BER Info": {
        "Effective Physical BER": "15E-255",
        "Effective Physical Errors": "0",
        "Link Down Counter": "0",
        "Link Error Recovery Counter": "0",
        "Raw Physical BER": "2E-8",
        "Raw Physical Errors Per Lane": {
          "values": [
            "5306170217",
            "119078402",
            "3662997258",
            "2001076904"
          ]
        },
        "Symbol BER": "15E-255",
        "Symbol Errors": "0",
        "Time Since Last Clear [Min]": "20724.4"
      },
      "Supported Info": {
        "Enabled Link Speed": {
          "mask": "0x00000080",
          "values": [
            "NDR"
          ]
        },
        "Supported Cable Speed": {
          "mask": "0x00000080",
          "values": [
            "NDR"
          ]
        }
      },
      "Tool Information": {
        "Firmware Version": "28.43.2566",
        "MFT Version": "mft 4.30.1-1210",
        "amBER Version": "3.6"
      },
      "Troubleshooting Info": {
        "Group Opcode": "N/A",
        "Recommendation": "No issue was observed",
        "Status Opcode": "0"
      }
    }
  },
  "status": {
    "code": 0,
    "message": "success"
  }
}

About

Prometheus exporter for InfiniBand optical module metrics (temperature, voltage, Tx/Rx power, bias, BER, link counters) via mlxreg/mlxlink, designed for minimal subnet-manager load

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages