InfiniBand Labels

CKS applies several topology-related and performance-related labels to instances equipped with InfiniBand, which are useful to control the placement of workloads and monitor the network's performance. The following labels are available:

Label	Description
`ib.coreweave.cloud/fabric`	The name of the InfiniBand fabric, representing a distinct network fabric.
`ib.coreweave.cloud/leafgroup`	The number identifying the leaf group within the InfiniBand topology.
`ib.coreweave.cloud/leaves.current`	The IDs of the current leaf switches connected to the Node.¹
`ib.coreweave.cloud/leaves.expected`	The IDs of the expected leaf switches that should be connected to the Node.¹
`ib.coreweave.cloud/neighbors.expected.{interface}.device`	The ID of the expected leaf connected to the Node at this physical interface.
`ib.coreweave.cloud/neighbors.expected.{interface}.port-id`	The ID of the expected port connected to the Node at this physical interface.
`ib.coreweave.cloud/neighbors.current.{interface}.device`	The ID of the current leaf connected to the Node at this physical interface.
`ib.coreweave.cloud/neighbors.current.{interface}.port-id`	The ID of the current port connected to the Node at this physical interface.
`ib.coreweave.cloud/ports.current`	The IDs of the current ports used by the Node.¹
`ib.coreweave.cloud/ports.expected`	The IDs of the expected ports that should be used by the Node.¹
`ib.coreweave.cloud/speed`	The overall speed of the InfiniBand network.
`ib.coreweave.cloud/speed.current`	The current speed of the InfiniBand network connected to the Node.
`ib.coreweave.cloud/speed.expected`	The expected speed of the InfiniBand network for optimal performance.
`ib.coreweave.cloud/superpod`	The number identifying the superpod within the InfiniBand topology.
`node.coreweave.cloud/rack`	The number identifying the physical rack where the Node is located.

¹ If the information in these labels exceeds the maximum allowed length, their values are replaced with an empty string and the /neighbors.current and /neighbors.expected labels are authoritative. Otherwise, they contain the same information as the per-interface labels, aggregated into a single label.

Setting Pod affinity

When scheduling Pods, the fabric, superpod, and rack labels may be used to set a Pod's affinity toward those Nodes, which is useful for some workloads and Node Pool configurations.

Learn more

GB200 and GB300 NVL72-powered instances may also use the ds.coreweave.com/nvlink.domain label to set a Pod's affinity toward one or more NVLink domains. See Control placement with NVLink domain for more information.

For example, this Pod affinity rule targets a specific InfiniBand fabric, superpod, and rack:

Example

affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
      - matchExpressions:
        - key: ib.coreweave.cloud/fabric
          operator: In
          values:
          - <FABRIC_NAME>
Your fabric name
        - key: ib.coreweave.cloud/superpod
          operator: In
          values:
          - <SUPERPOD_NUMBER>
Your superpod number
        - key: node.coreweave.cloud/rack
          operator: In
          values:
          - <RACK_NUMBER>
Your rack number

Setting Pod affinity​

Setting Pod affinity