Node Pools use the following API schema definitions:Documentation Index
Fetch the complete documentation index at: https://docs.coreweave.com/llms.txt
Use this file to discover all available pages before exploring further.
Node Pool
Node Pool is the schema for the Node Pools API.| Field | Type | Description | Default |
|---|---|---|---|
apiversion | string | The version of the Kubernetes API that the Node Pool uses. | compute.coreweave.com/v1alpha1 |
kind | string | The type of resource being defined, such as a Node Pool | Node PoolList |
metadata | ListMeta | See the Kubernetes API documentation for metadata | |
spec | Node PoolSpec | The desired state of the Node Pool. | |
status | Node PoolStatus | The observed state of the Node Pool. See Node PoolStatus. |
Node Pool spec
Node PoolSpec defines the desired state of Node Pool.| Field | Type | Description | Default | Validation |
|---|---|---|---|---|
computeClass | enum (default) | The type of Node Pool. default is used for Reserved instances and On-Demand instances and is the default | default | Optional |
instanceType | string | Instance Type for the Nodes in this Node Pool (Note: instanceType is immutable and unchangeable) | N/A | Required |
targetNodes | integer | The quantity of desired Nodes in the Node Pool | N/A | Required |
minNodes | integer | The minimum number of TargetNodes allowed by the autoscaler | N/A | Optional |
maxNodes | integer | The maximum number of TargetNodes allowed by the autoscaler | N/A | Optional |
nodeLabels | object (keys:string, values:string) | List of labels associated with the Nodes in the Node Pool | N/A | Optional |
nodeAnnotations | object (keys:string, values:string) | List of annotations associated with the Nodes in the Node Pool | N/A | Optional |
nodeTaints | Taint Array | List of taints associated with the Nodes in the Node Pool | N/A | Optional |
image | Image | The image to use for the Nodes in the Node Pool. (Note: image and gpu are mutually exclusive; if both are set, gpu will be ignored) For updates to take effect, perform a reconfigure reboot. | N/A | Optional |
gpu | Gpu | GPU driver configuration for the Nodes in the Node Pool. (Note: gpu and image are mutually exclusive; if both are set, gpu will be ignored) For updates to take effect, perform a reconfigure reboot. | N/A | Optional |
autoscaling | boolean | Enable or disable cluster autoscaling | false | Optional |
lifecycle.scaleDownStrategy | enum (IdleOnly, PreferIdle) | Options for removing Nodes when scaling down to reach targetNodes. IdleOnly removes only idle Nodes. PreferIdle removes idle Nodes first and then active Nodes if needed. | IdleOnly | Optional |
nodeConfigurationUpdateStrategy.type | enum (Manual, OnSpecUpdate, Always) | Defines how configuration updates get staged onto the Node Pool. See Node configuration update strategies for more information. | OnSpecUpdate | Optional |
Node configuration update strategies
| Strategy | Behavior |
|---|---|
| Manual | Updates require user intervention to stage onto NodePool. Updates can be inspected in the pendingNodeConfigurations and applied with cwic. |
| OnSpecUpdate | Updates are staged automatically only if triggered by a direct change to the NodePool spec (for example, modifying the GPU driver version). |
| Always | Updates are staged automatically for both user-initiated spec changes and any available upstream updates. |
Image
Image defines what boot image theNode PoolSpec uses.
| Field | Type | Description | Default | Validation |
|---|---|---|---|---|
kernel | string | Kernel version for this Node Pool | Optional | |
name | string | Name of the image for this Node Pool. If name is set, kernel must be empty and releaseTrain must be stable. | Optional | |
releaseTrain | string | The release channel or track for the image | stable | Optional Enum: [stable latest] |
NVSHMEM + GDRCopy supportIf you need an image with NVSHMEM and GDRCopy support, you can request to use ncore-image v2.10.1 by contacting support. You’ll need to apply a patch to
ibgda in your container, enable GDRCopy, then contact support to get access to this new image.Gpu
Thegpu field defines the GPU driver configuration for Nodes in the Node PoolSpec.
| Field | Type | Description | Default | Validation |
|---|---|---|---|---|
version | string | The specific GPU driver version to use. Note that only the major version is specified. | Optional |
Node PoolStatus
Node Pool Status is the observed state of aNode Pool.
| Field | Type | Description | Validation |
|---|---|---|---|
queuedNodes | integer | Number of queued Nodes waiting to be assigned. | Optional Minimum |
inProgress | integer | Number of Nodes that have been assigned, but have not yet fully booted into the cluster | Optional Minimum |
currentNodes | integer | Number of Nodes for this Node Pool present in the cluster | Optional Minimum |
nodeProfile | string | NodeProfile is the unique identifier for the active Node configuration | Optional |
pendingNodeConfiguration | object (createdAt: timestamp, nodeProfile: string, summary: string array) | Contains information about the staged Node configuration. This field is only present if NodePool spec changes were made that require a new Config, or if there are updates available such as a new ncore image. | Optional |
nodeConfigurationRevisions | nodeConfigurationRevision Array | Maintains a list of Node configurations that were applied to the NodePool, along with data about each one. | Optional |
conditions | Condition Array | All conditions associated with the Node Pool | Optional |
Pending Node Configuration
ThependingNodeConfiguration contains information about the configuration that is pending adoption on the NodePool. This configuration will not get set as the active configuration without an explicit upgrade from the user.
This field is only present if there are configuration updates available for your NodePool and your nodeConfigurationUpdateStrategy.type is set to Manual or OnSpecUpdate.
For example, if a new GPU driver or Kubernetes version is available, CKS will create a new configuration and set it as pending on the Node Pool. The summary field provides more info about the available updates. See Manage Node Pool Configuration for a guide on how to promote a pending configuration.
| Field | Type | Description |
|---|---|---|
createdAt | timestamp | When the Node configuration was created |
nodeProfile | string | The unique identifier for the configuration |
summary | string array | A summary of the features for the configuration, including things like ncore version, GPU driver version, K8s version, etc. |
Node Configuration Revisions
The revisions list holds history of the Node configurations that were applied to the NodePool, i.e. configurations that were at one point thestatus.nodeProfile.
The list is sorted by creation timestamp, and can be used as a reference to rollback to a previous Node configuration if desired. See Manage Node Pool Configuration for a guide on how to roll back to a previous configuration.
| Field | Type | Description |
|---|---|---|
activeNodes | integer | The count of Nodes in the NodePool that are currently booted into this configuration |
createdAt | timestamp | The creation timestamp for the configuration |
nodeProfile | string | The unique identifier for the configuration |
disabled | boolean | Indicates if the configuration is blocked from being applied to more Nodes. A configuration gets disabled if it experiences successive boot failures. Please contact support if this is true |
summary | string array | A summary of the features for the configuration, including things like ncore version, GPU driver version, K8s version, etc. |
Conditions
Node Conditions
CoreWeave sets the following conditions on Nodes in a Node Pool.| Condition Name | Type | Description |
|---|---|---|
CWActive | bool | true if the Node is active. If false, and the Node Pool is being scaled down, the Node may be selected for removal. |
CWRegistered | bool | true if the Node is registered. If false, the Node is not registered. Nodes are registered when they enter a customer cluster as part of the Node lifecycle. |
CWNodeRemoval | bool | true if the Node is pending removal. |
Node Pool Conditions
CoreWeave sets the following conditions on a Node Pool after the Node Pool resource has been created.Condition: Validated
This condition answers the question: “Is the Node Pool configuration valid?” It has three possible statuses:
| Status | Description |
|---|---|
Valid | The Node Pool configuration is correct. |
Invalid | The Node Pool configuration has errors, such as an unsupported instance type or incorrect Node affinity. |
InternalError | A system error occurred during validation, so the Node Pool couldn’t be fully checked. |
Condition: AtTarget
This condition shows whether the Node Pool has the expected number of active Nodes. The AtTarget condition has five possible values:
| Status | Description |
|---|---|
| TargetMet | The Node Pool has exactly the number of Nodes specified in the target. |
| PendingDelete | The Node Pool is being deleted, and its Nodes will be removed. |
| OverTarget | More Nodes exist than the target. Extra Nodes will be removed using the ScaleDownStrategy. |
| UnderTarget | Fewer Nodes exist than the target. New Nodes will be created if resources are available. |
| InternalError | A system error occurred while retrieving Node information for the Node Pool. |
Condition: NodesRemoved
The condition NodesRemoved is applied to a Node Pool when it is pending deletion and Nodes are in the process of being removed. Once all Nodes are removed, the Node Pool will be deleted. This response indicates one of the following conditions:
| Status | Meaning |
|---|---|
Complete | All Nodes have been removed from the Node Pool, and the Node Pool’s deletion is imminent. |
Pending | Nodes are in the process of being removed from the Node Pool. |
InternalError | An internal error has occurred while trying to remove Nodes from the Node Pool. |
Condition: Capacity
The Capacity condition indicates whether there is enough capacity available for the requested number of Nodes in the requested instance type. This response indicates one of the following conditions:
| Status | Meaning |
|---|---|
Sufficient | All Nodes have been removed from the Node Pool, and the pool’s deletion is imminent. |
Partial | Partial capacity exists in the designated Region to fulfill the request, but not to completely fulfill it. |
NoneAvailable | No Nodes are available of the requested type in the given Region. |
NoneAvailableNodeAffinity | No Nodes are available for the requested instance type due to Affinity constraints. |
PartialNodeAffinity | Partial capacity is available to fulfill the requested targetNodes, full capacity is not available due to Affinity constraints. |
QueuedAwaitingCapacity | Request for additional Nodes has been queued and is awaiting for additional capacity to become available. |
InternalError | An internal error has occurred while attempting to check capacity. |
Condition: Quota
Quota has four statuses.
| Status | Meaning |
|---|---|
Under | The organization is under quota for the Node Pool’s instance type. |
Over | The organization is over quota for the Node Pool’s instance type. |
NotSet | The organization does not have a quota set for the Node Pool’s instance type. |
InternalError | An internal error has occurred attempting to check the quota. |
Condition: NodeReconfigurationRequired
This condition answers the question: “Do any of my Nodes need to be reconfigured to boot into the active config (status.nodeProfile)?” It has three possible statuses:
| Status | Description |
|---|---|
StagedNodeConfig | The Node Pool has Nodes that need to be reconfigured to boot into the active config. |
AllNodesUpToDate | All of the Nodes in the Node Pool are booted into the active config. |
InternalError | A system error occurred fetching the active configuration for Nodes. It’s also possible there was a failure staging the Node Pool’s active Node configuration onto outdated Nodes. See the condition message for more information. |
Healthy Node Pools
A new Node Pool in a healthy state looks like this when described:new node pool, showing new conditions with 'describe'
Example output
Events
| Event Name | Resource | Description |
|---|---|---|
CWDrainNode | NodePool | Fired when a Node is being drained. |
CWInstanceTypeNotInZone | NodePool | Fired when a Node Pool has an instance type not in its Zone. |
CWInsufficientCapacity | NodePool | Fired when there is not sufficient capacity for a Node Pool. |
CWInvalidInstanceType | NodePool | Fired when a Node Pool contains an invalid instance type. |
CWNodeAssigned | NodePool | Fired when a Node is assigned to a Node Pool. |
CWNodeDeleted | NodePool | Fired to indicate a Node has been deleted. |
CWNodeDeletionRequestSuccess | NodePool | Fired when Node Pool Operator receives a request to delete a Node. |
CWNodeDeliverFail | NodePool | Fired when Node allocation to a Node Pool fails due to misconfigured Node Pool settings or internal issues. |
CWNodePoolCreated | NodePool | Fired when a Node Pool is created. |
CWNodePoolDeleted | NodePool | Fired when a Node Pool is deleted. |
CWNodePoolDisabled | NodePool | Fired when a Node Pool is disabled because the Node is misconfigured and is causing too many boot failures. Contact Support to diagnose and resolve. |
CWNodePoolMetadata | NodePool | Fired when metadata is updated for a Node Pool. |
CWNodePoolNodesRemoved | NodePool | Fired when Nodes are removed from the Node Pool. |
CWNodePoolNodesRemoveError | NodePool | Fired when an error occurs during Node removal. |
CWNodePoolNodesRequestFailed | NodePool | Fired when an error is returned when updating the Node Pool. |
CWNodePoolQuotaCheckFailed | NodePool | Fired when there is an internal error checking the quota. |
CWNodePoolRemoveNodes | NodePool | Fired when attempting to scale down a Node Pool. |
CWNodePoolScaledDown | NodePool | Fired when a Node Pool is scaled down. |
CWNodePoolScaledUp | NodePool | Fired when a Node Pool is scaled up. |
CWNodeRegistered | Node, NodePool | Fired when Node registration succeeds. |
CWNodeRegistrationFailed | Node | Fired when Node registration fails. See the message for additional details. |
CWNodeRequestQueued | NodePool | Fired when a request is submitted to add Nodes to a Node Pool. |
CWOverQuota | NodePool | Fired when the quota is insufficient for the Node Pool’s targetNodes. |
CWNodeCordoned | Node, NodePool | Fired when a Node is being cordoned as part of internal automation. |
CWNodeUncordoned | Node, NodePool | Fired when a Node is being uncordoned as part of internal automation. |
CWNodeMarkedPrepareForTerminate | Node, NodePool | Fired when a Node has been sent the signal to prepare for termination. See event message for details. |
CWNodeDraining | Node, NodePool | Fired when a Node is being drained as part of internal automation. |
CWNodeDrainingPDBViolation | Node, NodePool | Fired when a Node is being drained and certain pods cannot be evicted due to a Pod Disruption Budget. |
CWNodeDrainingForceDelete | Node, NodePool | Fired when a Node is being drained as part of internal automation and pods were deleted ungracefully. |
CWNodePreparedForTerminate | Node, NodePool | Fired when a Node is prepared for termination. |
CWNodeTainted | Node, NodePool | Fired when a Node is tainted as part of internal automation. |
CWNodeUntainted | Node, NodePool | Fired when a Node is untainted as part of internal automation. |
CWNodeConfigStaged | NodePool | Fired when the active Node Config (status.nodeProfile) for the NodePool has been updated. This might be a result of staging the pendingNodeConfiguration, new User-Initiated changes getting staged or rolling back to a previous configuration. |
CWPendingNodeConfig | NodePool | Fired when the status.pendingNodeConfiguration is updated on the NodePool. |
CWConfigStaged | Node | Fired when a new Node configuration is staged on the Node by setting it as the desired NodeProfile during Node boot. |
CWReconfigureRebooted | Node | Fired after the Node has been reconfigure rebooted and is determined to be up-to-date with the staged NodeProfile of the NodePool. |
CWNodePoolNodeConfigUpdatePending | NodePool | Fired when a Node Config update is pending due to nodes being mid-delivery. The update will complete once Nodes have booted into the cluster. |
CWNodePoolNodeConfigUpdated | NodePool | Fired when the active Node Config (status.nodeProfile) for the NodePool has been updated. This might be a result of promoting the pendingNodeConfiguration or rolling back to a previous configuration. |
CWNodePoolNoPendingConfiguration | NodePool | Fired when a user attempts to promote the status.pendingNodeConfiguration to become the active configuration (status.nodeProfile), but there is no staged configuration. |
CWNodePoolNodeConfigRollbackFailed | NodePool | Fired when there was an error rolling back the active Node configuration for the NodePool to a previous config. |
CWNodePoolNodeConfigUpdateConflict | NodePool | Fired when a user simulteaneouly attempts to upgrade to the pendingNodeConfig and rollback to a previous one. |
CWNodeConfigStagingFailed | Node | Fired when the Node Pool Operator attempted to stage the NodePool’s active configuration on an existing Node in the cluster to prepare it for a reconfigure reboot, but was not able to do so. It will keep retrying until successful. |
CWManualUpgradeTrigger | NodePool | Fired when a user upgrades a NodePool to the pending configuration. |
CWManualRollbackTrigger | NodePool | Fired when a user rolls back a NodePool to a previous configuration. |