Halloween DealsAmazon USHalloween week deals: useful picks before month-endAmazon US: current deals, useful picks and tech finds.Check DealsPC HealthRecommendedCrashes, freezes, slowdowns? Check your PC nowSpot repairable issues before they interrupt work.Check PCHalloween DealsAmazon USOctober picks are moving - compare before checkoutAmazon US: current deals, useful picks and tech finds.See Picks×
FREEDOM251

Freedom251 › Blog › AI infrastructure

Run:ai architecture explained: schedulers, node components, and GPU sharing

By Freedom251 Team · · 10 min read

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Run:ai is an AI-aware control and scheduling layer for Kubernetes. Kubernetes still creates, starts, stops, and monitors the pods; Run:ai adds the policies that decide which project receives GPU capacity, how queues share it, when workloads may be preempted, and how one GPU can be divided among several pods.

Its architecture has three cooperating areas: user interfaces, a centralized control plane, and components installed in each Kubernetes cluster. A request becomes recorded desired state, is synchronized into Kubernetes resources, passes through admission and scheduling, and is finally executed by ordinary Kubernetes machinery. GPU fractioning adds memory isolation and time-sliced compute sharing when several workloads use one device.

As an Amazon Associate I earn from qualifying purchases.

What Run:ai adds to Kubernetes

Kubernetes provides general-purpose orchestration: it accepts pod specifications, chooses nodes through its scheduler, launches containers through the kubelet and container runtime, and reports status. Those primitives do not natively understand an AI organization’s projects, departments, GPU quotas, fair-share history, gang requirements, or GPU-memory fractions.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Run:ai extends Kubernetes with those AI-specific functions while leaving Kubernetes as the underlying execution platform. The distinction matters when troubleshooting: a Run:ai decision can determine the queue, priority, resource shape, or target node, but Kubernetes remains responsible for binding and running the resulting pods.

#1 Best Overall
Sale
TOZO A1 Wireless Earbuds Bluetooth 5.3 Light Weight in Ear Headphones | IPX5 Waterproof,2 Mics for AI Calls,Immersive Premium Sound Bass Headset with Charging Case,32 EQ via App Black
  • [Ultra-Lightweight Ear Buds Designed for Small Ears] Each earbud weighs only 3.7g and features a compact, ergonomic in-ear design made especially for small ears. Secure, low-profile, and comfortable for workouts, all-day wear or overnight sleeping.The ultra-slim profile sits flush in your ear with no hard edges poking or rubbing, so you can even sleep on your side without discomfort.
  • [Immersive Stereo Sound with TOZO OrigX Technology] TOZO OrigX tuning delivers clear vocals, balanced mids, and natural stereo sound for music, podcasts, and videos.
  • [Long Battery Life for Daily Use] Get up to 7 hours of playtime on a single charge, with up to 32 hours total using the charging case—ideal for workdays, commuting, and extended listening sessions.
  • [Bluetooth 5.3 & Stable Connection] Bluetooth 5.3 provides fast pairing, stable wireless performance, and reduced dropouts as you move around home or office.
  • [Deep Bass with Clear Vocals] High-performance drivers produce punchy bass while keeping vocals clean and detailed for everyday listening.

The platform is Kubernetes-native rather than a separate cluster manager. Its resources are represented through Kubernetes APIs and custom resources, and an operator manages the components that reconcile those resources. This lets administrators apply Kubernetes security, namespaces, upgrades, and observability practices while adding GPU-specific policy.

Run:ai’s end-to-end architecture

A useful mental model is a control loop that carries intent from a user to a running pod and then carries status back to the user.

  1. Request: A user or automation submits through the web interface, command-line interface, REST API, or MCP server.
  2. Authentication and state: The control plane authenticates the request and stores organizational information, workload intent, and policy state.
  3. Synchronization: Cluster synchronization reconciles that desired state into resources in the selected Kubernetes cluster.
  4. Admission: The admission-controller adds Run:ai scheduling metadata and adjusts resource requirements before the pod is admitted.
  5. Placement: The Run:ai Scheduler evaluates queues, quotas, priorities, topology, and available capacity, then selects a placement or leaves the workload pending.
  6. Execution: Kubernetes binds the pod to a node and runs its containers. Run:ai controllers continue to enforce lifecycle and policy rules.
  7. Status: Cluster components report phase, placement, resource, and policy status back through synchronization to the control plane and user interfaces.

This separation allows one control plane to govern multiple clusters while each cluster retains local components that can react to Kubernetes events and node conditions.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

User interfaces and APIs

The web UI is intended for interactive administration and workload inspection. The CLI and REST API support scripts and platform integrations, while the MCP server provides another programmatic entry point. These interfaces do not directly start containers; they submit intent to the control plane.

Central control plane

The control plane is the system of record for users, projects, departments, policies, and workload intent. It authenticates requests, stores desired state, and coordinates synchronization with registered clusters. It is therefore the right place to reason about multi-cluster governance, while node-level execution remains local to each Kubernetes cluster.

Rank #2
Sale
Smart Watch for Men Women Compatible with iPhone Samsung Android Phone 1.69 inch Full Touch Screen IP68 Waterproof Bluetooh Fitness Tracker Heart Rate/Sleep Monitor (Black)
  • 【IP68 Waterproof Multifunctinal Smart Watch】Torjalph fitness activity tracker smart watch is IP68 waterproof technology ,you can wear it when swimming,handwashing,it's rain-proof which means that you perform multiple exercise modes outside without worrying about sweat or rain corroding your fitness smart watch,and this watch equipped with a 230 mAh large capacity battery magnetic charging ,and this sports activity tracker watch can be used 5-7 days If fully charged.
  • 【Larger 1.83'' Full Screen Watch & Multi-dial Selection】1.83 inch full HD color touch screen smart watch for men / women provides excellent 240*284 resolution image quality ,This Bluetooth smart watch offers a better touch experience and remarkable immersion and ,more than 200 watch faces for you to choose from App"Da Fit ", you can change the dial style according to your mood to wear keeping the world on your wrist
  • 【Accurate Activity Tracker Smart Watch 】sports watch activity tracker uses sweat and dust resistant breathable silicone strap, our Fitness Tracker Smart Watch for Android iOS Phones supports 12 sports modes,walking,running,bicycle,swimming,badminton,basketball,football,rope skipping. This bluetooth calls smart watch for men women is lightweight and stylish, bringing more convenient and relaxing sports experience while also providing more protection.
  • 【Your 24 Hour Personal Daily Health Monitor】Fitness watch for women men full touch screen IP67 waterproof bluetooth smartwatch for Android iOS Phones with heart rate blood pressure monitor multifunction Andriod smart watch with steps counter sleep monitor sports fitness watch, monitoring blood oxygen saturation allows you to understand your physical condition in real time to track fitness progress ,to know your health status in time.
  • 【More Functions Bluetooth Call Smart Watch】This fitness activity smart watch full screen for iPhone and Android phones includes a bunch of new features including:The bluetooth smart watch can hang up the phone, receive and display WhatsAPP /Text /Facebook /Twitter /Messenger reminder.Support sedentary reminder, alarm clock, stopwatch, timer,weather information,music control, remote camera control compatible with Android 5.0 and above or iOS 9.0 and above smartphones.

Cluster plane and reconciliation

Each managed cluster contains Run:ai components that watch the desired state and reconcile it with Kubernetes. The cluster plane is driven by custom resource definitions (CRDs) and an operator. Reconciliation is continuous: if a pod, policy object, or controller-managed resource changes unexpectedly, the relevant component works to restore the configured state.

Which components run in a cluster?

Not every component is a process on every GPU node. Some are cluster-wide controllers; the fractioning operator is the part that enforces GPU-memory boundaries on shared devices. The following map separates their responsibilities.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Component Primary responsibility Where it matters
Data synchronization Reconciles control-plane desired state with Kubernetes resources and returns cluster status. Cluster control plane
Admission-controller Injects Run:ai scheduling metadata and resource requirements before pod admission. Pod admission path
Scheduler Chooses placement using queues, quotas, priorities, preemption, gang rules, and topology. Cluster scheduling path
runtime-rules Enforces configured idle-time and maximum-duration limits. Workload lifecycle
anyworkload-controller Translates workload custom resources into pods and manages their lifecycle. Workload control loop
accessrule-controller Reconciles access-rule policy objects. Authorization and policy
shared-objects-controller Manages shared objects required by the platform across workloads or namespaces. Cluster resources
KAI Resource Management Provides projects, departments, node pools, queues, and workload-placement abstractions. Organizational and resource model
KAI Scheduler Supplies AI-aware scheduling logic for Kubernetes workloads. Placement decisions
gpu-fractioning operator Enforces per-pod GPU-memory boundaries and exports per-pod sharing metrics. Nodes containing shareable GPUs

A node pool is a logical grouping of hardware, not an organizational team. Projects and departments are organizational queues that consume resources from one or more pools. Scheduling decisions are scoped per node pool, so two hardware groups can have independent queue ranks, quotas, and fair-share behavior.

Run:ai Scheduler versus the Kubernetes Scheduler

The Kubernetes Scheduler answers a general question: which feasible node should receive this pod? Run:ai adds an AI-capacity and policy decision before that binding. It evaluates organizational demand and GPU-specific constraints, then cooperates with Kubernetes rather than replacing Kubernetes’ execution responsibilities.

Decision area Kubernetes Scheduler alone Run:ai scheduling layer
Resource model General Kubernetes resources and extended resources. GPU-aware projects, departments, node pools, queues, and workload abstractions through KAI Resource Management.
Quota semantics Namespace and resource-quota mechanisms. Deserved quota plus controlled over-quota borrowing within queue policy.
Fairness Not an AI queue fair-share system. Queue rank, weight, and numerical fair-share values, potentially including historical GPU-hour use.
Preemption Priority-based pod preemption. Priority and queue-aware preemption, including consolidation of preemptible workloads.
Gang placement Basic pod-by-pod scheduling behavior. All pods in a gang can be bound together or remain pending as a group.
Topology and packing General node-fit and affinity rules. AI-aware topology decisions plus bin-packing or spreading strategies for GPU capacity.
GPU sharing No built-in fractional GPU-memory isolation. Fractioning operator, memory boundaries, time-sliced compute, dynamic borrowing, and reclamation.

The practical consequence is that a pod can be technically schedulable yet remain pending because its project has exhausted deserved quota, a gang cannot fit simultaneously, or policy reserves the available GPU for a higher-ranked queue.

Rank #3
New Bee Guitar Cable 10ft Electric Instrument Bass AMP Cord | 6.35mm Jack, Durable Design, Stage and Home Studio Use, Low Noise Shielded Electric Cord for Bass, Keyboard, Amp, Mixer
  • [Low Noise High Fidelity] New bee guitar cord with 22AWG oxygen-free-copper center conductors provides enhanced signal transfer, the PVC insulation material and the high-density stranded copper double shield provide noise rejection for quiet operation
  • [Ideal for Choice] New bee 1/4 straight and right angle metal connectors, ensures a secure connection in any instrument, pedal, or amp jack. L shape great fro on stage guitar stands
  • [Wide Compatible] Suitable for mixer, power amplifier, microphone, effector, decoder, equalizer, electoronic organ, electric guitar, electric bass and other electronic equipments. Widely used in the installation of smart home, karaoke speaker, professional stage audio, etc
  • [Easy Disassemble] Making incidental repairs quick and easy! Very nice, flexible cable
  • [Dont Hesite to Order] 24 hours customer services and professional technology team are standing by

Quotas, fair-share, and over-quota borrowing

Run:ai evaluates demand inside each node pool. A project or department has a deserved quota: the capacity it is entitled to receive under policy. When several queues request more than their deserved amounts, the scheduler first honors those entitlements and then distributes unused capacity according to queue rank, configured weight, and fair-share state.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

What fair-share means

Fair-share is a numerical value calculated for a project or department in a particular node pool. It combines deserved resources with a share of currently unused resources. In configurations that track history, GPU-hour consumption over time can affect the value. A project that has recently consumed more than its intended share can therefore receive a lower opportunity to borrow idle capacity than a project that has consumed less.

Fair-share is not the same as a hard quota. A hard quota limits allocation; fair-share influences how available or borrowed capacity is divided. Administrators should document whether a queue’s number is a guaranteed entitlement, a weighting factor, or a historical usage signal so users understand why two pending jobs with similar GPU requests can wait different lengths of time.

Over-quota resources

After deserved quota is satisfied, unused GPUs may be assigned as over-quota resources. The scheduler ranks eligible queues using the configured queue order, weight, and fair-share calculation. Borrowing is opportunistic: when the owner of that capacity submits work, the borrower may be preempted if policy allows reclaiming the GPUs.

Placement, packing, topology, and preemption

Bin-packing

Bin-packing concentrates workloads on fewer nodes or devices. By filling existing allocations first, it preserves larger contiguous free regions for jobs that need substantial GPU memory or several devices. This can reduce fragmentation, although dense packing may leave fewer partially free nodes for small interactive jobs.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Rank #4
BSIMB 15.6" Digital Family Calendar & Chore Chart, No Subscription Fees
  • Digital Family Calendar: BSIMB digital calendar wall touch screen puts an end to your chaotic life. Designed for busy families, the digital calendar keeps you organised and in step with your family. Assign different colours and tasks to each family member to make family scheduling easier and more intuitive.
  • Easy Setup and Auto-Sync: Plug in, connect to WLAN and auto-syn your electric calendar in minutes. Digital wall calendar compatible with Google calendar, iCloud calendar, Outlook calendar, Cozi calendar and Yahoo calendar. Add events directly on the device or through the ecalendar app. Electronic calendar stays current whether you're at work or traveling—no manual updates needed.
  • All-In-One Home Assistant: Smart calendar builds habits and encourage independence with personalised chores. Display daily meal plan to reduce the daily ‘What's for dinner?’ stress. To-do lists keep track of everything you want. Turning chores into valuable achievements with stars and rewards, making life easier and more enjoyable.
  • Manage Anytime Anywhere: BSIMB electronic calendar for family stay connected with the powerful ecalendar app. Access and manage your digital wall calendar anytime, anywhere with free features like stars & rewards, photo screensaver, chore charts, meals plans, to-do lists etc. Electronic planner paid features like Magic Import tailor advanced tools for users with special needs.
  • Stay in Touch with Loved Ones: Share photos or videos from your phone to electronic wall calendar with app, instantly update your life moment, and keep in touch with your family, especially those who are far away; you can also surprise your smart calendar digital wall by sending him a wish card; it's also great for keeping tech-illiterate elders in touch with the youngsters.

Spreading

Spreading distributes workloads across nodes or devices. It can improve isolation and reduce the impact of a single node failure, but it may fragment free capacity. The choice is a policy decision: use packing when preserving large free regions matters, and spreading when distribution or isolation is more important.

Topology-aware placement

For multi-GPU jobs, placement can account for the relationship between devices and nodes. A request may be feasible in aggregate but unsuitable if the GPUs are too far apart for the workload’s communication pattern. Run:ai’s scheduler considers topology alongside queue policy and available resources rather than treating every GPU as interchangeable.

Priorities and preemption

Priority lets urgent workloads move ahead of lower-priority work. If no free capacity exists, the scheduler can preempt workloads that policy marks as interruptible. Consolidation may reschedule preemptible workloads so fragmented capacity becomes usable by a larger request. Preemption is not free: checkpointing, restart time, and lost progress should be accounted for when setting priorities.

Gang scheduling

A gang is a group of pods that must start together, such as distributed training workers. Run:ai can keep the entire group pending until all required placements are available or bind the group together. This avoids a partial launch in which some workers consume GPUs while others wait indefinitely.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

How GPU sharing works

Run:ai GPU fractions divide a device’s memory into dynamically managed chunks. A workload requests a fraction or an exact memory amount, and the local fractioning logic gives its pod a separate virtual memory address space. The intended boundary is that a pod cannot exceed its allocation or overwrite another pod’s GPU memory.

Memory allocation and compute sharing

Fractioning controls memory capacity; it does not by itself promise a proportional share of compute cycles. Compute sharing uses NVIDIA time-slicing by default, allowing multiple workloads to take turns on the same device. A separate time-slicing mode can split compute time dynamically. Consequently, two pods with equal memory fractions are not automatically guaranteed equal throughput if their kernels have different behavior or if compute-time policy differs.

Exact requests versus percentages

When an application needs a known capacity, an exact memory request is safer than a percentage. Percentage requests depend on the installed GPU’s total memory, so the same percentage produces different gigabyte capacities on different hardware generations.

Multi-GPU fractions

A multi-GPU fraction applies the same memory request to several devices on one node. For example, a request for 8 × 40 GB on eight 80 GB GPUs allocates 40 GB on each device and leaves 40 GB per device for other workloads. The scheduler must still find eight suitable GPUs together; a total of 320 GB scattered across incompatible placements is not equivalent to that topology.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Dynamic fractions and borrowing

Dynamic fractions use a request-and-limit model. In the documented example, a workload can request 0.25 GPU and set a 0.80-GPU limit. The 0.25 allocation is guaranteed; additional memory up to the limit may be loaned while capacity is free. When another workload needs that memory, the borrowed portion can be reclaimed.

GPUOOMKiller provides quality-of-service protection during reclamation. It is designed to terminate or protect workloads according to the configured policy rather than allowing an arbitrary memory overrun to damage neighboring pods. Teams should still design checkpointing and retry behavior because reclaiming borrowed capacity can interrupt work.

Operational limits and common failure modes

  • Memory fragmentation: Many small fractions can leave free memory in unusable shapes. Bin-packing and sensible fraction sizes reduce this risk.
  • MIG incompatibility: Run:ai GPU fractions cannot be used on the same node as MIG mode. Choose one partitioning model per node.
  • False compute expectations: A memory fraction is not a guaranteed percentage of compute throughput. Use time-slicing policy and workload measurements to set expectations.
  • Gang deadlock: A large gang can remain pending if the required devices are not simultaneously available. Check node-pool capacity, topology, and preemptible workloads.
  • Unexpected preemption: Borrowed over-quota capacity is reclaimable. Inspect queue rank, fair-share, priority, and whether the job is marked preemptible.
  • Idle or long-running jobs: runtime-rules can enforce idle and duration limits. A job that appears healthy may still be stopped when it exceeds those configured limits.
  • Stale status: Because state is synchronized between a control plane and cluster controllers, investigate both synchronization health and Kubernetes object status when the UI and cluster disagree.

A practical workflow for administrators and users

  1. Classify hardware: Create node pools that reflect meaningful hardware differences, such as GPU memory size, interconnect, or availability zone.
  2. Define organization: Place teams into projects or departments and assign deserved quotas, queue weights, priorities, and access rules.
  3. Choose placement policy: Select bin-packing or spreading, configure topology requirements, and decide which workloads may be preempted.
  4. Choose a GPU-sharing model: Use whole GPUs when isolation and predictable throughput are paramount; use exact-memory fractions for compatible workloads; use dynamic fractions when borrowing idle memory is valuable.
  5. Submit the workload: Use the UI, CLI, REST API, or MCP server to request the project, node pool, GPU count or memory, and any gang or priority requirements.
  6. Check admission and scheduling: Confirm that the admission-controller added the expected metadata and that the scheduler found a placement consistent with quota and topology.
  7. Observe execution: Inspect pod state in Kubernetes and per-pod GPU metrics exported by the fractioning operator. Compare requested, guaranteed, borrowed, and reclaimed capacity.
  8. Prepare for interruption: Distributed jobs should checkpoint, and dynamic or over-quota users should tolerate reclamation and retry.

Deployment and adoption choices

Run:ai’s current ecosystem combines open-source components with commercial support paths. Organizations can assemble an environment around open-source elements such as KAI Scheduler, KAI Resource Management, and the GPU-sharing operator. NVIDIA AI Enterprise offers an enterprise software route, while standalone Run:ai provides enterprise support. Certified partners provide another deployment and support option.

The right choice depends on operational requirements rather than scheduler features alone: evaluate who operates the control plane, how multi-cluster upgrades are handled, the support contract, security review, observability integration, and whether your GPU fleet requires fractional memory, MIG, or both. The marketing page’s figures such as “10x GPU Availability,” “20x Workloads Running,” and “5x GPU Utilization” are promotional claims without published study methodology in the available product material, so they should not be treated as independent benchmarks.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Why the architecture matters

Run:ai’s value comes from combining policy and execution without replacing Kubernetes. The control plane gives administrators a multi-cluster view; cluster controllers reconcile that intent; the scheduler turns organizational rules into placement decisions; and the fractioning operator makes shared GPU memory enforceable at the node. Understanding those boundaries makes diagnosis straightforward: queue and quota issues belong to policy, placement issues to scheduling and topology, pod execution to Kubernetes, and memory-sharing behavior to the fractioning and time-slicing layers.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

AI infrastructureGPU schedulingGPU sharingKAI SchedulerKubernetesRun:ai

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Recommended PC Tool
Recommended PC Tool
Crashes, No Sound, or Screen Glitches?Free driver scan
Windows Errors? Fix Them Before They SpreadFree repair scan

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.