Runpod.SDK 0.1.1

RunPod SDK for .NET

A modern, type-safe .NET SDK for the RunPod API. Manage GPU pods, run serverless endpoints, and scale your AI workloads with ease.

Features

Full RunPod API Coverage - Pods, GPUs, Serverless Endpoints, and more Dependency Injection Support - First-class support for ASP.NET Core Multi-tenant Architecture - Factory pattern for dynamic API keys Flexible Polling - Adaptive strategies for real-time and batch jobs Cancellation Support - Gracefully cancel long-running operations Resource Management - Proper IDisposable implementation Type Safety - Strongly-typed models with nullability support Rate Limiting - Built-in protection against API limits

Installation

dotnet add package Runpod.SDK

Or via NuGet Package Manager:

Install-Package Runpod.SDK

Quick Start

Basic Usage

using Runpod.SDK;

using var runpod = new RunpodClient("your_api_key_here");

// Run serverless endpoint
var endpoint = runpod.Endpoint("your-endpoint-id");
var result = await endpoint.RunSync<dynamic>(new {
    prompt = "A beautiful sunset over mountains"
});

Console.WriteLine($"Result: {result}");

ASP.NET Core - Single API Key

For applications using one RunPod account:

// Program.cs
using Runpod.SDK;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddRunpod(
    builder.Configuration["RunPod:ApiKey"]!
);

builder.Services.AddControllers();
var app = builder.Build();
app.MapControllers();
app.Run();

// Controller
[ApiController]
[Route("api/[controller]")]
public class ImageController : ControllerBase {
    private readonly IRunpodClient _runpod;

    public ImageController(IRunpodClient runpod) {
        _runpod = runpod;
    }

    [HttpPost("generate")]
    public async Task<IActionResult> Generate([FromBody] ImageRequest request) {
        var endpoint = _runpod.Endpoint("your-endpoint-id");

        var result = await endpoint.RunSync<ImageResult>(
            new { prompt = request.Prompt },
            pollingOptions: new PollingOptions {
                updateDelay = 500  // Check every 500ms for fast feedback
            },
            cancellationToken: HttpContext.RequestAborted
        );

        return Ok(result);
    }
}

ASP.NET Core - Multi-tenant (Factory)

For SaaS applications where each user has their own RunPod API key:

// Program.cs
builder.Services.AddRunpodFactory();  // Register factory instead
builder.Services.AddDbContext<AppDbContext>();

// Controller
public class ImageController : ControllerBase {
    private readonly IRunpodClientFactory _factory;
    private readonly AppDbContext _db;

    public ImageController(IRunpodClientFactory factory, AppDbContext db) {
        _factory = factory;
        _db = db;
    }

    [HttpPost("generate")]
    public async Task<IActionResult> Generate([FromBody] Request req) {
        // Get user's API key from database
        var userId = User.FindFirst(ClaimTypes.NameIdentifier)?.Value;
        var userSettings = await _db.UserSettings.FindAsync(userId);

        if (string.IsNullOrEmpty(userSettings?.RunpodApiKey)) {
            return BadRequest("RunPod API key not configured");
        }

        // Create client with user's key (shares HttpClient pool!)
        using var runpod = _factory.CreateClient(userSettings.RunpodApiKey);

        var endpoint = runpod.Endpoint(userSettings.PreferredEndpointId!);
        var result = await endpoint.RunSync<ImageResult>(
            new { prompt = req.Prompt },
            cancellationToken: HttpContext.RequestAborted
        );

        return Ok(result);
    }
}

Core Concepts

Serverless Endpoints

Synchronous Execution

Best for real-time tasks (image generation, LLM inference):

var result = await endpoint.RunSync<ImageResult>(new {
    prompt = "Beautiful landscape"
});

Asynchronous Execution

For longer-running tasks:

// Submit job
var job = await endpoint.Run(new {
    task = "process_video",
    url = "https://example.com/video.mp4"
});

// Wait for completion
var result = await job.Output<VideoResult>();

// Or check status periodically
var status = await job.Status();

Streaming Output

For LLM streaming responses:

var job = await endpoint.Run(new { prompt = "Tell me a story" });

await foreach (var chunk in job.Stream<string>()) {
    Console.Write(chunk);  // Real-time streaming
}

Polling Strategies

Real-time Tasks (Default)

Fast polling for immediate user feedback:

var result = await endpoint.RunSync<Result>(input);  // Default: 1 second polling

Long-running Batch Jobs

Progressive backoff to reduce API calls:

var job = await endpoint.Run(batchInput);

var result = await job.Output<BatchResult>(new PollingOptions {
    updateDelay = 1000,           	// Start with 1 second
    useProgressiveBackoff = true,  	// Enable adaptive polling
    maxDelay = 60000,             	// Max 60 seconds between checks
    timeout = 0                   	// No timeout (wait indefinitely)
});

Cancellation

Gracefully cancel operations when users disconnect. The SDK automatically cancels jobs on RunPod side when cancellation is requested:

[HttpPost("generate")]
public async Task<IActionResult> Generate([FromBody] Request req) {
    var result = await endpoint.RunSync<Result>(
        req,
        cancellationToken: HttpContext.RequestAborted  // Auto-cancel on disconnect
    );
    return Ok(result);
}
// When user disconnects:
// 1. Client stops polling
// 2. SDK automatically calls RunPod API to cancel the job
// 3. GPU resources are freed immediately

Manual cancellation:

var cts = new CancellationTokenSource();

var task = job.Output<Result>(cancellationToken: cts.Token);

// Cancel after 30 seconds
await Task.Delay(30000);
cts.Cancel();  // SDK will call RunPod cancel API automatically

Pod Management

List Available GPUs

var gpus = await runpod.cmd.GetGpus();
foreach (var gpu in gpus.AsArray()) {
    Console.WriteLine($"GPU: {gpu["displayName"]}, Memory: {gpu["memoryInGb"]}GB");
}

Create a Pod

var pod = await runpod.cmd.CreatePod(
    name: "training-pod",
    imageName: "runpod/pytorch:2.0.0",
    gpuTypeId: "NVIDIA RTX A6000",
    gpuCount: 2,
    volumeInGb: 50,
    env: new Dictionary<string, string> {
        ["WANDB_API_KEY"] = "your-wandb-key",
        ["HF_TOKEN"] = "your-hf-token"
    }
);

Console.WriteLine($"Pod created: {pod["id"]}");

Manage Pods

// List all pods
var pods = await runpod.cmd.GetPods();

// Get specific pod
var pod = await runpod.cmd.GetPod("pod-id");

// Stop pod
await runpod.cmd.StopPod("pod-id");

// Resume pod
await runpod.cmd.ResumePod("pod-id", gpuCount: 1);

// Terminate pod
await runpod.cmd.TerminatePod("pod-id");

Advanced Features

Endpoint Operations

var endpoint = runpod.Endpoint("endpoint-id");

// Check health
var health = await endpoint.Health();

// Cancel a job
await job.Cancel();

// Purge queue
await endpoint.PurgeQueue();

// Full purge
await endpoint.Purge();

Error Handling

try {
    var result = await endpoint.RunSync<Result>(input);
} catch (JobErrorException ex) {
    Console.WriteLine($"Job failed: {ex.status} - {ex.Message}");
} catch (TimeoutException ex) {
    Console.WriteLine($"Job timed out: {ex.Message}");
} catch (QueryException ex) {
    Console.WriteLine($"GraphQL error: {ex.Message}");
} catch (OperationCanceledException) {
    Console.WriteLine("Operation was cancelled");
}

User Management

// Get user info
var user = await runpod.cmd.GetUser();
Console.WriteLine($"User ID: {user["id"]}");
Console.WriteLine($"Network Volumes: {user["networkVolumes"]}");

// Update SSH key
await runpod.cmd.UpdateUserSettings(publicKey);

Container Registry

// Add registry authentication
var auth = await runpod.cmd.CreateContainerRegistryAuth(
    name: "docker-hub",
    username: "myuser",
    password: "mypassword"
);

// Update credentials
await runpod.cmd.UpdateContainerRegistryAuth(
    registryAuthId: auth["id"]!.ToString(),
    username: "newuser",
    password: "newpassword"
);

// Delete
await runpod.cmd.DeleteContainerRegistryAuth(auth["id"]!.ToString());

Configuration

Dependency Injection Options

builder.Services.AddRunpod(
    apiKey: configuration["RunPod:ApiKey"]!,
    configure: options => {
        options.enableRateLimiting = true;
        options.graphqlAddress = "https://api.runpod.io";
        options.endpointAddress = "https://api.runpod.ai";
    }
);

Factory Options

builder.Services.AddRunpodFactory(options => {
    options.enableRateLimiting = true;
    options.graphqlAddress = "https://api.runpod.io";
    options.endpointAddress = "https://api.runpod.ai";
});

When using AddRunpodFactory(), the SDK automatically uses IHttpClientFactory for proper HTTP connection management (DNS rotation, connection pooling). For standalone usage, the factory manages its own HttpClient pool.

Standalone Factory Usage

using Runpod.SDK;

// Create factory (remember to dispose)
using var factory = new RunpodClientFactory();

// Create clients for different users
var client1 = factory.CreateClient("user1-api-key");
var client2 = factory.CreateClient("user2-api-key");

// Use clients
var result1 = await client1.Endpoint("endpoint-id").RunSync<Result>(input);

Documentation

• RunPod API Docs - Official RunPod documentation

Best Practices

1. Always use using for resource cleanup:

   using var runpod = new RunpodClient(apiKey);
   

2. Use Factory for multi-tenant scenarios:

   • Shared HttpClient pool = better performance
   • Each user gets their own client with their API key

3. Choose appropriate polling strategy:

   • Real-time tasks: Default (1 second)
   • Batch jobs: Progressive backoff

4. Always pass CancellationToken in ASP.NET Core:

   cancellationToken: HttpContext.RequestAborted
   

5. Handle errors appropriately:

   • Catch JobErrorException for job failures
   • Catch TimeoutException for timeouts
   • Catch OperationCanceledException for cancellations

Requirements

• .NET 9.0 or higher
• RunPod API key (Get yours here)

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Support

• Report Issues
• Discussions
• Contact: Create an issue

Acknowledgments

• RunPod - Cloud GPU platform
• RunPod Python SDK - Official Python SDK for reference

Made with ❤️ for the AI/ML community