Getting Started with Horde: A Project Plan

Building and maintaining a production-ready Horde deployment can be a daunting task. This guide outlines the typical work involved in creating your own deployment.

Bronze

Core infrastructure and configuration required for a basic Horde deployment

Silver

Advanced optimization, monitoring, and reliability features for professional teams

Gold

Advanced automation, security, and scalability features for large organizations

Bronze

  • Configure Horde streams / templates, schedules, triggers
  • Set up Horde Dashboard and projects
  • Configure UnrealGameSync builds, updates, Horde server connection
  • Stand up Redis and Mongo servers
  • Configure Windows Horde agents base image
  • Configure Horde agent pools and fleet managers
  • Set up agent auto-scaling within cloud provider for Horde agents
  • Configure networking between agents and server
  • Set up authentication and configure SAML or OIDC integration to your IdP
  • Configure Horde artifact storage
  • Set up mac agent images (If shipping on mac or iOS)
  • Secure connection between between office/remote users and Horde server
  • Write documentation for setting up UGS, running preflights and other SOPs

Ongoing Maintenance

  • Regular and urgent security updates for Horde server and Horde agents
  • Regular and urgent security updates for Redis,Mongo, and other supporting services
  • Merge new versions of Unreal into Horde deployment. Apply Horde spot bugfixes as needed.

Silver

  • Create slack app and configure Horde to use it
  • Set up shared Zen server for DDC and configure cook agents to use it
  • Windows agent images to support GPUs for automated testing
  • Set up authorization and ACLs for end-users
  • Configure AutoSDK support for each target platform
  • Profile and optimize compile agent hardware (disk, CPU, memory) and software (OS configuration)
  • Profile and optimize cook agent hardware (disk, CPU, memory) and software (OS configuration)
  • Alerting and monitoring for each service: Horde server, agents, authentication, databases, etc
  • Profile and optimize networking between Horde agents, Horde coordinator, zen server
  • Configure and profile, optimize multiprocess cook for CI
  • Configure Horde Build Health monitoring and issue creation
  • Set up custom artifact types and retention policies for each. Plumb into BuildGraph.
  • Automated backups for Redis, Mongo, and Horde artifacts
  • Configure secrets storage and plumb into Horde
  • Set up load balancers to support zero-downtime deploys
  • Profile and optimize UBA agent hardware (disk, CPU, memory) and software (OS configuration)
  • Implement Unreal Build Accelerator support for development team and CI builds
  • Implement and optimize autoscaling for UBA agents

Gold

  • Profile and optimize Zen server hardware (disk, CPU, memory) and software (OS configuration)
  • Create images for Unreal Build Accelerator using Wine
  • Optimize Unreal Build Accelerator agent launch times
  • Automated CI/CD pipeline for Horde coordinator itself
  • Automated CI/CD pipeline for Agent Images
  • Automated CI/CD pipeline for UBA agent images
  • Automated deployment for Redis and Mongo servers
  • Profile and optimize different storage hardware and networking to it
  • Cost and usage reporting for teams to understand expenses
  • Configure UBA to handle remote shader compilation
  • Set up automated monitoring, alerting, and restarting for Agent hangs/stalls