Getting Started with Horde: A Project Plan
Building and maintaining a production-ready Horde deployment can be a daunting task. This guide outlines the typical work involved in creating your own deployment.
Bronze
Core infrastructure and configuration required for a basic Horde deployment
Silver
Advanced optimization, monitoring, and reliability features for professional teams
Gold
Advanced automation, security, and scalability features for large organizations
Bronze
- □ Configure Horde streams / templates, schedules, triggers
- □ Set up Horde Dashboard and projects
- □ Configure UnrealGameSync builds, updates, Horde server connection
- □ Stand up Redis and Mongo servers
- □ Configure Windows Horde agents base image
- □ Configure Horde agent pools and fleet managers
- □ Set up agent auto-scaling within cloud provider for Horde agents
- □ Configure networking between agents and server
- □ Set up authentication and configure SAML or OIDC integration to your IdP
- □ Configure Horde artifact storage
- □ Set up mac agent images (If shipping on mac or iOS)
- □ Secure connection between between office/remote users and Horde server
- □ Write documentation for setting up UGS, running preflights and other SOPs
Ongoing Maintenance
- □ Regular and urgent security updates for Horde server and Horde agents
- □ Regular and urgent security updates for Redis,Mongo, and other supporting services
- □ Merge new versions of Unreal into Horde deployment. Apply Horde spot bugfixes as needed.
Silver
- □ Create slack app and configure Horde to use it
- □ Set up shared Zen server for DDC and configure cook agents to use it
- □ Windows agent images to support GPUs for automated testing
- □ Set up authorization and ACLs for end-users
- □ Configure AutoSDK support for each target platform
- □ Profile and optimize compile agent hardware (disk, CPU, memory) and software (OS configuration)
- □ Profile and optimize cook agent hardware (disk, CPU, memory) and software (OS configuration)
- □ Alerting and monitoring for each service: Horde server, agents, authentication, databases, etc
- □ Profile and optimize networking between Horde agents, Horde coordinator, zen server
- □ Configure and profile, optimize multiprocess cook for CI
- □ Configure Horde Build Health monitoring and issue creation
- □ Set up custom artifact types and retention policies for each. Plumb into BuildGraph.
- □ Automated backups for Redis, Mongo, and Horde artifacts
- □ Configure secrets storage and plumb into Horde
- □ Set up load balancers to support zero-downtime deploys
- □ Profile and optimize UBA agent hardware (disk, CPU, memory) and software (OS configuration)
- □ Implement Unreal Build Accelerator support for development team and CI builds
- □ Implement and optimize autoscaling for UBA agents
Gold
- □ Profile and optimize Zen server hardware (disk, CPU, memory) and software (OS configuration)
- □ Create images for Unreal Build Accelerator using Wine
- □ Optimize Unreal Build Accelerator agent launch times
- □ Automated CI/CD pipeline for Horde coordinator itself
- □ Automated CI/CD pipeline for Agent Images
- □ Automated CI/CD pipeline for UBA agent images
- □ Automated deployment for Redis and Mongo servers
- □ Profile and optimize different storage hardware and networking to it
- □ Cost and usage reporting for teams to understand expenses
- □ Configure UBA to handle remote shader compilation
- □ Set up automated monitoring, alerting, and restarting for Agent hangs/stalls