Code Blind Documentation

• 1: Overview
• 2: Prerequisite Knowledge
• 3: Install and configure Code Blind on Kubernetes
• 3.1: Create Kubernetes Cluster
• 3.1.1: Google Kubernetes Engine
• 3.1.2: Amazon Elastic Kubernetes Service
• 3.1.3: Azure Kubernetes Service
• 3.1.4: Minikube
• 3.2: Install Code Blind
• 3.2.1: Install Code Blind using YAML
• 3.2.2: Install Code Blind using Helm
• 3.3: Deploy Kubernetes cluster and install Code Blind using Terraform
• 3.3.1: Installing Code Blind on Google Kubernetes Engine using Terraform
• 3.3.2: Installing Code Blind on AWS Elastic Kubernetes Service using Terraform
• 3.3.3: Installing Code Blind on Azure Kubernetes Service using Terraform
• 3.4: Confirming Code Blind Installation
• 3.5: Upgrading Code Blind and Kubernetes
• 4: Getting Started
• 4.1: Quickstart: Create a Game Server
• 4.2: Quickstart: Create a Game Server Fleet
• 4.3: Quickstart: Create a Fleet Autoscaler
• 4.4: Quickstart: Create a Fleet Autoscaler with Webhook Policy
• 4.5: Quickstart: Edit a Game Server
• 5: Guides
• 5.1: Feature Stages
• 5.2: Best Practices
• 5.2.1: Google Kubernetes Engine Best Practices
• 5.3: Code Blind Game Server Client SDKs
• 5.3.1: Unreal Engine Game Server Client Plugin
• 5.3.2: Unity Game Server Client SDK
• 5.3.3: C++ Game Server Client SDK
• 5.3.4: Go Game Server Client SDK
• 5.3.5: C# Game Server Client SDK
• 5.3.6: Node.js Game Server Client SDK
• 5.3.7: Rust Game Server Client SDK
• 5.3.8: REST Game Server Client API
• 5.3.9: Local Development
• 5.4: Windows Gameservers
• 5.5: Fleet Updates
• 5.6: GameServer Health Checking
• 5.7: Player Tracking
• 5.8: Local Game Server
• 5.9: Latency Testing with Multiple Clusters
• 5.10: Metrics
• 5.11: Access Code Blind via the Kubernetes API
• 5.12: Troubleshooting
• 6: Common Integration Patterns
• 6.1: Matchmaker requests a GameServer from a Fleet
• 6.2: Matchmaker requires game server process registration
• 6.3: Canary Testing a new Fleet
• 6.4: Reusing Allocated GameServers for more than one game session
• 6.5: High Density GameServers
• 6.6: Allocating based on GameServer Player Capacity
• 7: Tutorials
• 7.1: Tutorial Build and Run a Simple Gameserver (Rust)
• 7.2: Tutorial Build and Run a Simple Gameserver (node.js)
• 7.3: Tutorial Build and Run a Simple Gameserver (C++)
• 8: Reference
• 8.1: GameServer Specification
• 8.2: Fleet Specification
• 8.3: GameServerAllocation Specification
• 8.4: Fleet Autoscaler Specification
• 8.5: Code Blind Kubernetes API
• 9: Examples
• 10: Advanced
• 10.1: Scheduling and Autoscaling
• 10.2: High Availability Code Blind
• 10.3: Controlling Disruption
• 10.4: Limiting CPU & Memory
• 10.5: Out of Cluster Dev Server
• 10.6: Allocator Service
• 10.7: Multi-cluster Allocation
• 10.8: GameServer Pod Service Accounts
• 11: Frequently Asked Questions
• 12: Third Party Content
• 12.1: Third Party Videos and Presentations
• 12.2: Third Party Examples
• 12.3: Third Party Podcasts
• 12.4: Third Party Libraries and Tools
• 13: Documentation Editing and Contribution

1 - Overview

Agones, is derived from the Greek word agōn which roughly translates to “contest”, “competition at games” and “gathering” (source).

What is Code Blind?

Code Blind is an open source platform, for deploying, hosting, scaling, and orchestrating dedicated game servers for large scale multiplayer games, built on top of the industry standard, distributed system platform Kubernetes.

Code Blind replaces bespoke or proprietary cluster management and game server scaling solutions with an open source solution that can be utilized and communally developed - so that you can focus on the important aspects of building a multiplayer game, rather than developing the infrastructure to support it.

Built with both Cloud and on-premises infrastructure in mind, Code Blind can adjust its strategies as needed for Fleet management, autoscaling, and more to ensure the resources being used to host dedicated game servers are cost optimal for the environment that they are in.

Why Code Blind?

Some of Code Blind’ advantages:

• Lower development and operational costs for hosting, scaling and orchestrating multiplayer game servers.
• Any game server that can run on Linux can be hosted and orchestrated on Code Blind - in any language, or set of dependencies.
• Run Code Blind anywhere Kubernetes can run - in the cloud, on premise, on your local machine or anywhere else you need it.
• Game services and your game servers can be on the same foundational platform - simplifying your tooling and your operations knowledge.
• By extending Kubernetes, Code Blind allows you to take advantage of the thousands of developers that have worked on the features of Kubernetes, and the ecosystem of tools that surround it.
• Code Blind is free, open source and developed entirely in the public. Help shape its future by getting involved with the community.

Major Features

Code Blind incorporates these abilities:

• Code Blind extends Kubernetes, such that it gets native abilities to create, run, manage and scale dedicated game server processes within Kubernetes clusters using standard Kubernetes tooling and APIs.
• Run and update Fleets of Game Servers, without worrying about having Game Servers shutdown that have active players on them.
• Deploy game servers inside a Docker container, with any combination of dependencies or binaries.
• Integrated game server SDK for game server lifecycle managements, including health checking, state management, configuration and more.
• Autoscaling capabilities to ensure players always have a game server available to play on.
• Out of the box metrics and log aggregation to track and visualise what is happening across all your game server sessions.
• Modular architecture that can be tailored to your specific multiplayer game mechanics.
• Game server scheduling and allocation strategies to ensure cost optimisation across cloud and on-premise environments.

Code of Conduct

Participation in this project comes under the Contributor Covenant Code of Conduct

What Next?

• Review our Prerequisite Knowledge. Especially if the above sounds fantastic, but you aren’t yet familiar with technology like Kubernetes or terms such as “Game Servers”.
• Have a look at our installation guides, for setting up a Kubernetes cluster and installing Code Blind on it.
• Go through our Quickstart Guides to take you through setting up a simple game server on Code Blind.

2 - Prerequisite Knowledge

Code Blind is built on top of the foundation of multiple open source projects, as well as utilising several architectural patterns across both distributed and multiplayer game systems – which can make it complicated to get started with, if they are things you are not familiar with or have experience with already.

To make getting started easier to break down and digest, this guide was written to outline what concepts and technology that the documentation assumes that you have knowledge of, and the depth of that knowledge, as well as providing resource to help fill those knowledge gaps.

Docker and Containerisation

Docker and containerisation is the technological foundation of Code Blind, so if you aren’t familiar, we recommend you have knowledge in the following areas before getting started with Code Blind:

• Containers as a concept
• Running Docker containers
• Building your own container
• Registries as a concept
• Pushing and pulling containers from a registry

Resources

The following resources are great for learning these concepts:

• Docker Overview
• Docker “Quickstart” guide

Kubernetes

Kubernetes builds on top of Docker to run containers at scale, on lots of machines. If you have yet to learn about Kubernetes, we recommend that you have knowledge in the following areas before getting started with Code Blind:

• Kubernetes as a concept - you should take the basics tutorial
• Pods
• Deployments
• Services
• Creating a Deployment with a Service

Mappings in Code Blind

Code Blind extends the Kubernetes API to include new custom resources such as GameServer and Fleet. See the Reference documentation for more information.

Code Blind creates a backing Pod with the appropriate configuration parameters for each GameServer that is configured in a cluster. They both have the same name.

Resources

• You should totally read this comic and interactive tutorial
• Kubernetes concepts, explained

Dedicated Game Servers

Code Blind is a platform for dedicated game servers for multiplayer games. If “dedicated game servers” is a term that is not something you are familiar with, we recommend checking out some of the resources below, before getting started with Code Blind:

Resources

• Dedicated Game Servers, Drawn Badly (video)
• What Every Programmer Needs To Know About Game Networking
• Fast-Paced Multiplayer (Part I): Client-Server Game Architecture
• Game Server (wikipedia)
• Example simple gameserver that responds to UDP and/or TCP commands

Game Engine

If you are building a multiplayer game, you will eventually need to understand how your game engine will integrate with Code Blind. There are multiple possible solutions, but the engines that have out of the box SDK’s for Code Blind are:

• Unity
• Unreal

While you can integrate other engines with Code Blind, if you are new to developing on a game engine, you may want to start with the above.

3 - Install and configure Code Blind on Kubernetes

Usage Requirements

• Kubernetes cluster version 1.26, 1.27, 1.28
  • Google Kubernetes Engine, Azure Kubernetes Service, Amazon EKS and Minikube are supported.
  • If you are creating and managing your own Kubernetes cluster, the MutatingAdmissionWebhook, and ValidatingAdmissionWebhook admission controllers are required.
    • We also recommend following the recommended set of admission controllers.
• Firewall access for the range of ports that Game Servers can be connected to in the cluster.
• Game Servers must have the game server SDK integrated, to manage Game Server state, health checking, etc.

Supported Container Architectures

The following container operating systems and architectures can be utilised with Code Blind:

For all the platforms in Alpha, we would appreciate testing and bug reports on any issue found.

Code Blind and Kubernetes Supported Versions

Code Blind will support 3 releases of Kubernetes, targeting the newest version as being the latest available version in the GKE Rapid channel. However, we will ensure that at least one of the 3 versions chosen for each Code Blind release is supported by each of the major cloud providers (EKS and AKS). The vendored version of client-go will be aligned with the middle of the three supported Kubernetes versions. When a new version of Code Blind supports new versions of Kubernetes, it is explicitly called out in the release notes.

The following table lists recent Code Blind versions and their corresponding required Kubernetes versions:

Best Practices

For detailed guides on best practices running Code Blind in production, see Best Practices.

3.1 - Create Kubernetes Cluster

3.1.1 - Google Kubernetes Engine

Before you begin

Take the following steps to enable the Kubernetes Engine API:

1. Visit the Kubernetes Engine page in the Google Cloud Platform Console.
2. Create or select a project.
3. Wait for the API and related services to be enabled. This can take several minutes.
4. Enable billing for your project.

• If you are not an existing GCP user, you may be able to enroll for a $300 US Free Trial credit.

Choosing a shell

To complete this quickstart, we can use either Google Cloud Shell or a local shell.

Google Cloud Shell is a shell environment for managing resources hosted on Google Cloud Platform (GCP). Cloud Shell comes preinstalled with the gcloud and kubectl command-line tools. gcloud provides the primary command-line interface for GCP, and kubectl provides the command-line interface for running commands against Kubernetes clusters.

If you prefer using your local shell, you must install the gcloud and kubectl command-line tools in your environment.

Cloud shell

To launch Cloud Shell, perform the following steps:

1. Go to Google Cloud Platform Console
2. From the top-right corner of the console, click the Activate Google Cloud Shell button:
3. A Cloud Shell session opens inside a frame at the bottom of the console. Use this shell to run gcloud and kubectl commands.

Local shell

To install gcloud and kubectl, perform the following steps:

1. Install the Google Cloud SDK, which includes the gcloud command-line tool.
2. Initialize some default configuration by running the following command.
   • When asked Do you want to configure a default Compute Region and Zone? (Y/n)?, enter Y and choose a zone in your geographical region of choice.

   gcloud init
   

3. Install the kubectl command-line tool by running the following command:

   gcloud components install kubectl
   

Choosing a Regional or Zonal Cluster

You will need to pick a geographical region or zone where you want to deploy your cluster, and whether to create a regional or zonal cluster. We recommend using a Regional cluster, as the zonal GKE control plane can go down temporarily to adjust for cluster resizing, automatic upgrades and repairs.

After choosing a cluster type, choose a region or zone. The region you chose is COMPUTE_REGION below. (Note that if you chose a zone, replace --region=[COMPUTE_REGION] with --zone=[COMPUTE_ZONE] in commands below.)

Choosing a Release Channel and Optional Version

We recommend using the regular release channel, which offers a balance between stability and freshness. If you’d like to read more, see our guide on Release Channels. The release channel you chose is RELEASE_CHANNEL below.

(Optional) During cluster creation, to set a specific available version in the release channel, use the --cluster-version=[VERSION] flag, e.g. --cluster-version=1.27. Be sure to choose a version supported by Code Blind. (If you rely on release channels, the latest Code Blind release should be supported by the default versions of all channels.)

Choosing a GKE cluster mode

A cluster consists of at least one control plane machine and multiple worker machines called nodes. In Google Kubernetes Engine, nodes are Compute Engine virtual machine instances that run the Kubernetes processes necessary to make them part of the cluster.

Code Blind supports both GKE Standard mode and GKE Autopilot mode. Code Blind GameServer and Fleet manifests that work on Standard are compatible on Autopilot with some constraints, described in the following section. We recommend running GKE Autopilot clusters, if you meet the constraints.

You can’t convert existing Standard clusters to Autopilot; create new Autopilot clusters instead.

Code Blind on GKE Autopilot

Autopilot is GKE’s fully-managed mode. GKE configures, maintains, scales, and upgrades nodes for you, which can reduce your maintenance and operating overhead. You only pay for the resources requested by your running Pods, and you don’t pay for unused node capacity or Kubernetes system workloads.

This section describes the Code Blind-specific considerations in Autopilot clusters. For a general comparison between Autopilot and Standard, refer to Choose a GKE mode of operation.

Autopilot nodes are, by default, optimized for most workloads. If some of your workloads have broad compute requirements such as Arm architecture or a minimum CPU platform, you can also choose a compute class that meets that requirement. However, if you have specialized hardware needs that require fine-grained control over machine configuration, consider using GKE Standard.

Code Blind on Autopilot has pre-configured opinionated constraints. Evaluate whether these constraints impact your workloads:

• Operating system: No Windows containers.
• Resource requests: Autopilot has pre-determined minimum Pod resource requests. If your game servers require less than those minimums, use GKE Standard.
• Scheduling strategy: Packed is supported, which is the Code Blind default. Distributed is not supported.
• Host port policy: Dynamic is supported, which is the Code Blind default. Static and Passthrough are not supported.
• Seccomp profile: Code Blind sets the seccomp profile to Unconfined to avoid unexpected container creation delays that might occur because Autopilot enables the RuntimeDefault seccomp profile.
• Pod disruption policy: eviction.safe: Never is supported, which is the Code Blind default. eviction.safe: Always is supported. eviction.safe: OnUpgrade is not supported. If your game sessions exceed one hour, refer to Considerations for long sessions.

Choosing a GCP network

By default, gcloud and the Cloud Console use the VPC named default for all new resources. If you plan to create a dual-stack IPv4/IPv6 cluster cluster, special considerations need to be made. Dual-stack clusters require a dual-stack subnet, which are only supported in custom mode VPC networks. For a new dual-stack cluster, you can either:

• create a new custom mode VPC,

• or if you wish to continue using the default network, you must switch it to custom mode. After switching a network to custom mode, you will need to manually manage subnets within the default VPC.

Once you have a custom mode VPC, you will need to choose whether to use an existing subnet or create a new one - read VPC-native guide on creating a dual-stack cluster, but don’t create the cluster just yet - we’ll create the cluster later in this guide. To use the network and/or subnetwork you just created, you’ll need to add --network and --subnetwork, and for GKE Standard, possibly --stack-type and --ipv6-access-type, depending on whether you created the subnet simultaneously with the cluster.

Creating the firewall

We need a firewall to allow UDP traffic to nodes tagged as game-server via ports 7000-8000. These firewall rules apply to cluster nodes you will create in the next section.

gcloud compute firewall-rules create game-server-firewall \
  --allow udp:7000-8000 \
  --target-tags game-server \
  --description "Firewall to allow game server udp traffic"

Creating the cluster

Create a GKE cluster in which you’ll install Code Blind. You can use GKE Standard mode or GKE Autopilot mode.

Create a Standard mode cluster for Code Blind

Create the cluster:

gcloud container clusters create [CLUSTER_NAME] \
  --region=[COMPUTE_REGION] \
  --release-channel=[RELEASE_CHANNEL] \
  --tags=game-server \
  --scopes=gke-default \
  --num-nodes=4 \
  --enable-image-streaming \
  --machine-type=e2-standard-4

Replace the following:

• [CLUSTER_NAME]: The name of the cluster you want to create
• [COMPUTE_REGION]: The GCP region to create the cluster in, chosen above
• [RELEASE_CHANNEL]: The GKE release channel, chosen above

Flag explanations:

• --region: The compute region you chose above.
• --release-channel: The release channel you chose above.
• --tags: Defines the tags that will be attached to new nodes in the cluster. This is to grant access through ports via the firewall created above.
• --scopes: Defines the Oauth scopes required by the nodes.
• --num-nodes: The number of nodes to be created in each of the cluster’s zones. Default: 4. Depending on the needs of your game, this parameter should be adjusted.
• --enable-image-streaming: Use Image streaming to pull container images, which leads to significant improvements in initialization times. Limitations apply to enable this feature.
• --machine-type: The type of machine to use for nodes. Default: e2-standard-4. Depending on the needs of your game, you may wish to have smaller or larger machines.

(Optional) Creating a dedicated node pool

Create a dedicated node pool for the Code Blind resources to be installed in. If you skip this step, the Code Blind controllers will share the default node pool with your game servers, which is fine for experimentation but not recommended for a production deployment.

gcloud container node-pools create agones-system \
  --cluster=[CLUSTER_NAME] \
  --region=[COMPUTE_REGION] \
  --node-taints agones.dev/agones-system=true:NoExecute \
  --node-labels agones.dev/agones-system=true \
  --num-nodes=1 \
  --machine-type=e2-standard-4

Replace the following:

• [CLUSTER_NAME]: The name of the cluster you created
• [COMPUTE_REGION]: The GCP region to create the cluster in, chosen above

Flag explanations:

• --cluster: The name of the cluster you created.
• --region: The compute region you chose above.
• --node-taints: The Kubernetes taints to automatically apply to nodes in this node pool.
• --node-labels: The Kubernetes labels to automatically apply to nodes in this node pool.
• --num-nodes: The number of nodes per cluster zone. For regional clusters, --num-nodes=1 creates one node in 3 separate zones in the region, giving you faster recovery time in the event of a node failure.
• --machine-type: The type of machine to use for nodes. Default: e2-standard-4. Depending on the needs of your game, you may wish to have smaller or larger machines.

(Optional) Creating a metrics node pool

Create a node pool for Metrics if you want to monitor the Code Blind system using Prometheus with Grafana or Cloud Logging and Monitoring.

gcloud container node-pools create agones-metrics \
  --cluster=[CLUSTER_NAME] \
  --region=[COMPUTE_REGION] \
  --node-taints agones.dev/agones-metrics=true:NoExecute \
  --node-labels agones.dev/agones-metrics=true \
  --num-nodes=1 \
  --machine-type=e2-standard-4

Replace the following:

• [CLUSTER_NAME]: The name of the cluster you created
• [COMPUTE_REGION]: The GCP region to create the cluster in, chosen above

Flag explanations:

• --cluster: The name of the cluster you created.
• --region: The compute region you chose above.
• --node-taints: The Kubernetes taints to automatically apply to nodes in this node pool.
• --node-labels: The Kubernetes labels to automatically apply to nodes in this node pool.
• --num-nodes: The number of nodes per cluster zone. For regional clusters, --num-nodes=1 creates one node in 3 separate zones in the region, giving you faster recovery time in the event of a node failure.
• --machine-type: The type of machine to use for nodes. Default: e2-standard-4. Depending on the needs of your game, you may wish to have smaller or larger machines.

(Optional) Creating a node pool for Windows

If you run game servers on Windows, you need to create a dedicated node pool for those servers. Windows Server 2019 (WINDOWS_LTSC_CONTAINERD) is the recommended image for Windows game servers.

gcloud container node-pools create windows \
  --cluster=[CLUSTER_NAME] \
  --region=[COMPUTE_REGION] \
  --image-type WINDOWS_LTSC_CONTAINERD \
  --machine-type e2-standard-4 \
  --num-nodes=4

Replace the following:

• [CLUSTER_NAME]: The name of the cluster you created
• [COMPUTE_REGION]: The GCP region to create the cluster in, chosen above

Flag explanations:

• --cluster: The name of the cluster you created.
• --region: The compute region you chose above.
• --image-type: The image type of the instances in the node pool - WINDOWS_LTSC_CONTAINERD in this case.
• --machine-type: The type of machine to use for nodes. Default: e2-standard-4. Depending on the needs of your game, you may wish to have smaller or larger machines.
• --num-nodes: The number of nodes per cluster zone. For regional clusters, --num-nodes=1 creates one node in 3 separate zones in the region, giving you faster recovery time in the event of a node failure.

Create an Autopilot mode cluster for Code Blind

1. Choose a Release Channel (Autopilot clusters must be on a Release Channel).

2. Create the cluster:

   gcloud container clusters create-auto [CLUSTER_NAME] \
     --region=[COMPUTE_REGION] \
     --release-channel=[RELEASE_CHANNEL] \
     --autoprovisioning-network-tags=game-server
   

Replace the following:

• [CLUSTER_NAME]: The name of your cluster.
• [COMPUTE_REGION]: the GCP region to create the cluster in.
• [RELEASE_CHANNEL]: one of rapid, regular, or stable, chosen above. The default is regular.

Flag explanations:

• --region: The compute region you chose above.
• --release-channel: The release channel you chose above.
• --autoprovisioning-network-tags: Defines the tags that will be attached to new nodes in the cluster. This is to grant access through ports via the firewall created above.

Setting up cluster credentials

gcloud container clusters create configurates credentials for kubectl automatically. If you ever lose those, run:

gcloud container clusters get-credentials [CLUSTER_NAME] --region=[COMPUTE_REGION]

Next Steps

• Continue to Install Code Blind.

3.1.2 - Amazon Elastic Kubernetes Service

Create your EKS Cluster using the Getting Started Guide.

Possible steps are the following:

1. Create new IAM role for cluster management.
2. Run aws configure to authorize your awscli with proper AWS Access Key ID and AWS Secret Access Key.
3. Create an example cluster:

eksctl create cluster \
--name prod \
--version 1.28 \
--nodegroup-name standard-workers \
--node-type t3.medium \
--nodes 3 \
--nodes-min 3 \
--nodes-max 4

Allowing UDP Traffic

For Code Blind to work correctly, we need to allow UDP traffic to pass through to our EKS cluster worker nodes. To achieve this, we must update the workers’ nodepool SG (Security Group) with the proper rule. A simple way to do that is:

• Log in to the AWS Management Console
• Go to the VPC Dashboard and select Security Groups
• Find the Security Group for the workers nodepool, which will be named something like eksctl-[cluster-name]-nodegroup-[cluster-name]-workers/SG
• Select Inbound Rules
• Edit Rules to add a new Custom UDP Rule with a 7000-8000 port range and an appropriate Source CIDR range (0.0.0.0/0 allows all traffic)

Next Steps

• Continue to Install Code Blind.

3.1.3 - Azure Kubernetes Service

Choosing your shell

You can use either Azure Cloud Shell or install the Azure CLI on your local shell in order to install AKS in your own Azure subscription. Cloud Shell comes preinstalled with az and kubectl utilities whereas you need to install them locally if you want to use your local shell. If you use Windows 10, you can use the WIndows Subsystem for Windows as well.

Creating the AKS cluster

If you are using Azure CLI from your local shell, you need to log in to your Azure account by executing the az login command and following the login procedure.

Here are the steps you need to follow to create a new AKS cluster (additional instructions and clarifications are listed here):

# Declare necessary variables, modify them according to your needs
AKS_RESOURCE_GROUP=akstestrg     # Name of the resource group your AKS cluster will be created in
AKS_NAME=akstest                 # Name of your AKS cluster
AKS_LOCATION=westeurope          # Azure region in which you'll deploy your AKS cluster

# Create the Resource Group where your AKS resource will be installed
az group create --name $AKS_RESOURCE_GROUP --location $AKS_LOCATION

# Create the AKS cluster - this might take some time. Type 'az aks create -h' to see all available options

# The following command will create a four Node AKS cluster. Node size is Standard A1 v1 and Kubernetes version is 1.28.0. Plus, SSH keys will be generated for you, use --ssh-key-value to provide your values
az aks create --resource-group $AKS_RESOURCE_GROUP --name $AKS_NAME --node-count 4 --generate-ssh-keys --node-vm-size Standard_A4_v2 --kubernetes-version 1.28.0 --enable-node-public-ip

# Install kubectl
sudo az aks install-cli

# Get credentials for your new AKS cluster
az aks get-credentials --resource-group $AKS_RESOURCE_GROUP --name $AKS_NAME

Alternatively, you can use the Azure Portal to create a new AKS cluster (instructions).

Allowing UDP traffic

For Code Blind to work correctly, we need to allow UDP traffic to pass through to our AKS cluster. To achieve this, we must update the NSG (Network Security Group) with the proper rule. A simple way to do that is:

• Log in to the Azure Portal
• Find the resource group where the AKS(Azure Kubernetes Service) resources are kept, which should have a name like MC_resourceGroupName_AKSName_westeurope. Alternative, you can type az resource show --namespace Microsoft.ContainerService --resource-type managedClusters -g $AKS_RESOURCE_GROUP -n $AKS_NAME -o json | jq .properties.nodeResourceGroup
• Find the Network Security Group object, which should have a name like aks-agentpool-********-nsg (ie. aks-agentpool-55978144-nsg for dns-name-prefix agones)
• Select Inbound Security Rules
• Select Add to create a new Rule with UDP as the protocol and 7000-8000 as the Destination Port Ranges. Pick a proper name and leave everything else at their default values

Alternatively, you can use the following command, after modifying the RESOURCE_GROUP_WITH_AKS_RESOURCES and NSG_NAME values:

az network nsg rule create \
  --resource-group RESOURCE_GROUP_WITH_AKS_RESOURCES \
  --nsg-name NSG_NAME \
  --name AgonesUDP \
  --access Allow \
  --protocol Udp \
  --direction Inbound \
  --priority 520 \
  --source-port-range "*" \
  --destination-port-range 7000-8000

Getting Public IPs to Nodes

Kubernetes version prior to 1.18.19, 1.19.11 and 1.20.7

To find a resource’s public IP, search for Virtual Machine Scale Sets -> click on the set name(inside MC_resourceGroupName_AKSName_westeurope group) -> click Instances -> click on the instance name -> view Public IP address.

To get public IP via API look here.

For more information on Public IPs for VM NICs, see this document.

Kubernetes version starting 1.18.19, 1.19.11 and 1.20.7

Virtual Machines public IP is available directly in Kubernetes EXTERNAL-IP.

Next Steps

• Continue to Install Code Blind.

3.1.4 - Minikube

Installing Minikube

First, install Minikube, which may also require you to install a virtualisation solution, such as VirtualBox as well.

Starting Minikube

Minikube will need to be started with the supported version of Kubernetes that is supported with Code Blind, via the --kubernetes-version command line flag.

Optionally, we also recommend starting with an agones profile, using -p to keep this cluster separate from any other clusters you may have running with Minikube.

minikube start --kubernetes-version v1.27.6 -p agones

Check the official minikube start reference for more options that may be required for your platform of choice.

Known working drivers

Other operating systems and drivers may work, but at this stage have not been verified to work with UDP connections via Code Blind exposed ports.

Linux (amd64)

• Docker (default)
• kvm2

Mac (amd64)

• Docker (default)
• Hyperkit

Windows (amd64)

• hyper-v (might need this blog post and/or this comment for WSL support)

If you have successfully tested with other platforms and drivers, please click “edit this page” in the top right hand side and submit a pull request to let us know.

Local connection workarounds

Depending on your operating system and virtualization platform that you are using with Minikube, it may not be possible to connect directly to a GameServer hosted on Code Blind as you would on a cloud hosted Kubernetes cluster.

If you are unable to do so, the following workarounds are available, and may work on your platform:

minikube ip

Rather than using the published IP of a GameServer to connect, run minikube ip -p agones to get the local IP for the minikube node, and connect to that address.

Create a service

This would only be for local development, but if none of the other workarounds work, creating a Service for the GameServer you wish to connect to is a valid solution, to tunnel traffic to the appropriate GameServer container.

Use the following yaml:

apiVersion: v1
kind: Service
metadata:
  name: agones-gameserver
spec:
  type: LoadBalancer
  selector:
    agones.dev/gameserver: ${GAMESERVER_NAME}
  ports:
  - protocol: UDP
    port: 7000 # local port
    targetPort: ${GAMESERVER_CONTAINER_PORT}

Where ${GAMESERVER_NAME} is replaced with the GameServer you wish to connect to, and ${GAMESERVER_CONTAINER_PORT} is replaced with the container port GameServer exposes for connection.

Running minikube service list -p agones will show you the IP and port to connect to locally in the URL field.

To connect to a different GameServer, run kubectl edit service agones-gameserver and edit the ${GAMESERVER_NAME} value to point to the new GameServer instance and/or the ${GAMESERVER_CONTAINER_PORT} value as appropriate.

Use a different driver

If you cannot connect through the Serviceor use other workarounds, you may want to try a different minikube driver, and if that doesn’t work, connection via UDP may not be possible with minikube, and you may want to try either a different local Kubernetes tool or use a cloud hosted Kubernetes cluster.

Next Steps

• Continue to Install Code Blind.

3.2 - Install Code Blind

If you have not yet created a cluster, follow the instructions for the environment where you will be running Code Blind.

3.2.1 - Install Code Blind using YAML

Installing Code Blind

Installing Code Blind using the pre-generated install.yaml file is the quickest, simplest way to get Code Blind up and running in your Kubernetes cluster:

kubectl create namespace agones-system
kubectl apply --server-side -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/install/yaml/install.yaml

You can also find the install.yaml in the latest agones-install zip from the releases archive.

Customizing your install

To change the configurable parameters in the install.yaml file, you can use helm template to generate a custom file locally without needing to use helm to install Code Blind into your cluster.

The following example sets the featureGates and generateTLS helm parameters and creates a customized install-custom.yaml file (note that the pull command was introduced in Helm version 3):

helm pull --untar https://agones.dev/chart/stable/agones-1.38.0.tgz && \
cd agones && \
helm template agones-manual --namespace agones-system  . \
  --set agones.controller.generateTLS=false \
  --set agones.allocator.generateTLS=false \
  --set agones.allocator.generateClientTLS=false \
  --set agones.crds.cleanupOnDelete=false \
  --set agones.featureGates="PlayerTracking=true" \
  > install-custom.yaml

Uninstalling Code Blind

To uninstall/delete the Code Blind deployment and delete agones-system namespace:

kubectl delete fleets --all --all-namespaces
kubectl delete gameservers --all --all-namespaces
kubectl delete -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/install/yaml/install.yaml
kubectl delete namespace agones-system

Note: It may take a couple of minutes until all resources described in install.yaml file are deleted.

Next Steps

• Confirm Code Blind is up and running

3.2.2 - Install Code Blind using Helm

Prerequisites

• Helm package manager 3.2.3+
• Supported Kubernetes Cluster

Helm 3

Installing the Chart

To install the chart with the release name my-release using our stable helm repository:

helm repo add agones https://agones.dev/chart/stable
helm repo update
helm install my-release --namespace agones-system --create-namespace agones/agones

We recommend installing Code Blind in its own namespaces, such as agones-system as shown above. If you want to use a different namespace, you can use the helm --namespace parameter to specify.

When running in production, Code Blind should be scheduled on a dedicated pool of nodes, distinct from where Game Servers are scheduled for better isolation and resiliency. By default Code Blind prefers to be scheduled on nodes labeled with agones.dev/agones-system=true and tolerates node taint agones.dev/agones-system=true:NoExecute. If no dedicated nodes are available, Code Blind will run on regular nodes, but that’s not recommended for production use. For instructions on setting up a dedicated node pool for Code Blind, see the Code Blind installation instructions for your preferred environment.

The command deploys Code Blind on the Kubernetes cluster with the default configuration. The configuration section lists the parameters that can be configured during installation.

Namespaces

By default Code Blind is configured to work with game servers deployed in the default namespace. If you are planning to use another namespace you can configure Code Blind via the parameter gameservers.namespaces.

For example to use default and xbox namespaces:

kubectl create namespace xbox
helm install my-release agones/agones --set "gameservers.namespaces={default,xbox}" --namespace agones-system

If you want to add a new namespace afterward upgrade your release:

kubectl create namespace ps4
helm upgrade my-release agones/agones --reuse-values --set "gameservers.namespaces={default,xbox,ps4}" --namespace agones-system

Uninstalling the Chart

To uninstall/delete the my-release deployment:

helm uninstall my-release --namespace=agones-system

RBAC

By default, agones.rbacEnabled is set to true. This enables RBAC support in Code Blind and must be true if RBAC is enabled in your cluster.

The chart will take care of creating the required service accounts and roles for Code Blind.

If you have RBAC disabled, or to put it another way, ABAC enabled, you should set this value to false.

Configuration

The following tables lists the configurable parameters of the Code Blind chart and their default values.

General

Custom Resource Definitions

Metrics

Service Accounts

Container Images

Code Blind Controller

Ping Service

Allocator Service

Extensions

GameServers

Helm Installation

Specify each parameter using the --set key=value[,key=value] argument to helm install. For example,

helm install my-release --namespace agones-system \
  --set gameservers.minPort=1000,gameservers.maxPort=5000 agones

The above command will deploy Code Blind controllers to agones-system namespace. Additionally Code Blind will use a dynamic GameServers’ port allocation range of 1000-5000.

Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example,

helm install my-release --namespace agones-system -f values.yaml agones/agones

Helm test

This test would create a GameServer resource and delete it afterwards.

Check the Code Blind installation by running the following command:

helm test my-release -n agones-system

You should see a successful output similar to this :

NAME: my-release
LAST DEPLOYED: Wed Mar 29 06:13:23 2023
NAMESPACE: agones-system
STATUS: deployed
REVISION: 4
TEST SUITE:     my-release-test
Last Started:   Wed Mar 29 06:17:52 2023
Last Completed: Wed Mar 29 06:18:10 2023
Phase:          Succeeded

Controller TLS Certificates

By default agones chart generates tls certificates used by the admission controller, while this is handy, it requires the agones controller to restart on each helm upgrade command.

Manual

For most use cases the controller would have required a restart anyway (eg: controller image updated). However if you really need to avoid restarts we suggest that you turn off tls automatic generation (agones.controller.generateTLS to false) and provide your own certificates (certs/server.crt,certs/server.key).

Cert-Manager

Another approach is to use cert-manager.io solution for cluster level certificate management.

In order to use the cert-manager solution, first install cert-manager on the cluster. Then, configure an Issuer/ClusterIssuer resource and last configure a Certificate resource to manage controller Secret. Make sure to configure the Certificate based on your system’s requirements, including the validity duration.

Here is an example of using a self-signed ClusterIssuer for configuring controller Secret where secret name is my-release-cert or {{ template "agones.fullname" . }}-cert:

#!/bin/bash
# Create a self-signed ClusterIssuer
cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: selfsigned
spec:
  selfSigned: {}
EOF

# Create a Certificate with IP for the my-release-cert )
cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: my-release-agones-cert
  namespace: agones-system
spec:
  dnsNames:
    - agones-controller-service.agones-system.svc
  secretName: my-release-agones-cert
  issuerRef:
    name: selfsigned
    kind: ClusterIssuer
EOF

After the certificates are generated, we will want to inject caBundle into the controller and extensions webhook and disable the controller and extensions secret creation through the following values.yaml file.:

agones:
  controller:
    disableSecret: true
    customCertSecretPath:
    - key: ca.crt
      path: ca.crt
    - key: tls.crt
      path: server.crt
    - key: tls.key
      path: server.key
    allocationApiService:
      annotations:
        cert-manager.io/inject-ca-from: agones-system/my-release-agones-cert
      disableCaBundle: true
    validatingWebhook:
      annotations:
        cert-manager.io/inject-ca-from: agones-system/my-release-agones-cert
      disableCaBundle: true
    mutatingWebhook:
      annotations:
        cert-manager.io/inject-ca-from: agones-system/my-release-agones-cert
      disableCaBundle: true
  extensions:
    disableSecret: true
    customCertSecretPath:
    - key: ca.crt
      path: ca.crt
    - key: tls.crt
      path: server.crt
    - key: tls.key
      path: server.key
    allocationApiService:
      annotations:
        cert-manager.io/inject-ca-from: agones-system/my-release-agones-cert
      disableCaBundle: true
    validatingWebhook:
      annotations:
        cert-manager.io/inject-ca-from: agones-system/my-release-agones-cert
      disableCaBundle: true
    mutatingWebhook:
      annotations:
        cert-manager.io/inject-ca-from: agones-system/my-release-agones-cert
      disableCaBundle: true

After copying the above yaml into a values.yaml file, use below command to install Code Blind:

helm install my-release --namespace agones-system --create-namespace --values values.yaml agones/agones

Reserved Allocator Load Balancer IP

In order to reuse the existing load balancer IP on upgrade or install the agones-allocator service as a LoadBalancer using a reserved static IP, a user can specify the load balancer’s IP with the agones.allocator.http.loadBalancerIP helm configuration parameter value. By setting the loadBalancerIP value:

1. The LoadBalancer is created with the specified IP, if supported by the cloud provider.
2. A self-signed server TLS certificate is generated for the IP, used by the agones-allocator service.

Next Steps

• Confirm Code Blind is up and running

3.3 - Deploy Kubernetes cluster and install Code Blind using Terraform

Prerequisites

• Terraform v1.0.8
• Access to the the Kubernetes hosting provider you are using (e.g. gcloud, awscli, or az utility installed)
• Git

3.3.1 - Installing Code Blind on Google Kubernetes Engine using Terraform

Before you begin

Take the following steps to enable the Kubernetes Engine API:

1. Visit the Kubernetes Engine page in the Google Cloud Platform Console.
2. Create or select a project.
3. Wait for the API and related services to be enabled. This can take several minutes.
4. Enable billing for your project.

• If you are not an existing GCP user, you may be able to enroll for a $300 US Free Trial credit.

Choosing a shell

To complete this quickstart, we can use either Google Cloud Shell or a local shell.

Google Cloud Shell is a shell environment for managing resources hosted on Google Cloud Platform (GCP). Cloud Shell comes preinstalled with the gcloud and kubectl command-line tools. gcloud provides the primary command-line interface for GCP, and kubectl provides the command-line interface for running commands against Kubernetes clusters.

If you prefer using your local shell, you must install the gcloud and kubectl command-line tools in your environment.

Cloud shell

To launch Cloud Shell, perform the following steps:

1. Go to Google Cloud Platform Console
2. From the top-right corner of the console, click the Activate Google Cloud Shell button:
3. A Cloud Shell session opens inside a frame at the bottom of the console. Use this shell to run gcloud and kubectl commands.
4. Set a compute zone in your geographical region with the following command. The compute zone will be something like us-west1-a. A full list can be found here.

   gcloud config set compute/zone [COMPUTE_ZONE]
   

Local shell

To install gcloud and kubectl, perform the following steps:

1. Install the Google Cloud SDK, which includes the gcloud command-line tool.
2. Initialize some default configuration by running the following command.
   • When asked Do you want to configure a default Compute Region and Zone? (Y/n)?, enter Y and choose a zone in your geographical region of choice.

   gcloud init
   

3. Install the kubectl command-line tool by running the following command:

   gcloud components install kubectl
   

Installation

An example configuration can be found here: Terraform configuration with Code Blind submodule.

Copy this file into a local directory where you will execute the terraform commands.

The GKE cluster created from the example configuration will contain 3 Node Pools:

• "default" node pool with "game-server" tag, containing 4 nodes.
• "agones-system" node pool for Code Blind Controller.
• "agones-metrics" for monitoring and metrics collecting purpose.

Configurable parameters:

• project - your Google Cloud Project ID (required)
• name - the name of the GKE cluster (default is “agones-terraform-example”)
• agones_version - the version of agones to install (an empty string, which is the default, is the latest version from the Helm repository)
• machine_type - machine type for hosting game servers (default is “e2-standard-4”)
• node_count - count of game server nodes for the default node pool (default is “4”)
• enable_image_streaming - whether or not to enable image streaming for the "default" node pool (default is true)
• zone - (Deprecated, use location) the name of the zone you want your cluster to be created in (default is “us-west1-c”)
• network - the name of the VPC network you want your cluster and firewall rules to be connected to (default is “default”)
• subnetwork - the name of the subnetwork in which the cluster’s instances are launched. (required when using non default network)
• log_level - possible values: Fatal, Error, Warn, Info, Debug (default is “info”)
• feature_gates - a list of alpha and beta version features to enable. For example, “PlayerTracking=true&ContainerPortAllocation=true”
• gameserver_minPort - the lower bound of the port range which gameservers will listen on (default is “7000”)
• gameserver_maxPort - the upper bound of the port range which gameservers will listen on (default is “8000”)
• gameserver_namespaces - a list of namespaces which will be used to run gameservers (default is ["default"]). For example ["default", "xbox-gameservers", "mobile-gameservers"]
• force_update - whether or not to force the replacement/update of resource (default is true, false may be required to prevent immutability errors when updating the configuration)
• location - the name of the location you want your cluster to be created in (default is “us-west1-c”)
• autoscale - whether you want to enable autoscale for the gameserver nodepool (default is false)
• min_node_count - the minimum number of nodes for a nodepool when autoscale is enabled (default is “1”)
• max_node_count - the maximum number of nodes for a nodepool when autoscale is enabled (default is “5”)

Creating the cluster

In the directory where you created module.tf, run:

terraform init

This will cause terraform to clone the Code Blind repository and use the ./install/terraform folder as the starting point of the Code Blind submodule, which contains all necessary Terraform configuration files.

Next, make sure that you can authenticate using gcloud:

gcloud auth application-default login

Option 1: Creating the cluster in the default VPC

To create your GKE cluster in the default VPC just specify the project variable.

terraform apply -var project="<YOUR_GCP_ProjectID>"

Option 2: Creating the cluster in a custom VPC

To create the cluster in a custom VPC you must specify the project, network and subnetwork variables.

terraform apply -var project="<YOUR_GCP_ProjectID>" -var network="<YOUR_NETWORK_NAME>" -var subnetwork="<YOUR_SUBNETWORK_NAME>"

To verify that the cluster was created successfully, set up your kubectl credentials:

gcloud container clusters get-credentials --zone us-west1-c agones-terraform-example

Then check that you have access to the Kubernetes cluster:

kubectl get nodes

You should have 6 nodes in Ready state.

Uninstall the Code Blind and delete GKE cluster

To delete all resources provisioned by Terraform:

terraform destroy -var project="<YOUR_GCP_ProjectID>"

Next Steps

• Confirm Code Blind is up and running

3.3.2 - Installing Code Blind on AWS Elastic Kubernetes Service using Terraform

Installation

You can use Terraform to provision your Amazon EKS (Elastic Kubernetes Service) cluster and install Code Blind on it using the Helm Terraform provider.

An example of the EKS submodule config file can be found here: Terraform configuration with Code Blind submodule

Copy this file into a separate folder.

Configure your AWS CLI tool CLI configure:

aws configure

Initialise your terraform:

terraform init

Creating Cluster

By editing modules.tf you can change the parameters that you need to. For instance, the - machine_type variable.

Configurable parameters:

• cluster_name - the name of the EKS cluster (default is “agones-terraform-example”)
• agones_version - the version of agones to install (an empty string, which is the default, is the latest version from the Helm repository)
• machine_type - EC2 instance type for hosting game servers (default is “t2.large”)
• region - the location of the cluster (default is “us-west-2”)
• node_count - count of game server nodes for the default node pool (default is “4”)
• log_level - possible values: Fatal, Error, Warn, Info, Debug (default is “info”)
• feature_gates - a list of alpha and beta version features to enable. For example, “PlayerTracking=true&ContainerPortAllocation=true”
• gameserver_minPort - the lower bound of the port range which gameservers will listen on (default is “7000”)
• gameserver_maxPort - the upper bound of the port range which gameservers will listen on (default is “8000”)
• gameserver_namespaces - a list of namespaces which will be used to run gameservers (default is ["default"]). For example ["default", "xbox-gameservers", "mobile-gameservers"]
• force_update - whether or not to force the replacement/update of resource (default is true, false may be required to prevent immutability errors when updating the configuration)

Now you can create an EKS cluster and deploy Code Blind on EKS:

terraform apply [-var agones_version="1.38.0"]

After deploying the cluster with Code Blind, you can get or update your kubeconfig by using:

aws eks --region us-west-2 update-kubeconfig --name agones-cluster

With the following output:

Added new context arn:aws:eks:us-west-2:601646756426:cluster/agones-cluster to /Users/user/.kube/config

Switch kubectl context to the recently created one:

kubectl config use-context arn:aws:eks:us-west-2:601646756426:cluster/agones-cluster

Check that you are authenticated against the recently created Kubernetes cluster:

kubectl get nodes

Uninstall the Code Blind and delete EKS cluster

Run the following commands to delete all Terraform provisioned resources:

terraform destroy -target module.helm_agones.helm_release.agones -auto-approve && sleep 60
terraform destroy

3.3.3 - Installing Code Blind on Azure Kubernetes Service using Terraform

Installation

Install az utility by following these instructions.

The example of AKS submodule configuration could be found here: Terraform configuration with Code Blind submodule

Copy module.tf file into a separate folder.

Log in to Azure CLI:

az login

Configure your terraform:

terraform init

Create a service principal and configure its access to Azure resources:

az ad sp create-for-rbac

Now you can deploy your cluster (use values from the above command output):

terraform apply -var client_id="<appId>" -var client_secret="<password>"

Once you created all resources on AKS you can get the credentials so that you can use kubectl to configure your cluster:

az aks get-credentials --resource-group agonesRG --name test-cluster

Check that you have access to the Kubernetes cluster:

kubectl get nodes

Configurable parameters:

• log_level - possible values: Fatal, Error, Warn, Info, Debug (default is “info”)
• cluster_name - the name of the AKS cluster (default is “agones-terraform-example”)
• agones_version - the version of agones to install (an empty string, which is the default, is the latest version from the Helm repository)
• machine_type - node machine type for hosting game servers (default is “Standard_D2_v2”)
• disk_size - disk size of the node
• region - the location of the cluster
• node_count - count of game server nodes for the default node pool (default is “4”)
• feature_gates - a list of alpha and beta version features to enable. For example, “PlayerTracking=true&ContainerPortAllocation=true”
• gameserver_minPort - the lower bound of the port range which gameservers will listen on (default is “7000”)
• gameserver_maxPort - the upper bound of the port range which gameservers will listen on (default is “8000”)
• gameserver_namespaces - a list of namespaces which will be used to run gameservers (default is ["default"]). For example ["default", "xbox-gameservers", "mobile-gameservers"]
• force_update - whether or not to force the replacement/update of resource (default is true, false may be required to prevent immutability errors when updating the configuration)

Uninstall the Code Blind and delete AKS cluster

Run next command to delete all Terraform provisioned resources:

terraform destroy

Reference

Details on how you can authenticate your AKS terraform provider using official instructions.

Next Steps

• Confirm Code Blind is up and running

3.4 - Confirming Code Blind Installation

To confirm Code Blind is up and running, run the following command:

kubectl describe --namespace agones-system pods

It should describe six pods created in the agones-system namespace, with no error messages or status. All Conditions sections should look like this:

Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True

All this pods should be in a RUNNING state:

kubectl get pods --namespace agones-system

NAME                                 READY   STATUS    RESTARTS   AGE
agones-allocator-5c988b7b8d-cgtbs    1/1     Running   0          8m47s
agones-allocator-5c988b7b8d-hhhr5    1/1     Running   0          8m47s
agones-allocator-5c988b7b8d-pv577    1/1     Running   0          8m47s
agones-controller-7db45966db-56l66   1/1     Running   0          8m44s
agones-ping-84c64f6c9d-bdlzh         1/1     Running   0          8m37s
agones-ping-84c64f6c9d-sjgzz         1/1     Running   0          8m47s

That’s it!

Now with Code Blind installed, you can utilise its Custom Resource Definitions to create resources of type GameServer, Fleet and more!

What’s next

• Go through the Create a Game Server Quickstart

3.5 - Upgrading Code Blind and Kubernetes

Upgrading Code Blind

The following are strategies for safely upgrading Code Blind from one version to another. They may require adjustment to your particular game architecture but should provide a solid foundation for updating Code Blind safely.

The recommended approach is to use multiple clusters, such that the upgrade can be tested gradually with production load and easily rolled back if the need arises.

Upgrading Code Blind: Multiple Clusters

We essentially want to transition our GameServer allocations from a cluster with the old version of Code Blind, to a cluster with the upgraded version of Code Blind while ensuring nothing surprising happens during this process.

This also allows easy rollback to the previous infrastructure that we already know to be working in production, with minimal interruptions to player experience.

The following are steps to implement this:

1. Create a new cluster of the same size or smaller as the current cluster.
2. Install the new version of Code Blind on the new cluster.
3. Deploy the same set of Fleets, GameServers and FleetAutoscalers from the old cluster into the new cluster.
4. With your matchmaker, start sending a small percentage of your matched players’ game sessions to the new cluster.
5. Assuming everything is working successfully on the new cluster, slowly increase the percentage of matched sessions to the new cluster, until you reach 100%.
6. Once you are comfortable with the stability of the new cluster with the new Code Blind version, shut down the old cluster.
7. Congratulations - you have now upgraded to a new version of Code Blind! 👍

Upgrading Code Blind: Single Cluster

If you are upgrading a single cluster, we recommend creating a maintenance window, in which your game goes offline for the period of your upgrade, as there will be a short period in which Code Blind will be non-responsive during the upgrade.

Installation with install.yaml

If you installed Code Blind with install.yaml, then you will need to delete the previous installation of Code Blind before upgrading to the new version, as we need to remove all of Code Blind before installing the new version.

1. Start your maintenance window.
2. Delete the current set of Fleets, GameServers and FleetAutoscalers in your cluster.
3. Make sure to delete the same version of Code Blind that was previously installed, for example: kubectl delete -f https://raw.githubusercontent.com/googleforgames/agones/<old-release-version>/install/yaml/install.yaml
4. Install Code Blind with install.yaml.
5. Deploy the same set of Fleets, GameServers and FleetAutoscalers back into the cluster.
6. Run any other tests to ensure the Code Blind installation is working as expected.
7. Close your maintenance window.
8. Congratulations - you have now upgraded to a new version of Code Blind! 👍

Installation with Helm

Helm features capabilities for upgrading to newer versions of Code Blind without having to uninstall Code Blind completely.

For details on how to use Helm for upgrades, see the helm upgrade documentation.

Given the above, the steps for upgrade are simpler:

1. Start your maintenance window.
2. Delete the current set of Fleets, GameServers and FleetAutoscalers in your cluster.
3. Run helm upgrade with the appropriate arguments, such a --version, for your specific upgrade
4. Deploy the same set of Fleets, GameServers and FleetAutoscalers back into the cluster.
5. Run any other tests to ensure the Code Blind installation is working as expected.
6. Close your maintenance window.
7. Congratulations - you have now upgraded to a new version of Code Blind! 👍

Upgrading Kubernetes

The following are strategies for safely upgrading the underlying Kubernetes cluster from one version to another. They may require adjustment to your particular game architecture but should provide a solid foundation for updating your cluster safely.

The recommended approach is to use multiple clusters, such that the upgrade can be tested gradually with production load and easily rolled back if the need arises.

Code Blind has multiple supported Kubernetes versions for each version. You can stick with a minor Kubernetes version until it is not supported by Code Blind, but it is recommended to do supported minor (e.g. 1.12.1 ➡ 1.13.2) Kubernetes version upgrades at the same time as a matching Code Blind upgrades.

Patch upgrades (e.g. 1.12.1 ➡ 1.12.3) within the same minor version of Kubernetes can be done at any time.

Multiple Clusters

This process is very similar to the Upgrading Code Blind: Multiple Clusters approach above.

We essentially want to transition our GameServer allocations from a cluster with the old version of Kubernetes, to a cluster with the upgraded version of Kubernetes while ensuring nothing surprising happens during this process.

This also allows easy rollback to the previous infrastructure that we already know to be working in production, with minimal interruptions to player experience.

The following are steps to implement this:

1. Create a new cluster of the same size or smaller as the current cluster, with the new version of Kubernetes
2. Install the same version of Code Blind on the new cluster, as you have on the previous cluster.
3. Deploy the same set of Fleets and/or GameServers from the old cluster into the new cluster.
4. With your matchmaker, start sending a small percentage of your matched players’ game sessions to the new cluster.
5. Assuming everything is working successfully on the new cluster, slowly increase the percentage of matched sessions to the new cluster, until you reach 100%.
6. Once you are comfortable with the stability of the new cluster with the new Kubernetes version, shut down the old cluster.
7. Congratulations - you have now upgraded to a new version of Kubernetes! 👍

Single Cluster

If you are upgrading a single cluster, we recommend creating a maintenance window, in which your game goes offline for the period of your upgrade, as there will be a short period in which Code Blind will be non-responsive during the node upgrades.

1. Start your maintenance window.
2. Scale your Fleets down to 0 and/or delete your GameServers. This is a good safety measure so there aren’t race conditions between the Code Blind controller being recreated and GameServers being deleted doesn’t occur, and GameServers can end up stuck in erroneous states.
3. Start and complete your control plane upgrade(s).
4. Start and complete your node upgrades.
5. Scale your Fleets back up and/or recreate your GameServers.
6. Run any other tests to ensure the Code Blind installation is still working as expected.
7. Close your maintenance window.
8. Congratulations - you have now upgraded to a new version of Kubernetes! 👍

4 - Getting Started

4.1 - Quickstart: Create a Game Server

Prerequisites

The following prerequisites are required to create a GameServer:

1. A Kubernetes cluster with the UDP port range 7000-8000 open on each node.
2. Code Blind controller installed in the targeted cluster
3. kubectl properly configured
4. Netcat which is already installed on most Linux/macOS distributions, for windows you can use WSL.

If you don’t have a Kubernetes cluster you can follow these instructions to create a cluster on Google Kubernetes Engine (GKE), Minikube or Azure Kubernetes Service (AKS), and install Code Blind.

For the purpose of this guide we’re going to use the simple-game-server example as the GameServer container. This example is a very simple UDP server written in Go. Don’t hesitate to look at the code of this example for more information.

Objectives

• Create a GameServer in Kubernetes using Code Blind custom resource.
• Get information about the GameServer such as IP address, port and state.
• Connect to the GameServer.

1. Create a GameServer

Let’s create a GameServer using the following command :

kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/gameserver.yaml

You should see a successful output similar to this :

gameserver.agones.dev/simple-game-server-4ss4j created

This has created a GameServer record inside Kubernetes, which has also created a backing Pod to run our simple udp game server code in. If you want to see all your running GameServers you can run:

kubectl get gameservers

It should look something like this:

NAME                       STATE     ADDRESS       PORT   NODE     AGE
simple-game-server-7pjrq   Ready   35.233.183.43   7190   agones   3m

You can also see the Pod that got created by running kubectl get pods, the Pod will be prefixed by simple-game-server.

NAME                        READY     STATUS    RESTARTS   AGE
simple-game-server-7pjrq    2/2       Running   0          5m

As you can see above it says READY: 2/2 this means there are two containers running in this Pod, this is because Code Blind injected the SDK sidecar for readiness and health checking of your Game Server.

For the full details of the YAML file head to the GameServer Specification Guide

2. Fetch the GameServer Status

Let’s wait for the GameServer state to become Ready. You can use the watch tool to see the state change. If your operating system does not have watch, manually run kubectl describe gameserver until the state changes.

watch kubectl describe gameserver

Name:         simple-game-server-7pjrq
Namespace:    default
Labels:       <none>
Annotations:  agones.dev/sdk-version: 0.9.0-764fa53
API Version:  agones.dev/v1
Kind:         GameServer
Metadata:
  Creation Timestamp:  2019-02-27T15:06:20Z
  Finalizers:
    agones.dev
  Generate Name:     simple-game-server-
  Generation:        1
  Resource Version:  30377
  Self Link:         /apis/agones.dev/v1/namespaces/default/gameservers/simple-game-server-7pjrq
  UID:               3d7ac3e1-3aa1-11e9-a4f5-42010a8a0019
Spec:
  Container:  simple-game-server
  Health:
    Failure Threshold:      3
    Initial Delay Seconds:  5
    Period Seconds:         5
  Ports:
    Container Port:  7654
    Host Port:       7190
    Name:            default
    Port Policy:     Dynamic
    Protocol:        UDP
  Scheduling:        Packed
  Template:
    Metadata:
      Creation Timestamp:  <nil>
    Spec:
      Containers:
        Image:  us-docker.pkg.dev/codeblind/examples/simple-server:0.27
        Name:   simple-game-server
        Resources:
          Limits:
            Cpu:     20m
            Memory:  32Mi
          Requests:
            Cpu:     20m
            Memory:  32Mi
Status:
  Address:    35.233.183.43
  Node Name:  agones
  Ports:
    Name:  default
    Port:  7190
  State:   Ready
Events:
  Type    Reason          Age   From                   Message
  ----    ------          ----  ----                   -------
  Normal  PortAllocation  34s   gameserver-controller  Port allocated
  Normal  Creating        34s   gameserver-controller  Pod simple-game-server-7pjrq created
  Normal  Scheduled       34s   gameserver-controller  Address and port populated
  Normal  Ready           27s   gameserver-controller  SDK.Ready() executed

If you look towards the bottom, you can see there is a Status > State value. We are waiting for it to move to Ready, which means that the game server is ready to accept connections.

You might also be interested to see the Events section, which outlines when various lifecycle events of the GameServer occur. We can also see when the GameServer is ready on the event stream as well - at which time the Status > Address and Status > Ports > Port have also been populated, letting us know what IP and port our client can now connect to!

Let’s retrieve the IP address and the allocated port of your Game Server :

kubectl get gs

This should output your Game Server IP address and ports, eg:

NAME                       STATE   ADDRESS         PORT   NODE     AGE
simple-game-server-7pjrq   Ready   35.233.183.43   7190   agones   4m

3. Connect to the GameServer

You can now communicate with the Game Server :

nc -u {IP} {PORT}
Hello World !
ACK: Hello World !
EXIT

You can finally type EXIT which tells the SDK to run the Shutdown command, and therefore shuts down the GameServer.

If you run kubectl describe gameserver again - either the GameServer will be gone completely, or it will be in Shutdown state, on the way to being deleted.

Next Step

If you want to use your own GameServer container make sure you have properly integrated the Code Blind SDK.

4.2 - Quickstart: Create a Game Server Fleet

Prerequisites

The following prerequisites are required to create a GameServer:

1. A Kubernetes cluster with the UDP port range 7000-8000 open on each node.
2. Code Blind controller installed in the targeted cluster
3. kubectl properly configured
4. Netcat which is already installed on most Linux/macOS distributions, for windows you can use WSL.

If you don’t have a Kubernetes cluster you can follow these instructions to create a cluster on Google Kubernetes Engine (GKE), Minikube or Azure Kubernetes Service (AKS), and install Code Blind.

For the purpose of this guide we’re going to use the simple-game-server example as the GameServer container. This example is a very simple UDP server written in Go. Don’t hesitate to look at the code of this example for more information.

While not required, you may wish to go through the Create a Game Server quickstart before this one.

Objectives

• Create a Fleet in Kubernetes using an Code Blind custom resource.
• Scale the Fleet up from its initial configuration.
• Request a GameServer allocation from the Fleet to play on.
• Connect to the allocated GameServer.
• Deploy a new GameServer configuration to the Fleet.

1. Create a Fleet

Let’s create a Fleet using the following command:

kubectl apply -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/fleet.yaml

You should see a successful output similar to this :

fleet.agones.dev/simple-game-server created

This has created a Fleet record inside Kubernetes, which in turn creates two warm GameServers that are available to be allocated for a game session.

kubectl get fleet

It should look something like this:

NAME                 SCHEDULING   DESIRED   CURRENT   ALLOCATED   READY     AGE
simple-game-server   Packed       2         3         0           2         9m

You can also see the GameServers that have been created by the Fleet by running kubectl get gameservers, the GameServer will be prefixed by simple-game-server.

NAME                             STATE     ADDRESS            PORT   NODE      AGE
simple-game-server-llg4x-rx6rc   Ready     192.168.122.205    7752   minikube   9m
simple-game-server-llg4x-v6g2r   Ready     192.168.122.205    7623   minikube   9m

For the full details of the YAML file head to the Fleet Specification Guide

2. Fetch the Fleet status

Let’s wait for the two GameServers to become ready.

watch kubectl describe fleet simple-game-server

Name:         simple-game-server
Namespace:    default
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"agones.dev/v1","kind":"Fleet","metadata":{"annotations":{},"name":"simple-game-server","namespace":"default"},"spec":{"replicas":2,...
API Version:  agones.dev/v1
Kind:         Fleet
Metadata:
  Cluster Name:
  Creation Timestamp:  2018-07-01T18:55:35Z
  Generation:          1
  Resource Version:    24685
  Self Link:           /apis/agones.dev/v1/namespaces/default/fleets/simple-game-server
  UID:                 56710a91-7d60-11e8-b2dd-08002703ef08
Spec:
  Replicas:  2
  Strategy:
    Rolling Update:
      Max Surge:        25%
      Max Unavailable:  25%
    Type:               RollingUpdate
  Template:
    Metadata:
      Creation Timestamp:  <nil>
    Spec:
      Health:
      Ports:
        Container Port:  7654
        Name:            default
        Port Policy:     Dynamic
      Template:
        Metadata:
          Creation Timestamp:  <nil>
        Spec:
          Containers:
            Image:  us-docker.pkg.dev/codeblind/examples/simple-server:0.27
            Name:   simple-game-server
            Resources:
Status:
  Allocated Replicas:  0
  Ready Replicas:      2
  Replicas:            2
Events:
  Type    Reason                 Age   From              Message
  ----    ------                 ----  ----              -------
  Normal  CreatingGameServerSet  13s   fleet-controller  Created GameServerSet simple-game-server-wlqnd

If you look towards the bottom, you can see there is a section of Status > Ready Replicas which will tell you how many GameServers are currently in a Ready state. After a short period, there should be 2 Ready Replicas.

3. Scale up the Fleet

Let’s scale up the Fleet from 2 replicates to 5.

Run kubectl scale fleet simple-game-server --replicas=5 to change Replicas count from 2 to 5.

If we now run kubectl get gameservers we should see 5 GameServers prefixed by simple-game-server.

NAME                             STATE    ADDRESS           PORT    NODE       AGE
simple-game-server-sdhzn-kcmh6   Ready    192.168.122.205   7191    minikube   52m
simple-game-server-sdhzn-pdpk5   Ready    192.168.122.205   7752    minikube   53m
simple-game-server-sdhzn-r4d6x   Ready    192.168.122.205   7623    minikube   52m
simple-game-server-sdhzn-wng5k   Ready    192.168.122.205   7709    minikube   53m
simple-game-server-sdhzn-wnhsw   Ready    192.168.122.205   7478    minikube   52m

4. Allocate a Game Server from the Fleet

Since we have a fleet of warm gameservers, we need a way to request one of them for usage, and mark that it has players accessing it (and therefore, it should not be deleted until they are finished with it).

We can do the allocation of a GameServer for usage through a GameServerAllocation, which will both return to us the details of a GameServer (assuming one is available), and also move it to the Allocated state, which demarcates that it has players on it, and should not be removed until SDK.Shutdown() is called, or it is manually deleted.

It is worth noting that there is nothing specific that ties a GameServerAllocation to a fleet. A GameServerAllocation uses a label selector to determine what group of GameServers it will attempt to allocate out of. That being said, a Fleet and GameServerAllocation are often used in conjunction.

This example uses the label selector to specifically target the simple-game-server fleet that we just created.

kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/gameserverallocation.yaml -o yaml

For the full details of the YAML file head to the GameServerAllocation Specification Guide

You should get back a response that looks like the following:

apiVersion: allocation.agones.dev/v1
kind: GameServerAllocation
metadata:
  creationTimestamp: 2019-02-19T02:13:12Z
  name: simple-game-server-dph9b-hfk24
  namespace: default
spec:
  metadata: {}
  required:
    matchLabels:
      agones.dev/fleet: simple-game-server
  scheduling: Packed
status:
  address: 192.168.122.152
  gameServerName: simple-game-server-dph9b-hfk24
  nodeName: minikube
  ports:
  - name: default
    port: 7714
  state: Allocated

If you look at the status section, there are several things to take note of. The state value will tell if a GameServer was allocated or not. If a GameServer could not be found, this will be set to UnAllocated. If there are too many concurrent requests overwhelmed the system, state will be set to Contention even though there are available GameServers.

However, we see that the status.state value was set to Allocated. This means you have been successfully allocated a GameServer out of the fleet, and you can now connect your players to it!

You can see various immutable details of the GameServer in the status - the address, ports and the name of the GameServer, in case you want to use it to retrieve more details.

We can also check to see how many GameServers you have Allocated vs Ready with the following command (“gs” is shorthand for “gameserver”).

kubectl get gs

This will get you a list of all the current GameServers and their Status.State.

NAME                             STATE       ADDRESS           PORT   NODE      AGE
simple-game-server-sdhzn-kcmh6   Ready       192.168.122.205   7191   minikube  52m
simple-game-server-sdhzn-pdpk5   Ready       192.168.122.205   7752   minikube  53m
simple-game-server-sdhzn-r4d6x   Allocated   192.168.122.205   7623   minikube  52m
simple-game-server-sdhzn-wng5k   Ready       192.168.122.205   7709   minikube  53m
simple-game-server-sdhzn-wnhsw   Ready       192.168.122.205   7478   minikube  52m

A handy trick for checking to see how many GameServers you have Allocated vs Ready, run the following:

kubectl get gs

This will get you a list of all the current GameServers and their Status > State.

NAME                             STATE       ADDRESS          PORT   NODE        AGE
simple-game-server-tfqn7-c9tqz   Ready       192.168.39.150   7136   minikube    52m
simple-game-server-tfqn7-g8fhq   Allocated   192.168.39.150   7148   minikube    53m
simple-game-server-tfqn7-p8wnl   Ready       192.168.39.150   7453   minikube    52m
simple-game-server-tfqn7-t6bwp   Ready       192.168.39.150   7228   minikube    53m
simple-game-server-tfqn7-wkb7b   Ready       192.168.39.150   7226   minikube    52m

5. Scale down the Fleet

Not only can we scale our fleet up, but we can scale it down as well.

The nice thing about Code Blind is that it is smart enough to know when GameServers have been moved to Allocated and will automatically leave them running on scale down – as we assume that players are playing on this game server, and we shouldn’t disconnect them!

Let’s scale down our Fleet to 0 (yep! you can do that!), and watch what happens.

Run kubectl scale fleet simple-game-server --replicas=0 to change Replicas count from 5 to 0.

It may take a moment for all the GameServers to shut down, so let’s watch them all and see what happens:

watch kubectl get gs

Eventually, one by one they will be removed from the list, and you should simply see:

NAME                             STATUS      ADDRESS          PORT    NODE       AGE
simple-game-server-tfqn7-g8fhq   Allocated   192.168.39.150   7148    minikube   55m

That lone Allocated GameServer is left all alone, but still running!

If you would like, try editing the Fleet configuration replicas field and watch the list of GameServers grow and shrink.

6. Connect to the GameServer

Since we’ve only got one allocation, we’ll just grab the details of the IP and port of the only allocated GameServer:

kubectl get gameservers | grep Allocated | awk '{print $3":"$4 }'

This should output your Game Server IP address and port. (eg 10.130.65.208:7936)

You can now communicate with the GameServer:

nc -u {IP} {PORT}
Hello World !
ACK: Hello World !
EXIT

You can finally type EXIT which tells the SDK to run the Shutdown command, and therefore shuts down the GameServer.

If you run kubectl describe gs | grep State again - either the GameServer will be replaced with a new, Ready GameServer , or it will be in Shutdown state, on the way to being deleted.

Since we are running a Fleet, Code Blind will always do it’s best to ensure there are always the configured number of GameServers in the pool in either a Ready or Allocated state.

7. Deploy a new version of the GameServer on the Fleet

We can also change the configuration of the GameServer of the running Fleet, and have the changes roll out, without interrupting the currently Allocated GameServers.

Let’s take this for a spin! Run kubectl scale fleet simple-game-server --replicas=5 to return Replicas count back to 5.

Let’s also allocate ourselves a GameServer:

kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/gameserverallocation.yaml -o yaml

We should now have four Ready GameServers and one Allocated.

We can check this by running kubectl get gs.

NAME                             STATE       ADDRESS          PORT   NODE       AGE
simple-game-server-tfqn7-c9tz7   Ready       192.168.39.150   7136   minikube   5m
simple-game-server-tfqn7-g8fhq   Allocated   192.168.39.150   7148   minikube   5m
simple-game-server-tfqn7-n0wnl   Ready       192.168.39.150   7453   minikube   5m
simple-game-server-tfqn7-hiiwp   Ready       192.168.39.150   7228   minikube   5m
simple-game-server-tfqn7-w8z7b   Ready       192.168.39.150   7226   minikube   5m

In production, we’d likely be changing a containers > image configuration to update our Fleet to run a new game server process, but to make this example simple, change containerPort from 7654 to 6000.

Run kubectl edit fleet simple-game-server, and make the necessary changes, and then save and exit your editor.

This will start the deployment of a new set of GameServers running with a Container Port of 6000.

Run kubectl describe gs | grep "Container Port" until you can see that there is one with a containerPort of 7654, which is the Allocated GameServer, and four instances with a containerPort of 6000 which is the new configuration. You can also run kubectl get gs and look at the Age column to see that one GameServer is much older than the other four.

You have now deployed a new version of your game!

Next Steps

• Have a look at the GameServerAllocation specification, and see how the extra functionality can enable smoke testing, server information communication, and more.
• You can now create a fleet autoscaler to automatically resize your fleet based on the actual usage. See Create a Fleet Autoscaler.
• Have a look at the GameServer Integration Patterns, to give you a set of examples on how all the pieces fit together with your matchmaker and other systems.
• Or if you want to try to use your own GameServer container make sure you have properly integrated the Code Blind SDK.
• If you would like to learn how to programmatically allocate a Game Server from the fleet, see how to Access Code Blind via the Kubernetes API or alternatively use the Allocator Service, depending on your needs.

4.3 - Quickstart: Create a Fleet Autoscaler

Prerequisites

It is assumed that you have followed the instructions to Create a Game Server Fleet and you have a running fleet of game servers.

Objectives

• Create a Fleet Autoscaler in Kubernetes using Code Blind custom resource.
• Watch the Fleet scale up when allocating GameServers
• Watch the Fleet scale down when shutting down allocated GameServers
• Edit the autoscaler specification to apply live changes

1. Create a Fleet Autoscaler

Let’s create a Fleet Autoscaler using the following command :

kubectl apply -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/fleetautoscaler.yaml

You should see a successful output similar to this :

fleetautoscaler.autoscaling.agones.dev/simple-game-server-autoscaler created

This has created a FleetAutoscaler record inside Kubernetes.

2. See the autoscaler status.

kubectl describe fleetautoscaler simple-game-server-autoscaler

It should look something like this:

Name:         simple-game-server-autoscaler
Namespace:    default
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"au
toscaling.agones.dev/v1","kind":"FleetAutoscaler","metadata":{"annotations":{},
"name":"simple-game-server-autoscaler","namespace":"default"},...
API Version:  autoscaling.agones.dev/v1
Kind:         FleetAutoscaler
Metadata:
  Cluster Name:
  Creation Timestamp:  2018-10-02T15:19:58Z
  Generation:          1
  Owner References:
    API Version:           autoscaling.agones.dev/v1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Fleet
    Name:                  simple-game-server
    UID:                   9960762e-c656-11e8-933e-fa163e07a1d4
  Resource Version:        6123197
  Self Link:               /apis/autoscaling.agones.dev/v1/namespaces/default/fleetautoscalers/simple-game-server-autoscaler
  UID:                     9fd0efa1-c656-11e8-933e-fa163e07a1d4
Spec:
  Fleet Name:  simple-game-server
  Policy:
    Buffer:
      Buffer Size:   2
      Max Replicas:  10
      Min Replicas:  2
    Type:            Buffer
Status:
  Able To Scale:     true
  Current Replicas:  2
  Desired Replicas:  2
  Last Scale Time:   <nil>
  Scaling Limited:   false
Events:              <none>

You can see the status (able to scale, not limited), the last time the fleet was scaled (nil for never) and the current and desired fleet size.

The autoscaler works by changing the desired size, and the fleet creates/deletes game server instances to achieve that number. The convergence is achieved in time, which is usually measured in seconds.

3. Allocate a Game Server from the Fleet

If you’re interested in more details for game server allocation, you should consult the Create a Game Server Fleet page. In here we are only interested in triggering allocations to see the autoscaler in action.

kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/gameserverallocation.yaml -o yaml

You should get in return the allocated game server details, which should end with something like:

status:
  address: 34.94.118.237
  gameServerName: simple-game-server-v6jwb-6bzkz
  nodeName: gke-test-cluster-default-f11755a7-5km3
  ports:
  - name: default
    port: 7832
  state: Allocated

Note the address and port, you might need them later to connect to the server.

4. See the autoscaler in action

Now let’s wait a few seconds to allow the autoscaler to detect the change in the fleet and check again its status

kubectl describe fleetautoscaler simple-game-server-autoscaler

The last part should look something like this:

Spec:
  Fleet Name:  simple-game-server
  Policy:
    Buffer:
      Buffer Size:   2
      Max Replicas:  10
      Min Replicas:  2
    Type:            Buffer
Status:
  Able To Scale:     true
  Current Replicas:  3
  Desired Replicas:  3
  Last Scale Time:   2018-10-02T16:00:02Z
  Scaling Limited:   false
Events:
  Type    Reason            Age   From                        Message
  ----    ------            ----  ----                        -------
  Normal  AutoScalingFleet  2m    fleetautoscaler-controller  Scaling fleet simple-game-server from 2 to 3

You can see that the fleet size has increased, the autoscaler having compensated for the allocated instance. Last Scale Time has been updated, and a scaling event has been logged.

Double-check the actual number of game server instances and status by running

kubectl get gs

This will get you a list of all the current GameServers and their Status > State.

NAME                             STATE       ADDRESS        PORT     NODE        AGE
simple-game-server-mzhrl-hz8wk   Allocated   10.30.64.99    7131     minikube    5m
simple-game-server-mzhrl-k6jg5   Ready       10.30.64.100   7243     minikube    5m  
simple-game-server-mzhrl-n2sk2   Ready       10.30.64.168   7658     minikube    5m

5. Shut the allocated instance down

Since we’ve only got one allocation, we’ll just grab the details of the IP and port of the only allocated GameServer:

kubectl get gameservers | grep Allocated | awk '{print $3":"$4 }'

This should output your Game Server IP address and port. (eg 10.130.65.208:7936)

You can now communicate with the GameServer:

nc -u {IP} {PORT}
Hello World !
ACK: Hello World !
EXIT

You can finally type EXIT which tells the SDK to run the Shutdown command, and therefore shuts down the GameServer.

6. See the fleet scaling down

Now let’s wait a few seconds to allow the autoscaler to detect the change in the fleet and check again its status

kubectl describe fleetautoscaler simple-game-server-autoscaler

It should look something like this:

Spec:
  Fleet Name:  simple-game-server
  Policy:
    Buffer:
      Buffer Size:   2
      Max Replicas:  10
      Min Replicas:  2
    Type:            Buffer
Status:
  Able To Scale:     true
  Current Replicas:  3
  Desired Replicas:  2
  Last Scale Time:   2018-10-02T16:09:02Z
  Scaling Limited:   false
Events:
  Type    Reason            Age   From                        Message
  ----    ------            ----  ----                        -------
  Normal  AutoScalingFleet  9m    fleetautoscaler-controller  Scaling fleet simple-game-server from 2 to 3
  Normal  AutoScalingFleet  45s   fleetautoscaler-controller  Scaling fleet simple-game-server from 3 to 2

You can see that the fleet size has decreased, the autoscaler adjusting to game server instance being de-allocated, the Last Scale Time and the events have been updated. Note that simple-game-server game server instance you just closed earlier might stay a bit in ‘Unhealthy’ state (and its pod in ‘Terminating’ until it gets removed.

Double-check the actual number of game server instances and status by running

kubectl get gs

This will get you a list of all the current GameServers and their Status > State.

NAME                             STATE     ADDRESS        PORT    NODE       AGE
simple-game-server-mzhrl-k6jg5   Ready     10.30.64.100   7243    minikube   5m
simple-game-server-mzhrl-t7944   Ready     10.30.64.168   7561    minikube   5m

7. Change autoscaling parameters

We can also change the configuration of the FleetAutoscaler of the running Fleet, and have the changes applied live, without interruptions of service.

Run kubectl edit fleetautoscaler simple-game-server-autoscaler and set the bufferSize field to 5.

Let’s look at the list of game servers again. Run watch kubectl get gs until you can see that are 5 ready server instances:

NAME                             STATE     ADDRESS        PORT    NODE         AGE
simple-game-server-mzhrl-7jpkp   Ready     10.30.64.100   7019    minikube     5m
simple-game-server-mzhrl-czt8v   Ready     10.30.64.168   7556    minikube     5m
simple-game-server-mzhrl-k6jg5   Ready     10.30.64.100   7243    minikube     5m
simple-game-server-mzhrl-nb8h2   Ready     10.30.64.168   7357    minikube     5m
simple-game-server-mzhrl-qspb6   Ready     10.30.64.99    7859    minikube     5m
simple-game-server-mzhrl-zg9rq   Ready     10.30.64.99    7745    minikube     5m

Next Steps

Read the advanced Scheduling and Autoscaling guide, for more details on autoscaling.

If you want to use your own GameServer container make sure you have properly integrated the Code Blind SDK.

4.4 - Quickstart: Create a Fleet Autoscaler with Webhook Policy

In some cases, your game servers may need to use custom logic for scaling your fleet that is more complex than what can be expressed using the Buffer policy in the fleetautoscaler. This guide shows how you can extend Code Blind with an autoscaler webhook to implement a custom autoscaling policy.

When you use an autoscaler webhook the logic computing the number of target replicas is delegated to an external HTTP/S endpoint, such as one provided by a Kubernetes deployment and service in the same cluster (as shown in the examples below). The fleetautoscaler will send a request to the webhook autoscaler’s /scale endpoint every sync period (currently 30s) with a JSON body, and scale the target fleet based on the data that is returned.

Chapter 1 Configuring HTTP fleetautoscaler webhook

Prerequisites

It is assumed that you have completed the instructions to Create a Game Server Fleet and have a running fleet of game servers.

Objectives

• Run a fleet
• Deploy the Webhook Pod and service for autoscaling
• Create a Fleet Autoscaler with Webhook policy type in Kubernetes using Code Blind custom resource
• Watch the Fleet scales up when allocating GameServers
• Watch the Fleet scales down after GameServer shutdown

1. Deploy the fleet

Run a fleet in a cluster:

kubectl apply -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/fleet.yaml

2. Deploy a Webhook service for autoscaling

In this step we would deploy an example webhook that will control the size of the fleet based on allocated gameservers portion in a fleet. You can see the source code for this example webhook server here. The fleetautoscaler would trigger this endpoint every 30 seconds. More details could be found also here. We need to create a pod which will handle HTTP requests with json payload FleetAutoscaleReview and return back it with FleetAutoscaleResponse populated.

The Scale flag and Replicas values returned in the FleetAutoscaleResponse tells the FleetAutoscaler what target size the backing Fleet should be scaled up or down to. If Scale is false - no scaling occurs.

Run next command to create a service and a Webhook pod in a cluster:

kubectl apply -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/autoscaler-webhook/autoscaler-service.yaml

To check that it is running and liveness probe is fine:

kubectl describe pod autoscaler-webhook

Name:           autoscaler-webhook-86944884c4-sdtqh
Namespace:      default
Node:           gke-test-cluster-default-1c5dec79-h0tq/10.138.0.2
...
Status:         Running

3. Create a Fleet Autoscaler

Let’s create a Fleet Autoscaler using the following command:

kubectl apply -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/webhookfleetautoscaler.yaml

You should see a successful output similar to this:

fleetautoscaler.autoscaling.agones.dev "webhook-fleet-autoscaler" created

This has created a FleetAutoscaler record inside Kubernetes. It has the link to Webhook service we deployed above.

4. See the fleet and autoscaler status.

In order to track the list of gameservers which run in your fleet you can run this command in a separate terminal tab:

 watch "kubectl get gs -n default"

In order to get autoscaler status use the following command:

kubectl describe fleetautoscaler webhook-fleet-autoscaler

It should look something like this:

Name:         webhook-fleet-autoscaler
Namespace:    default
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration={"apiVersion":
"autoscaling.agones.dev/v1","kind":"FleetAutoscaler","metadata":{"annotations"
:{},"name":"webhook-fleet-autoscaler","namespace":"default...
API Version:  autoscaling.agones.dev/v1
Kind:         FleetAutoscaler
etadata:
  Cluster Name:
  Creation Timestamp:  2018-12-22T12:52:23Z
  Generation:          1
  Resource Version:    2274579
  Self Link:           /apis/autoscaling.agones.dev/v1/namespaces/default/fleet
autoscalers/webhook-fleet-autoscaler
  UID:                 6d03eae4-05e8-11e9-84c2-42010a8a01c9
Spec:
  Fleet Name:  simple-game-server
  Policy:
    Type:  Webhook
    Webhook:
      Service:
        Name:       autoscaler-webhook-service
        Namespace:  default
        Path:       scale
      URL:
Status:
  Able To Scale:     true
  Current Replicas:  2
  Desired Replicas:  2
  Last Scale Time:   <nil>
  Scaling Limited:   false
Events:              <none>

You can see the status (able to scale, not limited), the last time the fleet was scaled (nil for never), current and desired fleet size.

The autoscaler makes a query to a webhoook service deployed on step 1 and on response changing the target Replica size, and the fleet creates/deletes game server instances to achieve that number. The convergence is achieved in time, which is usually measured in seconds.

5. Allocate Game Servers from the Fleet to trigger scale up

If you’re interested in more details for game server allocation, you should consult the Create a Game Server Fleet page. Here we only interested in triggering allocations to see the autoscaler in action.

kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/gameserverallocation.yaml -o yaml

You should get in return the allocated game server details, which should end with something like:

status:
  address: 34.94.118.237
  gameServerName: simple-game-server-v6jwb-6bzkz
  nodeName: gke-test-cluster-default-f11755a7-5km3
  ports:
  - name: default
    port: 7832
  state: Allocated

Note the address and port, you might need them later to connect to the server.

Run the kubectl command one more time so that we have both servers allocated:

kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/gameserverallocation.yaml -o yaml

6. Check new Autoscaler and Fleet status

Now let’s wait a few seconds to allow the autoscaler to detect the change in the fleet and check again its status

kubectl describe fleetautoscaler webhook-fleet-autoscaler

The last part should look similar to this:

Spec:
  Fleet Name:  simple-game-server
  Policy:
    Type:  Webhook
    Webhook:
      Service:
        Name:       autoscaler-webhook-service
        Namespace:  default
        Path:       scale
      URL:
Status:
  Able To Scale:     true
  Current Replicas:  4
  Desired Replicas:  4
  Last Scale Time:   2018-12-22T12:53:47Z
  Scaling Limited:   false
Events:
  Type    Reason            Age   From                        Message
  ----    ------            ----  ----                        -------
  Normal  AutoScalingFleet  35s   fleetautoscaler-controller  Scaling fleet simple-game-server from 2 to 4

You can see that the fleet size has increased in particular case doubled to 4 gameservers (based on our custom logic in our webhook), the autoscaler having compensated for the two allocated instances. Last Scale Time has been updated and a scaling event has been logged.

Double-check the actual number of game server instances and status by running:

 kubectl get gs -n default

This will get you a list of all the current GameServers and their Status > State.

NAME                     STATE       ADDRESS         PORT     NODE        AGE
simple-game-server-dmkp4-8pkk2   Ready       35.247.13.175   7386     minikube     5m
simple-game-server-dmkp4-b7x87   Allocated   35.247.13.175   7219     minikube     5m
simple-game-server-dmkp4-r4qtt   Allocated   35.247.13.175   7220     minikube     5m
simple-game-server-dmkp4-rsr6n   Ready       35.247.13.175   7297     minikube     5m

7. Check downscaling using Webhook Autoscaler policy

Based on our custom webhook deployed earlier, if the fraction of allocated replicas in whole Replicas count would be less than threshold (0.3) then the fleet would scale down by scaleFactor, in our example by 2.

Note that the example webhook server has a limitation that it would not decrease fleet replica count under minReplicasCount, which is equal to 2.

We need to run EXIT command on one gameserver (Use IP address and port of the allocated gameserver from the previous step) in order to decrease the number of allocated gameservers in a fleet (<0.3).

nc -u 35.247.13.175 7220
EXIT

Server would be in shutdown state. Wait about 30 seconds. Then you should see scaling down event in the output of next command:

kubectl describe fleetautoscaler webhook-fleet-autoscaler

You should see these lines in events:

  Normal   AutoScalingFleet  11m                fleetautoscaler-controller  Scaling fleet simple-game-server from 2 to 4
  Normal   AutoScalingFleet  1m                 fleetautoscaler-controller  Scaling fleet simple-game-server from 4 to 2

And get gameservers command output:

kubectl get gs -n default

NAME                             STATUS      ADDRESS          PORT     NODE       AGE
simple-game-server-884fg-6q5sk   Ready       35.247.117.202   7373     minikube   5m
simple-game-server-884fg-b7l58   Allocated   35.247.117.202   7766     minikube   5m

8. Cleanup

You can delete the autoscaler service and associated resources with the following commands.

kubectl delete -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/autoscaler-webhook/autoscaler-service.yaml

Removing the fleet:

kubectl delete -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/fleet.yaml

Chapter 2 Configuring HTTPS fleetautoscaler webhook with CA Bundle

Objectives

Using TLS and a certificate authority (CA) bundle we can establish trusted communication between Fleetautoscaler and an HTTPS server running the autoscaling webhook that controls the size of the fleet (Replicas count). The certificate of the autoscaling webhook must be signed by the CA provided in fleetautoscaler yaml configuration file. Using TLS eliminates the possibility of a man-in-the-middle attack between the fleetautoscaler and the autoscaling webhook.

1. Deploy the fleet

Run a fleet in a cluster:

kubectl apply -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/fleet.yaml

2. Create X509 Root and Webhook certificates

The procedure of generating a Self-signed CA certificate is as follows:

The first step is to create the private root key:

openssl genrsa -out rootCA.key 2048

The next step is to self-sign this certificate:

openssl req -x509 -new -nodes -key rootCA.key -sha256 -days 1024 -out rootCA.pem

This will start an interactive script that will ask you for various bits of information. Fill it out as you see fit.

Every webhook that you wish to install a trusted certificate will need to go through this process. First, just like with the root CA step, you’ll need to create a private key (different from the root CA):

openssl genrsa -out webhook.key 2048

Next create configuration file cert.conf for the certificate signing request:

[req]
distinguished_name = req_distinguished_name
req_extensions = v3_req
prompt = no
[req_distinguished_name]
CN = autoscaler-tls-service.default.svc
[v3_req]
keyUsage = digitalSignature
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1 = autoscaler-tls-service.default.svc

Generate the certificate signing request, use valid hostname which in this case will be autoscaler-tls-service.default.svc as Common Name (eg, fully qualified host name) as well as DNS.1 in the alt_names section of the config file.

Check the Kubernetes documentation to see how Services get assigned DNS entries.

openssl req -new -out webhook.csr -key webhook.key -config cert.conf

Once that’s done, you’ll sign the CSR, which requires the CA root key:

openssl x509 -req -in webhook.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out webhook.crt -days 500 -sha256 -extfile cert.conf -extensions v3_req

This would generate webhook.crt certificate

Add secret which later would be mounted to autoscaler-webhook-tls pod.

kubectl create secret tls autoscalersecret --cert=webhook.crt --key=webhook.key

You need to put Base64-encoded string into caBundle field in your fleetautoscaler yaml configuration:

base64 -i ./rootCA.pem

Copy the output of the command above and replace the caBundle field in your text editor (say vim) with the new value:

wget https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/webhookfleetautoscalertls.yaml
vim ./webhookfleetautoscalertls.yaml

3. Deploy a Webhook service for autoscaling

Run next command to create a service and a Webhook pod in a cluster:

kubectl apply -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/autoscaler-webhook/autoscaler-service-tls.yaml

To check that it is running and liveness probe is fine:

kubectl describe pod autoscaler-webhook-tls

Wait for the Running status results:

Name:               autoscaler-webhook-tls-f74c9bff7-ssrsc
Namespace:          default
...
Status:         Running

4. Create a Fleet Autoscaler

Let’s create a Fleet Autoscaler using the following command (caBundle should be set properly on Step 2):

kubectl apply -f ./webhookfleetautoscalertls.yaml

5. See the fleet and autoscaler status.

In order to track the list of gameservers which run in your fleet you can run this command in a separate terminal tab:

 watch "kubectl get gs -n default"

6. Allocate two Game Servers from the Fleet to trigger scale up

If you’re interested in more details for game server allocation, you should consult the Create a Game Server Fleet page. Here we only interested in triggering allocations to see the autoscaler in action.

for i in {0..1} ; do kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/gameserverallocation.yaml -o yaml ; done

7. Check new Autoscaler and Fleet status

Now let’s wait a few seconds to allow the autoscaler to detect the change in the fleet and check again its status

kubectl describe fleetautoscaler  webhook-fleetautoscaler-tls

The last part should look similar to this:

Spec:
  Fleet Name:  simple-game-server
  Policy:
    Type:  Webhook
    Webhook:
      Ca Bundle:  LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN1RENDQWFBQ0NRQ29kcEFNbTlTd0pqQU5CZ2txaGtpRzl3MEJBUXNGQURBZU1Rc3dDUVlEVlFRR0V3SlYKVXpFUE1BMEdBMVVFQ3d3R1FXZHZibVZ6TUI0WERURTVNREV3TkRFeE5URTBORm9YRFRJeE1UQXlOREV4TlRFMApORm93SGpFTE1Ba0dBMVVFQmhNQ1ZWTXhEekFOQmdOVkJBc01Ca0ZuYjI1bGN6Q0NBU0l3RFFZSktvWklodmNOCkFRRUJCUUFEZ2dFUEFEQ0NBUW9DZ2dFQkFOQ0h5dndDOTZwZDlTdkFhMUIvRWg2ekcxeDBLS1dPaVhtNzhJcngKKzZ5WHd5YVpsMVo1cVExbUZoOThMSGVZUmQwWVgzRTJnelZ5bFpvUlUra1ZESzRUc0VzV0tNUFVpdVo0MUVrdApwbythbEN6alAyaXZzRGZaOGEvdnByL3dZZ2FrWGtWalBUaGpKUk9xTnFIdWROMjZVcUFJYnNOTVpoUkxkOVFFCnFLSjRPNmFHNVMxTVNqZFRGVHFlbHJiZitDcXNKaHltZEIzZmxGRUVvdXExSmoxS0RoQjRXWlNTbS9VSnpCNkcKNHUzY3BlQm1jTFVRR202ZlFHb2JFQSt5SlpMaEVXcXBrd3ZVZ2dCNmRzWE8xZFNIZXhhZmlDOUVUWGxVdFRhZwo1U2JOeTVoYWRWUVV3Z253U0J2djR2R0t1UUxXcWdXc0JyazB5Wll4Sk5Bb0V5RUNBd0VBQVRBTkJna3Foa2lHCjl3MEJBUXNGQUFPQ0FRRUFRMkgzaWJRcWYzQTNES2l1eGJISURkbll6TlZ2Z0dhRFpwaVZyM25ocm55dmxlNVgKR09hRm0rMjdRRjRWV29FMzZDTGhYZHpEWlM4bEpIY09YUW5KOU83Y2pPYzkxVmh1S2NmSHgwS09hU1oweVNrVAp2bEtXazlBNFdoNGE0QXFZSlc3Z3BUVHR1UFpydnc4VGsvbjFaWEZOYVdBeDd5RU5OdVdiODhoNGRBRDVaTzRzCkc5SHJIdlpuTTNXQzFBUXA0Q3laRjVyQ1I2dkVFOWRkUmlKb3IzM3pLZTRoRkJvN0JFTklZZXNzZVlxRStkcDMKK0g4TW5LODRXeDFUZ1N5Vkp5OHlMbXFpdTJ1aThjaDFIZnh0OFpjcHg3dXA2SEZLRlRsTjlBeXZUaXYxYTBYLwpEVTk1eTEwdi9oTlc0WHpuMDJHNGhrcjhzaUduSEcrUEprT3hBdz09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
      Service:    <nil>
      URL:        https://autoscaler-tls-service.default.svc:8000/scale
Events:
  Type    Reason            Age   From                        Message
  ----    ------            ----  ----                        -------
  Normal  AutoScalingFleet  5s   fleetautoscaler-controller  Scaling fleet simple-game-server from 2 to 4

You can see that the fleet size has increased in particular case doubled to 4 gameservers (based on our custom logic in our webhook), the autoscaler having compensated for the two allocated instances. Last Scale Time has been updated and a scaling event has been logged.

Double-check the actual number of game server instances and status by running:

 kubectl get gs -n default

This will get you a list of all the current GameServers and their Status > State.

NAME                     STATE       ADDRESS         PORT      NODE      AGE
simple-game-server-njmr7-2t4nx   Ready       35.203.159.68   7330      minikube   1m
simple-game-server-njmr7-65rp6   Allocated   35.203.159.68   7294      minikube   4m

8. Cleanup

You can delete the autoscaler service and associated resources with the following commands.

kubectl delete -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/autoscaler-webhook/autoscaler-service-tls.yaml

Removing x509 key secret:

kubectl delete secret autoscalersecret

Removing the fleet:

kubectl delete -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/fleet.yaml

Comments

Note that secure communication has been established and we can trust that communication between the fleetautoscaler and the autoscaling webhook. If you need to run the autoscaling webhook outside of the Kubernetes cluster, you can use another root certificate authority as long as you put it into the caBundle parameter in fleetautoscaler configuration (in pem format, base64-encoded).

Troubleshooting Guide

If you run into problems with the configuration of your fleetautoscaler and webhook service the easiest way to debug them is to run:

kubectl describe fleetautoscaler <FleetAutoScalerName>

and inspect the events at the bottom of the output.

Common error messages.

If you have configured the wrong service Path for the FleetAutoscaler you will see a message like

Error calculating desired fleet size on FleetAutoscaler simple-fleet-r7fdv-autoscaler. Error: bad status code 404 from the server: https://autoscaler-tls-service.default.svc:8000/scale

If you are using a hostname other than autoscaler-tls-service.default.svc as the Common Name (eg, fully qualified host name) when creating a certificate using openssl tool you will see a message like

Post https://autoscaler-tls-service.default.svc:8000/scale: x509: certificate is not valid for any names, but wanted to match autoscaler-tls-service.default.svc

If you see errors like the following in autoscaler-webhook-tls pod logs:

http: TLS handshake error from 10.48.3.125:33374: remote error: tls: bad certificate

Then there could be an issue with your ./rootCA.pem.

You can repeat the process from step 2, in order to fix your certificates setup.

Next Steps

Read the advanced Scheduling and Autoscaling guide, for more details on autoscaling.

If you want to use your own GameServer container make sure you have properly integrated the Code Blind SDK.

4.5 - Quickstart: Edit a Game Server

This guide addresses Google Kubernetes Engine and Minikube. We would welcome a Pull Request to expand this to include other platforms as well.

Prerequisites

1. A Go environment
2. Docker
3. Code Blind installed on GKE or Minikube
4. kubectl properly configured

To install on GKE, follow the install instructions (if you haven’t already) at Setting up a Google Kubernetes Engine (GKE) cluster. Also complete the “Enabling creation of RBAC resources” and “Installing Code Blind” sets of instructions on the same page.

To install locally on Minikube, read Setting up a Minikube cluster. Also complete the “Enabling creation of RBAC resources” and “Installing Code Blind” sets of instructions on the same page.

Modify the code and push another new image

Modify the simple-game-server example source code

Modify the main.go file. For example:

Change the following line in main.go:

From:

respond(conn, sender, "ACK: "+txt+"\n")

To:

respond(conn, sender, "ACK: Echo says "+txt+"\n")

Build Server

Since Docker image is using Alpine Linux, the “go build” command has to include few more environment variables.

go get agones.dev/agones/pkg/sdk
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o bin/server -a -v main.go

Using Docker File

Create a new docker image

docker build -t gcr.io/[PROJECT_ID]/agones-simple-game-server:modified .

Note: you can change the image name “agones-simple-game-server” to something else.

If using GKE, push the image to GCP Registry

docker push gcr.io/[PROJECT_ID]/agones-simple-game-server:modified

Note: Review Authentication Methods for additional information regarding use of gcloud as a Docker credential helper and advanced authentication methods to the Google Container Registry.

If using Minikube, load the image into Minikube

minikube cache add gcr.io/[PROJECT_ID]/agones-agones-simple-game-server:modified

Modify gameserver.yaml

Modify the following line from gameserver.yaml to use the new configuration.

spec:
  containers:
  - name: agones-simple-game-server
    image: gcr.io/[PROJECT_ID]/agones-simple-game-server:modified

If using GKE, deploy Server to GKE

Apply the latest settings to the Kubernetes container.

gcloud config set container/cluster [CLUSTER_NAME]
gcloud container clusters get-credentials [CLUSTER_NAME]
kubectl apply -f gameserver.yaml

If using Minikube, deploy the Server to Minikube

kubectl apply -f gameserver.yaml

Check the GameServer Status

kubectl describe gameserver

Verify

Let’s retrieve the IP address and the allocated port of your Game Server:

kubectl get gs simple-game-server -o jsonpath='{.status.address}:{.status.ports[0].port}'

You can now communicate with the Game Server :

nc -u {IP} {PORT}
Hello World!
ACK:  Echo says  Hello World!
EXIT

You can finally type EXIT which tells the SDK to run the Shutdown command, and therefore shuts down the GameServer.

If you run kubectl describe gameserver again - either the GameServer will be gone completely, or it will be in Shutdown state, on the way to being deleted.

Next Steps

If you want to perform rolling updates of modified game servers, see Quickstart Create a Game Server Fleet.

5 - Guides

5.1 - Feature Stages

Supported Versions

Code Blind versions are expressed as x.y.z, where x is the major version, y is the minor version, and z is the patch version following Semantic Versioning terminology.

Code Blind Features

A feature within Code Blind can be in Alpha, Beta or Stable stage.

Feature Gates

Alpha and Beta features can be enabled or disabled through the agones.featureGates configuration option that can be found in the Helm configuration documentation.

The current set of alpha and beta feature gates:

Description of Stages

Alpha

An Alpha feature means:

• Disabled by default.
• Might be buggy. Enabling the feature may expose bugs.
• Support for this feature may be dropped at any time without notice.
• The API may change in incompatible ways in a later software release without notice.
• Recommended for use only in short-lived testing clusters, due to increased risk of bugs and lack of long-term support.

Beta

A Beta feature means:

• Enabled by default, but able to be disabled through a feature gate.
• The feature is well tested. Enabling the feature is considered safe.
• Support for the overall feature will not be dropped, though details may change.
• The schema and/or semantics of objects may change in incompatible ways in a subsequent beta or stable releases. When this happens, we will provide instructions for migrating to the next version. This may require deleting, editing, and re-creating API objects. The editing process may require some thought. This may require downtime for applications that rely on the feature.
• Recommended for only non-business-critical uses because of potential for incompatible changes in subsequent releases. If you have multiple clusters that can be upgraded independently, you may be able to relax this restriction.

Stable

A Stable feature means:

• The feature is enabled and the corresponding feature gate no longer exists.
• Stable versions of features will appear in released software for many subsequent versions.

Feature Stage Indicators

There are a variety of features with Code Blind, how can we determine what stage each feature is in?

Below are indicators for each type of functionality that can be used to determine the feature stage for a given aspect of Code Blind.

Custom Resource Definitions (CRDs)

This refers to Kubernetes resource for Code Blind, such as GameServer, Fleet and GameServerAllocation.

New CRDs

For new resources, the stage of the resource will be indicated by the apiVersion of the resource.

For example: apiVersion: "agones.dev/v1" is a stable resource, apiVersion: "agones.dev/v1beta1" is a beta stage resource, and apiVersion: "agones.dev/v1alpha1" is an alpha stage resource.

New CRD attributes

When alpha and beta attributes are added to an existing stable Code Blind CRD, we will follow the Kubernetes Adding Unstable Features to Stable Versions Guide to optimise on the least amount of breaking changes for users as attributes progress through feature stages.

alpha and beta attributes will be added to the existing CRD as optional and documented with their feature stage. Attempting to populate these alpha and beta attributes on an Code Blind CRD will return a validation error if their accompanying Feature Flag is not enabled.

alpha and beta attributes can be subject to change of name and structure, and will result in breaking changes before moving to a stable stage. These changes will be outlined in release notes and feature documentation.

Code Blind Game Server SDK

Any alpha or beta Game Server SDK functionality will be a subpackage of the sdk package. For example , functionality found in a sdk.alphav1 package should be considered at the alpha feature stage.

Only experimental functionality will be found in any alpha and beta SDK packages, and as such may change as development occurs.

As SDK features move to through feature stages towards stable, the previous version of the SDK API will remain for at least one release to enable easy migration to the more stable feature stage (i.e. from alpha -> beta, beta -> stable)

Any other SDK functionality not marked as alpha or beta is assumed to be stable.

REST & gRPC APIs

REST and gRPC API will have versioned paths where appropriate to indicate their feature stage.

For example, a REST API with a prefix of v1alpha1 is an alpha stage feature: http://api.example.com/v1alpha1/exampleaction.

Similar to the SDK, any alpha or beta gRPC functionality will be a subpackage of the main API package. For example, functionality found in a api.alphav1 package should be considered at the alpha feature stage.

5.2 - Best Practices

Overview

Running Code Blind in production takes consideration, from planning your launch to figuring out the best course of action for cluster and Code Blind upgrades. On this page, we’ve collected some general best practices. We also have cloud specific pages for:

• Google Kubernetes Engine (GKE)

If you are interested in submitting best practices for your cloud prodiver / on-prem, please contribute!

Separation of Code Blind from GameServer nodes

When running in production, Code Blind should be scheduled on a dedicated pool of nodes, distinct from where Game Servers are scheduled for better isolation and resiliency. By default Code Blind prefers to be scheduled on nodes labeled with agones.dev/agones-system=true and tolerates the node taint agones.dev/agones-system=true:NoExecute. If no dedicated nodes are available, Code Blind will run on regular nodes. See taints and tolerations for more information about Kubernetes taints and tolerations.

If you are collecting Metrics using our standard Prometheus installation, see the installation guide for instructions on configuring a separate node pool for the agones.dev/agones-metrics=true taint.

See Creating a Cluster for initial set up on your cloud provider.

Redundant Clusters

Allocate Across Clusters

Code Blind supports Multi-cluster Allocation, allowing you to allocate from a set of clusters, versus a single point of potential failure. There are several other options for multi-cluster allocation:

• Anthos Service Mesh can be used to route allocation traffic to different clusters based on arbitrary criteria. See Global Multiplayer Demo for an example where the match maker influences which cluster the allocation is routed to.
• Allocation Endpoint can be used in Cloud Run to proxy allocation requests.
• Or peruse the Third Party Examples

Spread

You should consider spreading your game servers in two ways:

• Across geographic fault domains (GCP regions, AWS availability zones, separate datacenters, etc.): This is desirable for geographic fault isolation, but also for optimizing client latency to the game server.
• Within a fault domain: Kubernetes Clusters are single points of failure. A single misconfigured RBAC rule, an overloaded Kubernetes Control Plane, etc. can prevent new game server allocations, or worse, disrupt existing sessions. Running multiple clusters within a fault domain also allows for easier upgrades.

5.2.1 - Google Kubernetes Engine Best Practices

Overview

On this page, we’ve collected several Google Kubernetes Engine (GKE) best practices.

Release Channels

Why?

We recommned using Release Channels for all GKE clusters. Using Release Channels has several advantages:

• Google automatically manages the version and upgrade cadence for your Kubernetes Control Plane and its nodes.
• Clusters on a Release Channel are allowed to use the No minor upgrades and No minor or node upgrades scope of maintenance exclusions - in other words, enrolling a cluster in a Release Channel gives you more control over node upgrades.
• Clusters enrolled in rapid channel have access to the newest Kubernetes version first. Code Blind strives to support the newest release in rapid channel to allow you to test the newest Kubernetes soon after it’s available in GKE.

What channel should I use?

We recommend the regular channel, which offers a balance between stability and freshness. See this guide for more discussion.

If you need to disallow minor version upgrades for more than 6 months, consider choosing the freshest Kubernetes version possible: Choosing the freshest version on rapid or regular will extend the amount of time before your cluster reaches end of life.

What versions are available on a given channel?

You can query the versions available across different channels using gcloud:

gcloud container get-server-config \
  --region=[COMPUTE_REGION] \
  --flatten="channels" \
  --format="yaml(channels)"

Replace the following:

• COMPUTE_REGION: the Google Cloud region where you will create the cluster.

Managing Game Server Disruption on GKE

If your game session length is less than an hour, use the eviction API to configure your game servers appropriately - see Controlling Disruption.

For sessions longer than an hour, there are currently two possible approaches to manage disruption:

• (GKE Standard/Autopilot) Blue/green deployment at the cluster level: If you are using an automated deployment process, you can:

  • create a new, green cluster within a release channel e.g. every week,
  • use maintenance exclusions to prevent node upgrades for 30d, and
  • scale the Fleet on the old, blue cluster down to 0, and
  • use multi-cluster allocation on Code Blind, which will then direct new allocations to the new green cluster (since blue has 0 desired), then
  • delete the old, blue cluster when the Fleet successfully scales down.

• (GKE Standard only) Use node pool blue/green upgrades

5.3 - Code Blind Game Server Client SDKs

Overview

The client SDKs are required for a game server to work with Code Blind.

The current supported SDKs are:

• Unreal Engine
• Unity
• C++
• C#
• Node.js
• Go
• Rust
• REST

You can also find some externally supported SDKs in our Third Party Content.

The SDKs are relatively thin wrappers around gRPC generated clients, or an implementation of the REST API (exposed via grpc-gateway), where gRPC client generation and compilation isn’t well supported.

They connect to a small process that Code Blind coordinates to run alongside the Game Server in a Kubernetes Pod. This means that more languages can be supported in the future with minimal effort (but pull requests are welcome! 😊 ).

There is also local development tooling for working against the SDK locally, without having to spin up an entire Kubernetes infrastructure.

Connecting to the SDK Server

Starting with Code Blind 1.1.0, the port that the SDK Server listens on for incoming gRPC or HTTP requests is configurable. This provides flexibility in cases where the default port conflicts with a port that is needed by the game server.

Code Blind will automatically set the following environment variables on all game server containers:

• AGONES_SDK_GRPC_PORT: The port where the gRPC server is listening (defaults to 9357)
• AGONES_SDK_HTTP_PORT: The port where the grpc-gateway is listening (defaults to 9358)

The SDKs will automatically discover and connect to the gRPC port specified in the environment variable.

If your game server requires using a REST client, it is advised to use the port from the environment variable, otherwise your game server will not be able to contact the SDK server if it is configured to use a non-default port.

Function Reference

While each of the SDKs are canonical to their languages, they all have the following functions that implement the core responsibilities of the SDK.

For language specific documentation, have a look at the respective source (linked above), and the examples.

Calling any of state changing functions mentioned below does not guarantee that GameServer Custom Resource object would actually change its state right after the call. For instance, it could be moved to the Shutdown state elsewhere (for example, when a fleet scales down), which leads to no changes in GameServer object. You can verify the result of this call by waiting for the desired state in a callback to WatchGameServer() function.

Functions which changes GameServer state or settings are:

1. Ready()
2. Shutdown()
3. SetLabel()
4. SetAnnotation()
5. Allocate()
6. Reserve()
7. Alpha().SetCapacity()
8. Alpha().PlayerConnect()
9. Alpha().PlayerDisconnect()
10. Alpha().SetCounterCount()
11. Alpha().IncrementCounter()
12. Alpha().DecrementCounter()
13. Alpha().SetCounterCapacity()
14. Alpha().AppendListValue()
15. Alpha().DeleteListValue()
16. Alpha().SetListCapacity()

Lifecycle Management

Ready()

This tells Code Blind that the Game Server is ready to take player connections. Once a Game Server has specified that it is Ready, then the Kubernetes GameServer record will be moved to the Ready state, and the details for its public address and connection port will be populated.

While Code Blind prefers that Shutdown() is run once a game has completed to delete the GameServer instance, if you want or need to move an Allocated GameServer back to Ready to be reused, you can call this SDK method again to do this.

Health()

This sends a single ping to designate that the Game Server is alive and healthy. Failure to send pings within the configured thresholds will result in the GameServer being marked as Unhealthy.

See the gameserver.yaml for all health checking configurations.

Reserve(seconds)

With some matchmaking scenarios and systems it is important to be able to ensure that a GameServer is unable to be deleted, but doesn’t trigger a FleetAutoscaler scale up. This is where Reserve(seconds) is useful.

Reserve(seconds) will move the GameServer into the Reserved state for the specified number of seconds (0 is forever), and then it will be moved back to Ready state. While in Reserved state, the GameServer will not be deleted on scale down or Fleet update, and also it could not be Allocated using GameServerAllocation.

This is often used when a game server process must register itself with an external system, such as a matchmaker, that requires it to designate itself as available for a game session for a certain period. Once a game session has started, it should call SDK.Allocate() to designate that players are currently active on it.

Calling other state changing SDK commands such as Ready or Allocate will turn off the timer to reset the GameServer back to the Ready state or to promote it to an Allocated state accordingly.

Allocate()

With some matchmakers and game matching strategies, it can be important for game servers to mark themselves as Allocated. For those scenarios, this SDK functionality exists.

There is a chance that GameServer does not actually become Allocated after this call. Please refer to the general note in Function Reference above.

The agones.dev/last-allocated annotation will be set on the GameServer to an RFC3339 formatted timestamp of the time of allocation, even if the GameServer was already in an Allocated state.

Note that if using SDK.Allocate() in combination with GameServerAllocations, it’s possible for the agones.dev/last-allocated timestamp to move backwards if clocks are not synchronized between the Code Blind controller and the GameServer pod.

Shutdown()

This tells Code Blind to shut down the currently running game server. The GameServer state will be set Shutdown and the backing Pod will be Terminated.

It’s worth reading the Termination of Pods Kubernetes documentation, to understand the termination process, and the related configuration options.

As a rule of thumb, implement a graceful shutdown in your game sever process when it receives the TERM signal from Kubernetes when the backing Pod goes into Termination state.

Be aware that if you use a variation of System.exit(0) after calling SDK.Shutdown(), your game server container may restart for a brief period, inline with our Health Checking policies.

If the SDK server receives a TERM signal before calling SDK.Shutdown(), the SDK server will stay alive for the period of the terminationGracePeriodSeconds until SDK.Shutdown() has been called.

Configuration Retrieval

GameServer()

This returns most of the backing GameServer configuration and Status. This can be useful for instances where you may want to know Health check configuration, or the IP and Port the GameServer is currently allocated to.

Since the GameServer contains an entire PodTemplate the returned object is limited to that configuration that was deemed useful. If there are areas that you feel are missing, please file an issue or pull request.

The easiest way to see what is exposed, is to check the sdk.proto, specifically at the message GameServer.

For language specific documentation, have a look at the respective source (linked above), and the examples.

WatchGameServer(function(gameserver){…})

This executes the passed in callback with the current GameServer details whenever the underlying GameServer configuration is updated. This can be useful to track GameServer > Status > State changes, metadata changes, such as labels and annotations, and more.

In combination with this SDK, manipulating Annotations and Labels can also be a useful way to communicate information through to running game server processes from outside those processes. This is especially useful when combined with GameServerAllocation applied metadata.

Since the GameServer contains an entire PodTemplate the returned object is limited to that configuration that was deemed useful. If there are areas that you feel are missing, please file an issue or pull request.

The easiest way to see what is exposed, is to check the sdk.proto, specifically at the message GameServer.

For language specific documentation, have a look at the respective source (linked above), and the examples.

Metadata Management

SetLabel(key, value)

This will set a Label value on the backing GameServer record that is stored in Kubernetes.

To maintain isolation, the key value is automatically prefixed with the value “agones.dev/sdk-”. This is done for two main reasons:

• The prefix allows the developer to always know if they are accessing or reading a value that could have come, or may be changed by the client SDK. Much like private vs public scope in a programming language, the Code Blind SDK only gives you access to write to part of the set of labels and annotations that exist on a GameServer.
• The prefix allows for a smaller attack surface if the GameServer container gets compromised. Since the game container is generally externally exposed, and the Code Blind project doesn’t control the binary that is run within it, limiting exposure if the game server becomes compromised is worth the extra development friction that comes with having this prefix in place.

Setting GameServer labels can be useful if you want information from your running game server process to be observable or searchable through the Kubernetes API.

SetAnnotation(key, value)

This will set an Annotation value on the backing GameServer record that is stored in Kubernetes.

To maintain isolation, the key value is automatically prefixed with “agones.dev/sdk-” for the same reasons as in SetLabel(…) above. The isolation is also important as Code Blind uses annotations on the GameServer as part of its internal processing.

Setting GameServer annotations can be useful if you want information from your running game server process to be observable through the Kubernetes API.

Counters And Lists

The Counters and Lists features in the SDK offer a flexible configuration for tracking various entities like players, rooms, and sessions.

Declared keys and default values for Counters and Lists are specified in GameServer.Spec.Counters and GameServer.Spec.Lists respectively.

Modified Counter and List values and capacities will be updated in GameServer.Status.Counters and GameServer.Status.Lists respectively.

Counters

All functions will return an error if the specified key is not predefined in the GameServer.Spec.Counters resource configuration.

Note: For Counters, the default setting for the capacity is preset to 1000. It is recommended to avoid configuring the capacity to max(int64), as doing so could cause problems with JSON Patch operations.

Alpha().GetCounterCount(key)

This function retrieves either the GameServer.Status.Counters[key].Count or the SDK awaiting-batch value for a given key, whichever is most up to date.

Alpha().SetCounterCount(key, amount)

This function sets the value of GameServer.Status.Counters[key].Count for the given key to the passed in amount. This operation overwrites any previous values and the new value cannot exceed the Counter’s capacity.

Alpha().IncrementCounter(key, amount)

This function increments GameServer.Status.Counters[key].Count for the given key by the passed in non-negative amount. The function returns an error if the Counter is already at capacity (at time of operation), indicating no increment will occur.

Alpha().DecrementCounter(key, amount)

This function decreases GameServer.Status.Counters[key].Count for the given key by the passed in non-negative amount. It returns an error if the Counter’s count is already at zero.

Alpha().SetCounterCapacity(key, amount)

This function sets the maximum GameServer.Status.Counters[key].Capacity for the given key by the passed in non-negative amount. A capacity value of 0 indicates no capacity limit.

Alpha().GetCounterCapacity(key)

This function retrieves either the GameServer.Status.Counters[key].Capacity or the SDK awaiting-batch value for the given key, whichever is most up to date.

Lists

All functions will return an error if the specified key is not predefined in the GameServer.Spec.Lists resource configuration.

Alpha().AppendListValue(key, value)

This function appends the specified string value to the List in GameServer.Status.Lists[key].Values.

An error is returned if the string already exists in the list or if the list is at capacity.

Alpha().DeleteListValue(key, value)

This function removes the specified string value from the List in GameServer.Status.Lists[key].Values.

An error is returned if the string does not exist in the list.

Alpha().SetListCapacity(key, amount)

This function sets the maximum capacity for the List at GameServer.Status.Lists[key].Capacity.

The capacity value is required to be between 0 and 1000.

Alpha().GetListCapacity(key)

This function retrieves either the GameServer.Status.Lists[key].Capacity or the SDK awaiting-batch value for the given key, whichever is most up to date.

Alpha().GetListValues(key)

This function retrieves either the GameServer.Status.Lists[key].Values or the SDK awaiting-batch values array for the given key, whichever is most up to date.

Alpha().ListContains(key, value)

Convenience function, which returns if the specific string value exists in the results of Alpha().GetListValues(key).

Alpha().GetListLength(key)

Convenience function, which retrieves the length of the results of Alpha().GetListValues(key).

Player Tracking

Alpha().PlayerConnect(playerID)

This function increases the SDK’s stored player count by one, and appends this playerID to GameServer.Status.Players.IDs.

GameServer.Status.Players.Count and GameServer.Status.Players.IDs are then set to update the player count and id list a second from now, unless there is already an update pending, in which case the update joins that batch operation.

PlayerConnect() returns true and adds the playerID to the list of playerIDs if this playerID was not already in the list of connected playerIDs.

If the playerID exists within the list of connected playerIDs, PlayerConnect() will return false, and the list of connected playerIDs will be left unchanged.

An error will be returned if the playerID was not already in the list of connected playerIDs but the player capacity for the server has been reached. The playerID will not be added to the list of playerIDs.

Alpha().PlayerDisconnect(playerID)

This function decreases the SDK’s stored player count by one, and removes the playerID from GameServer.Status.Players.IDs.

GameServer.Status.Players.Count and GameServer.Status.Players.IDs are then set to update the player count and id list a second from now, unless there is already an update pending, in which case the update joins that batch operation.

PlayerDisconnect() will return true and remove the supplied playerID from the list of connected playerIDs if the playerID value exists within the list.

If the playerID was not in the list of connected playerIDs, the call will return false, and the connected playerID list will be left unchanged.

Alpha().SetPlayerCapacity(count)

Update the GameServer.Status.Players.Capacity value with a new capacity.

Alpha().GetPlayerCapacity()

This function retrieves the current player capacity. This is always accurate from what has been set through this SDK, even if the value has yet to be updated on the GameServer status resource.

Alpha().GetPlayerCount()

This function retrieves the current player count. This is always accurate from what has been set through this SDK, even if the value has yet to be updated on the GameServer status resource.

Alpha().IsPlayerConnected(playerID)

This function returns if the playerID is currently connected to the GameServer. This is always accurate from what has been set through this SDK, even if the value has yet to be updated on the GameServer status resource.

Alpha().GetConnectedPlayers()

This function returns the list of the currently connected player ids. This is always accurate from what has been set through this SDK, even if the value has yet to be updated on the GameServer status resource.

Writing your own SDK

If there isn’t an SDK for the language and platform you are looking for, you have several options:

gRPC Client Generation

If client generation is well supported by gRPC, then generate client(s) from the proto files found in the proto/sdk, directory and look at the current sdks to see how the wrappers are implemented to make interaction with the SDK server simpler for the user.

REST API Implementation

If client generation is not well supported by gRPC, or if there are other complicating factors, implement the SDK through the REST HTTP+JSON interface. This could be written by hand, or potentially generated from the Swagger/OpenAPI Specifications.

Finally, if you build something that would be usable by the community, please submit a pull request!

SDK Conformance Test

There is a tool SDK server Conformance checker which will run Local SDK server and record all requests your client is performing.

In order to check that SDK is working properly you should write simple SDK test client which would use all methods of your SDK.

Also to test that SDK client is receiving valid Gameserver data, your binary should set the same Label value as creation timestamp which you will receive as a result of GameServer() call and Annotation value same as gameserver UID received by Watch gameserver callback.

Complete list of endpoints which should be called by your test client is the following:

ready,allocate,setlabel,setannotation,gameserver,health,shutdown,watch

In order to run this test SDK server locally use:

SECONDS=30 make run-sdk-conformance-local

Docker container would timeout in 30 seconds and give your the comparison of received requests and expected requests

For instance you could run Go SDK conformance test and see how the process goes:

SDK_FOLDER=go make run-sdk-conformance-test

In order to add test client for your SDK, write sdktest.sh and Dockerfile. Refer to Golang SDK Conformance testing directory structure.

Building the Tools

If you wish to build the binaries from source the make target build-agones-sdk-binary will compile the necessary binaries for all supported operating systems (64 bit windows, linux and osx).

You can find the binaries in the bin folder in `cmd/sdk-server` once compilation is complete.

See Developing, Testing and Building Code Blind for more details.

5.3.1 - Unreal Engine Game Server Client Plugin

Check the Client SDK Documentation for more details on each of the SDK functions and how to run the SDK locally.

SDK Functionality

Additional methods have been added for ease of use (both of which are enabled by default):

• Connect
  • will call /gameserver till a succesful response is returned and then call /ready.
  • disabled by setting bDisableAutoConnect to true.
  • An event is broadcast with the GameServer data once the /gameserver call succeeds.
• Health
  • calls /health endpoint on supplied rate
  • enabled by default with 10 second rate
  • disabled by default by setting HealthRateSeconds to 0.

Both of the above are automatically kicked off in the BeginPlay of the component.

Download

Download the source from the Releases Page or directly from GitHub.

Resources

Unreal is a game engine that is used by anyone from hobbyists all the way through to huge AAA Game Stuidos.

With this in mind there is a vast amount to learn to run a production game using Unreal, even before you get to learning how it integrates with Code Blind. If you want to kick the tires with a starter project you will probably be fine with one of the starter projects out of the box.

However as your Unreal/Code Blind project gets more advanced you will want to understand more about the engine itself and how it can be used to integrate with this project. There will be different ways of interacting via in Play In Editor (PIE) versus running as an actual dedicated game server packaged into a container.

There are few helpful links for latest Unreal Engine 5:

• UE5 Documentation Site
• UE5 Dedicated Servers
  • useful guide to getting started with dedicated servers in Unreal
• UE5 Game Mode and Game State
• UE5 Game Mode API Reference
  • useful for hooking up calls to Code Blind
• UE5 Game Session API Reference
  • as above there are hooks in Game Session that can be used to call into Code Blind
• UE5 Building & Packaging Games
  • only building out Unreal game servers / clients, will also need to package into a container

If you use Unreal Engine 4, There are few helpful links for it:

• UE4 Documentation Site
• UE4 Dedicated Servers
• UE4 Game Flow
• UE4 Game Mode
• UE4 Game Session
• UE4 Building & Packaging Games

Getting Started

This is a SDK inspired by the REST API to the Code Blind sidecars that allows engineers to communicate with the sidecar from either C++ or Blueprints.

Getting the Code

Easiest way to get this code is to clone the repository and drop the entire plugin folder into your own Plugins folder. This runs the plugin as a Project plugin rather than an engine plugin.

We could however turn this into a marketplace plugin that can be retrived from the marketplace directly into the UE editor.

Using C++ (UE5/UE4)

• Add Plugin (in your own .uproject file)

  "Plugins": [
    {
      "Enabled": true,
      "Name": "Code Blind"
    }
  ],

• Add Plugin (in your own *.Build.cs)

PublicDependencyModuleNames.AddRange(
    new[]
    {
        "Code Blind",
    });

• Add component in header

#include "AgonesComponent.h"

UPROPERTY(EditAnywhere, BlueprintReadWrite)
UAgonesComponent* AgonesSDK;

• Initialize component in GameMode

#include "AgonesComponent.h"
#include "Classes.h"

ATestGameMode::ATestGameMode()
{
	AgonesSDK = CreateDefaultSubobject<UAgonesComponent>(TEXT("AgonesSDK"));
}

• Use the Code Blind component to call PlayerReady

void APlatformGameSession::PostLogin(APlayerController* NewPlayer)
{
  // Empty brances are for callbacks on success and errror.
  AgonesSDK->PlayerConnect("netspeak-player", {}, {});
}

Using Blueprints (UE5)

• Add Component to your Blueprint GameMode

• This will automatically call /health every 10 seconds and once /gameserver calls are succesful it will call /ready.

• Accessing other functionality of Code Blind can be done via adding a node in Blueprints.

Using Blueprints (UE4)

• Add Component to your Blueprint GameMode

• This will automatically call /health every 10 seconds and once /gameserver calls are succesful it will call /ready.

• Accessing other functionality of Code Blind can be done via adding a node in Blueprints.

Configuration Options

A number of options can be altered via config files in Unreal these are supplied via Game configuration eg. DefaultGame.ini.

[/Script/Code Blind.AgonesComponent]
HttpPort=1337
HealthRateSeconds=5.0
bDisableAutoConnect=true

Unreal Hooks

Within the Unreal GameMode and GameSession exist a number of useful existing funtions that can be used to fit in with making calls out to Code Blind.

A few examples are:

• RegisterServer to call SetLabel, SetPlayerCapacity
• PostLogin to call PlayerConnect
• NotifyLogout to call PlayerDisconnect

5.3.2 - Unity Game Server Client SDK

Check the Client SDK Documentation for more details on each of the SDK functions and how to run the SDK locally.

SDK Functionality

Additional methods have been added for ease of use:

• Connect

Installation

The client SDK code can be manually downloaded and added to your project hierarchy.

It can also be imported into your project via the Unity Package Manager (UPM). To do that, open your project’s manifest.json file, and add the following line to the dependencies section:

{
  "dependencies": {
        "com.googleforgames.agones": "https://github.com/googleforgames/agones.git?path=/sdks/unity",
...

If you want a specific release, the dependency can be pinned to that version. For example:

"com.googleforgames.agones": "https://github.com/googleforgames/agones.git?path=/sdks/unity#v1.38.0",

Download

Download the source directly from GitHub.

Prerequisites

• Unity >= 2018.x (.NET 4.x)

Usage

Import this script to your unity project and attach it to GameObject.

To begin working with the SDK, get an instance of it.

var agones = agonesGameObject.GetComponent<Code Blind.AgonesSdk>();

To connect to the SDK server, either local or when running on Code Blind, run the async Connect() method. This will wait for up to 30 seconds if the SDK server has not yet started and the connection cannot be made, and will return false if there was an issue connecting.

bool ok = await agones.Connect();

To mark the game server as ready to receive player connections, call the async method Ready().

async void SomeMethod()
{
    bool ok = await agones.Ready();
}

To get the details on the backing GameServer call GameServer().

Will return null if there is an error in retrieving the GameServer record.

var gameserver = await agones.GameServer();

To mark the GameServer as Reserved for a duration call Reserve(TimeSpan duration).

ok = await agones.Reserve(duration);

To mark that the game session is completed and the game server should be shut down call Shutdown().

bool ok = await agones.Shutdown();

Similarly SetAnnotation(string key, string value) and SetLabel(string key, string value) are async methods that perform an action.

And there is no need to call Health(), it is automatically called.

To watch when the backing GameServer configuration changes call WatchGameServer(callback), where the delegate function callback will be executed every time the GameServer configuration changes.

agones.WatchGameServer(gameServer => Debug.Log($"Server - Watch {gameServer}"));

Player Tracking

To use alpha features use the AgonesAlphaSDK class.

var agones = agonesGameObject.GetComponent<Code Blind.AgonesAlphaSdk>();

Alpha: PlayerConnect

This method increases the SDK’s stored player count by one, and appends this playerID to GameServer.Status.Players.IDs. Returns true and adds the playerID to the list of playerIDs if the playerIDs was not already in the list of connected playerIDs.

bool ok = await agones.PlayerConnect(playerId);

Alpha: PlayerDisconnect

This function decreases the SDK’s stored player count by one, and removes the playerID from GameServer.Status.Players.IDs. Will return true and remove the supplied playerID from the list of connected playerIDs if the playerID value exists within the list.

bool ok = await agones.PlayerDisconnect(playerId);

Alpha: SetPlayerCapacity

Update the GameServer.Status.Players.Capacity value with a new capacity.

var capacity = 100;
bool ok = await agones.SetPlayerCapacity(capacity);

Alpha: GetPlayerCapacity

This function retrieves the current player capacity GameServer.Status.Players.Capacity. This is always accurate from what has been set through this SDK, even if the value has yet to be updated on the GameServer status resource.

long capacity = await agones.GetPlayerCapacity();

Alpha: GetPlayerCount

Returns the current player count

long count = await agones.GetPlayerCount();

Alpha: IsPlayerConnected

This returns if the playerID is currently connected to the GameServer. This is always accurate, even if the value hasn’t been updated to the GameServer status yet.

bool isConnected = await agones.IsPlayerConnected(playerId);

Alpha: GetConnectedPlayers

This returns a list of the playerIDs that are currently connected to the GameServer.

List<string> players = await agones.GetConnectedPlayers();

void Deadlock()
{
    Task<bool> t = agones.Shutdown();
    t.Wait(); // deadlock!!!
}

Settings

The properties for the Unity Code Blind SDK can be found in the Inspector.

• Health Interval Second
  • Interval of the server sending a health ping to the Code Blind sidecar. (default: 5)
• Health Enabled
  • Whether the server sends a health ping to the Code Blind sidecar. (default: true)
• Log Enabled
  • Debug Logging Enabled. Debug logging for development of this Plugin. (default: false)

5.3.3 - C++ Game Server Client SDK

Check the Client SDK Documentation for more details on each of the SDK functions and how to run the SDK locally.

SDK Functionality

Installation

Download

Download the source from the Releases Page or directly from GitHub.

Building the Libraries from source

CMake is used to build SDK for all supported platforms (Linux/Window/macOS).

Prerequisites

• CMake >= 3.15.0
• Git
• C++17 compiler

Dependencies

Code Blind SDK only depends on the gRPC library.

If CMake cannot find gRPC with find_package(), it downloads and builds gRPC. There are some extra prerequisites for OpenSSL on Windows, see documentation:

• Perl
• NASM

Note that OpenSSL is not used in Code Blind SDK, but it is required to have a successful build of gRPC.

Options

Following options are available:

• AGONES_THIRDPARTY_INSTALL_PATH (default is CMAKE_INSTALL_PREFIX) - installation path for Code Blind prerequisites (used only if gRPC and Protobuf are not found by find_package)
• AGONES_ZLIB_STATIC (default is ON) - use static version of zlib for gRPC

(Windows only):

• AGONES_BUILD_THIRDPARTY_DEBUG (default is OFF) - build both debug and release versions of SDK’s prerequisites. Option is not used if you already have built gRPC.
• AGONES_OPENSSL_CONFIG_STRING (default is VC-WIN64A) - arguments to configure OpenSSL build (documentation). Used only if OpenSSL and gRPC is built by Code Blind.

Linux / MacOS

mkdir -p .build
cd .build
cmake .. -DCMAKE_BUILD_TYPE=Release -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=./install
cmake --build . --target install

Windows

Building with Visual Studio:

md .build
cd .build
cmake .. -G "Visual Studio 15 2017 Win64" -DCMAKE_INSTALL_PREFIX=./install
cmake --build . --config Release --target install

Building with NMake

md .build
cd .build
cmake .. -G "NMake Makefiles" -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=./install
cmake --build . --target install

CMAKE_INSTALL_PREFIX may be skipped if it is OK to install Code Blind SDK to a default location (usually /usr/local or c:/Program Files/Code Blind).

CMake option -Wno-dev is specified to suppress CMP0048 deprecation warning for gRPC build.

If AGONES_ZLIB_STATIC is set to OFF, ensure that you have installed zlib. For Windows, it’s enough to copy zlib.dll near to gameserver executable. For Linux/Mac usually no actions are needed.

Usage

Using SDK

In CMake-based projects it’s enough to specify a folder where SDK is installed with CMAKE_PREFIX_PATH and use find_package(agones CONFIG REQUIRED) command. For example: cpp-simple. It may be useful to disable some protobuf warnings in your project.

Usage

The C++ SDK is specifically designed to be as simple as possible, and deliberately doesn’t include any kind of singleton management, or threading/asynchronous processing to allow developers to manage these aspects as they deem appropriate for their system.

We may consider these types of features in the future, depending on demand.

To begin working with the SDK, create an instance of it:

agones::SDK *sdk = new agones::SDK();

To connect to the SDK server, either local or when running on Code Blind, run the sdk->Connect() method. This will block for up to 30 seconds if the SDK server has not yet started and the connection cannot be made, and will return false if there was an issue connecting.

bool ok = sdk->Connect();

To send a health check call sdk->Health(). This is a synchronous request that will return false if it has failed in any way. Read GameServer Health Checking for more details on the game server health checking strategy.

bool ok = sdk->Health();

To mark the game server as ready to receive player connections, call sdk->Ready(). This will return a grpc::Status object, from which we can call status.ok() to determine if the function completed successfully.

For more information you can also look at the gRPC Status reference.

grpc::Status status = sdk->Ready();
if (!status.ok()) { ... }

To mark the game server as allocated, call sdk->Allocate(). This will return a grpc::Status object, from which we can call status.ok() to determine if the function completed successfully.

For more information you can also look at the gRPC Status reference.

grpc::Status status = sdk->Allocate();
if (!status.ok()) { ... }

To mark the game server as reserved, call sdk->Reserve(seconds). This will return a grpc::Status object, from which we can call status.ok() to determine if the function completed successfully.

For more information you can also look at the gRPC Status reference.

grpc::Status status = sdk->Reserve(std::chrono::seconds(N));
if (!status.ok()) { ... }

To mark that the game session is completed and the game server should be shut down call sdk->Shutdown(). This will return a grpc::Status object, from which we can call status.ok() to determine if the function completed successfully.

For more information you can also look at the gRPC Status reference.

grpc::Status status = sdk->Shutdown();
if (!status.ok()) { ... }

To set a Label on the backing GameServer call sdk->SetLabel(key, value). This will return a grpc::Status object, from which we can call status.ok() to determine if the function completed successfully.

For more information you can also look at the gRPC Status reference.

grpc::Status status = sdk->SetLabel("test-label", "test-value");
if (!status.ok()) { ... }

To set an Annotation on the backing GameServer call sdk->SetAnnotation(key, value). This will return a grpc::Status object, from which we can call status.ok() to determine if the function completed successfully.

For more information you can also look at the gRPC Status reference.

status = sdk->SetAnnotation("test-annotation", "test value");
if (!status.ok()) { ... }

To get the details on the backing GameServer call sdk->GameServer(&gameserver), passing in a agones::dev::sdk::GameServer* to push the results of the GameServer configuration into.

This function will return a grpc::Status object, from which we can call status.ok() to determine if the function completed successfully.

agones::dev::sdk::GameServer gameserver;
grpc::Status status = sdk->GameServer(&gameserver);
if (!status.ok()) {...}

To get updates on the backing GameServer as they happen, call sdk->WatchGameServer([](const agones::dev::sdk::GameServer& gameserver){...}).

This will call the passed in std::function synchronously (this is a blocking function, so you may want to run it in its own thread) whenever the backing GameServer is updated.

sdk->WatchGameServer([](const agones::dev::sdk::GameServer& gameserver){
  std::cout << "GameServer Update:\n"                                 //
            << "\tname: " << gameserver.object_meta().name() << "\n"  //
            << "\tstate: " << gameserver.status().state() << "\n"
            << std::flush;
});

Next Steps

• Read the SDK Overview to review all SDK functionality
• Look at the C++ example for a full build template.
• Check out sdk.h.

5.3.4 - Go Game Server Client SDK

Check the Client SDK Documentation for more details on each of the SDK functions and how to run the SDK locally.

SDK Functionality

Installation

go get the source, directly from GitHub

Usage

Review the GoDoc for usage instructions

5.3.5 - C# Game Server Client SDK

Check the Client SDK Documentation for more details on each of the SDK functions and how to run the SDK locally.

SDK Functionality

Download

Download the source directly from GitHub.

Install using NuGet

• Download the nuget package directly
• Install the latest version using the Package Manager: Install-Package AgonesSDK
• Install the latest version using the .NET CLI: dotnet add package AgonesSDK

To select a specific version, append --version, for example: --version 1.8.0 to either commands.

Prerequisites

• .Net Standard 2.0 compliant framework.

Usage

Reference the SDK in your project & create a new instance of the SDK wrapper:

Initialization

To use the AgonesSDK, you will need to import the namespace by adding using Code Blind; at the beginning of your relevant files.

var agones = new AgonesSDK();

Connection

To connect to the SDK server, either locally or when running on Code Blind, run the ConnectAsync() method. This will wait for up to 30 seconds if the SDK server has not yet started and the connection cannot be made, and will return false if there was an issue connecting.

bool ok = await agones.ConnectAsync();

Ready

To mark the game server as ready to receive player connections, call the async method ReadyAsync().

async void SomeMethod()
{
    var status = await agones.ReadyAsync();
}

Health

To send Health pings, call the async method HealthAsync()

await agones.HealthAsync();

GetGameServer

To get the details on the backing GameServer call GetGameServerAsync().

Will return null if there is an error in retrieving the GameServer record.

var gameserver = await agones.GetGameServerAsync();

Reserve

To mark the GameServer as Reserved for a duration call ReserveAsync(long duration).

long duration = 30;
var status = await agones.ReserveAsync(duration);

ShutDown

To mark that the game session is completed and the game server should be shut down call ShutdownAsync().

var status = await agones.ShutdownAsync();

SetAnnotation & SetLabel

Similarly SetAnnotation(string key, string value) and SetLabel(string key, string value) are async methods that perform an action & return a Status object.

WatchGameServer

To watch when the backing GameServer configuration changes call WatchGameServer(callback), where the delegate function callback of type Action<GameServer> will be executed every time the GameServer configuration changes. This process is non-blocking internally.

agonesSDK.WatchGameServer((gameServer) => { Console.WriteLine($"Server - Watch {gameServer}");});

Player Tracking

Alpha: PlayerConnect

This method increases the SDK’s stored player count by one, and appends this playerID to GameServer.Status.Players.IDs. Returns true and adds the playerID to the list of playerIDs if the playerIDs was not already in the list of connected playerIDs.

bool ok = await agones.Alpha().PlayerConnectAsync(playerId);

Alpha: PlayerDisconnect

This function decreases the SDK’s stored player count by one, and removes the playerID from GameServer.Status.Players.IDs. Will return true and remove the supplied playerID from the list of connected playerIDs if the playerID value exists within the list.

bool ok = await agones.Alpha().PlayerDisconnectAsync(playerId);

Alpha: SetPlayerCapacity

Update the GameServer.Status.Players.Capacity value with a new capacity.

var capacity = 100;
var status = await agones.Alpha().SetPlayerCapacityAsync(capacity);

Alpha: GetPlayerCapacity

This function retrieves the current player capacity GameServer.Status.Players.Capacity. This is always accurate from what has been set through this SDK, even if the value has yet to be updated on the GameServer status resource.

long cap = await agones.Alpha().GetPlayerCapacityAsync();

Alpha: GetPlayerCount

Returns the current player count

long count = await agones.Alpha().GetPlayerCountAsync();

Alpha: IsPlayerConnected

This returns if the playerID is currently connected to the GameServer. This is always accurate, even if the value hasn’t been updated to the GameServer status yet.

var playerId = "player1";
bool isConnected = await agones.Alpha().IsPlayerConnectedAsync(playerId);

Remarks

• All requests other than ConnectAsync will wait for up to 15 seconds before giving up, time to wait can also be set in the constructor.
• Default host & port are localhost:9357
• Methods that do not return a data object such as GameServer will return a gRPC Grpc.Core.Status object. To check the state of the request, check Status.StatusCode & Status.Detail. Ex:

if(status.StatusCode == StatusCode.OK)
    //do stuff

5.3.6 - Node.js Game Server Client SDK

Check the Client SDK Documentation for more details on each of the SDK functions and how to run the SDK locally.

SDK Functionality