Getting Started

• 1: Quickstart: Create a Game Server
• 2: Quickstart: Create a Game Server Fleet
• 3: Quickstart: Create a Fleet Autoscaler
• 4: Quickstart: Create a Fleet Autoscaler with Webhook Policy
• 5: Quickstart: Edit a Game Server

1 - Quickstart: Create a Game Server

Prerequisites

The following prerequisites are required to create a GameServer:

1. A Kubernetes cluster with the UDP port range 7000-8000 open on each node.
2. Code Blind controller installed in the targeted cluster
3. kubectl properly configured
4. Netcat which is already installed on most Linux/macOS distributions, for windows you can use WSL.

If you don’t have a Kubernetes cluster you can follow these instructions to create a cluster on Google Kubernetes Engine (GKE), Minikube or Azure Kubernetes Service (AKS), and install Code Blind.

For the purpose of this guide we’re going to use the simple-game-server example as the GameServer container. This example is a very simple UDP server written in Go. Don’t hesitate to look at the code of this example for more information.

Objectives

• Create a GameServer in Kubernetes using Code Blind custom resource.
• Get information about the GameServer such as IP address, port and state.
• Connect to the GameServer.

1. Create a GameServer

Let’s create a GameServer using the following command :

kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/gameserver.yaml

You should see a successful output similar to this :

gameserver.agones.dev/simple-game-server-4ss4j created

This has created a GameServer record inside Kubernetes, which has also created a backing Pod to run our simple udp game server code in. If you want to see all your running GameServers you can run:

kubectl get gameservers

It should look something like this:

NAME                       STATE     ADDRESS       PORT   NODE     AGE
simple-game-server-7pjrq   Ready   35.233.183.43   7190   agones   3m

You can also see the Pod that got created by running kubectl get pods, the Pod will be prefixed by simple-game-server.

NAME                        READY     STATUS    RESTARTS   AGE
simple-game-server-7pjrq    2/2       Running   0          5m

As you can see above it says READY: 2/2 this means there are two containers running in this Pod, this is because Code Blind injected the SDK sidecar for readiness and health checking of your Game Server.

For the full details of the YAML file head to the GameServer Specification Guide

2. Fetch the GameServer Status

Let’s wait for the GameServer state to become Ready. You can use the watch tool to see the state change. If your operating system does not have watch, manually run kubectl describe gameserver until the state changes.

watch kubectl describe gameserver

Name:         simple-game-server-7pjrq
Namespace:    default
Labels:       <none>
Annotations:  agones.dev/sdk-version: 0.9.0-764fa53
API Version:  agones.dev/v1
Kind:         GameServer
Metadata:
  Creation Timestamp:  2019-02-27T15:06:20Z
  Finalizers:
    agones.dev
  Generate Name:     simple-game-server-
  Generation:        1
  Resource Version:  30377
  Self Link:         /apis/agones.dev/v1/namespaces/default/gameservers/simple-game-server-7pjrq
  UID:               3d7ac3e1-3aa1-11e9-a4f5-42010a8a0019
Spec:
  Container:  simple-game-server
  Health:
    Failure Threshold:      3
    Initial Delay Seconds:  5
    Period Seconds:         5
  Ports:
    Container Port:  7654
    Host Port:       7190
    Name:            default
    Port Policy:     Dynamic
    Protocol:        UDP
  Scheduling:        Packed
  Template:
    Metadata:
      Creation Timestamp:  <nil>
    Spec:
      Containers:
        Image:  us-docker.pkg.dev/codeblind/examples/simple-server:0.27
        Name:   simple-game-server
        Resources:
          Limits:
            Cpu:     20m
            Memory:  32Mi
          Requests:
            Cpu:     20m
            Memory:  32Mi
Status:
  Address:    35.233.183.43
  Node Name:  agones
  Ports:
    Name:  default
    Port:  7190
  State:   Ready
Events:
  Type    Reason          Age   From                   Message
  ----    ------          ----  ----                   -------
  Normal  PortAllocation  34s   gameserver-controller  Port allocated
  Normal  Creating        34s   gameserver-controller  Pod simple-game-server-7pjrq created
  Normal  Scheduled       34s   gameserver-controller  Address and port populated
  Normal  Ready           27s   gameserver-controller  SDK.Ready() executed

If you look towards the bottom, you can see there is a Status > State value. We are waiting for it to move to Ready, which means that the game server is ready to accept connections.

You might also be interested to see the Events section, which outlines when various lifecycle events of the GameServer occur. We can also see when the GameServer is ready on the event stream as well - at which time the Status > Address and Status > Ports > Port have also been populated, letting us know what IP and port our client can now connect to!

Let’s retrieve the IP address and the allocated port of your Game Server :

kubectl get gs

This should output your Game Server IP address and ports, eg:

NAME                       STATE   ADDRESS         PORT   NODE     AGE
simple-game-server-7pjrq   Ready   35.233.183.43   7190   agones   4m

3. Connect to the GameServer

You can now communicate with the Game Server :

nc -u {IP} {PORT}
Hello World !
ACK: Hello World !
EXIT

You can finally type EXIT which tells the SDK to run the Shutdown command, and therefore shuts down the GameServer.

If you run kubectl describe gameserver again - either the GameServer will be gone completely, or it will be in Shutdown state, on the way to being deleted.

Next Step

If you want to use your own GameServer container make sure you have properly integrated the Code Blind SDK.

2 - Quickstart: Create a Game Server Fleet

Prerequisites

The following prerequisites are required to create a GameServer:

1. A Kubernetes cluster with the UDP port range 7000-8000 open on each node.
2. Code Blind controller installed in the targeted cluster
3. kubectl properly configured
4. Netcat which is already installed on most Linux/macOS distributions, for windows you can use WSL.

If you don’t have a Kubernetes cluster you can follow these instructions to create a cluster on Google Kubernetes Engine (GKE), Minikube or Azure Kubernetes Service (AKS), and install Code Blind.

For the purpose of this guide we’re going to use the simple-game-server example as the GameServer container. This example is a very simple UDP server written in Go. Don’t hesitate to look at the code of this example for more information.

While not required, you may wish to go through the Create a Game Server quickstart before this one.

Objectives

• Create a Fleet in Kubernetes using an Code Blind custom resource.
• Scale the Fleet up from its initial configuration.
• Request a GameServer allocation from the Fleet to play on.
• Connect to the allocated GameServer.
• Deploy a new GameServer configuration to the Fleet.

1. Create a Fleet

Let’s create a Fleet using the following command:

kubectl apply -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/fleet.yaml

You should see a successful output similar to this :

fleet.agones.dev/simple-game-server created

This has created a Fleet record inside Kubernetes, which in turn creates two warm GameServers that are available to be allocated for a game session.

kubectl get fleet

It should look something like this:

NAME                 SCHEDULING   DESIRED   CURRENT   ALLOCATED   READY     AGE
simple-game-server   Packed       2         3         0           2         9m

You can also see the GameServers that have been created by the Fleet by running kubectl get gameservers, the GameServer will be prefixed by simple-game-server.

NAME                             STATE     ADDRESS            PORT   NODE      AGE
simple-game-server-llg4x-rx6rc   Ready     192.168.122.205    7752   minikube   9m
simple-game-server-llg4x-v6g2r   Ready     192.168.122.205    7623   minikube   9m

For the full details of the YAML file head to the Fleet Specification Guide

2. Fetch the Fleet status

Let’s wait for the two GameServers to become ready.

watch kubectl describe fleet simple-game-server

Name:         simple-game-server
Namespace:    default
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"agones.dev/v1","kind":"Fleet","metadata":{"annotations":{},"name":"simple-game-server","namespace":"default"},"spec":{"replicas":2,...
API Version:  agones.dev/v1
Kind:         Fleet
Metadata:
  Cluster Name:
  Creation Timestamp:  2018-07-01T18:55:35Z
  Generation:          1
  Resource Version:    24685
  Self Link:           /apis/agones.dev/v1/namespaces/default/fleets/simple-game-server
  UID:                 56710a91-7d60-11e8-b2dd-08002703ef08
Spec:
  Replicas:  2
  Strategy:
    Rolling Update:
      Max Surge:        25%
      Max Unavailable:  25%
    Type:               RollingUpdate
  Template:
    Metadata:
      Creation Timestamp:  <nil>
    Spec:
      Health:
      Ports:
        Container Port:  7654
        Name:            default
        Port Policy:     Dynamic
      Template:
        Metadata:
          Creation Timestamp:  <nil>
        Spec:
          Containers:
            Image:  us-docker.pkg.dev/codeblind/examples/simple-server:0.27
            Name:   simple-game-server
            Resources:
Status:
  Allocated Replicas:  0
  Ready Replicas:      2
  Replicas:            2
Events:
  Type    Reason                 Age   From              Message
  ----    ------                 ----  ----              -------
  Normal  CreatingGameServerSet  13s   fleet-controller  Created GameServerSet simple-game-server-wlqnd

If you look towards the bottom, you can see there is a section of Status > Ready Replicas which will tell you how many GameServers are currently in a Ready state. After a short period, there should be 2 Ready Replicas.

3. Scale up the Fleet

Let’s scale up the Fleet from 2 replicates to 5.

Run kubectl scale fleet simple-game-server --replicas=5 to change Replicas count from 2 to 5.

If we now run kubectl get gameservers we should see 5 GameServers prefixed by simple-game-server.

NAME                             STATE    ADDRESS           PORT    NODE       AGE
simple-game-server-sdhzn-kcmh6   Ready    192.168.122.205   7191    minikube   52m
simple-game-server-sdhzn-pdpk5   Ready    192.168.122.205   7752    minikube   53m
simple-game-server-sdhzn-r4d6x   Ready    192.168.122.205   7623    minikube   52m
simple-game-server-sdhzn-wng5k   Ready    192.168.122.205   7709    minikube   53m
simple-game-server-sdhzn-wnhsw   Ready    192.168.122.205   7478    minikube   52m

4. Allocate a Game Server from the Fleet

Since we have a fleet of warm gameservers, we need a way to request one of them for usage, and mark that it has players accessing it (and therefore, it should not be deleted until they are finished with it).

We can do the allocation of a GameServer for usage through a GameServerAllocation, which will both return to us the details of a GameServer (assuming one is available), and also move it to the Allocated state, which demarcates that it has players on it, and should not be removed until SDK.Shutdown() is called, or it is manually deleted.

It is worth noting that there is nothing specific that ties a GameServerAllocation to a fleet. A GameServerAllocation uses a label selector to determine what group of GameServers it will attempt to allocate out of. That being said, a Fleet and GameServerAllocation are often used in conjunction.

This example uses the label selector to specifically target the simple-game-server fleet that we just created.

kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/gameserverallocation.yaml -o yaml

For the full details of the YAML file head to the GameServerAllocation Specification Guide

You should get back a response that looks like the following:

apiVersion: allocation.agones.dev/v1
kind: GameServerAllocation
metadata:
  creationTimestamp: 2019-02-19T02:13:12Z
  name: simple-game-server-dph9b-hfk24
  namespace: default
spec:
  metadata: {}
  required:
    matchLabels:
      agones.dev/fleet: simple-game-server
  scheduling: Packed
status:
  address: 192.168.122.152
  gameServerName: simple-game-server-dph9b-hfk24
  nodeName: minikube
  ports:
  - name: default
    port: 7714
  state: Allocated

If you look at the status section, there are several things to take note of. The state value will tell if a GameServer was allocated or not. If a GameServer could not be found, this will be set to UnAllocated. If there are too many concurrent requests overwhelmed the system, state will be set to Contention even though there are available GameServers.

However, we see that the status.state value was set to Allocated. This means you have been successfully allocated a GameServer out of the fleet, and you can now connect your players to it!

You can see various immutable details of the GameServer in the status - the address, ports and the name of the GameServer, in case you want to use it to retrieve more details.

We can also check to see how many GameServers you have Allocated vs Ready with the following command (“gs” is shorthand for “gameserver”).

kubectl get gs

This will get you a list of all the current GameServers and their Status.State.

NAME                             STATE       ADDRESS           PORT   NODE      AGE
simple-game-server-sdhzn-kcmh6   Ready       192.168.122.205   7191   minikube  52m
simple-game-server-sdhzn-pdpk5   Ready       192.168.122.205   7752   minikube  53m
simple-game-server-sdhzn-r4d6x   Allocated   192.168.122.205   7623   minikube  52m
simple-game-server-sdhzn-wng5k   Ready       192.168.122.205   7709   minikube  53m
simple-game-server-sdhzn-wnhsw   Ready       192.168.122.205   7478   minikube  52m

A handy trick for checking to see how many GameServers you have Allocated vs Ready, run the following:

kubectl get gs

This will get you a list of all the current GameServers and their Status > State.

NAME                             STATE       ADDRESS          PORT   NODE        AGE
simple-game-server-tfqn7-c9tqz   Ready       192.168.39.150   7136   minikube    52m
simple-game-server-tfqn7-g8fhq   Allocated   192.168.39.150   7148   minikube    53m
simple-game-server-tfqn7-p8wnl   Ready       192.168.39.150   7453   minikube    52m
simple-game-server-tfqn7-t6bwp   Ready       192.168.39.150   7228   minikube    53m
simple-game-server-tfqn7-wkb7b   Ready       192.168.39.150   7226   minikube    52m

5. Scale down the Fleet

Not only can we scale our fleet up, but we can scale it down as well.

The nice thing about Code Blind is that it is smart enough to know when GameServers have been moved to Allocated and will automatically leave them running on scale down – as we assume that players are playing on this game server, and we shouldn’t disconnect them!

Let’s scale down our Fleet to 0 (yep! you can do that!), and watch what happens.

Run kubectl scale fleet simple-game-server --replicas=0 to change Replicas count from 5 to 0.

It may take a moment for all the GameServers to shut down, so let’s watch them all and see what happens:

watch kubectl get gs

Eventually, one by one they will be removed from the list, and you should simply see:

NAME                             STATUS      ADDRESS          PORT    NODE       AGE
simple-game-server-tfqn7-g8fhq   Allocated   192.168.39.150   7148    minikube   55m

That lone Allocated GameServer is left all alone, but still running!

If you would like, try editing the Fleet configuration replicas field and watch the list of GameServers grow and shrink.

6. Connect to the GameServer

Since we’ve only got one allocation, we’ll just grab the details of the IP and port of the only allocated GameServer:

kubectl get gameservers | grep Allocated | awk '{print $3":"$4 }'

This should output your Game Server IP address and port. (eg 10.130.65.208:7936)

You can now communicate with the GameServer:

nc -u {IP} {PORT}
Hello World !
ACK: Hello World !
EXIT

You can finally type EXIT which tells the SDK to run the Shutdown command, and therefore shuts down the GameServer.

If you run kubectl describe gs | grep State again - either the GameServer will be replaced with a new, Ready GameServer , or it will be in Shutdown state, on the way to being deleted.

Since we are running a Fleet, Code Blind will always do it’s best to ensure there are always the configured number of GameServers in the pool in either a Ready or Allocated state.

7. Deploy a new version of the GameServer on the Fleet

We can also change the configuration of the GameServer of the running Fleet, and have the changes roll out, without interrupting the currently Allocated GameServers.

Let’s take this for a spin! Run kubectl scale fleet simple-game-server --replicas=5 to return Replicas count back to 5.

Let’s also allocate ourselves a GameServer:

kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/gameserverallocation.yaml -o yaml

We should now have four Ready GameServers and one Allocated.

We can check this by running kubectl get gs.

NAME                             STATE       ADDRESS          PORT   NODE       AGE
simple-game-server-tfqn7-c9tz7   Ready       192.168.39.150   7136   minikube   5m
simple-game-server-tfqn7-g8fhq   Allocated   192.168.39.150   7148   minikube   5m
simple-game-server-tfqn7-n0wnl   Ready       192.168.39.150   7453   minikube   5m
simple-game-server-tfqn7-hiiwp   Ready       192.168.39.150   7228   minikube   5m
simple-game-server-tfqn7-w8z7b   Ready       192.168.39.150   7226   minikube   5m

In production, we’d likely be changing a containers > image configuration to update our Fleet to run a new game server process, but to make this example simple, change containerPort from 7654 to 6000.

Run kubectl edit fleet simple-game-server, and make the necessary changes, and then save and exit your editor.

This will start the deployment of a new set of GameServers running with a Container Port of 6000.

Run kubectl describe gs | grep "Container Port" until you can see that there is one with a containerPort of 7654, which is the Allocated GameServer, and four instances with a containerPort of 6000 which is the new configuration. You can also run kubectl get gs and look at the Age column to see that one GameServer is much older than the other four.

You have now deployed a new version of your game!

Next Steps

• Have a look at the GameServerAllocation specification, and see how the extra functionality can enable smoke testing, server information communication, and more.
• You can now create a fleet autoscaler to automatically resize your fleet based on the actual usage. See Create a Fleet Autoscaler.
• Have a look at the GameServer Integration Patterns, to give you a set of examples on how all the pieces fit together with your matchmaker and other systems.
• Or if you want to try to use your own GameServer container make sure you have properly integrated the Code Blind SDK.
• If you would like to learn how to programmatically allocate a Game Server from the fleet, see how to Access Code Blind via the Kubernetes API or alternatively use the Allocator Service, depending on your needs.

3 - Quickstart: Create a Fleet Autoscaler

Prerequisites

It is assumed that you have followed the instructions to Create a Game Server Fleet and you have a running fleet of game servers.

Objectives

• Create a Fleet Autoscaler in Kubernetes using Code Blind custom resource.
• Watch the Fleet scale up when allocating GameServers
• Watch the Fleet scale down when shutting down allocated GameServers
• Edit the autoscaler specification to apply live changes

1. Create a Fleet Autoscaler

Let’s create a Fleet Autoscaler using the following command :

kubectl apply -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/fleetautoscaler.yaml

You should see a successful output similar to this :

fleetautoscaler.autoscaling.agones.dev/simple-game-server-autoscaler created

This has created a FleetAutoscaler record inside Kubernetes.

2. See the autoscaler status.

kubectl describe fleetautoscaler simple-game-server-autoscaler

It should look something like this:

Name:         simple-game-server-autoscaler
Namespace:    default
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"au
toscaling.agones.dev/v1","kind":"FleetAutoscaler","metadata":{"annotations":{},
"name":"simple-game-server-autoscaler","namespace":"default"},...
API Version:  autoscaling.agones.dev/v1
Kind:         FleetAutoscaler
Metadata:
  Cluster Name:
  Creation Timestamp:  2018-10-02T15:19:58Z
  Generation:          1
  Owner References:
    API Version:           autoscaling.agones.dev/v1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Fleet
    Name:                  simple-game-server
    UID:                   9960762e-c656-11e8-933e-fa163e07a1d4
  Resource Version:        6123197
  Self Link:               /apis/autoscaling.agones.dev/v1/namespaces/default/fleetautoscalers/simple-game-server-autoscaler
  UID:                     9fd0efa1-c656-11e8-933e-fa163e07a1d4
Spec:
  Fleet Name:  simple-game-server
  Policy:
    Buffer:
      Buffer Size:   2
      Max Replicas:  10
      Min Replicas:  2
    Type:            Buffer
Status:
  Able To Scale:     true
  Current Replicas:  2
  Desired Replicas:  2
  Last Scale Time:   <nil>
  Scaling Limited:   false
Events:              <none>

You can see the status (able to scale, not limited), the last time the fleet was scaled (nil for never) and the current and desired fleet size.

The autoscaler works by changing the desired size, and the fleet creates/deletes game server instances to achieve that number. The convergence is achieved in time, which is usually measured in seconds.

3. Allocate a Game Server from the Fleet

If you’re interested in more details for game server allocation, you should consult the Create a Game Server Fleet page. In here we are only interested in triggering allocations to see the autoscaler in action.

kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/gameserverallocation.yaml -o yaml

You should get in return the allocated game server details, which should end with something like:

status:
  address: 34.94.118.237
  gameServerName: simple-game-server-v6jwb-6bzkz
  nodeName: gke-test-cluster-default-f11755a7-5km3
  ports:
  - name: default
    port: 7832
  state: Allocated

Note the address and port, you might need them later to connect to the server.

4. See the autoscaler in action

Now let’s wait a few seconds to allow the autoscaler to detect the change in the fleet and check again its status

kubectl describe fleetautoscaler simple-game-server-autoscaler

The last part should look something like this:

Spec:
  Fleet Name:  simple-game-server
  Policy:
    Buffer:
      Buffer Size:   2
      Max Replicas:  10
      Min Replicas:  2
    Type:            Buffer
Status:
  Able To Scale:     true
  Current Replicas:  3
  Desired Replicas:  3
  Last Scale Time:   2018-10-02T16:00:02Z
  Scaling Limited:   false
Events:
  Type    Reason            Age   From                        Message
  ----    ------            ----  ----                        -------
  Normal  AutoScalingFleet  2m    fleetautoscaler-controller  Scaling fleet simple-game-server from 2 to 3

You can see that the fleet size has increased, the autoscaler having compensated for the allocated instance. Last Scale Time has been updated, and a scaling event has been logged.

Double-check the actual number of game server instances and status by running

kubectl get gs

This will get you a list of all the current GameServers and their Status > State.

NAME                             STATE       ADDRESS        PORT     NODE        AGE
simple-game-server-mzhrl-hz8wk   Allocated   10.30.64.99    7131     minikube    5m
simple-game-server-mzhrl-k6jg5   Ready       10.30.64.100   7243     minikube    5m  
simple-game-server-mzhrl-n2sk2   Ready       10.30.64.168   7658     minikube    5m

5. Shut the allocated instance down

Since we’ve only got one allocation, we’ll just grab the details of the IP and port of the only allocated GameServer:

kubectl get gameservers | grep Allocated | awk '{print $3":"$4 }'

This should output your Game Server IP address and port. (eg 10.130.65.208:7936)

You can now communicate with the GameServer:

nc -u {IP} {PORT}
Hello World !
ACK: Hello World !
EXIT

You can finally type EXIT which tells the SDK to run the Shutdown command, and therefore shuts down the GameServer.

6. See the fleet scaling down

Now let’s wait a few seconds to allow the autoscaler to detect the change in the fleet and check again its status

kubectl describe fleetautoscaler simple-game-server-autoscaler

It should look something like this:

Spec:
  Fleet Name:  simple-game-server
  Policy:
    Buffer:
      Buffer Size:   2
      Max Replicas:  10
      Min Replicas:  2
    Type:            Buffer
Status:
  Able To Scale:     true
  Current Replicas:  3
  Desired Replicas:  2
  Last Scale Time:   2018-10-02T16:09:02Z
  Scaling Limited:   false
Events:
  Type    Reason            Age   From                        Message
  ----    ------            ----  ----                        -------
  Normal  AutoScalingFleet  9m    fleetautoscaler-controller  Scaling fleet simple-game-server from 2 to 3
  Normal  AutoScalingFleet  45s   fleetautoscaler-controller  Scaling fleet simple-game-server from 3 to 2

You can see that the fleet size has decreased, the autoscaler adjusting to game server instance being de-allocated, the Last Scale Time and the events have been updated. Note that simple-game-server game server instance you just closed earlier might stay a bit in ‘Unhealthy’ state (and its pod in ‘Terminating’ until it gets removed.

Double-check the actual number of game server instances and status by running

kubectl get gs

This will get you a list of all the current GameServers and their Status > State.

NAME                             STATE     ADDRESS        PORT    NODE       AGE
simple-game-server-mzhrl-k6jg5   Ready     10.30.64.100   7243    minikube   5m
simple-game-server-mzhrl-t7944   Ready     10.30.64.168   7561    minikube   5m

7. Change autoscaling parameters

We can also change the configuration of the FleetAutoscaler of the running Fleet, and have the changes applied live, without interruptions of service.

Run kubectl edit fleetautoscaler simple-game-server-autoscaler and set the bufferSize field to 5.

Let’s look at the list of game servers again. Run watch kubectl get gs until you can see that are 5 ready server instances:

NAME                             STATE     ADDRESS        PORT    NODE         AGE
simple-game-server-mzhrl-7jpkp   Ready     10.30.64.100   7019    minikube     5m
simple-game-server-mzhrl-czt8v   Ready     10.30.64.168   7556    minikube     5m
simple-game-server-mzhrl-k6jg5   Ready     10.30.64.100   7243    minikube     5m
simple-game-server-mzhrl-nb8h2   Ready     10.30.64.168   7357    minikube     5m
simple-game-server-mzhrl-qspb6   Ready     10.30.64.99    7859    minikube     5m
simple-game-server-mzhrl-zg9rq   Ready     10.30.64.99    7745    minikube     5m

Next Steps

Read the advanced Scheduling and Autoscaling guide, for more details on autoscaling.

If you want to use your own GameServer container make sure you have properly integrated the Code Blind SDK.

4 - Quickstart: Create a Fleet Autoscaler with Webhook Policy

In some cases, your game servers may need to use custom logic for scaling your fleet that is more complex than what can be expressed using the Buffer policy in the fleetautoscaler. This guide shows how you can extend Code Blind with an autoscaler webhook to implement a custom autoscaling policy.

When you use an autoscaler webhook the logic computing the number of target replicas is delegated to an external HTTP/S endpoint, such as one provided by a Kubernetes deployment and service in the same cluster (as shown in the examples below). The fleetautoscaler will send a request to the webhook autoscaler’s /scale endpoint every sync period (currently 30s) with a JSON body, and scale the target fleet based on the data that is returned.

Chapter 1 Configuring HTTP fleetautoscaler webhook

Prerequisites

It is assumed that you have completed the instructions to Create a Game Server Fleet and have a running fleet of game servers.

Objectives

• Run a fleet
• Deploy the Webhook Pod and service for autoscaling
• Create a Fleet Autoscaler with Webhook policy type in Kubernetes using Code Blind custom resource
• Watch the Fleet scales up when allocating GameServers
• Watch the Fleet scales down after GameServer shutdown

1. Deploy the fleet

Run a fleet in a cluster:

kubectl apply -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/fleet.yaml

2. Deploy a Webhook service for autoscaling

In this step we would deploy an example webhook that will control the size of the fleet based on allocated gameservers portion in a fleet. You can see the source code for this example webhook server here. The fleetautoscaler would trigger this endpoint every 30 seconds. More details could be found also here. We need to create a pod which will handle HTTP requests with json payload FleetAutoscaleReview and return back it with FleetAutoscaleResponse populated.

The Scale flag and Replicas values returned in the FleetAutoscaleResponse tells the FleetAutoscaler what target size the backing Fleet should be scaled up or down to. If Scale is false - no scaling occurs.

Run next command to create a service and a Webhook pod in a cluster:

kubectl apply -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/autoscaler-webhook/autoscaler-service.yaml

To check that it is running and liveness probe is fine:

kubectl describe pod autoscaler-webhook

Name:           autoscaler-webhook-86944884c4-sdtqh
Namespace:      default
Node:           gke-test-cluster-default-1c5dec79-h0tq/10.138.0.2
...
Status:         Running

3. Create a Fleet Autoscaler

Let’s create a Fleet Autoscaler using the following command:

kubectl apply -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/webhookfleetautoscaler.yaml

You should see a successful output similar to this:

fleetautoscaler.autoscaling.agones.dev "webhook-fleet-autoscaler" created

This has created a FleetAutoscaler record inside Kubernetes. It has the link to Webhook service we deployed above.

4. See the fleet and autoscaler status.

In order to track the list of gameservers which run in your fleet you can run this command in a separate terminal tab:

 watch "kubectl get gs -n default"

In order to get autoscaler status use the following command:

kubectl describe fleetautoscaler webhook-fleet-autoscaler

It should look something like this:

Name:         webhook-fleet-autoscaler
Namespace:    default
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration={"apiVersion":
"autoscaling.agones.dev/v1","kind":"FleetAutoscaler","metadata":{"annotations"
:{},"name":"webhook-fleet-autoscaler","namespace":"default...
API Version:  autoscaling.agones.dev/v1
Kind:         FleetAutoscaler
etadata:
  Cluster Name:
  Creation Timestamp:  2018-12-22T12:52:23Z
  Generation:          1
  Resource Version:    2274579
  Self Link:           /apis/autoscaling.agones.dev/v1/namespaces/default/fleet
autoscalers/webhook-fleet-autoscaler
  UID:                 6d03eae4-05e8-11e9-84c2-42010a8a01c9
Spec:
  Fleet Name:  simple-game-server
  Policy:
    Type:  Webhook
    Webhook:
      Service:
        Name:       autoscaler-webhook-service
        Namespace:  default
        Path:       scale
      URL:
Status:
  Able To Scale:     true
  Current Replicas:  2
  Desired Replicas:  2
  Last Scale Time:   <nil>
  Scaling Limited:   false
Events:              <none>

You can see the status (able to scale, not limited), the last time the fleet was scaled (nil for never), current and desired fleet size.

The autoscaler makes a query to a webhoook service deployed on step 1 and on response changing the target Replica size, and the fleet creates/deletes game server instances to achieve that number. The convergence is achieved in time, which is usually measured in seconds.

5. Allocate Game Servers from the Fleet to trigger scale up

If you’re interested in more details for game server allocation, you should consult the Create a Game Server Fleet page. Here we only interested in triggering allocations to see the autoscaler in action.

kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/gameserverallocation.yaml -o yaml

You should get in return the allocated game server details, which should end with something like:

status:
  address: 34.94.118.237
  gameServerName: simple-game-server-v6jwb-6bzkz
  nodeName: gke-test-cluster-default-f11755a7-5km3
  ports:
  - name: default
    port: 7832
  state: Allocated

Note the address and port, you might need them later to connect to the server.

Run the kubectl command one more time so that we have both servers allocated:

kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/gameserverallocation.yaml -o yaml

6. Check new Autoscaler and Fleet status

Now let’s wait a few seconds to allow the autoscaler to detect the change in the fleet and check again its status

kubectl describe fleetautoscaler webhook-fleet-autoscaler

The last part should look similar to this:

Spec:
  Fleet Name:  simple-game-server
  Policy:
    Type:  Webhook
    Webhook:
      Service:
        Name:       autoscaler-webhook-service
        Namespace:  default
        Path:       scale
      URL:
Status:
  Able To Scale:     true
  Current Replicas:  4
  Desired Replicas:  4
  Last Scale Time:   2018-12-22T12:53:47Z
  Scaling Limited:   false
Events:
  Type    Reason            Age   From                        Message
  ----    ------            ----  ----                        -------
  Normal  AutoScalingFleet  35s   fleetautoscaler-controller  Scaling fleet simple-game-server from 2 to 4

You can see that the fleet size has increased in particular case doubled to 4 gameservers (based on our custom logic in our webhook), the autoscaler having compensated for the two allocated instances. Last Scale Time has been updated and a scaling event has been logged.

Double-check the actual number of game server instances and status by running:

 kubectl get gs -n default

This will get you a list of all the current GameServers and their Status > State.

NAME                     STATE       ADDRESS         PORT     NODE        AGE
simple-game-server-dmkp4-8pkk2   Ready       35.247.13.175   7386     minikube     5m
simple-game-server-dmkp4-b7x87   Allocated   35.247.13.175   7219     minikube     5m
simple-game-server-dmkp4-r4qtt   Allocated   35.247.13.175   7220     minikube     5m
simple-game-server-dmkp4-rsr6n   Ready       35.247.13.175   7297     minikube     5m

7. Check downscaling using Webhook Autoscaler policy

Based on our custom webhook deployed earlier, if the fraction of allocated replicas in whole Replicas count would be less than threshold (0.3) then the fleet would scale down by scaleFactor, in our example by 2.

Note that the example webhook server has a limitation that it would not decrease fleet replica count under minReplicasCount, which is equal to 2.

We need to run EXIT command on one gameserver (Use IP address and port of the allocated gameserver from the previous step) in order to decrease the number of allocated gameservers in a fleet (<0.3).

nc -u 35.247.13.175 7220
EXIT

Server would be in shutdown state. Wait about 30 seconds. Then you should see scaling down event in the output of next command:

kubectl describe fleetautoscaler webhook-fleet-autoscaler

You should see these lines in events:

  Normal   AutoScalingFleet  11m                fleetautoscaler-controller  Scaling fleet simple-game-server from 2 to 4
  Normal   AutoScalingFleet  1m                 fleetautoscaler-controller  Scaling fleet simple-game-server from 4 to 2

And get gameservers command output:

kubectl get gs -n default

NAME                             STATUS      ADDRESS          PORT     NODE       AGE
simple-game-server-884fg-6q5sk   Ready       35.247.117.202   7373     minikube   5m
simple-game-server-884fg-b7l58   Allocated   35.247.117.202   7766     minikube   5m

8. Cleanup

You can delete the autoscaler service and associated resources with the following commands.

kubectl delete -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/autoscaler-webhook/autoscaler-service.yaml

Removing the fleet:

kubectl delete -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/fleet.yaml

Chapter 2 Configuring HTTPS fleetautoscaler webhook with CA Bundle

Objectives

Using TLS and a certificate authority (CA) bundle we can establish trusted communication between Fleetautoscaler and an HTTPS server running the autoscaling webhook that controls the size of the fleet (Replicas count). The certificate of the autoscaling webhook must be signed by the CA provided in fleetautoscaler yaml configuration file. Using TLS eliminates the possibility of a man-in-the-middle attack between the fleetautoscaler and the autoscaling webhook.

1. Deploy the fleet

Run a fleet in a cluster:

kubectl apply -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/fleet.yaml

2. Create X509 Root and Webhook certificates

The procedure of generating a Self-signed CA certificate is as follows:

The first step is to create the private root key:

openssl genrsa -out rootCA.key 2048

The next step is to self-sign this certificate:

openssl req -x509 -new -nodes -key rootCA.key -sha256 -days 1024 -out rootCA.pem

This will start an interactive script that will ask you for various bits of information. Fill it out as you see fit.

Every webhook that you wish to install a trusted certificate will need to go through this process. First, just like with the root CA step, you’ll need to create a private key (different from the root CA):

openssl genrsa -out webhook.key 2048

Next create configuration file cert.conf for the certificate signing request:

[req]
distinguished_name = req_distinguished_name
req_extensions = v3_req
prompt = no
[req_distinguished_name]
CN = autoscaler-tls-service.default.svc
[v3_req]
keyUsage = digitalSignature
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1 = autoscaler-tls-service.default.svc

Generate the certificate signing request, use valid hostname which in this case will be autoscaler-tls-service.default.svc as Common Name (eg, fully qualified host name) as well as DNS.1 in the alt_names section of the config file.

Check the Kubernetes documentation to see how Services get assigned DNS entries.

openssl req -new -out webhook.csr -key webhook.key -config cert.conf

Once that’s done, you’ll sign the CSR, which requires the CA root key:

openssl x509 -req -in webhook.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out webhook.crt -days 500 -sha256 -extfile cert.conf -extensions v3_req

This would generate webhook.crt certificate

Add secret which later would be mounted to autoscaler-webhook-tls pod.

kubectl create secret tls autoscalersecret --cert=webhook.crt --key=webhook.key

You need to put Base64-encoded string into caBundle field in your fleetautoscaler yaml configuration:

base64 -i ./rootCA.pem

Copy the output of the command above and replace the caBundle field in your text editor (say vim) with the new value:

wget https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/webhookfleetautoscalertls.yaml
vim ./webhookfleetautoscalertls.yaml

3. Deploy a Webhook service for autoscaling

Run next command to create a service and a Webhook pod in a cluster:

kubectl apply -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/autoscaler-webhook/autoscaler-service-tls.yaml

To check that it is running and liveness probe is fine:

kubectl describe pod autoscaler-webhook-tls

Wait for the Running status results:

Name:               autoscaler-webhook-tls-f74c9bff7-ssrsc
Namespace:          default
...
Status:         Running

4. Create a Fleet Autoscaler

Let’s create a Fleet Autoscaler using the following command (caBundle should be set properly on Step 2):

kubectl apply -f ./webhookfleetautoscalertls.yaml

5. See the fleet and autoscaler status.

In order to track the list of gameservers which run in your fleet you can run this command in a separate terminal tab:

 watch "kubectl get gs -n default"

6. Allocate two Game Servers from the Fleet to trigger scale up

If you’re interested in more details for game server allocation, you should consult the Create a Game Server Fleet page. Here we only interested in triggering allocations to see the autoscaler in action.

for i in {0..1} ; do kubectl create -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/gameserverallocation.yaml -o yaml ; done

7. Check new Autoscaler and Fleet status

Now let’s wait a few seconds to allow the autoscaler to detect the change in the fleet and check again its status

kubectl describe fleetautoscaler  webhook-fleetautoscaler-tls

The last part should look similar to this:

Spec:
  Fleet Name:  simple-game-server
  Policy:
    Type:  Webhook
    Webhook:
      Ca Bundle:  LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN1RENDQWFBQ0NRQ29kcEFNbTlTd0pqQU5CZ2txaGtpRzl3MEJBUXNGQURBZU1Rc3dDUVlEVlFRR0V3SlYKVXpFUE1BMEdBMVVFQ3d3R1FXZHZibVZ6TUI0WERURTVNREV3TkRFeE5URTBORm9YRFRJeE1UQXlOREV4TlRFMApORm93SGpFTE1Ba0dBMVVFQmhNQ1ZWTXhEekFOQmdOVkJBc01Ca0ZuYjI1bGN6Q0NBU0l3RFFZSktvWklodmNOCkFRRUJCUUFEZ2dFUEFEQ0NBUW9DZ2dFQkFOQ0h5dndDOTZwZDlTdkFhMUIvRWg2ekcxeDBLS1dPaVhtNzhJcngKKzZ5WHd5YVpsMVo1cVExbUZoOThMSGVZUmQwWVgzRTJnelZ5bFpvUlUra1ZESzRUc0VzV0tNUFVpdVo0MUVrdApwbythbEN6alAyaXZzRGZaOGEvdnByL3dZZ2FrWGtWalBUaGpKUk9xTnFIdWROMjZVcUFJYnNOTVpoUkxkOVFFCnFLSjRPNmFHNVMxTVNqZFRGVHFlbHJiZitDcXNKaHltZEIzZmxGRUVvdXExSmoxS0RoQjRXWlNTbS9VSnpCNkcKNHUzY3BlQm1jTFVRR202ZlFHb2JFQSt5SlpMaEVXcXBrd3ZVZ2dCNmRzWE8xZFNIZXhhZmlDOUVUWGxVdFRhZwo1U2JOeTVoYWRWUVV3Z253U0J2djR2R0t1UUxXcWdXc0JyazB5Wll4Sk5Bb0V5RUNBd0VBQVRBTkJna3Foa2lHCjl3MEJBUXNGQUFPQ0FRRUFRMkgzaWJRcWYzQTNES2l1eGJISURkbll6TlZ2Z0dhRFpwaVZyM25ocm55dmxlNVgKR09hRm0rMjdRRjRWV29FMzZDTGhYZHpEWlM4bEpIY09YUW5KOU83Y2pPYzkxVmh1S2NmSHgwS09hU1oweVNrVAp2bEtXazlBNFdoNGE0QXFZSlc3Z3BUVHR1UFpydnc4VGsvbjFaWEZOYVdBeDd5RU5OdVdiODhoNGRBRDVaTzRzCkc5SHJIdlpuTTNXQzFBUXA0Q3laRjVyQ1I2dkVFOWRkUmlKb3IzM3pLZTRoRkJvN0JFTklZZXNzZVlxRStkcDMKK0g4TW5LODRXeDFUZ1N5Vkp5OHlMbXFpdTJ1aThjaDFIZnh0OFpjcHg3dXA2SEZLRlRsTjlBeXZUaXYxYTBYLwpEVTk1eTEwdi9oTlc0WHpuMDJHNGhrcjhzaUduSEcrUEprT3hBdz09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
      Service:    <nil>
      URL:        https://autoscaler-tls-service.default.svc:8000/scale
Events:
  Type    Reason            Age   From                        Message
  ----    ------            ----  ----                        -------
  Normal  AutoScalingFleet  5s   fleetautoscaler-controller  Scaling fleet simple-game-server from 2 to 4

You can see that the fleet size has increased in particular case doubled to 4 gameservers (based on our custom logic in our webhook), the autoscaler having compensated for the two allocated instances. Last Scale Time has been updated and a scaling event has been logged.

Double-check the actual number of game server instances and status by running:

 kubectl get gs -n default

This will get you a list of all the current GameServers and their Status > State.

NAME                     STATE       ADDRESS         PORT      NODE      AGE
simple-game-server-njmr7-2t4nx   Ready       35.203.159.68   7330      minikube   1m
simple-game-server-njmr7-65rp6   Allocated   35.203.159.68   7294      minikube   4m

8. Cleanup

You can delete the autoscaler service and associated resources with the following commands.

kubectl delete -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/autoscaler-webhook/autoscaler-service-tls.yaml

Removing x509 key secret:

kubectl delete secret autoscalersecret

Removing the fleet:

kubectl delete -f https://raw.githubusercontent.com/googleforgames/agones/release-1.38.0/examples/simple-game-server/fleet.yaml

Comments

Note that secure communication has been established and we can trust that communication between the fleetautoscaler and the autoscaling webhook. If you need to run the autoscaling webhook outside of the Kubernetes cluster, you can use another root certificate authority as long as you put it into the caBundle parameter in fleetautoscaler configuration (in pem format, base64-encoded).

Troubleshooting Guide

If you run into problems with the configuration of your fleetautoscaler and webhook service the easiest way to debug them is to run:

kubectl describe fleetautoscaler <FleetAutoScalerName>

and inspect the events at the bottom of the output.

Common error messages.

If you have configured the wrong service Path for the FleetAutoscaler you will see a message like

Error calculating desired fleet size on FleetAutoscaler simple-fleet-r7fdv-autoscaler. Error: bad status code 404 from the server: https://autoscaler-tls-service.default.svc:8000/scale

If you are using a hostname other than autoscaler-tls-service.default.svc as the Common Name (eg, fully qualified host name) when creating a certificate using openssl tool you will see a message like

Post https://autoscaler-tls-service.default.svc:8000/scale: x509: certificate is not valid for any names, but wanted to match autoscaler-tls-service.default.svc

If you see errors like the following in autoscaler-webhook-tls pod logs:

http: TLS handshake error from 10.48.3.125:33374: remote error: tls: bad certificate

Then there could be an issue with your ./rootCA.pem.

You can repeat the process from step 2, in order to fix your certificates setup.

Next Steps

Read the advanced Scheduling and Autoscaling guide, for more details on autoscaling.

If you want to use your own GameServer container make sure you have properly integrated the Code Blind SDK.

5 - Quickstart: Edit a Game Server

This guide addresses Google Kubernetes Engine and Minikube. We would welcome a Pull Request to expand this to include other platforms as well.

Prerequisites

1. A Go environment
2. Docker
3. Code Blind installed on GKE or Minikube
4. kubectl properly configured

To install on GKE, follow the install instructions (if you haven’t already) at Setting up a Google Kubernetes Engine (GKE) cluster. Also complete the “Enabling creation of RBAC resources” and “Installing Code Blind” sets of instructions on the same page.

To install locally on Minikube, read Setting up a Minikube cluster. Also complete the “Enabling creation of RBAC resources” and “Installing Code Blind” sets of instructions on the same page.

Modify the code and push another new image

Modify the simple-game-server example source code

Modify the main.go file. For example:

Change the following line in main.go:

From:

respond(conn, sender, "ACK: "+txt+"\n")

To:

respond(conn, sender, "ACK: Echo says "+txt+"\n")

Build Server

Since Docker image is using Alpine Linux, the “go build” command has to include few more environment variables.

go get agones.dev/agones/pkg/sdk
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o bin/server -a -v main.go

Using Docker File

Create a new docker image

docker build -t gcr.io/[PROJECT_ID]/agones-simple-game-server:modified .

Note: you can change the image name “agones-simple-game-server” to something else.

If using GKE, push the image to GCP Registry

docker push gcr.io/[PROJECT_ID]/agones-simple-game-server:modified

Note: Review Authentication Methods for additional information regarding use of gcloud as a Docker credential helper and advanced authentication methods to the Google Container Registry.

If using Minikube, load the image into Minikube

minikube cache add gcr.io/[PROJECT_ID]/agones-agones-simple-game-server:modified

Modify gameserver.yaml

Modify the following line from gameserver.yaml to use the new configuration.

spec:
  containers:
  - name: agones-simple-game-server
    image: gcr.io/[PROJECT_ID]/agones-simple-game-server:modified

If using GKE, deploy Server to GKE

Apply the latest settings to the Kubernetes container.

gcloud config set container/cluster [CLUSTER_NAME]
gcloud container clusters get-credentials [CLUSTER_NAME]
kubectl apply -f gameserver.yaml

If using Minikube, deploy the Server to Minikube

kubectl apply -f gameserver.yaml

Check the GameServer Status

kubectl describe gameserver

Verify

Let’s retrieve the IP address and the allocated port of your Game Server:

kubectl get gs simple-game-server -o jsonpath='{.status.address}:{.status.ports[0].port}'

You can now communicate with the Game Server :

nc -u {IP} {PORT}
Hello World!
ACK:  Echo says  Hello World!
EXIT

You can finally type EXIT which tells the SDK to run the Shutdown command, and therefore shuts down the GameServer.

If you run kubectl describe gameserver again - either the GameServer will be gone completely, or it will be in Shutdown state, on the way to being deleted.

Next Steps

If you want to perform rolling updates of modified game servers, see Quickstart Create a Game Server Fleet.