diff --git a/.gitignore b/.gitignore index ed9d900..c8418b8 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,5 @@ ec2gaming.ovpn ec2gaming.rdp ec2gaming.auth +ec2gaming.bat +ec2gaming-permissionpolicy.json diff --git a/README.md b/README.md index f82d98b..35f1a45 100644 --- a/README.md +++ b/README.md @@ -61,7 +61,9 @@ Use `ec2gaming.cfg` to configure options such as the bid price over the minimum # First-time configuration -The goal is to take the base [ec2gaming AMI](http://lg.io/2015/07/05/revised-and-much-faster-run-your-own-highend-cloud-gaming-service-on-ec2.html) and create a reusable image. These steps assume that you already have: +The goal is to take the base [ec2gaming AMI](http://lg.io/2015/07/05/revised-and-much-faster-run-your-own-highend-cloud-gaming-service-on-ec2.html) and create a reusable image. This configuration differs from the original blog post, in that the goal is to keep the AMI immutable from session to session (call it an [institituional bias](http://techblog.netflix.com/2016/03/how-we-build-code-at-netflix.html) ;) + +These steps assume that you already have: - Homebrew and Homebrew Cask (but feel free to install the required software any way you like) - An Amazon AWS account, and have generated credentials from the AWS Console @@ -80,6 +82,13 @@ From a terminal: - Install Microsoft Remote Desktop from the App Store - Run `aws configure` to configure your AWS credentials and region +- Create a `ec2gaming.auth` file in the `ec2gaming` location (it's `.gitignored`) with two lines, the `administrator` username and the password you plan to use to login to the instance. It's used for later configuration and needs to be available before you proceed to the next step: + + ``` + administrator + + ``` + - Run `ec2gaming start` to bootstrap an instance ## EC2 Spot Instances @@ -111,13 +120,17 @@ Edit `C:\Program Files\OpenVPN\config\server.ovpn` and add `cipher none` to the - Install and run several of the games you intend to play to `Z:\`. This performs first-time installation, avoiding the redistributable installation overhead. Delete local files once you're done - Exit Steam from the system tray, otherwise Steam will forget your password when the instance next starts -## Steam remote install to ephemeral storage +## Ephemeral storage + +The `Z:\` is SSD instance storage, that's destroyed when the instance is stopped, but is the ideal place for installing games and data for your gaming session thanks to it's high performance. -The Steam remote install feature assumes the default Stream library, even if a second library is available and set to default. So, to install games to the emphemeral storage on `Z:\` remotely, we create a junction on instance startup: +The Steam remote install feature assumes the default Stream library, even if a second library is available and set to default. So, to install games to the emphemeral storage on `Z:\` remotely, we create a junction on instance startup. Unfortunately, you can't install games larger than the freespace on `C:` with this approach, but in that case you can use RDP/VNC to install the game. -- Copy `bootstrap/steamapps-junction.bat` to `C:\Users\Administrator\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup` +Steam Cloud will also do a decent job of saving, but it's good to have coverage for games that don't cloud save, or if your instance terminates and Steam doesn't have a chance to perform the cloud sync. So, we also want to configure a periodic sync to S3 to save Documents from `Z:\Documents`. -Unfortunately, you can't install games larger than the freespace on `C:` with this approach, but in that case you can use RDP and this avoids having to RDP for every installation. +- Install the 64-bit AWS CLI from https://aws.amazon.com/cli/ +- Copy `ec2gaming.bat` to `C:\Users\Administrator\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup`. This file creates the junction, performs an initial one-way S3 sync, and schedules a task that runs every minute to keep the directory in sync +- Right-click the Documents folder from the file explorer, and set the location to `Z:\Documents` ## Windows automatic login @@ -158,22 +171,10 @@ Driver 373.06 is the latest driver version that has the K520 included in the dev - Restart Windows and attach a VNC session (`ec2gaming vnc`) - Right-click the desktop, select Screen resolution and select the highest available resolution -## Cloud sync My Documents - -WIP - need to figure out how to allow sync that doesn't delete files if they're missing on the host. - -Steam Cloud will do a decent job, but it's good to have coverage for games that don't cloud save, or if your instance terminates and Steam doesn't have a chance to perform the cloud sync. - ## Final steps - Run `ec2gaming snapshot` to snapshot the EBS volume and create your AMI - Run `ec2gaming terminate` to terminate the instance -- Create a `ec2gaming.auth` file in the `ec2gaming` location (it's `.gitignored`) with two lines, it'll be used to authenticate the VPN for gaming: - - ``` - administrator - - ``` ## Steam client configuration @@ -194,7 +195,7 @@ On your Mac, go to Steam Home-Streaming settings and: # Periodic maintenance -This configuration differs from the original blog post, in that the goal is to keep the AMI immutable from session to session (call it an [institituional bias](http://techblog.netflix.com/2016/03/how-we-build-code-at-netflix.html) ;). You'll want to periodically update Steam, run Windows Update etc. and re-snapshot and replace your AMI using the `ec2gaming snapshot` command. +Because we treat the AMI as immutable, you'll want to periodically update Steam, run Windows Update etc. and re-snapshot and replace your AMI using the `ec2gaming snapshot` command. # Help diff --git a/bootstrap/steamapps-junction.bat b/bootstrap/steamapps-junction.bat deleted file mode 100644 index 709c5ac..0000000 --- a/bootstrap/steamapps-junction.bat +++ /dev/null @@ -1,4 +0,0 @@ -@echo off -rmdir /Q /S "C:\Program Files (x86)\Steam\steamapps" -md Z:\SteamLibrary\steamapps -cmd /c mklink /j "C:\Program Files (x86)\Steam\steamapps" Z:\SteamLibrary\steamapps diff --git a/ec2gaming-permissionpolicy.json.template b/ec2gaming-permissionpolicy.json.template new file mode 100644 index 0000000..76ee1c3 --- /dev/null +++ b/ec2gaming-permissionpolicy.json.template @@ -0,0 +1,19 @@ +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": ["s3:ListBucket"], + "Resource": ["arn:aws:s3:::BUCKET"] + }, + { + "Effect": "Allow", + "Action": [ + "s3:PutObject", + "s3:GetObject", + "s3:DeleteObject" + ], + "Resource": ["arn:aws:s3:::BUCKET/*"] + } + ] +} diff --git a/ec2gaming-start.sh b/ec2gaming-start.sh index fa34a90..aebbe22 100755 --- a/ec2gaming-start.sh +++ b/ec2gaming-start.sh @@ -36,9 +36,34 @@ if [ -z "$EC2_SECURITY_GROUP_ID" ]; then aws ec2 authorize-security-group-ingress --group-name ec2gaming --protocol udp --port 1194 --cidr "0.0.0.0/0" EC2_SECURITY_GROUP_ID=$(describe_security_group ec2gaming) fi - echo "$EC2_SECURITY_GROUP_ID" +echo -n "Looking for S3 bucket... " +ACCOUNT_ID=$(aws iam get-user | jq '.User.Arn' | cut -d ':' -f 5) +BUCKET="ec2gaming-$ACCOUNT_ID" +if ! aws s3api head-bucket --bucket "$BUCKET" &> /dev/null; then + echo -n "not found. Creating... " + REGION=$(aws configure get region) + aws s3api create-bucket --bucket "$BUCKET" --region "$REGION" --create-bucket-configuration LocationConstraint="$REGION" > /dev/null +fi +USERNAME=$(head -1 ec2gaming.auth) +PASSWORD=$(tail -1 ec2gaming.auth) +sed "s/BUCKET/$BUCKET/g;s/USERNAME/$USERNAME/g;s/PASSWORD/$PASSWORD/g" ec2gaming.bat.template > ec2gaming.bat +echo "$BUCKET" + +PROFILE_NAME="ec2gaming" +echo -n "Looking for instance profile... " +if ! aws iam get-instance-profile --instance-profile-name ec2gaming &> /dev/null; then + echo -n "not found. Creating... " + aws iam create-role --role-name "$PROFILE_NAME" --assume-role-policy-document file://ec2gaming-trustpolicy.json > /dev/null + sed "s/BUCKET/$BUCKET/g" ec2gaming-permissionpolicy.json.template > ec2gaming-permissionpolicy.json > /dev/null + aws iam put-role-policy --role-name "$PROFILE_NAME" --policy-name "$PROFILE_NAME" --policy-document file://ec2gaming-permissionpolicy.json > /dev/null + aws iam create-instance-profile --instance-profile-name "$PROFILE_NAME" + aws iam add-role-to-instance-profile --instance-profile-name "$PROFILE_NAME" --role-name "$PROFILE_NAME" +fi +INSTANCE_PROFILE_ARN=$(aws iam get-instance-profile --instance-profile-name ec2gaming | jq -r '.InstanceProfile.Arn') +echo "$INSTANCE_PROFILE_ARN" + echo -n "Creating spot instance request... " SPOT_INSTANCE_ID=$(aws ec2 request-spot-instances --spot-price "$FINAL_SPOT_PRICE" --launch-specification " { @@ -47,6 +72,9 @@ SPOT_INSTANCE_ID=$(aws ec2 request-spot-instances --spot-price "$FINAL_SPOT_PRIC \"InstanceType\": \"$INSTANCE_TYPE\", \"Placement\": { \"AvailabilityZone\": \"$ZONE\" + }, + \"IamInstanceProfile\": { + \"Arn\": \"$INSTANCE_PROFILE_ARN\" } }" | jq --raw-output '.SpotInstanceRequests[0].SpotInstanceRequestId') echo "$SPOT_INSTANCE_ID" diff --git a/ec2gaming-trustpolicy.json b/ec2gaming-trustpolicy.json new file mode 100644 index 0000000..eb59115 --- /dev/null +++ b/ec2gaming-trustpolicy.json @@ -0,0 +1,8 @@ +{ + "Version": "2012-10-17", + "Statement": { + "Effect": "Allow", + "Principal": {"Service": "ec2.amazonaws.com"}, + "Action": "sts:AssumeRole" + } +} diff --git a/ec2gaming-vnc.sh b/ec2gaming-vnc.sh index 718bad7..5ade617 100755 --- a/ec2gaming-vnc.sh +++ b/ec2gaming-vnc.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash source "$(dirname "$0")/ec2gaming.header" -USER=$(head -1 ec2gaming.auth) -PASSWD=$(tail -1 ec2gaming.auth) -open "vnc://$USER:$PASSWD@10.8.0.1" +USERNAME=$(head -1 ec2gaming.auth) +PASSWORD=$(tail -1 ec2gaming.auth) +open "vnc://$USERNAME:$PASSWORD@10.8.0.1" diff --git a/ec2gaming.bat.template b/ec2gaming.bat.template new file mode 100644 index 0000000..f2c829b --- /dev/null +++ b/ec2gaming.bat.template @@ -0,0 +1,7 @@ +@echo off +rmdir /Q /S "C:\Program Files (x86)\Steam\steamapps" +md Z:\SteamLibrary\steamapps +cmd /c mklink /j "C:\Program Files (x86)\Steam\steamapps" Z:\SteamLibrary\steamapps +md Z:\Documents +aws s3 sync Z:\Documents s3://BUCKET/Documents +schtasks /Create /RU USERNAME /RP PASSWORD /SC MINUTE /MO 1 /TN "Sync Documents with S3" /TR "aws s3 sync Z:\Documents s3://ec2gaming-639801188054/Documents --delete"