Skip to content
This repository was archived by the owner on Jan 25, 2023. It is now read-only.

Resurrection of #173 - Consul Service Mesh #202

Open
wants to merge 34 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
ff3a7e7
First commit, updated READMEs
7hacker May 14, 2020
09b16dd
Is Atom snipping off ws?
7hacker May 14, 2020
16781ed
Is Atom snipping off ws?
7hacker May 14, 2020
02a4c57
revert it works?
7hacker May 14, 2020
09656be
disabled ws package
7hacker May 14, 2020
995b63e
set up connect
7hacker May 14, 2020
893e24c
Example scripts
7hacker May 14, 2020
22cb4d9
Basic test that only verifies that a consul cluster with connect turn…
7hacker May 14, 2020
548eda7
testing if a CA comes up
7hacker May 14, 2020
cef1e03
fix tests
7hacker May 14, 2020
05fa6ed
I think this is a good first refactor to tease out the connect tests.…
7hacker May 14, 2020
042ec0a
clean up names
7hacker May 14, 2020
351283f
start 2 servies and their proxies in a user-data client
7hacker May 14, 2020
4b618ca
need to register services before starting the proxies
7hacker May 14, 2020
aee822c
addressing some comments in PR
7hacker May 14, 2020
4bf4132
Updated the readme with production notes
7hacker May 14, 2020
ea11497
clean up my keys/amis
7hacker May 14, 2020
6bb88de
more cleanup
7hacker May 14, 2020
b80d789
Update modules/run-consul/README.md
7hacker May 15, 2020
943b3dd
Update examples/example-with-consul-connect/user-data-client.sh
7hacker May 15, 2020
4a3a982
added a paragraph on running services and proxies in the run-consul r…
7hacker May 15, 2020
6156505
Update examples/example-with-consul-connect/main.tf
7hacker May 15, 2020
7be86c1
Update examples/example-with-consul-connect/README.md
7hacker May 15, 2020
306f9c6
:twisted_rightwards_arrows: Merge branch 'master' of https://github.c…
MatthiasScholzTW Jan 12, 2021
ff16c3a
:recycle: making enable_connect local
MatthiasScholzTW Jan 12, 2021
494ec73
:wastebasket: cleanup default ssh key
MatthiasScholzTW Jan 12, 2021
d8dff71
:recycle: complete tf 0.12 migration
MatthiasScholzTW Jan 12, 2021
539b521
:art: removed trailing space
MatthiasScholzTW Jan 12, 2021
35b324c
:art: minor beautification
MatthiasScholzTW Jan 12, 2021
a7c21e9
:bug: copy and paste error - test naming
MatthiasScholzTW Jan 12, 2021
b64e85d
:twisted_rightwards_arrows: Merge branch 'master' of https://github.c…
MatthiasScholzTW Jan 22, 2021
a4295b8
:white_check_mark: test consul connect configuration creation
MatthiasScholzTW Apr 17, 2021
2a9aea2
Merge branch 'master' into master
MatthiasScholz Apr 17, 2021
8c97591
:bug: missing parameter shifting
MatthiasScholzTW Apr 18, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
disabled ws package
7hacker committed May 14, 2020
commit 09656be75dac880389d7d1281da112c4b4fda2b5
68 changes: 34 additions & 34 deletions modules/run-consul/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Consul Run Script

This folder contains a script for configuring and running Consul on an [AWS](https://aws.amazon.com/) server. This
This folder contains a script for configuring and running Consul on an [AWS](https://aws.amazon.com/) server. This
script has been tested on the following operating systems:

* Ubuntu 16.04
@@ -14,8 +14,8 @@ There is a good chance it will work on other flavors of Debian, CentOS, and RHEL

## Quick start

This script assumes you installed it, plus all of its dependencies (including Consul itself), using the [install-consul
module](https://github.com/hashicorp/terraform-aws-consul/tree/master/modules/install-consul). The default install path is `/opt/consul/bin`, so to start Consul in server mode,
This script assumes you installed it, plus all of its dependencies (including Consul itself), using the [install-consul
module](https://github.com/hashicorp/terraform-aws-consul/tree/master/modules/install-consul). The default install path is `/opt/consul/bin`, so to start Consul in server mode,
you run:

```
@@ -33,22 +33,22 @@ This will:
1. Generate a Consul configuration file called `default.json` in the Consul config dir (default: `/opt/consul/config`).
See [Consul configuration](#consul-configuration) for details on what this configuration file will contain and how
to override it with your own configuration.

1. Generate a [systemd](https://www.freedesktop.org/wiki/Software/systemd/) configuration file called `consul.service` in the systemd
config dir (default: `/etc/systemd/system`) with a command that will run Consul:
`consul agent -config-dir=/opt/consul/config -data-dir=/opt/consul/data`.

1. Tell systemd to load the new configuration file, thereby starting Consul.

We recommend using the `run-consul` command as part of [User
We recommend using the `run-consul` command as part of [User
Data](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html#user-data-shell-scripts), so that it executes
when the EC2 Instance is first booting. After runing `run-consul` on that initial boot, the `systemd` configuration
when the EC2 Instance is first booting. After runing `run-consul` on that initial boot, the `systemd` configuration
will automatically restart Consul if it crashes or the EC2 instance reboots.

Note that `systemd` logs to its own journal by default. To view the Consul logs, run `journalctl -u consul.service`. To change
the log output location, you can specify the `StandardOutput` and `StandardError` options by using the `--systemd-stdout` and `--systemd-stderr`
options. See the [`systemd.exec` man pages](https://www.freedesktop.org/software/systemd/man/systemd.exec.html#StandardOutput=) for available
options, but note that the `file:path` option requires [systemd version >= 236](https://stackoverflow.com/a/48052152), which is not provided
options, but note that the `file:path` option requires [systemd version >= 236](https://stackoverflow.com/a/48052152), which is not provided
in the base Ubuntu 16.04 and Amazon Linux 2 images.

See the [consul-cluster example](https://github.com/hashicorp/terraform-aws-consul/tree/master/examples/root-example) for fully-working sample code.
@@ -107,38 +107,38 @@ Example:

## Consul configuration

`run-consul` generates a configuration file for Consul called `default.json` that tries to figure out reasonable
defaults for a Consul cluster in AWS. Check out the [Consul Configuration Files
`run-consul` generates a configuration file for Consul called `default.json` that tries to figure out reasonable
defaults for a Consul cluster in AWS. Check out the [Consul Configuration Files
documentation](https://www.consul.io/docs/agent/options.html#configuration-files) for what configuration settings are
available.


### Default configuration

`run-consul` sets the following configuration values by default:

* [advertise_addr](https://www.consul.io/docs/agent/options.html#advertise_addr): Set to the EC2 Instance's private IP
* [advertise_addr](https://www.consul.io/docs/agent/options.html#advertise_addr): Set to the EC2 Instance's private IP
address, as fetched from [Metadata](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html).

* [bind_addr](https://www.consul.io/docs/agent/options.html#bind_addr): Set to the EC2 Instance's private IP address,
* [bind_addr](https://www.consul.io/docs/agent/options.html#bind_addr): Set to the EC2 Instance's private IP address,
as fetched from [Metadata](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html).

* [bootstrap_expect](https://www.consul.io/docs/agent/options.html#bootstrap_expect): If `--server` is set,
set this config based on the EC2 Instance's tags (using the
[describe-tags API](https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-tags.html)):
* If there is a `aws:autoscaling:groupName` tag, that means this EC2 Instance is part of an Auto Scaling Group
(ASG), so set this config to the desired capacity of the ASG (fetched via the [describe-auto-scaling-groups
API](https://docs.aws.amazon.com/cli/latest/reference/autoscaling/describe-auto-scaling-groups.html)).
* [bootstrap_expect](https://www.consul.io/docs/agent/options.html#bootstrap_expect): If `--server` is set,
set this config based on the EC2 Instance's tags (using the
[describe-tags API](https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-tags.html)):
* If there is a `aws:autoscaling:groupName` tag, that means this EC2 Instance is part of an Auto Scaling Group
(ASG), so set this config to the desired capacity of the ASG (fetched via the [describe-auto-scaling-groups
API](https://docs.aws.amazon.com/cli/latest/reference/autoscaling/describe-auto-scaling-groups.html)).
* Otherwise, log a warning, and set this to 1. This fallback is not recommended!

* [client_addr](https://www.consul.io/docs/agent/options.html#client_addr): Set to 0.0.0.0 so you can access the client
and UI endpoint on each EC2 Instance from the outside.

* [datacenter](https://www.consul.io/docs/agent/options.html#datacenter): Set to the current AWS region (e.g.
* [datacenter](https://www.consul.io/docs/agent/options.html#datacenter): Set to the current AWS region (e.g.
`us-east-1`), as fetched from [Metadata](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html).
If the `--datacenter` flag is provided, then that value is used instead.

* [node_name](https://www.consul.io/docs/agent/options.html#node_name): Set to the instance id, as fetched from
* [node_name](https://www.consul.io/docs/agent/options.html#node_name): Set to the instance id, as fetched from
[Metadata](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html).

* [retry_join_ec2](https://www.consul.io/docs/agent/options.html#retry_join_ec2): Look up the EC2 Instances tags
@@ -148,21 +148,21 @@ available.
argument.
* [tag_value](https://www.consul.io/docs/agent/options.html#tag_value): Set to the value this EC2 Instance has for
the `tag_key`. If the key is not set, then the `retry_join_ec2` setting will NOT be included in the config file.
* [region](https://www.consul.io/docs/agent/options.html#region): Set to the current AWS region (e.g. `us-east-1`),
* [region](https://www.consul.io/docs/agent/options.html#region): Set to the current AWS region (e.g. `us-east-1`),
as fetched from [Metadata](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html).

* [server](https://www.consul.io/docs/agent/options.html#server): Set to true if `--server` is set.

* [ui](https://www.consul.io/docs/agent/options.html#ui): Set to true.


### Overriding the configuration

To override the default configuration, simply put your own configuration file in the Consul config folder (default:
`/opt/consul/config`), but with a name that comes later in the alphabet than `default.json` (e.g.
`my-custom-config.json`). Consul will load all the `.json` configuration files in the config dir and
[merge them together in alphabetical order](https://www.consul.io/docs/agent/options.html#_config_dir), so that
settings in files that come later in the alphabet will override the earlier ones.
To override the default configuration, simply put your own configuration file in the Consul config folder (default:
`/opt/consul/config`), but with a name that comes later in the alphabet than `default.json` (e.g.
`my-custom-config.json`). Consul will load all the `.json` configuration files in the config dir and
[merge them together in alphabetical order](https://www.consul.io/docs/agent/options.html#_config_dir), so that
settings in files that come later in the alphabet will override the earlier ones.

For example, to override the default `retry_join_ec2` settings, you could create a file called `tags.json` with the
contents:
@@ -187,7 +187,7 @@ at all using the `--skip-consul-config` flag:

### Required permissions

The `run-consul` script assumes it is running on an EC2 Instance with an [IAM
The `run-consul` script assumes it is running on an EC2 Instance with an [IAM
Role](http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html) that has the following permissions:

* `ec2:DescribeInstances`
@@ -201,8 +201,8 @@ These permissions are automatically added by the [consul-cluster module](https:/

## How do you handle encryption?

Consul can encrypt all of its network traffic (see the [encryption docs for
details](https://www.consul.io/docs/agent/encryption.html)), but by default, encryption is not enabled in this
Consul can encrypt all of its network traffic (see the [encryption docs for
details](https://www.consul.io/docs/agent/encryption.html)), but by default, encryption is not enabled in this
Module. To enable encryption, you need to do the following:

1. [Gossip encryption: provide an encryption key](#gossip-encryption-provide-an-encryption-key)
@@ -248,7 +248,7 @@ dir (default location: `/opt/consul/config`):
```

You will also want to set the [verify_incoming](https://www.consul.io/docs/agent/options.html#verify_incoming) and
[verify_outgoing](https://www.consul.io/docs/agent/options.html#verify_outgoing) settings to verify TLS certs on
[verify_outgoing](https://www.consul.io/docs/agent/options.html#verify_outgoing) settings to verify TLS certs on
incoming and outgoing connections, respectively:

```json
@@ -277,4 +277,4 @@ track other servers. A server is considered healthy when:
There are Autopilot settings called [upgrade migrations](https://www.consul.io/docs/guides/autopilot.html#upgrade-migrations)
that are useful when adding new members to the cluster either with newer configurations or using
newer versions of Consul. These configurations manage how Consul will promote new servers and demote
old ones. These settings, however, are only available at the Consul Enterprise version.
old ones. These settings, however, are only available at the Consul Enterprise version.