Ex
Ratione
About • GitHub • Blog • Archives • Feed

Deploying Streisand from a Configuration File, without Prompts

By Reason November 1st, 2017 Permalink

Streisand is a useful project. Quite aside from its value as a tool of freedom, a thumb in the eye of those who would pen us all inside the panopticon if they had their way, it is a good, low-effort, packaged way to set up a cloud VPN and proxy server for a household or other small community. It offers support for mobile as well as desktop users. The scripts provided deploy a server and generate friendly documentation describing how to connect to and make use of the various applications running on that server.

That said, the development team rather strangely assembled the Bash and Ansible deployment to prompt for user input. The whole process is interactive, top to bottom; every piece of information must be provided in response to a prompt. I can see why this might be thought of as more friendly to non-technical users, but is it really more friendly than providing an example configuration file with inline documentation and expecting those users to edit it? As someone who wants to keep records and configuration in files, this just won't do. So I provided another option.

Override as Many Prompts as Possible with Variables

By providing extra-vars to Ansible, the various var_prompt entries can be bypassed. So I created a suitable file, for Digital Ocean deployment in this case: 

---
# Example site specific configuration for a noninteractive Digital Ocean
# deployment.
#
# Copy this and edit it as needed before running streisand-new-cloud-server.
#

streisand_noninteractive: true
confirmation: true

# The SSH private key that Ansible will use to connect to the Streisand node.
#
# The corresponding public key must be added to the Digital Ocean control panel
# and the name given to it referenced below in the do_ssh_name variable.
# The corresponding public key must be uploaded to Digital Ocean and the name
# given to it referenced below in the do_ssh_name variable.
streisand_ssh_private_key: "~/.ssh/id_rsa"

vpn_clients: 5

streisand_l2tp_enabled: yes
streisand_openconnect_enabled: yes
streisand_openvpn_enabled: yes
streisand_shadowsocks_enabled: yes
streisand_ssh_forward_enabled: yes
streisand_stunnel_enabled: yes
streisand_tinyproxy_enabled: yes
streisand_tor_enabled: yes
streisand_wireguard_enabled: yes

# The Digital Ocean region number.
#
# 1.  Amsterdam        (Datacenter 2)
# 2.  Amsterdam        (Datacenter 3)
# 3.  Bangalore
# 4.  Frankfurt
# 5.  London
# 6.  New York         (Datacenter 1)
# 7.  New York         (Datacenter 2)
# 8.  New York         (Datacenter 3)
# 9.  San Francisco    (Datacenter 1)
# 10. San Francisco    (Datacenter 2)
# 11. Singapore
# 12. Toronto
#
# Note that this must be a string representation of a number, not a number.
do_region: "2"

do_server_name: streisand

# Add the Digital Ocean access token here.
do_access_token_entry: ""

# The name given to the key in the DigitalOcean control panel.
do_ssh_name: streisand

# Definitions needed for Let's Encrypt SSH certificate setup.
#
# If these are both left as empty strings, Let's Encrypt will not be set up and
# a self-signed certificate will be used instead.
#
# The domain to use for Let's Encrypt certificate.
streisand_domain: ""
# The admin email address for Let's Encrypt certificate registration.
streisand_admin_email: ""

Write New Bash Scripts

The streisand Bash script also prompts for user input, so I wrote the necessary replacements: one to deploy a new server, and one to reprovision an existing server. These scripts accept a path to the configuration file above as an argument. They also bypass some of the now unnecessary configuration-focused Ansible playbooks invoked by the main Bash script. 

#!/usr/bin/env bash
#
# Run a noninteractive Streisand installation that creates a new cloud server.
#
# This requires an expanded extra-vars file specific to the provider type that
# sets all of the values gathered by prompts in the interactive installation.
# See the contents of global_vars/noninteractive for examples that can be copied
# and modified.
#
# Usage:
# $0 --provider digitalocean --site-config path/to/digitalocean-site.yml
#

set -o errexit
set -o nounset

DIR="$( cd "$( dirname "$0" )" && pwd)"

VALID_PROVIDERS="amazon|azure|digitalocean|google|linode|rackspace"
DEFAULT_SITE_VARS="${DIR}/global_vars/default-site.yml"
GLOBAL_VARS="${DIR}/global_vars/vars.yml"

# Include the check_ansible function from ansible_check.sh
source util/ansible_check.sh

# --------------------------------------------------------------------------
# Reading options.
# --------------------------------------------------------------------------

function usage () {
  cat <<EOF
Usage:
$0 \\
  --provider ${VALID_PROVIDERS} \\
  --site-config path/to/site.yml
EOF
}

PROVIDER=""
SITE_VARS=""

while [[ ${#} -gt 0 ]]; do
  case "${1}" in
    # Required.
    --provider)      PROVIDER="${2}"; shift;;
    --site-config)   SITE_VARS="${2}"; shift;;

    # Utility.
    -h|--help)       usage; exit 0;;
    --)              break;;
    -*)              echo "Unrecognized option ${1}"; usage; exit 1;;
  esac

  shift
done

# --------------------------------------------------------------------------
# Fail if required options are not set.
# --------------------------------------------------------------------------

if [ -z "${PROVIDER}" ] || [ -z "${SITE_VARS}" ]; then
  usage
  exit 1
fi

# --------------------------------------------------------------------------
# Fail for other reasons.
# --------------------------------------------------------------------------

# Make sure the alleged configuration file exists.
if [ ! -f "${SITE_VARS}" ]; then
  echo "No such config file: ${SITE_VARS}"
  exit 1
fi

# Check validity of the provider name.
if [[ ! "${PROVIDER}" =~ ${VALID_PROVIDERS} ]]; then
  echo "Invalid provider: ${PROVIDER}"
  exit 1
fi

# --------------------------------------------------------------------------
# Onwards to launch and provision the server.
# --------------------------------------------------------------------------

GENESIS_INVENTORY="${DIR}/inventories/inventory"
GENESIS_PLAYBOOK="${DIR}/playbooks/${PROVIDER}.yml"

# Validate the settings.
ansible-playbook \
  --extra-vars="@$GLOBAL_VARS" \
  --extra-vars="@$DEFAULT_SITE_VARS" \
  --extra-vars="@$SITE_VARS" \
  playbooks/validate.yml

# Deploy.
ansible-playbook \
  -i "${GENESIS_INVENTORY}" \
  --extra-vars="@$GLOBAL_VARS" \
  --extra-vars="@$DEFAULT_SITE_VARS" \
  --extra-vars="@$SITE_VARS" \
  "${GENESIS_PLAYBOOK}"

#!/usr/bin/env bash
#
# Provision an existing cloud server.
#
# This requires an expanded extra-vars file specific to the provider type that
# sets all of the values gathered by prompts in the interactive installation.
# See the contents of global_vars/noninteractive for examples that can be copied
# and modified.
#
# Usage:
# $0 --provider digitalocean --site-config path/to/digitalocean-site.yml
#

set -o errexit
set -o nounset

DIR="$( cd "$( dirname "$0" )" && pwd)"

DEFAULT_SITE_VARS="${DIR}/global_vars/default-site.yml"
GLOBAL_VARS="${DIR}/global_vars/vars.yml"

# Include the check_ansible function from ansible_check.sh
source util/ansible_check.sh

# --------------------------------------------------------------------------
# Reading options.
# --------------------------------------------------------------------------

function usage () {
  cat <<EOF
Usage:
$0 \\
  --ssh-user root \\
  --ip-address 10.10.10.10 \\
  --site-config path/to/site.yml
EOF
}

SSH_USER=""
IP_ADDRESS=""
SITE_VARS=""

while [[ ${#} -gt 0 ]]; do
  case "${1}" in
    # Required.
    --ip-address)    IP_ADDRESS="${2}"; shift;;
    --site-config)   SITE_VARS="${2}"; shift;;
    --ssh-user)      SSH_USER="${2}"; shift;;

    # Utility.
    -h|--help)       usage; exit 0;;
    --)              break;;
    -*)              echo "Unrecognized option ${1}"; usage; exit 1;;
  esac

  shift
done

# --------------------------------------------------------------------------
# Fail if required options are not set.
# --------------------------------------------------------------------------

if [ -z "${IP_ADDRESS}" ] || [ -z "${SITE_VARS}" ] || [ -z "${SSH_USER}" ]; then
  usage
  exit 1
fi

# --------------------------------------------------------------------------
# Fail for other reasons.
# --------------------------------------------------------------------------

# Make sure the alleged configuration file exists.
if [ ! -f "${SITE_VARS}" ]; then
  echo "No such config file: ${SITE_VARS}"
  exit 1
fi

# --------------------------------------------------------------------------
# Onwards to launch and provision the server.
# --------------------------------------------------------------------------

# Create an inventory file on the fly.
cat > inventories/inventory-existing <<EOF
[localhost]
localhost ansible_connection=local ansible_python_interpreter=python
[streisand-host]
${IP_ADDRESS} ansible_user=${SSH_USER}
EOF

# Validate the settings.
ansible-playbook \
  --extra-vars="@$GLOBAL_VARS" \
  --extra-vars="@$DEFAULT_SITE_VARS" \
  --extra-vars="@$SITE_VARS" \
  playbooks/validate.yml

# Update the server.
ansible-playbook \
  -i "${DIR}/inventories/inventory-existing" \
  --extra-vars="@$GLOBAL_VARS" \
  --extra-vars="@$DEFAULT_SITE_VARS" \
  --extra-vars="@$SITE_VARS" \
  playbooks/existing-server.yml

Make the Remaining Prompts Optional

Some editing of the Ansible setup is still needed, however. There are a couple of places in the Streisand Ansible roles where pause blocks prompt the user to continue. These can be bypassed by adding a when condition that checks one of the variables set in the configuration file. E.g.: 

  - name: Warn about manual provisioning
    pause:
      prompt: "..."
    when: not streisand_noninteractive

Wrap it up into a Pull Request

Since it is polite to share, I wrapped up my changes into a pull request for the Streisand volunteers to consider. I can't be the only one who prefers to work with configuration files, and the PR adds that option for those who want it.

Deploy the Server

Then it just remains to edit the configuration file to add secrets and names, and run the script to create a new cloud server: 

./streisand-new-cloud-server \
  --provider digitalocean \
  --site-config global_vars/noninteractive/digitalocean-site.yml

This generates documentation in the generated-docs that you can then copy to a safe place, and which provides the credentials and instructions needed to access the server and its applications.