First Look at the new BOSH CLI

The new and improved version 2 of the bosh CLI has some great features that make deploying a bosh director (among other things) a breeze.

This document will walk through the steps of deploying a bosh director to a vSphere environment using the v2 CLI.

Install CLI v2

The first step for those following along is to make sure you have v2 of the CLI.

Downloads for Mac and Linux are available on bosh.io.

Alternatively, you can use Homebrew on macOS: brew install cloudfoundry/tap/bosh-cli.

Note: Homebrew installs the binary as bosh2 - you’ll want to alias it or replace all bosh commands in this document with bosh2 if you go this route.

Make sure bosh -v shows a version number starting with 2:

$ bosh -v
version 2.0.17-ef55792-2017-05-11T20:37:11Z

Succeeded

Deployment Manifest

Deployments in bosh are described with deployment manifests. Thankfully, there are a variety of open source templates available to get you started. You can obtain them from https://github.com/cloudfoundry/bosh-deployment.

For a vSphere deployment, you’ll need:

I chose to copy these files out into a separate working directory to reduce clutter, but this step is optional.

If you’ve never used the v2 CLI, you may find a few unfamiliar items in the bosh.yml manifest that you downloaded. Specifically, there is a variables section towards the bottom, and a variety of variables scattered throughout the manifest. They look like this:

   password: ((admin_password))

With the new CLI we can store the values of these fields outside the deployment manifest. This allows us to share and reuse the deployment manifest without:

  1. leaking sensitive information.
  2. changing environment-specific details

The v2 CLI performs a process called variable interpolation to replace these placeholders with concrete values. You can specify these variables directly on the command line with the -v flag (-v admin_password=password), but I find it easier to store them in separate YAML files.

I created two files, misc.yml for misc properties, and vsphere.yml for the IaaS-specific properties.

misc.yml is pretty straightforward:

---
director_name: bosh-1
internal_cidr: 10.0.0.0/24
internal_gw: 10.0.0.1
internal_ip: 10.0.0.6
network_name: 'VM Network'

The only gotcha here is the single quotes around 'VM Network' since there’s a space in the network name.

vsphere.yml will require values specific to your environment:

---
vcenter_dc: Datacenter
vcenter_ds: <redacted>
vcenter_ip: <redacted>
vcenter_user: <redacted>
vcenter_password: <redacted>
vcenter_templates: bosh-1-templates
vcenter_vms: bosh-1-vms
vcenter_disks: bosh-1-disks
vcenter_cluster: Cluster

Note: in addition to reading variables from the command line and from files the CLI can also get them from environment variables, and in the future from a configuration server.

Create Environment

With these files in place, you’re ready to deploy.

The create-env command is roughly analogous to bosh-init:

$ bosh create-env bosh.yml -l vsphere.yml -l misc.yml -o cpi.yml --vars-store creds.yml

A brief description of this command:

The variables section at the bottom of the deployment manifest explicitly declares several variables and their types. This allows the CLI to generate them automatically if they are not provided. In this case, creds.yml doesn’t need to exist because we’re providing all required variables in the other files.

Operations Files

A major goal of CLI v2 is to make deployment manifests more portable (creating them from scratch is a pain!) Variable interpolation works great for simple placeholders, but sometimes you need to perform more sophisticated transformations. This is where operations files come in.

Operations files are a way to specify transformations to a deployment manifest at the time of deployment. This eliminates the need to reapply the transformations whenever the base document changes.

For this deployment the configuration for the vSphere CPI is specified in an operations file, which keeps the deployment manifest IaaS-agnostic.

Alias Environment

Once you’ve created an environment, you’ll want to create an alias for it to simplify future commands.

$ bosh alias-env bosh-1 -e 10.0.0.6 --ca-cert <(bosh int ./creds.yml --path /director_ssl/ca)
Using environment '10.0.0.6' as anonymous user

Name      bosh-1
UUID      f7e6bbd5-d4d2-4328-88f1-e5a549183a24
Version   261.4.0 (00000000)
CPI       vsphere_cpi
Features  compiled_package_cache: disabled
          config_server: disabled
          dns: disabled
          snapshots: disabled
User      (not logged in)

Succeeded

Take note of the bosh int command, which was used here to retrieve the generated CA cert from the creds.yml file. You can view this new environment with bosh environments, (or bosh envs for short):

$ bosh envs
URL             Alias
10.0.0.6        bosh-1

1 environments

Succeeded

From now on, you can use the -e flag with the bosh command to target this environment.

For example, you can use the cloud config from bosh-deployments/vsphere/cloud-config.yml:

$ bosh -e bosh-1 update-cloud-config -l misc.yml -l vsphere.yml cloud-config.yml

Gotchas

For the most part, this was a pretty painless exercise and definitely beats tweaking bosh-init manifests by hand.

There were a few things that caught me by surprise that you should look out for.

First, the default bosh.yml that we used from the bosh-deployments repo uses Google’s public DNS. You may need to tweak this for your environment.

More importantly, the user experience is lacking when you omit variables, either by forgetting to supply them on the command line or by loading them from a file. The CLI ends up leaving the ((variable)) placeholder in the manifest and attempting to deploy, resulting in an error that can be difficult to troubleshoot.

There’s plenty of discussion on the github issues, but the BOSH team seems hesitant to address the issue at this point, as their long term plan is to also allow the director to perform variable interpolation.

The bosh interpolate command takes --var-errs and --var-errs-unused flags which causes the interpolation to fail if variables are missing or unused respectively. This is helpful for troubleshooting, but bosh deploy and bosh create-env don’t have similar flags. Until this area improves, I’d recommend running a bosh int --var-errs when you encounter an odd error - it may just point you to a missing variable.

Summary

A few important points from this exercise:

We used variable interpolation and operations files to deploy a bosh director without ever modifying a deployment manifest. Instead of generating our own manifest, we were able to use one that was provided for us, applying customizations in a non-destructive manner.

We didn’t need to make use of bosh-init. The V2 CLI’s create-env command is a suitable replacement.

The CLI was able to generate passwords and certs for us and store them in a file. This is great for a manifest consumer. Additionally, the manifest contained a list of variables and their types. This is great for a manifest author as it allows them to specify their requirements in a way that makes them easy to satisfy.