stuffs - an AWS inventory tool

problem

I wanted to create a simple inventory website. I wanted to have a collection of information, including all systems, installed software on those systems, and any other relevent stats and details on those systems. The available tools out there, however, I didn’t really like. I just wanted a simple static HTML site, with possible some search features. Why not use some free software out there to generate my own?

solution

With most inventory management solutions appearing meh, I thought about cooking up my own solution. Why not use Ansible to gather data, generate a Markdown formatted file, then use some static web page generator to generate the sit?. Afterwards, upload the generated content to an S3 bucket. With that, there’s easy reporting and searching of all software that needs to be updated. We can add more checks to it as well, but first, let’s just get a list of apt packages. So that’s what I built. An Ansible playbook that scans all available instances and creates a EC2 inventory and software inventory.

In the end it didn’t turn out half bad. Some screen shots…

First the index. Note it’s just 3 instances; an OpenVPN instance, a “DB” instance, and a “web” instance. Inventory page is generated when Hugo is run. Since lunr-hugo is installed, this is somewhat searchable.

index

Next, an instance. The Ansible playbook gathered EC2 facts, such as IP address and private DNS name. More can be gathered, this was just a start. After that, Ansible gathered the installed apt packages and if there’s an update for it. More software checks can be added, like checking for Docker imgs.

db1

Not too shabby. Note that this playbook will find ALL instances based on the AWS tag named Environment, using the Ansible env variable. Now on some of the details of deploying this.

Terraform deployment

The Terraform files can be found here. In this case, I left the bucket public, to make a demo/proof of concept easier to play with. Obviously, in a real production environment, you won’t want this public. A couple files to modify include:

  • s3.tf - this has the public S3 bucket settings.

  • policy.json - this file is the template for the S3 bucket policy. It allows public web read-only access.

  • variables.tf - modify the S3 bucket name.

  • Terraform:

cd terraforms/stuffs
terraform plan
terraform apply

This creates an OpenVPN instance and example instances, such as a “web” instance and a “db” instance. The example instance is an old AMI, which has out of date packages. It creates an S3 bucket (again, public readable, not a production quality setting, but it’s useful for the demo) that will serve the Hugo generated web pages.

Ansible deployment

  • Ansible playbook:
cd playbooks
ansible-playbook openvpn_terraform.yml -e "env=corp keypair=id_mod" # note this playbook will modify the local ~/.ssh/config to allow SSH proxying
ansible-playbook stuffs-example-instance.yml -e "env=corp"

And with this, the test environment should be deployed. SSH to an instance (note, SSH proxy config is applied to ~/.ssh/config) if you are interested in verifying.

The real guts is the next step, the stuffs.yml playbook, which will gather the facts on all instances it finds with the Environment=corp AWS tag:

ansible-playbook stuffs.yml -e "env=corp"

This runs a few roles:

  • localhost.aws_ssh_keys - Gathers AWS SSH keys from the AWS System Log.
  • ansible.groups_init - Adds found instances to Ansible groups dynamically, such as instances_private. All instance private IP addresses are added to this group.
  • instance.get_inventory - The meat. Gathers information about the instance and generates a Markdown inventory file, which it copies locally using fetch.
  • localhost.generate_hugo_and_upload - Using those markdown files, generates the Hugo static web site and uploads it to AWS S3. Uses the Hugo Learn Theme

And with that, a very simple inventory site has been generated.

summary

I’m not really sure this is a good way to go. There are limitations to using a static website engine. But now someone potentially has a simple, effective start at gathering system inventory, a core tenant to managing infrastructure. It was a fun little PoC though, I might revisit it and improve it in the future.

links

bill cawthra avatar
bill cawthra
DevOps/Cloud Infrastructure Person.