Setting up WSL, Ansible, and Packer for DevOps
WSL stands for Windows Subsystem for Linux. It allows Windows to run a lightweight Linux environment for running Linux binaries on Windows. I have not set it up from scratch by myself but have used it in the past with lab VM's like the one associated with Black Hills Information Security's Pay What You Can courses. However, over the holidays I got myself an upgraded laptop and so I thought I would give it a try!
WSL
There are a few different ways to install WSL but the easiest is by using the wsl.exe command with a few options that we have available. If you don't feel comfortable using the command line at least for this step, you can also install WSL via the common Turn Windows features on and off found in the Control Panel under Programs and Features or in Settings under Apps - related settings. For this blog I am going to use the command line option.
First start by running wsl.exe --list --online from command line (or PowerShell) as an Administrator.
--listwill show a list of WSL environments.--onlinewill tell Windows that we want to see the online distros we have available to us. If we leave off this option the command will show us WSL environments that are installed locally.
Listing WSL Distros
This list shows us all the distros we can download. I am going to use Ubuntu-18.04 I can install it by using the --install and -d switch and specifying the distro name!
wsl.exe --install -d Ubuntu-18.04
It might take a bit to install based on your download speed and such. As part of this process, the latest WSL kernel is also installed. The last step will be rebooting to properly install the kernel otherwise our WSL environment will not boot. To launch our environment we can run wsl.exe, or bash.exe on the commandline or in the search bar. Take note that the user account for Windows is mounted under /mnt/c/Users/username we will get to the importance of this in the future.
Awesome! We now have a fully functional WSL environment. We will now need to set up a user/password for the account. Alright now onto the fun part, Installing tools!
Ansible
My primary inspiration for this blog is Ansible. For those that aren't familiar, Ansible is an open source software that can be used for configuration management, and software provisioning. Simply put, it's goal is to automate IT. I will not go into specifics about Ansible here as there are multiple blog posts and tools that use Ansible. Instead I will go over what I intend to do with it. I have used Ansible in the past most times in conjuction with setting up some open source project on GitHub but it has never had great compatibility with Windows. While it is possible with Cgywin is is pretty clunky and not at all convenient to "make Ansible work" on Windows. Until now!
Installing Ansible in WSL
To install Ansible I am going to use Pip, but it needs to be installed first.
sudo apt-get -y install python-pip python-dev libffi-dev libssl-dev
Next we need to run Pip to install Ansible.
pip install ansible --user
--user- This switch will allow us to install our packages local to the user account instead of globally.
- Since the
ansiblecommands are installed under~/.local/bin, we need to add that to the$PATH.
echo 'PATH=$HOME/.local/bin:$PATH' >> ~/.bashrc
From here we can launch a new instance of bash by closing the current window or we can run source .bashrc. Personally, I like closing and reopening the window. We can validate that Ansible is mapped correctly by running which ansible and then run ansible --version.
Awesome! Fairly painless to install.
Dealing with DevOps
With my involvement in open source I am a contributer of a few projects which utilize Vagrant and optionally Packer. Here lately I have noticed that when using the latest version of Vagrant (2.2.19) and the lateslt VirtualBox (6.1.30), boxes running Centos 7/8 will not deploy with Vagrant due to some SSH issue that intermittedly affects macOS and WIndows that hasn't fully been solved yet. This is a problem when the boxes are available via VagrantCloud to make a deployment automagic but they can't be brought up with Vagrant once they are on your workstation. However instead of pivoting to my NUC for launching a Packer build, I thought "why not throw this at WSL?"
Packer
For those that don't know, Packer allows us to automate the creation of any type of machine image via several different providers. To simplify, I can use Packer to create a lightweight VM, set it up with software, and then configure that software as well. There are multiple ways of doing this from bash scripts to using tools like Chef or in our case Ansible.
Installing Packer in WSL
This is fairly straight forward to do thankfully. Packer is just a binary we can grab and install from Hashicorp. Here is their process as it is documented on https://packer.io
1curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -
2
3sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
4
5sudo apt-get update && sudo apt-get install packer
and voila we have Packer now. Can run it via packer to make sure it is working as we expect.
Verifying Packer is installed
Testing
Alright so now it's time to grab our project. In this instance I am using ThremulationStation. ThremulationStation is an project built on free and open tools for small scale threat-emulation with the goal that you can run it on your laptop. ThremulationStation uses Vagrant to build out a small lab of three machines running Elastic Security, Caldera, and Atomic Red Team. By running attacks against the victim machines inside of the lab, (Windows 10, Centos 7) the telemetry of thise attacks will logged by the Elastic Endpoint agent for detection and response.
First we will git clone the project somewhere on disk and then cd into the proper directory. For ease of use and locating I opted for threm-wsl in my user directory.
1mkdir threm-wsl
2cd threm-wsl
3git clone --branch devel https://github.com/thremulation-station/thremulation-station
- In this example I am using the devel branch.
Alright now we need to cd to the Packer directory.
cd thremulation-station/packer/
Here we can see multiple folders representative of each machine in ThremulationStation. Let's start with vbox-elastic.
cd vbox-elastic
There are a few files here and the one we are interested in is elastic.json
This is the Packer configuration that we have to declare for Packer when we are ready to try building it. So let's kick off a build!
packer build elastic.json
Running a build
Well that didn't work. We can start troubleshooting at this point but the key takeaway here is the way Packer is executing VBoxManage. See if we run VBoxManage in our terminal Ubuntu will tell us it isn't installed. However, I will not be able to use VirtualBox in the terminal if I had it installed anyway because of a few reasons:
- There is no GUI/display for WSL. That isn't how it was designed and quite frankly it wouldn't make any sense.
- Paths in Windows.
Remember above when I told you to take note of what your working directory was? Well this is another byproduct of the way WSL works. Below are two different path notations, first in Linux and then in Windows:
Linux: /mnt/c/Users/Username/threm-wsl/thremulation-station/packer/vbox-elastic
Windows: C:\Users\Username\threm-wsl\thremulation-station\packer\vbox-elastic
This paths are exactly the same which means that WSL is required to alter them to pass them off to Windows and vice versa. This is done using a tool called wslpath the likes of which I don't want to discuss here. All that matters for our purpose is no matter how hard we try to make this work it will not work easily as Packer is going to be running several instances and variations of the VBoxManage command to get us to our end state.
In fact Elastic Security is running on my workstation and I can use a data table to demonstrate what I am referring to.
Multiple instance of VBoxManage
Here I was testing a different box called redops but you can see the variations of the VBoxManage command options that are being passed to the process. Several commands are getting passed as a result of building a box with Packer. This problem compounds when we start passing Windows paths for things like ISO's taking into consideration spacing and even the disk itself for the VM when the box is being packaged.
Full process command line
So.... Now what? Well after some digging I found someone that created a wrapper for WSL. GitHub - finarfin/wsl-virtualbox: VBoxManage wrapper to use Virtualbox from WSL for Packer
What this script does is really exactly what we need. It passes the full path of VBoxManage to xargs and converts process command line as needed. All we have to do is drop it in usr/local/bin and create a symbolic link back to the Linux equivalent for VBoxManage in /usr/bin/VBoxManage.
1sudo su
2mkdir -p /usr/local/bin
3wget -O /usr/local/bin/VBoxManage.sh https://raw.githubusercontent.com/finarfin/wsl-virtualbox/master/VBoxManage.sh
4chmod +x /usr/local/bin/VBoxManage.sh
5ln -s /usr/local/bin/VBoxManage.sh /usr/bin/VBoxManage
6exit
Now that we are back in our user directory running VBoxManage should call the script located in /usr/local/bin. One last thing I noticed after testing is that sometimes the script fails to account for a storage medium from being ejected. So I added a quick patch to it here: GitHub - webhead404/wsl-virtualbox: VBoxManage wrapper to use Virtualbox from WSL for Packer
You should now be able to successfully build Packer boxes with Ansible!
Tips and tricks
Once your box is built, from the bash terminal you can run explorer.exe . this will allow you to see the .box files that are created.
Special Thanks
Special thanks to Jeff Geerling and Can Hanhan!