In production we will hopefully have a secrets management system, or at a very
minimum private repos to store encrypted secrets in. For the purpose of this
demo though I will be using a public repo and do not want to expose any sensitive
data that we may need to add to this repo. WARNING: To be clear, storing any kind
of secrets in a git repo, encrypted or not, may not be a good idea. Consult your
local security team for advice. We however, don’t have any state secrets for
this demo and very few options.
Transcrypt is a shell script that uses OpenSSL to encrypt and decrypt files in
your git repo that are noted in the .gitattributes file. Let’s initialize our
repo and confirm that it is working.
We will initialize Transcrypt within the repo on a clean branch. This will
hopefully ensure that we can configure Transcrypt prior to adding any secrets
to the repo.
Add the generated PASSWORD to your LastPass, keychain, text document on your
desktop, or wherever you store secure passwords. It is very important to not
lose that passphrase.
We now want to get the .gitattributes file added up and merged in so we can
test if secrets encryption is working as expected so let’s branch, commit, push
Let’s create a secrets dir and initialize it with a README to confirm that
transcrypt is working as expected without potentially exposing secrets on
the internet. The way transcrypt works is that it looks at the .gitattributes
in the repo root for files to encrypt. If it sees that there are paths to
encrypt, it will encrypt them before adding to the tree. It does still allow
you to view the changes as plaintext locally which sometimes can lead to
confusion as to whether or not its working.
When opening the PR, you should notice that there are two files changed:
You should also notice that the contents of README.md is encrypted. If this is
the case, give yourself a thumbs up, merge the PR, and update your local branch.
NOTE: If you see that README.md is not encrypted, do not open the PR. Instead,
delete your branch and start over. Follow the instructions on the Transcrypt site
if you need more help or information. It is important to confirm that the encryption
mechanism is working the way we expect before we put secrets into the repo.
Let’s take this time to add a line to the PULL_REQUEST_TEMPLATE.md file reminding
us to check for unencrypted secrets:
Now that the repo has been initialized with Transcrypt, we can begin adding some
configuration that depends on credentials. In the next segment, we will create
a machine user for GitHub and configure the Git plugin to use those credentials
Currently it just so happens that the Jenkins URL in the management console does
have the proper config as it is set to http://localhost:8080. This is merely a
coincidence that the default value and the current address match though. Once we
start moving this thing around, it will be very important that it is set to the
right value else we’ll have all kinds of strange issues. Since it will definitely
have to be configured, let’s start here. It doesn’t hurt that it is a fairly
simple example of configuring Jenkins with an environment variable passed by
Passing from Docker Compose
This is a setting that will change from environment to environment and so I
think the best place to set it is in the compose file. Let’s create a variable
in there that we can read it in during init and make the right configuration.
Edit the compose file and create a JENKINS_UI_URL env variable as well as volumes
for the configs themselves.
Now that it is set, we should be able to write a groovy init script to read it.
But first, we will have to restart Jenkins to pick up the new environment:
With the environment variable now being available to us, we can use the script
console at localhost to develop our script that will set the URL of our
We now have a working script that will manage the URL of Jenkins in any
environment that we would be deploying into simply by setting a variable
in the compose file. Now we need to make this available to the master so that
it can be executed on startup. To do that, we will add a director to the
plugins image and then mount it into the master in a similar way to how the
war and plugins work.
Add the full directory (instead of the individual scripts) of Groovy
configuration to the Docker image and then export it as a volume.
We can now rebuild the image and pick up our new config that should (hopefully)
configure the URL of our Jenkins on boot!
Ok you should now see that the URL in the Jenkins management console is set to
http://localhost:8080! Ahhhh, it was like that before? Hmm. Ok well then
let’s break it to confirm it is working. Modify the value in the compose file to
something different and restart Jenkins:
Did that work? A typo you say? I can’t imagine it. I’ve typed docker-compose
over 1000 times today, it’s not possible for me to misspell it. In addition, if
you notice the difference between this set of commands and the one earlier, we
seem to be drifting to chaos. Let’s take note of that, but address it after we
confirm that this current change is working as expected.
Gooood, it does work :) The URL in the management console has been updated to the
new, wrong, custom value so we can confirm it works. Let’s commit our code now,
but attend to that tiny little pile of tech debt we found (starting and stopping
the system differently every time is definitely tech debt). NOTE: Don’t forget
to set the JENKINS_UI_URL back to http://localhost:8080
So we’ve noticed something a bit stinky in our code as we’ve been going about
our business. This happens very often in our work lives and attending to little
tech debts like this is critical to having quality software. I certainly
encourage everyone to leave the code better than they found it and to refactor
things when they see something turning into a turd like object.
I also encourage you to make note of these things and take care of them after
you are in a place to save game. Switching context between one problem and
another can be very expensive mind and time wise so feel free to take a note,
create a ticket or something, then finish what you are doing (unless there is
an actual issue that needs to be addressed before your code will work). When
you submit your PR for review, then jump on that Jira and refactor your heart
We need to do everything, but we can only do one thing at a time. Try to be
aware of time management.
Let’s get that squirrel
What we noticed was that we were beginning to type the command differently
every time we did it. That seems like it should be replaced by a shell script.
Let’s create a start script so that this thing starts consistently every time:
Write that guy out to deploy/master/start.sh and make it executable with
chmod +x deploy/master/start.sh and we’ve now got ourselves a script that
will consistently restart our app.
Add that onto the branch with a good message and push, PR, merge, etc..
See, you’re getting the hang of it!
Next let’s get ready to handle some secrets. Shhhhhh!
Configuring the Jenkins Master on Boot with Groovy
One of the greatest strengths of Jenkins is it’s ability to do almost anything.
This comes from it’s extremely customizable nature. I think of it as a scheduler
that can do any given task on any given trigger with enough configuration. What
we are building specifically though is a system to build, test, and deploy our
piece of custom software that most likely is at least a little bit different
than anything else out there requiring the same tasks. We will need to use
Jenkins’ ultra powerful customization toolkit, but do it in a way that strives
Deterministic: Given a set of inputs, there should be only one output.
If that output is not as we expect, there is a problem that we should spend
time fixing. ie: removing “flaky” tests that fail for “no reason”.
Reliable: The system should have high availability to the users who depend
on it. Having a system that is sometimes down and sometimes up does not
encourage teams to inject automated builds into their workflows.
Repeatable: This system should be able to be recreated without persistent
data from the repo.
Agile: The system should evolve to meet the needs of it’s consumers in a
sustainable way. If one team’s jobs or configs are breaking another team’s
pipelines, it is a good indication that it is time to split the monolith
into two independent systems.
Scalable: As the system becomes more popular, more people are going to
utilize it. When this happens, it’s critical to be able to support the
increased capacity in not only job runners, but also collaboration from
Luckily we can treat the code that configures the system in the same way we
treat the code the builds and runs the system :)
Not super impressive documentation considering how powerful this mechanism is.
Using this init system you are able to configure any aspect of the master that
you are able to using “Manage Jenkins”. This includes (but is not limited to):
The URL and name of this instance
Authentication and security settings
Secrets in the credential store
Global plugin configuration
Global tool configuration
Adding and removing nodes
Creation of jobs (though we’ll only use it to create one special job)
The Groovy Script Console
Not only does it support configuring so much of the system, it has a direct
REPL like environment to code in. This is called the “Script Console” (available
at http://localhost:8080/script on our instance) and can be considered a shell
into the Jenkins system. This shell has the same abilities of the init system’s
shell so it makes for super quick and easy development of the scripts that we
Jenkins Groovy Hello World
Let’s kill two stones with one bird. We will do a quick little Hello World that
will introduce you to bot the syntax of groovy as well as how to use the script
Stand up your development Jenkins (cd deploy/master && docker-compose up -d)
Check out the cool message for all the users of your system to see. I bet your
boss will love it!
Nothing too crazy, but this should give you a good idea of how we are going to
configure our master to build our particular brand of software. Inside Old Mr.
Jenkins is just a series of objects (I think of them as his organs) that we can
grab and squeeze and modify to fit our needs. I hope Janky is ready to play