Validate server configuration with Serverspec

The other day I had to validate if a bunch of servers was configured according to specification. Now normally I would log in to every server and inspect each detail of the configuration by hand, but in this case that would be too time-consuming. Also, if the configuration would change later I would do the same thing again. And again. You may know the drill.

Suddenly realised that I work in the automation business, I figured I might as well automate this validation process. Why not. After a quick search one tool named Serverspec came to my attention. Serverspec is a Ruby-based (RSpec) command-line tool that validates server configuration for you. You write the server specifications in RSpec files which can get messy, but are at least easier to read than the shell commands you would write as an alternative.

The big pro is that you can run the validation again and again. You can save the server specifications as plain-text files meaning you can manage them in version control easily.

There are a couple of things that were “out of my comfort zone”. Or cons, as you may call them. For example Serverspec is built with validating one server specification per run. Running the tests against multiple servers requires some extra work. Not hard to do, but documentation on this is lacking. Apart from this the documentation is alright!

Being a coach at WeCamp 2015

When Stefan Koopmanschap asked me if I wanted to coach a group of developers at WeCamp 2015, I immediately said yes! Of course! That was a no-brainer.
And then I started thinking. What does a coach do? What makes a good coach?

But then it struck me. A coach helps others to see what they’re capable of. A coach is not the drill instructor that pushes the team to their destination. A coach is not the leader of the pack, who leads the way and protects them from harm. A coach is the person who tells you to get out there while you’re (probably) still procrastinating to make the first step. A coach is the person who tells you that you can do the thing you want, because you make the rules.

“Give a man a fish, and you feed him for a day. Coach a man to grab his fishing gear, and you feed him for a lifetime.” – Richard Tuin

So that’s what I’ll do at WeCamp this year, and the rest of my life, too.
Thanks for the opportunity and coaching me to do this, Stefan!

“You can do it, it’s all up to you.” – Mr. Mackey

Oh by the way! There are still tickets available for this great event!

First attempt at sketchnoting

Inspired by this tweet I decided to give sketchnoting a go. It’s great to get the creative juice flowing and for me it is a great way to get in the “thinking” zone. It’s a lot easier than I thought too!

Sketchnotes: Ideas, not art

Now I’m not really a great visual artist (not even close), nor do I aspire to become one. But this is just fun to do! I think sketchnoting is great to work on (technical) ideas, information or thoughts, and it is not hard at all. Everyone can do it. Really.

After I bought some fineliner pencils I practiced while listening to Peter Doolittle’s TED talk “How your “working memory” makes sense of the world”.
Which ironically explains why sketchnoting can help you learn, concentrate, remember, and instruct.

This is what I came up with:

As soon as I make another one that’s worth sharing I’ll post it here!

Automatically installing all Ansible role dependencies

I think it was a couple of months ago now that I first started to fiddle around with Ansible. Being used to work with Puppet as a system provision tool before, Ansible seemed to make this task I disliked so bad before very easy. Almost fun I guess.

One thing that I found missing though, was a way to automatically install all the role dependencies. This still seems to be a task you have to do manually. Or anyway, I can’t find a way how to do it with the current Ansible version. So, being a programmer and all, I wrote a script for it.

The script itself is very simple:

  1. Loops through every role installed in roles/
  2. For every role, it fetches dependencies from meta/main.yml
  3. Installs the dependencies via Ansible galaxy

Personally I haven’t found a use case for other features, but if you do, please let me know.

The script source is available here: GitHub link
Available for download here: Direct download

Happy coding!

Charles is my superhero!

As a true geek I am a big fan of several superheroes. Actually, Iron Man is by far my favourite one. Or actually, the coolest part about Iron Man is J.A.R.V.I.S. if you ask me. The Just A Rather Very Intelligent System, is a clever system that assists Tony Stark in virtually everything that he considers “everyday tasks”.

Unfortonately I do not embody a superhero, but I do have several tools that assist me with my everyday tasks, and one of these tools is Charles Proxy.
Like my IDE helps me with being productive in writing, testing and troubleshooting code, Charles Web Debugging Proxy helps me with most of the things that are related to HTTP. And as a web developer, you know, that’s a lot. Here are some of the most significant contributions of Charles in my everyday work.

Communication insight

Specifically when developing APIs, a lot of communication is happening “under water”. A website does a request to an API, or a mobile device calls some webservices, and you want to know what’s going on. As with most proxies, Charles can show the HTTP-client requests. What’s connecting when, to where, and with what headers, cookies, request/response bodies etc.

Request tampering

Screen Shot 2013-07-25 at 20.34.59By setting a breakpoint on a specific request, you can modify the entire HTTP request the client is sending to the server. Change headers, cookies, request bodies, query parameters. In short, everything that you need to troubleshoot most common HTTP-related issues. Activate this in the context menu of a specific request -> Breakpoints, and the next time your client requests this URL, charles will break on the request.

Give colleagues and devices access to your dev environment

Another very convenient feature is that if you set your proxy server as a system-wide proxy on let’s say, your mobile device or colleagues’ machine, that they can see everything web-accessible on the machine you’re running Charles on.
So in practice, it means that you can check out your application that’s running on let’s say, a host-only connected virtual machine. This is something that we use a lot here in the office. Pretty neat, huh?

But there’s more…

Actually, the list above is just the tip of the iceberg of Charles’ features. There are a lot more features in the application, like: Bandwidth throttling, Request filtering, Recording/Replaying and saving sessions, DNS spoofing, SSL proxying, and there’s still more.

So, why a blog about Charles’ features while I could also visit it’s website?

Well, if you’ve read this far, I’ve got your attention. And that’s exactly what I was after. My goal with this post is to show that there’s a tool that can help you with things that can be very cumbersome without a proxy. And more! Charles is just great, and every web developer should at least know about it’s existence.

Of course, Charles is not the only Proxy program out there, and probably there are tools that have more or less the same features. If you know such a tool, could you let me know in the comments?
Oh, and Charles runs on virtually every desktop platform!

When and what to test in software development

Image courtesy of http://www.sxc.hu/photo/418215

I hear people around me often ask this question. Confused whether and when they should unit-test their classes, do integration tests on their components, functionally test their to be delivered software, and execute a regression test on their previously delivered product.

And really, there are a lot of answers to all these questions. And unfortunately, each of them are subject to the infamous answer of all questions in software development: It depends*.

But actually, there is a simple answer to all of these questions when you think about why you test things. Simple enough to give you food for thought. Tests reassure you that something (still) works. It gives you confidence. Confidence in the code that you write as a software developer. In the application that you deploy as a development team. In the product that you accept as a product owner/client.

Think about it. Think about your current project, and about what you currently test (or not) and why.
I often get the impression that people test, or want to test “because it is good for quality”. And it sure is. But think about what quality really is. Being confident about the solution that you built for your clients problem, and that you’ve delivered it well. That’s quality in a nutshell.

So actually the answer to the question “When and what to test in software development?” is really simple:

Test what gives you confidence about the (to be delivered) solution.

How you do it, is up to you.

Disclaimer: Some statements in this post may be over-simplified.

* actually, as in all of life, the universe, and everything, the answer should be 42. But that seemed a bit inappropriate in the context of this post.

Image courtesy of http://www.sxc.hu/photo/418215

New writing tool: Getting in the zone

Usually when I am writing, I get distracted a lot. And I mean a lot. And this is one of the reasons that i don’t write as much as I want to.

However tonight I found this tool called OmmWriter. And it is specifically designed to zone yourself out from all distractions and write. Just write. It helps me a lot to do productive writing.

ommwriter-logo-main

When starting the application, which runs in full-screen mode only, it suggests that it works best with your headphones on. And this is where the zoning out begins. After the splash screen the application plays a soothing ambient song in the background. And it helps me focus on my screen.
Music normally distracts my thought from the task at hand, but this music does not. Excellent!

As soon as you start typing you hear the sound of your keyboard. A little something that the application adds. I think it works motivating to hear the ticks of the keyboard. It motivates to stick to the rhythm of typing.

Proof that this works is that last night I’ve written a 900 word blogpost in under an hour. Possibly one of the longest blogposts that I’ve ever written. Now that’s some productive writing!

If you are interested in this application, you can download the application for free at the OmmWriter website. It is available for Mac, Windows, and iPad.

See, another 200+ word blogpost done in under 10 minutes. Way to go!

An example on running Behat from Jenkins

Time for a very short article. I just want to share a little trick that helped me run my Behat testsuite from a Jenkins build.
Most of the articles that I found on Google did not really cut to the chase. Here it is:

1) Create an “execute shell” build step that executes Behat, and outputs it’s results to a jUnit format:

2) Create a post-build “Publish JUnit test result report” action that reads the build/ directory for the jUnit files.

That’s it! If you want to know more about the formatting features that Behat provides, click here.

Screen Shot 2013-05-10 at 00.47.42

Javascript minification in capistrano/webistrano

Photo: http://www.sxc.hu/photo/347930

Recently i had to automate the minification of specific javascripts. Since we don’t have a real build tool, i thought it was best to do this during deployment.
For (automatic) deployments our tool of choice is Webistrano, which is based on Capistrano, which is written in Ruby.

Capistrano has this concept of recipes, which are small scripts that you write to do specific tasks during deployment.
So, one of these tasks could be the minification of some javascript.

But for the minification and deployment of that, i was bound to some constraints:

  • We’re using the yuicompressor (java)
  • Deploy target machines do not have java installed
  • So we have to do the minification locally

After doing some initial research it was quite easy to do, and i came up with the following code:

The approach can be considered somewhat unusual, as the project’s files are first uploaded to the server. And after that the javascript is minified and uploaded seperately. This is mainly because I couldn’t find another hook within capistrano that fits my need.
Of course, you can rewrite this script to work with any other local command too.

Note/disclaimer: This script uses the copy_cache directive. I’m not a capistrano expert, but this variable might not be available to all deployment strategies. This script might not work for you if copied 1:1. But i suppose you get the idea.

How to make Git ignore different line endings

warning: CRLF will be replaced by LF in [filename].
The file will have its original line endings in your working directory.

Getting errors like these, and want to deliberately ignore them?

If you don’t know why you would, you should read this great blogpost about line endings in Git by Tim Clem.

Don’t look at line endings!

In some cases you want to ignore these errors. In my specific situation i don’t want Git to warn me about the line endings, because my IDE takes care of that. I work in a project with multiple people, and *somewhere* in the process some files with different line endings than the project standard slipped into the repo. It happens. Your Git could be configured so that it automatically corrects the file line endings for you when you pull this code.  And this can be pretty annoying.

You can change this behaviour, but again, to avoid problems you have to know the reason why you get these errors. Be sure you know, see the link earlier in this post.

You can disable the CRLF behaviour completely, or per filetype by changing entries in your .gitattributes file. In my case, i put this:

* -crlf

This tells git to ignore the line endings for all files. And does not change the files in your working directory. Even if you have the core.autocrlf set to true, false, or input.