Richard Tuin's mindspins

A weblog on web development, tools, and productivity

Navigation

Automatically installing all Ansible role dependencies

I think it was a couple of months ago now that I first started to fiddle around with Ansible. Being used to work with Puppet as a system provision tool before, Ansible seemed to make this task I disliked so bad before very easy. Almost fun I guess.

One thing that I found missing though, was a way to automatically install all the role dependencies. This still seems to be a task you have to do manually. Or anyway, I can’t find a way how to do it with the current Ansible version. So, being a programmer and all, I wrote a script for it.

The script itself is very simple:

  1. Loops through every role installed in roles/
  2. For every role, it fetches dependencies from meta/main.yml
  3. Installs the dependencies via Ansible galaxy

Personally I haven’t found a use case for other features, but if you do, please let me know.

The script source is available here: GitHub link
Available for download here: Direct download

Happy coding!

Charles is my superhero!

As a true geek I am a big fan of several superheroes. Actually, Iron Man is by far my favourite one. Or actually, the coolest part about Iron Man is J.A.R.V.I.S. if you ask me. The Just A Rather Very Intelligent System, is a clever system that assists Tony Stark in virtually everything that he considers “everyday tasks”.

Unfortonately I do not embody a superhero, but I do have several tools that assist me with my everyday tasks, and one of these tools is Charles Proxy.
Like my IDE helps me with being productive in writing, testing and troubleshooting code, Charles Web Debugging Proxy helps me with most of the things that are related to HTTP. And as a web developer, you know, that’s a lot. Here are some of the most significant contributions of Charles in my everyday work.

Communication insight

Specifically when developing APIs, a lot of communication is happening “under water”. A website does a request to an API, or a mobile device calls some webservices, and you want to know what’s going on. As with most proxies, Charles can show the HTTP-client requests. What’s connecting when, to where, and with what headers, cookies, request/response bodies etc.

Request tampering

Screen Shot 2013-07-25 at 20.34.59By setting a breakpoint on a specific request, you can modify the entire HTTP request the client is sending to the server. Change headers, cookies, request bodies, query parameters. In short, everything that you need to troubleshoot most common HTTP-related issues. Activate this in the context menu of a specific request -> Breakpoints, and the next time your client requests this URL, charles will break on the request.

Give colleagues and devices access to your dev environment

Another very convenient feature is that if you set your proxy server as a system-wide proxy on let’s say, your mobile device or colleagues’ machine, that they can see everything web-accessible on the machine you’re running Charles on.
So in practice, it means that you can check out your application that’s running on let’s say, a host-only connected virtual machine. This is something that we use a lot here in the office. Pretty neat, huh?

But there’s more…

Actually, the list above is just the tip of the iceberg of Charles’ features. There are a lot more features in the application, like: Bandwidth throttling, Request filtering, Recording/Replaying and saving sessions, DNS spoofing, SSL proxying, and there’s still more.

So, why a blog about Charles’ features while I could also visit it’s website?

Well, if you’ve read this far, I’ve got your attention. And that’s exactly what I was after. My goal with this post is to show that there’s a tool that can help you with things that can be very cumbersome without a proxy. And more! Charles is just great, and every web developer should at least know about it’s existence.

Of course, Charles is not the only Proxy program out there, and probably there are tools that have more or less the same features. If you know such a tool, could you let me know in the comments?
Oh, and Charles runs on virtually every desktop platform!

When and what to test in software development

Image courtesy of http://www.sxc.hu/photo/418215

I hear people around me often ask this question. Confused whether and when they should unit-test their classes, do integration tests on their components, functionally test their to be delivered software, and execute a regression test on their previously delivered product.

And really, there are a lot of answers to all these questions. And unfortunately, each of them are subject to the infamous answer of all questions in software development: It depends*.

But actually, there is a simple answer to all of these questions when you think about why you test things. Simple enough to give you food for thought. Tests reassure you that something (still) works. It gives you confidence. Confidence in the code that you write as a software developer. In the application that you deploy as a development team. In the product that you accept as a product owner/client.

Think about it. Think about your current project, and about what you currently test (or not) and why.
I often get the impression that people test, or want to test “because it is good for quality”. And it sure is. But think about what quality really is. Being confident about the solution that you built for your clients problem, and that you’ve delivered it well. That’s quality in a nutshell.

So actually the answer to the question “When and what to test in software development?” is really simple:

Test what gives you confidence about the (to be delivered) solution.

How you do it, is up to you.

Disclaimer: Some statements in this post may be over-simplified.

* actually, as in all of life, the universe, and everything, the answer should be 42. But that seemed a bit inappropriate in the context of this post.

Image courtesy of http://www.sxc.hu/photo/418215

New writing tool: Getting in the zone

Usually when I am writing, I get distracted a lot. And I mean a lot. And this is one of the reasons that i don’t write as much as I want to.

However tonight I found this tool called OmmWriter. And it is specifically designed to zone yourself out from all distractions and write. Just write. It helps me a lot to do productive writing.

ommwriter-logo-main

When starting the application, which runs in full-screen mode only, it suggests that it works best with your headphones on. And this is where the zoning out begins. After the splash screen the application plays a soothing ambient song in the background. And it helps me focus on my screen.
Music normally distracts my thought from the task at hand, but this music does not. Excellent!

As soon as you start typing you hear the sound of your keyboard. A little something that the application adds. I think it works motivating to hear the ticks of the keyboard. It motivates to stick to the rhythm of typing.

Proof that this works is that last night I’ve written a 900 word blogpost in under an hour. Possibly one of the longest blogposts that I’ve ever written. Now that’s some productive writing!

If you are interested in this application, you can download the application for free at the OmmWriter website. It is available for Mac, Windows, and iPad.

See, another 200+ word blogpost done in under 10 minutes. Way to go!

An example on running Behat from Jenkins

Time for a very short article. I just want to share a little trick that helped me run my Behat testsuite from a Jenkins build.
Most of the articles that I found on Google did not really cut to the chase. Here it is:

1) Create an “execute shell” build step that executes Behat, and outputs it’s results to a jUnit format:

2) Create a post-build “Publish JUnit test result report” action that reads the build/ directory for the jUnit files.

That’s it! If you want to know more about the formatting features that Behat provides, click here.

Screen Shot 2013-05-10 at 00.47.42

Javascript minification in capistrano/webistrano

Photo: http://www.sxc.hu/photo/347930

Recently i had to automate the minification of specific javascripts. Since we don’t have a real build tool, i thought it was best to do this during deployment.
For (automatic) deployments our tool of choice is Webistrano, which is based on Capistrano, which is written in Ruby.

Capistrano has this concept of recipes, which are small scripts that you write to do specific tasks during deployment.
So, one of these tasks could be the minification of some javascript.

But for the minification and deployment of that, i was bound to some constraints:

  • We’re using the yuicompressor (java)
  • Deploy target machines do not have java installed
  • So we have to do the minification locally

After doing some initial research it was quite easy to do, and i came up with the following code:

The approach can be considered somewhat unusual, as the project’s files are first uploaded to the server. And after that the javascript is minified and uploaded seperately. This is mainly because I couldn’t find another hook within capistrano that fits my need.
Of course, you can rewrite this script to work with any other local command too.

Note/disclaimer: This script uses the copy_cache directive. I’m not a capistrano expert, but this variable might not be available to all deployment strategies. This script might not work for you if copied 1:1. But i suppose you get the idea.

How to make Git ignore different line endings

warning: CRLF will be replaced by LF in [filename].
The file will have its original line endings in your working directory.

Getting errors like these, and want to deliberately ignore them?

If you don’t know why you would, you should read this great blogpost about line endings in Git by Tim Clem.

Don’t look at line endings!

In some cases you want to ignore these errors. In my specific situation i don’t want Git to warn me about the line endings, because my IDE takes care of that. I work in a project with multiple people, and *somewhere* in the process some files with different line endings than the project standard slipped into the repo. It happens. Your Git could be configured so that it automatically corrects the file line endings for you when you pull this code.  And this can be pretty annoying.

You can change this behaviour, but again, to avoid problems you have to know the reason why you get these errors. Be sure you know, see the link earlier in this post.

You can disable the CRLF behaviour completely, or per filetype by changing entries in your .gitattributes file. In my case, i put this:

* -crlf

This tells git to ignore the line endings for all files. And does not change the files in your working directory. Even if you have the core.autocrlf set to true, false, or input.

PHP CLI remote debugging with PhpStorm & Zend Debugger

This post aims to give an overview of how to get remote cli debugging working with PhpStorm and Zend Debugger. You can find more detailed information under the links in this article.

Some time ago i wanted to debug a PHPUnit test in PhpStorm, debugging PHP scripts using the cli sapi turned out to require some extra effort.

The situation is that my project files reside in a virtualbox, and open the project files -via a mounted directory- in PhpStorm. This means that, for normal web debugging, you have to specify some server settings, including the path mapping.

Since you work on the server directly, you have to set up a remote debugging connection to your IDE. In his blog post, Kevin Schroeder explains how to do this.
You can do this by running the following command, as copied from Kevin’s post:

Don’t forget to change the IP-address and port to the address that you’re IDE is running on.

Now there is one more thing you have to do to make PhpStorm understand where the files are, otherwise you still wouldn’t hit those breakpoints.
According to this article from the JetBrains team, this can be done by setting the PHP_IDE_CONFIG environment variable to “serverName=name-of-server” where name-of-server is the name as configured in Project Settings -> PHP -> Servers.

And oh, make sure your “Listen to Debug connections” button is green!

As i work with different projects on the same virtualbox, i have created this small script that you can put in your .bashrc

You can enable debugging by running:

$ bugon server-name

And disable by running:

$ bugoff

Happy debugging!

Software Development and the Broken Windows Theory

There is a theory called the “Broken Windows Theory“. This theory states that if there is a disorder in a neighbourhood for a substantial period of time, the mess will only get worse. The authors give the example of a broken window in an abandoned building. If it isn’t replaced within some time, the chances are that vandals will break another window. And another. Gradually making the mess worse, and therefore making the issue bigger.

How the broken windows theory relates to software development

This theory applies to quality in software development as well. Over time the quality of our software may decrease for a variety of reasons. For instance, when under the pressure of a deadline we tend to do concessions to the quality of our software. Or when some tests fail, and no-one acts, it doesn’t motivate to write good tests and more tests will fail over time. Effectively making the testsuite useless. This disorder is also known as technical debt.

How it works

The mechanic about this effect is that you unconsciously and gradually lower the norm of quality of your software. This is of course not what you want for your customer. But also for your team.
Not responding to the “broken windows” in your project will effectively change the mindset about quality, and it leads to passive/reactive behaviour. Do you know that feeling that you’re constantly running into problems with existing code? Then you might have had some broken windows for too long!

Fix the windows!

The broken windows theory in full effect
©2012 iStockphoto

The solution that the broken windows theory describes is to actively monitor and report any disorder like a broken window, litter, holes in the road, or graffiti. In software development their equals would be poor code, bugs, failing tests, or bad design implementations.

The key to success is to maintain a zero tolerance policy on technical debt. And to clean up the list of disorders within a timely fashion.
Of course, you and your team have to decide on what you consider technical debt within the project. This will usually be affected by customer expectations and/or budget. But make sure you all agree on what the quality standard is. Otherwise maintaining a zero tolerance policy will be ineffective, and may cause frustration.

It’s in your hands

Implementing this strategy is something you can start doing today. If you start pointing out and act to the so-called technical debt, you will soon start a movement within your team. You will create awareness and team members will follow your example. Social control on the subject will grow, which encourages to stick to the set quality standards.

Ultimately leading to a better product, happy teams, and happy customers!

Productivity ABC: A

Automation

Gears

© 2011 foxbit – sxc.hu

Automation can be very profitable when it comes to boosting productivity. Not only because you don’t have to do an action manually, but there are other productivity gains.

Eliminating the possibility of human error is one of them. Automation is especially useful for repetitive tasks. Tasks which are more often than not boring as waiting for grass to grow.
Waiting, or being bored decreases focus and causes the need for distraction. And losing focus on the task at hand is the worst thing that can happen to your productivity.

There are a lot of tools that can help you with automating different type of tasks. As a software developer i usually make small scripts for tasks i need to automate. Therefore i don’t use many automation tools. However, if you have suggestions for productivity tools, please leave a comment!
Some of my favourite automation languages: PHP & shell.

Attention span

Oh look, a butterfly!

Distraction: “Oh look, a butterfly!” – © 2012 irum – sxc.hu

Try to find out what your average attention span is, and always take a short break after that period.
It doesn’t have to be a long break. Just grab a coffee, stretch, do a small (but different) task, or anything like that. You need this to re-focus on the task. If you start on the original task again, work while your attention span lasts, and then take a break again to refocus after that.

It may sound that you will lose a lot of time because of all the breaks, but this is not true. (if your breaks are short enough, hehe).
Working longer on a task longer than your full attention lasts will result in working less efficient, and what you do is possibly more prone to errors. So taking a break every now and then will most likely boost your productivity.

Finding out how long your attention span lasts can be quite a challenge. It might be good to know that the maximum attention span lies somewhere between 3 minutes (young children) and 20 minutes (adults).
The easiest way to find your attention span is to start a stopwatch, and start working on the task you want to complete. Stop the stopwatch and remember what time is on the stopwatch. Reset and start the stopwatch and continue on the task. Do this a couple of times and you’ve found your average attention span.

For more information on attention span visit Wikipedia.