Getting Involved in the Operations Working Group
For the last few years I've been trying to get more people involved in the Operations Working Group - the team within the OpenStreetMap Foundation that runs all of our services. Each time I think "why aren't more people involved", I try to figure out some plausible barriers to entry, and then work on fixing them.
One of the reasons is that we're very quiet about what we do - there's a pride in keeping OpenStreetMap humming along and not causing too much of a fuss. But the lack of publicity hurts us when we're trying to get more people involved. Hence this blog post, among other reasons.
I've been working recently (as in, for the last few years) on making as much of our OWG activities public as possible, rather than hidden away on our mailing list or in our meetings. So we now have a public issue tracker showing our tasks, and we also publish a monthly summary of our activities.
To make OpenStreetMap work, we run a surprisingly large number of servers. For many years we maintained a list of our hardware and what each server was being used for on the OpenStreetMap wiki, which helped new people find out how everything works. Maintaining this information was a lot of work and the wiki was often outdated. For my own projects I use Jekyll to create static websites (which are easier to find hosting for than websites that need databases), and information on Jekyll sites can be generated with the help of data files. Since OWG uses Chef to configure all of our servers, and Chef knows both the hardware configuration and also what the machines are used for, the idea came that we could automate these server information pages entirely. That website is now live on hardware.openstreetmap.org so we have a public, accurate and timely list of all of our hardware and the services running on it.
Now my attention has moved to our Chef configuration. Although the configuration has been public for years, currently the barrier to entry is substantial. One straightforward (but surprisingly time-consuming) improvement was to simply write a README for each cookbook - 77 in all. I finished that project last week.
Unless you have administrator rights to OSMF hardware (and even I don't have that!) you need to write the chef configuration 'blind' - that is, you can propose a change but you can't realistically test that it works before you make a pull request. That makes proposing changes close to impossible, so it's not surprising that few non-administrators have ever contributed changes. I have experience with a few tools that can help, the most important being test-kitchen. This allows a developer to locally check that their changes work, and have the desired effect, before making a pull request, and also it allows the administrators to check that the PR works before deploying the updated cookbook. Both Matt and I have been working on this recently, and today I proposed a basic test-kitchen configuration.
This will only be the start of a long process, since eventually most of those 77 cookbooks will need test-kitchen configurations. Even in my initial attempts to test the serverinfo cookbook (that generates hardware.openstreetmap.org) I found a bunch of problems, some of which I haven't yet figured out how to work around. There will be many more of these niggles found, but the goal is to allow future developers to improve each cookbook using only their own laptops.
All of these are small steps on the long path to getting more people involved in the Operations Working Group. If you're interested in helping, get stuck in to both our task list and our chef repo, and let me know when you get stuck.
This post was posted on 28 July 2016 and tagged chef, development, OpenStreetMap