On updating chef nodes

After almost a year of starting instances we got the problem of keeping chef-clients recent on instances.

Couple times we had problems due to bugs in old chefs.

First task was to get list of all installed chef-clients on our boxes.

We found the article about it. But the method is dirty:

Get response in Ruby, transform to json, parse it using Ruby.

Yak.

Here is simpler way using Ruby

require 'chef/knife'
Chef::Knife.new.configure_chef
result = Chef::Search::Query.new.search('node', 'name:*')[0]
versions = result.each_with_object(Hash.new(0)) {|node, hash| hash[node['chef_packages']['chef']['version']] += 1 rescue nil}

Additionally with that way we can get the list of broken nodes, those won’t have node[‘chef_packages’] at all, that is why I used rescue nil.

For cleaning, I used couple more lines:

result.select do |node|
  node['chef_packages'].nil?
end.each do |node|
  Chef::ApiClient.load(node.name).destroy if Chef::ApiClient.list.keys.include?(node.name)
  node.destroy
end

Updating was also tricky. There is omnibus_updater cookbook for keeping chef up-to-date. There is a fork which allows update windows nodes. But the easiest way I found is just update chef gem on nodes using winrm/ssh. It is not cleanest way (old versions of Ruby and old gems), but still it is fast and clear.

Advertisements