Chef has been announced. Listen to this podcast at Cloud Cafe. There’s no way around comparing puppet and chef. Sure, they’re both configuration management tools. It’s simplest to put it this way:
We’re replacing puppet with chef.
And why? A little while ago I wrote about problems I’ve been having scaling puppet. Off the top of my head, the biggest issues for me working with puppet have been:
- Dependencies graphs
- Limited capabilities of the language (DSL)
- Templates are evaluated on the server
Dependency Graphs
There’s a talk about vertically scaling puppet, but not a lot of it about horizontally scaling. I tend to run everything under puppet. People argue that it’s too much work to put single servers in puppet, and you should only use it for systems you intend to clone. I disagree. Puppet recipe’s are self documenting. The same people who don’t want to take the time to write puppet recipes for the single services are the people you have to beat with a sucker rod to get to document anything. Sometimes if I don’t have the time to put into fully testing a puppet recipe for a new machine, I’ll at least write the recipe as I’m working to server both as documentation and a starting point for if/when I come back to it.
The point is that as I scale out in this fashion, more often puppet will fail with a dependency problem on one run, and be fine on the next. I asked Luke about this at a BoF at OSCON 2008, and he basically told me that he really only focuses on the problems his paid customers have and was anxious to leave and get a beer. That’s fine, I understand it, but since it does nothing to fix my problem it drove me away from the puppet community.
While in theory having puppet do all this work to resolve depency issues seems fine, it is more complexity and trouble than it is worth. As a systems administrator I know what the dependancies are. As you build a system you simply write your recipe in the same order as the steps you’re taking.
Chef takes this idea and runs with it. Recipes are parsed top to bottom. If a package needs to be installed before a service is started, you simply put the package in the recipe first. This not only makes a lot of sense, it makes depencies in a complex recipe visually understandable. With puppet you can end up with spaghetti code remincisent of “goto”, jumping around a number of recipes in an order that’s difficult to understand.
Language
Before the recent 0.24.6, you could not even do:
if $ram > 1024 {
$maxclient = 500
}
The support for conditionals was rudimentary. I run into a lot of languages and the biggest problem I have is remembering how to do the same thing in each language. The puppet language does not do what a lot of lot of other languages do. I didn’t need another language to learn, let alone one written from scratch. It was just silly doing something like:
# iclassify script addes vmware-guest tag based on facter facts
$is_vmware = tagged('vmware-guest')
if $is_vmware {
include vmware
}
Chef uses ruby for it’s recipes. This makes the above stupidly simple with something like:
include_recipe "vmware" if node[:manufacturer] =~ /VMware/
Templates
Puppet evaluates recipes and templates on the server. I ended up with this block of code once when I need to specify the client node’s IP Address in a configuration file:
require '/srv/icagent/lib/iclassify'
ic = IClassify::Client.new("https://iclassify", iclassify_user, iclassify_password)
query = [ "hostname:", hostname].to_s
mip = nil
nodes = ic.search(query)
nodes.each do |node|
# node.attribs is an array of hashes. keys is 'name' value is 'values'
node.attribs.each do |attrib|
if attrib[:name].match(/ipaddress/)
ip = attrib[:values].to_s
if ip.match(/10.0.0./)
mip = ip
break
elsif ip.match(/10.0.1./)
mip = ip
break
end
end
end
end
This was so much work. Of course with chef you can easily get this information in the recipe because it’s parsed on the node, let alone the ease of doing it in the template if that’s more appropriate. Since the template’s parsed on the client, you grab the information out of variables or directly from the system.
As time goes on I’ll surely write more about using chef. We’re using it production now, and happy with it. In the interim, come to #chef on freenode if you have any questions.
Pingback: Hell’s Kitchen | IT Management and Cloud Blog
I hope I never actually said that I only fixed problems for customers – certainly 99% of my development is non-customer development, and I still do lots of bug-fixing and trouble-shooting in the community. It’s true that I can’t *guarantee* bug fixes for non-customers, but what open source project does? There’s a reason that the GPL specifies it comes with no warranty.
As to the rest, it’s pretty clear you’ve decided these problems are too much; the ip address is trivially available in server-side templates, and I’ve already expressed my willingness to work with Adam to solve the dependency issues. Apparently Adam was more interested in replacing than working with me.
It was more along the lines that you didn’t have the time to focus on the problem. I can’t speak to your motives… you knew I paid HJK money, perhaps you were encouraging me to get them to fix the problem, and you also strongly wanted beer. In any case it doesn’t matter a whole lot, I just figured you weren’t interested in looking at my problem right there at the BoF, which I believe James spearheaded. You’re free to do what you want to do, and like most people, someone offering you money can influence that. And that is just fine.
I’m a puppet fan and vocal supporter. The problems I’ve run into are frustrating and time consuming. Sometimes I’ve taken the time to look at fixing the problem: in some cases it was too complicated for my time (depgraphs), sometimes I was doing something stupid, and other times the magic (code) just wasn’t there yet to do what I wanted. I’m a systems administrator first, so my time tends to get allocated with that priority. My commits tend to have alignment with my needs, ease of entry to the community, and my skill.
The way Chef handles dependencies I think is indicative that Adam has a different idea about how to deal with this problem. “… some pieces that are differences in philosophy.” Most of the conversations I’ve had with Adam about these sorts of differences he has said that he looked at the code to try to find a way to fix the problems he encountered but it would be too much a fundamental change. Would you want to remove the dependency graph and move to a top down execution? It’s a change in both philosophy and a significant change to the code.
I think it’s unfair to claim HJK has given little back to puppet because of their lack of commits. I saw myself that much time was put into #1010 specifically, iClassify greatly increases the value of puppet and doesn’t do a whole lot without it, and they converted many a shop over to puppet, increasing its exposure. It’s silly to argue the value of these. I know that everyone I’ve met at HJK would hate to see a world where puppet never existed.
I certainly learned a lot from Puppet. Every day of my job is different now because of it and I believe Chef is a child of puppet. Experience showed that certain features didn’t work the way we expected them to, these were fundamental design differences, and so it was time for a rewrite with the benefit of that experience. That’s sorta how open source works, right? If everything isn’t going the way you feel it should you can fork. And hopefully when the dust settles the user gets a product that works the way they want it to.
I’m glad you mentioned the dependency issue. I used Puppet for developing a hosting setup on EC2 and really liked what I got from it, but the biggest problem I had was having to run the configuration multiple time to get everything installed. I assumed I was doing something wrong, but now this sounds like it’s (currently) and inherent issue with Puppet.
I plan to evaluate both products head to head in the next two weeks.
I appreciate your efforts at introducing Puppet to others, and I’m sorry you’ve had problems with dependencies in Puppet. As you say, Adam has a different philosophy of how they should work; I thought we had come to an agreement about how they could work in Puppet, but in looking at the commit logs for Chef, our discussion was after Chef was already created, and, of course, I never got any code to go with the agreement.
Anyway, I’ve done what I can to build an open development community, and I’ll continue doing so. Hopefully we’ll win you back someday.
Pingback: Learning to cook at btm.geek
Pingback: People Over Process » Links for January 26th
Pingback: tba » Blog Archive » Alternatives…
is there a forum on the site?
No forum, there are the mailing lists and #chef on freenode.
Chef is less compelling as it dodges the hard questions. Looks like it is probably great for Ruby coders but in that sense, this is much like an already solved problem redone in Ruby: how to execute scripts to configure systems on remote machines. Nothing is really “modeled,” you just have bits of code that get assembled, order dependent, and run on the remote machine (I’m sure the Chef supporters would argue that point, which is fine but my institution already had an answer for that when we switched to Puppet). There is nothing wrong with that. But it dodges the hard part of dependencies, as the project readily admits. I don’t think dependency graphs are an insurmountable problem and I think dodging it is something that’s been done already by other tools.
@DK, I’m not entirely sure what you’re talking about. It’s frustrating that CNET linked to this post as ‘chef versus puppet’ mantra, and that you comment with a similar tone.
You mention hard questions, but only bring up dependencies. I have no idea what ‘the hard part of dependencies’ that you claim Chef dodges is, and at least a link to where the project ‘readily admits’ that it dodges this question would be helpful rather than being hearsay.
The way Chef handles dependencies is a matter of philosophy, and centers mostly around a “top down” philosophy. Which is to say, you specify dependencies with order. Puppet is the only other configuration management language I have countable experience with. In that case, excluding a little bit of automatic dependencies like this file resource ought to have the directory it exists in, you generally still specify all of your dependencies by hand with requires. At this point, Puppet tries to make sense of all of these requires, which failed to work for me in frustrating ways. Which is sort of the point of this old post, recently renewed with foolish rivalry.
If you have an issue with Chef’s dependency design, I urge you to write your own thoughts out in a post, and pingback here, and perhaps we can help you address them.
Pingback: Top ten humorous linux tidbits | Nuclear Rooster
I know it’s a little late to the party, but there has been a very interesting discussion in the comments on my Puppet vs Chef article about Puppet and Chef’s respective approaches to dependency resolution.
There’s clearly a philosophical divide between those who think resources should be deterministically ordered where necessary and not otherwise (like Puppet), and those who believe they should always be predictably ordered (like Chef). It’s clear from the article which side I come down on, but I’m starting to think Chef’s design has some interesting features which I’d like to write about. I think a follow-up article is needed called something like ’10 Great Reasons to Use Chef’!
@John,
I think there’s clearly a philosophical divide about much more than ordering. Before the ‘Puppet vs Chef’ debates there were the ‘Why fork Puppet’ debates. I don’t believe the attitudes and answers have changed significantly since then.
Aside from my personal reasons for using chef a year ago in this article, there was a nice writeup by Adam Jacob entitled 9 things to like about Chef around the same period.
Thus, I’m against a follow-up article about the strengths of Chef over Puppet. I don’t believe we can add anything to that discussion in a positive way at this time, and this is why my response was ‘Configuration Management vs Meatcloud: 5 reasons CM wins‘.
While the dependency model arguments are academically interesting, it underscores that it is ultimately a matter of personal preference for the end user; which is more important, a short list of possibilities that make it easier to understand an entire system up front, or a more complicated system that leaves more options available to you in the long run?
If you read this thread very carefully, you’ll find most of the roots of the choices that made me leave puppet. If you’d like to discuss these in detail, you’re welcome to email me, but that is not an appropriate public discussion for the community.
Pingback: Why Puppet Should Manage Your Infrastructure | Engine Yard Ruby on Rails Blog