Alexey Vasiliev aka leopard
leopard.in.ua
Cooking
infrastructure
by
Chef
Contents
Contents 1
1 Introduction 5
2 So, what is Chef? 6
2.1 What are the core principles? . . . . . . . . . . . . . . . . . . . 7
Idempotence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Thick Clients, Thin Server . . . . . . . . . . . . . . . . . . . . . 7
Order Matters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 Why you should use Chef? . . . . . . . . . . . . . . . . . . . . . 7
2.3 What doesn’t Chef do? . . . . . . . . . . . . . . . . . . . . . . . 8
2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3 Chef Solo 10
3.1 Required software . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Rubygems and bundler . . . . . . . . . . . . . . . . . . . . . . . 11
Knife-solo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Berkshelf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.2 Creation of kitchen (chef-repo) . . . . . . . . . . . . . . . . . . . 12
3.3 .Chef folder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.4 Vendor cookbooks and berkshelf . . . . . . . . . . . . . . . . . . 13
3.5 Defining nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.6 Vagrant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.7 Idempotence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.8 Defining roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.9 Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.10 Defining environments . . . . . . . . . . . . . . . . . . . . . . . 23
3.11 Defining data bags . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.12 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4 Chef Server 30
1
Contents
4.1 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.2 Knife . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.3 Bootstrap first node . . . . . . . . . . . . . . . . . . . . . . . . 36
Node in Vagrant . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.4 Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Attribute Types . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Automatic (Ohai) . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Attribute Precedence . . . . . . . . . . . . . . . . . . . . . . . . 41
4.5 Role . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.6 Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.7 Knife ssh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Chef-client cookbook . . . . . . . . . . . . . . . . . . . . . . . . 47
4.8 Data bags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
5 Writing Cookbooks 51
5.1 Cookbook file organization . . . . . . . . . . . . . . . . . . . . . 51
5.2 Metadata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.3 Resources and Providers . . . . . . . . . . . . . . . . . . . . . . 54
Bash . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Cron . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Directory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Git . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
Link . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
Cookbook_file . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
User . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Deploy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
5.4 Recipes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
Assign Dependencies . . . . . . . . . . . . . . . . . . . . . . . . 60
Create Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Include Recipes . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Reload Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Accessor Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 62
5.5 Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
5.6 Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
File Specificity . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
Partial Templates . . . . . . . . . . . . . . . . . . . . . . . . . . 68
5.7 LWRPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
Providers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
DSL Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Using LWRP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
2
Contents
5.8 HWRPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
HWRPs and LWRPS . . . . . . . . . . . . . . . . . . . . . . . . 81
Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.9 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
5.10 Ohai . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
5.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
6 Testing Cookbooks 97
6.1 Test Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Unit Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Integration Testing . . . . . . . . . . . . . . . . . . . . . . . . . 97
Acceptance Testing . . . . . . . . . . . . . . . . . . . . . . . . . 98
6.2 ChefSpec . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
Installing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
6.3 Fauxhai . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
6.4 Test Kitchen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Installing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Running Kitchen Converge . . . . . . . . . . . . . . . . . . . . . 107
Manually Verifying . . . . . . . . . . . . . . . . . . . . . . . . . 108
Writing a Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
Running Kitchen Test . . . . . . . . . . . . . . . . . . . . . . . 110
Adding a Platform . . . . . . . . . . . . . . . . . . . . . . . . . 111
Fixing Converge . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
Adding a Suite . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
Writing a Server Test . . . . . . . . . . . . . . . . . . . . . . . . 117
6.5 Chef Zero . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Installing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Using with ChefSpec . . . . . . . . . . . . . . . . . . . . . . . . 122
Using with Test Kitchen . . . . . . . . . . . . . . . . . . . . . . 128
6.6 Minitest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
Test Kitchen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
Minitest Chef Handler . . . . . . . . . . . . . . . . . . . . . . . 133
6.7 Cucumber . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
6.8 Static Analysis and Linting Tools . . . . . . . . . . . . . . . . . 138
Foodcritic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
RuboCop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
Strainer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
6.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
7 Tips and Tricks 146
3
Contents
7.1 Wrapper cookbook . . . . . . . . . . . . . . . . . . . . . . . . . 146
Codifying Standards in Your Organization . . . . . . . . . . . . 146
Modifying Upstream Cookbook Behavior . . . . . . . . . . . . . 146
Advanced Upstream Cookbook Modification . . . . . . . . . . . 147
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
7.2 Knife Plugins . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
7.3 Chef Metal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
7.4 Chef Sugar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
7.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
Bibliography 157
4
2
So, what is Chef?
Chef is a configuration management tool written in Ruby and Erlang. It
uses a pure-Ruby, domain-specific language (DSL) for writing system configu-
ration «recipes». Chef is used to streamline the task of configuring and main-
taining a company’s servers, and can integrate with cloud-based platforms such
as Rackspace and Amazon EC2 to automatically provision and configure new
machines.
The user writes «recipes» that describe how Chef manages server applica-
tions (such as Apache, MySQL, or Hadoop) and how they are to be configured.
These recipes describe a series of resources that should be in a particular state:
packages that should be installed, services that should be running, or files that
should be written. Chef makes sure each resource is properly configured and
corrects any resources that are not in the desired state.
Traditionally, Chef is used to manage GNU/Linux but later versions sup-
port running on Windows as well.
6
2.1. What are the core principles?
2.1 What are the core principles?
Idempotence
A recipe can run multiple times on the same system and the results will
always be identical. A resource is defined in a recipe, which then defines the
actions to be performed on the system. The chef-client ensures that actions
are not performed if the resources have not changed and that any action that
is performed is done the same way each time. If a recipe is re-run and nothing
has changed, then the chef-client will not do anything.
Thick Clients, Thin Server
Chef does as much work as possible on the node and as little as possible on
the server. A chef-client runs on each node and it only interacts with the server
when it needs to. The server is designed to distribute the data to each node
easily, including all cookbooks, recipes, templates, files, and so on. The server
also retains a copy of the state of the node at the conclusion of every chef-
client run. This approach ensures that the actual work needed to configure
each node in your infrastructure is distributed across the organization, rather
than centralized on smaller set of configuration management servers.
Order Matters
When the chef-client configures each node in the system, the order in which
that configuration occurs is very important. For example, if Apache is not
installed, then it cannot be configured and the daemon cannot be started.
Configuration management tools have struggled with this problem for a long
time. For each node a list of recipes is applied. Within a recipe, resources
are applied in the order in which they are listed. At any point in a recipe
other recipes may be included, which assures that all resources are applied.
The chef-client will never apply the same recipe twice. Dependencies are only
applied at the recipe level (and never the resource level). This means that
dependencies can be tracked between high-level concepts like «I need to install
Apache before I can start my Django application!» It also means that given
the same set of cookbooks, the chef-client will always execute resources in the
same exact order.
2.2 Why you should use Chef?
There are several reasons for using Chef:
∙ Efficiency: It’s more effective to use Chef, which will contain all your
servers configuration in one place
7