Skip to content

Commit 928ad14

Browse files
author
Patrick Wright
committed
Merge of Chef Stack (https://github.com/ncerny/chef_stack) resource and helpers into Chef Ingredient (https://github.com/chef-cookbooks/chef-ingredient)
Specific files were selected in their entirety versus merging the chef_stack repo to keep the commits clean and avoid conflicts. Background Chef Stack is a series of primitives for installing, configuring, and managing over time all of Chef's products. Its intention is to be very prescriptive about how Chef's software stack is configured and maintained, while being very non-prescriptive about how its deployed, or where its configuration comes from. It was born out of a Customer need to focus on the complexities of their environment, without having to focus on Chef's core software. Moving the management of Chef's software to easy-to-consume primitives allows the Customer to focus on the things that are important to them -- their environment and their workflow. Contributions Special Thanks to Edmund DeSmet <[email protected]> who heavily influenced the design of Chef Stack. andy-dufour <[email protected]> (10): Fixing push jobs client config Fix Issue #5: Run delivery token before delivery api so output is only JSON Version bump Adding non-standard platform support Adding platform options to client resource Adding readme Fixing formating Fixing spacing Adding descriptions to custom resources Fixing some formatting Brandon Raabe <[email protected]> (3): Fix tables in the readme Remove default value of `nil` Fix regex to determine if source is a URI Jeremy J. Miller <[email protected]> (5): adding package_source to automate resource changing property_is_set call adding second guard for resource fixing 'source is required' error Merge pull request #10 from itmustbejj/wip Josh Hudson <[email protected]> (20): Merge pull request #6 from andy-dufour/ad/v2-runner-token-fix Move write_vault to helpers Update write_vault to reflect renamed cookbook Add gather_secrets action for chef_backend resource. Use node['chef_stack']['admin'] defaulted to workflow instead of hardcoded string in write_vault function Implement read_vault helper function Change the location of the chef vault encryption key Use an attribute to determine the client key Default chef_backend_secrets to nil Don't have non-bootstrap try to join-cluster if they already have. Allow :chef_config_file property for wf_builder resource instead of hardcoding /etc/chef/client.rb Don't explicitly pass in node name when tagging runner Use Chef::Config['node_name'] instead of node['fqdn'] in wf_builder resource Reverting a bunch of changes we didn't need to make Don't write chef-backend-secrets.json if it's not set (bootstrap) Revert partial implementation of package_source property for resources for another feature branch. Fix chef_backend_secrets property for use case of bootstrap backend generating it's own secrets. Lint and syntax fixes Merge pull request #11 from ncerny/fix_source Merge pull request #15 from andy-dufour/ad/multi_platform Nathan Cerny <[email protected]> (55): Add generated delivery configuration Add generated cookbook content Add generated delivery build cookbook Merge branch 'add-delivery-configuration' Initial build out of cookbook for managing Chef-Client, Chef-Server, and Chef-Automate. Update v1 build node search string to delivery-build-node to better match legacy workflows. Supermarket recipe and tweaks for performance and workflow. Move client config to shared recipe, and update all components to latest. cleanup and switch all components to latest Fix issues with spinning up fresh cluster. Create directory and place config before reconfigure of automate so nginx picks up config. Bug fixes. Add initial chef-backend resource and recipe Fix bug with key on chef_user resource. Same bug in chef_org Be consistent in declaration of new_resource in chef_user Pull client.rb template from this cookbook. Rename 'key' to 'key_path' to be more explicit. Lay down validation pem in install, if it's set. property error - test if new_resource works in this context. Lazy-load contents of delivery-cmd for v2 builders. Ensure authorized_keys file exists for v2 job runner. Enterprise should be configurable in workflow_builder. Merge pull request #1 from andy-dufour/ad/push_jobs_config Rename Cookbook to chef_stack Merge pull request #2 from ncerny/nc/rename-cookbook Move ensurekv to helpers to avoid duplication. Testing should be done with chef-services Merge pull request #3 from ncerny/nc/cleanup Merge pull request #4 from ncerny/lauck/fix_cookbook_name Merge pull request #16 from andy-dufour/ad/add_readme Merge pull request #18 from brandocorp-cookbooks/chef-backend-secrets-deprecation Merge pull request #17 from brandocorp-cookbooks/readme-fixes Version Bump for PRs - fix deprecation on chef_backend, and fix README tables. Switch chef_backend to use peers instead of bootstrap. Change chef-backend-secrets to default to empty string. suppress error messages in chef-backend guard Fix lint failures. Remove old delivery configuration. Merge pull request #23 from ncerny/nc/remove-delivery Remove old .chef directory. Merge pull request #24 from ncerny/nc/remove-chef Merge branch 'master' into nc/fix-lints Default chef_file source to chef_file if source property is not set. Fix incorrect log_location in wf_builder. Version bump - 0.8.0 - Clear up ambiguous versions Merge pull request #26 from ncerny/nc/fix-source-req-error Merge branch 'master' into nc/wf-log-bug Merge pull request #25 from ncerny/nc/wf-log-bug Merge branch 'master' into nc/fix-lints Merge pull request #21 from ncerny/nc/fix-lints Merge branch 'master' into nc/allow-backend-peers Merge pull request #19 from ncerny/nc/allow-backend-peers Merge pull request #31 from brandocorp-cookbooks/chef-file-regex Release 0.8.1 Stephen Lauck <[email protected]> (1): Rename cookbook chef to chef_stack in client resource Signed-off-by: Patrick Wright <[email protected]> Signed-off-by: Patrick Wright <[email protected]>
1 parent 5543cf5 commit 928ad14

File tree

16 files changed

+1265
-0
lines changed

16 files changed

+1265
-0
lines changed

.kitchen.yml

+6
Original file line numberDiff line numberDiff line change
@@ -78,3 +78,9 @@ suites:
7878
<% end %>
7979
- recipe[test]
8080
- recipe[test::inspec]
81+
82+
- name: chef_server
83+
includes: [ 'ubuntu-16.04' ]
84+
run_list:
85+
- recipe[test]
86+
- recipe[test::chef_server]

CHEFSTACK.md

+142
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
# chef_stack
2+
3+
Chef stack is a library cookbook that provides custom resources to build and manage your Chef infrastructure.
4+
5+
An accompanying project, [Chef-Services](https//github.com/stephenlauck/chef-services) exists as an example implementation of Chef Stack.
6+
7+
## Custom Resources
8+
9+
Below are the custom resources provided by this cookbook.
10+
11+
### General Properties
12+
13+
These properties exist for all resources
14+
15+
| Name | Type | Default Value | Description |
16+
|---|---|---|---|
17+
| name | String | N/A | A name for the resource |
18+
| channel | Symbol | stable | The channel from our package repository to install. Most of the time you want stable. |
19+
| version | [String, Symbol] | latest | The version of Automate you want to install |
20+
| config | String | N/A | The configuration that will be written to the appropriate configuration file for the product. |
21+
| accept_license | [TrueClass, FalseClass] | false | Do you accept Chef's license agreements. |
22+
| platform | String | Auto-detected | Use only if you need to over-ride the default platform. |
23+
| platform_version | String | Auto-detected | Use only if you need to over-ride the default platform. |
24+
25+
26+
### chef_automate
27+
28+
Installs Chef Automate.
29+
30+
#### Properties
31+
| Name | Type | Default Value | Description |
32+
|---|---|---|---|
33+
| enterprise | [String, Array] | chef | The Enterprise to create in Automate|
34+
| license | String | N/A | Your license file | we recommend using the chef_file resource |
35+
| chef_user | String | workflow | The user you will connect to the Chef server as |
36+
| chef_user_pem | String | N/A | The private key of the above Chef user |
37+
| validation_pem | String | N/A | The validator key of the Chef org we're connecting to |
38+
| builder_pem | String | N/A | The private key of the build nodes |
39+
40+
### chef_backend
41+
42+
#### Properties
43+
44+
| Name | Type | Default Value | Description |
45+
|---|---|---|---|
46+
| bootstrap_node | String | N/A | The node we'll bootstrap secrets with. |
47+
| publish_address | String | node['ipaddress'] | The address you want Chef-Backend to listen on. |
48+
| chef_backend_secrets | String | nil | A location where your secrets are | we recommend using the chef_file resource. |
49+
50+
### chef_org
51+
52+
#### Properties
53+
54+
55+
| Name | Type | Default Value | Description |
56+
|---|---|---|---|
57+
| org | String | N/A | The short name of the org. |
58+
| org_full_name | String | node['ipaddress'] | The full name of the org you want to create. |
59+
| admins | Array | N/A | An array of admins for the org. |
60+
| users | Array | [] | An array of users for the org. |
61+
| remove_users | Array | [] | An array of users to remove from the org. |
62+
| key_path | String | N/A | Where to store the validator key that is created with the org. |
63+
64+
### chef_user
65+
66+
| Name | Type | Default Value | Description |
67+
|---|---|---|---|
68+
| username | String | N/A | The username of the user. |
69+
| first_name | String | N/A | The first name of the user. |
70+
| last_name | Array | N/A | The last name of the user. |
71+
| email | Array | [] | N/A | The users e-mail. |
72+
| password | Array | [] | The users password. |
73+
| key_path | String | N/A | Where to store the users private key that is created with the user. |
74+
| serveradmin | [TrueClass, FalseClass] | F | Is the user a serveradmin? |
75+
76+
### chef_client
77+
| Name | Type | Default Value | Description |
78+
|---|---|---|---|
79+
| node_name | String | true | The name of the node. |
80+
| version | [String, Symbol] | latest | The version of chef-client to install. |
81+
| chefdk | [TrueClass, FalseClass] | false | Do you want to install chefdk? |
82+
| chef_server_url | [String, Symbol] | local | What is hte Chef server URL to connect to. |
83+
| ssl_verify | [TrueClass, FalseClass] | true | Validate ssl certificates? |
84+
| log_location | String | 'STDOUT' | Where to log. |
85+
| log_level | Symbol | auto | Log level. |
86+
| config | String | | Any configuration for client.rb. |
87+
| run_list | Array | | The clients runlist. |
88+
| environment | String | | Which Chef Environment the client belongs to. |
89+
| validation_pem | String | | The validation pem to validate with. |
90+
| validation_client_name | String | | The validation client name. |
91+
| tags | [String, Array] | '' | Any tags for the node. |
92+
| interval | Integer | 1800 | The interval to run chef-client on. |
93+
| splay | Integer | 1800 | The randomization to add to the interval. |
94+
| data_collector_token | String | '93a49a4f2482c64126f7b6015e6b0f30284287ee4054ff8807fb63d9cbd1c506' | The data collector token to talk to Visibility. |
95+
| data_collector_url | String | | The Visibility URL to send data. |
96+
97+
### chef_file
98+
99+
100+
| Name | Type | Default Value | Description |
101+
|---|---|---|---|
102+
| filename | String | | The name of the resource. |
103+
| source | String | | The source of the file. |
104+
| user | String | default 'root' | The owner of the file. |
105+
| group | String | default 'root' | The group owner of the file. |
106+
| mode | String | default '0600' | The mode for the file. |
107+
108+
### chef_server
109+
110+
| Name | Type | Default Value | Description |
111+
|---|---|---|---|
112+
| addons | Hash | | A set of addons to install with the Chef Server. |
113+
| data_collector_token | String | default '93a49a4f2482c64126f7b6015e6b0f30284287ee4054ff8807fb63d9cbd1c506' | The data collector token to authenticate with Chef Visiblity. |
114+
| data_collector_url | String | | The URL to connect to Visibility. |
115+
116+
### chef_supermarket
117+
118+
| Name | Type | Default Value | Description |
119+
|---|---|---|---|
120+
| chef_server_url | String | Chef::Config['chef_server_url'] | The Chef server's URL. |
121+
| chef_oauth2_app_id | String | | The oauth2 app id from the Chef server. |
122+
| chef_oauth2_secret | String | | The oauth2 secret from the Chef server. |
123+
| chef_oauth2_verify_ssl | [TrueClass, FalseClass] | true | Whether to validate SSL certificates. |
124+
125+
### workflow_builder
126+
127+
| Name | Type | Default Value | Description |
128+
|---|---|---|---|
129+
| pj_version | [String, Symbol] | :latest | The version of Push-Jobs to install. |
130+
| chef_user | String | 'workflow' | The Chef user to authenticate with the Chef Server. |
131+
| chef_user_pem | String | | The private key of the Chef user to authenticate with the Chef Server. |
132+
| builder_pem | String | | The builder users private key to communicate with Chef Automate. |
133+
| chef_fqdn | String | URI.parse(Chef::Config['chef_server_url']).host | The FQDN of the Chef server. |
134+
| automate_fqdn | String | | | The FQDN of the automate server. |
135+
| supermarket_fqdn | String | | The FQDN of the Supermarket server. |
136+
| job_dispatch_version | String | 'v2' | Which job dispatch version to use. V1 is push-jobs, V2 is SSH runners. |
137+
| automate_user | String | 'admin' | What is the Automate user we're connecting to Automate as. |
138+
| automate_password | String | | The password for the user above. |
139+
| automate_enterprise | String | 'chef' | The Enterprise to connect to. |
140+
| chef_config_path | String | '/etc/chef/client.rb' | The config path for chef-client. |
141+
142+
## Contributers

libraries/helpers.rb

+24
Original file line numberDiff line numberDiff line change
@@ -300,3 +300,27 @@ def installer
300300
end
301301
end
302302
end
303+
304+
#
305+
# Chef Stack Helpers
306+
#
307+
def prefix
308+
(platform_family?('windows') ? 'C:/Chef/' : '/etc/chef/')
309+
end
310+
311+
def ensurekv(config, hash)
312+
hash.each do |k, v|
313+
if v.is_a?(Symbol)
314+
v = v.to_s
315+
str = v
316+
else
317+
str = "'#{v}'"
318+
end
319+
if config =~ /^ *#{v}.*$/
320+
config.sub(/^ *#{v}.*$/, "#{k} #{str}")
321+
else
322+
config << "\n#{k} #{str}"
323+
end
324+
end
325+
config
326+
end

resources/automate.rb

+119
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
#
2+
# Author:: Nathan Cerny <[email protected]>
3+
#
4+
# Cookbook Name:: chef_stack
5+
# Resource:: automate
6+
#
7+
# Copyright 2017 Chef Software Inc
8+
#
9+
# Licensed under the Apache License, Version 2.0 (the "License");
10+
# you may not use this file except in compliance with the License.
11+
# You may obtain a copy of the License at
12+
#
13+
# http://www.apache.org/licenses/LICENSE-2.0
14+
#
15+
# Unless required by applicable law or agreed to in writing, software
16+
# distributed under the License is distributed on an "AS IS" BASIS,
17+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
18+
# See the License for the specific language governing permissions and
19+
# limitations under the License.
20+
21+
resource_name 'chef_automate'
22+
default_action :create
23+
24+
property :name, String, name_property: true
25+
property :channel, Symbol, default: :stable
26+
property :version, [String, Symbol], default: :latest
27+
property :config, String, required: true
28+
property :accept_license, [TrueClass, FalseClass], default: false
29+
property :enterprise, [String, Array], default: 'chef'
30+
property :license, String
31+
property :chef_user, String, default: 'workflow'
32+
property :chef_user_pem, String, required: true
33+
property :validation_pem, String, required: true
34+
property :builder_pem, String, required: true
35+
property :platform, String
36+
property :platform_version, String
37+
38+
load_current_value do
39+
# node.run_state['chef-users'] ||= Mixlib::ShellOut.new('chef-server-ctl user-list').run_command.stdout
40+
# node.run_state['chef-orgs'] ||= Mixlib::ShellOut.new('chef-server-ctl org-list').run_command.stdout
41+
# current_value_does_not_exist! unless node.run_state['chef-orgs'].index(/^#{org}$/)
42+
end
43+
44+
action :create do
45+
# https://github.com/chef/delivery/issues/469
46+
new_resource.config << "\ndelivery['chef_server_proxy'] = false" unless new_resource.config.include?('chef_server_proxy')
47+
48+
# Always make sure user and key provided to resource is at the bottom of the config, overriding duplicates.
49+
new_resource.config << "\ndelivery['chef_username'] = '#{new_resource.chef_user}'"
50+
new_resource.config << "\ndelivery['chef_private_key'] = '/etc/delivery/#{new_resource.chef_user}.pem'"
51+
52+
# Hardcode v1 runner search to automate-build-node
53+
new_resource.config << "\ndelivery['default_search'] = 'tags:delivery-build-node'"
54+
55+
chef_ingredient 'automate' do
56+
action :upgrade
57+
channel new_resource.channel
58+
version new_resource.version
59+
config new_resource.config
60+
accept_license new_resource.accept_license
61+
platform new_resource.platform if new_resource.platform
62+
platform_version new_resource.platform_version if new_resource.platform_version
63+
end
64+
65+
directory '/etc/delivery'
66+
directory '/etc/chef'
67+
68+
directory '/var/opt/delivery/license/' do
69+
recursive true
70+
end
71+
72+
{
73+
'/var/opt/delivery/license/delivery.license' => new_resource.license,
74+
"/etc/delivery/#{new_resource.chef_user}.pem" => new_resource.chef_user_pem,
75+
'/etc/chef/validation.pem' => new_resource.validation_pem,
76+
'/etc/delivery/builder_key' => new_resource.builder_pem,
77+
}.each do |file, src|
78+
chef_file file do
79+
source src
80+
user 'root'
81+
group 'root'
82+
mode '0600'
83+
end
84+
end
85+
86+
file '/etc/delivery/builder_key.pub' do
87+
content lazy { "ssh-rsa #{[OpenSSL::PKey::RSA.new(::File.read('/etc/delivery/builder_key')).to_blob].pack('m0')}" }
88+
user 'root'
89+
group 'root'
90+
mode '0644'
91+
end
92+
93+
directory '/var/opt/delivery/nginx/etc/addon.d/' do
94+
recursive true
95+
end
96+
97+
file '/var/opt/delivery/nginx/etc/addon.d/99-installer_internal.conf' do
98+
content <<-EOF
99+
location /installer {
100+
alias /opt/delivery/embedded/service/omnibus-ctl/installer;
101+
}
102+
EOF
103+
end
104+
105+
ingredient_config 'automate' do
106+
notifies :reconfigure, 'chef_ingredient[automate]', :immediately
107+
end
108+
109+
if new_resource.enterprise.is_a?(String)
110+
new_resource.enterprise = [new_resource.enterprise]
111+
new_resource.enterprise.each do |ent|
112+
execute "create enterprise #{ent}" do
113+
command "delivery-ctl create-enterprise #{ent} --ssh-pub-key-file=/etc/delivery/builder_key.pub > /etc/delivery/#{ent}.creds"
114+
not_if "delivery-ctl list-enterprises --ssh-pub-key-file=/etc/delivery/builder_key.pub | grep -w #{ent}"
115+
only_if 'delivery-ctl status'
116+
end
117+
end
118+
end
119+
end

resources/backend.rb

+93
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
#
2+
# Author:: Nathan Cerny <[email protected]>
3+
#
4+
# Cookbook Name:: chef_stack
5+
# Resource:: backend
6+
#
7+
# Copyright 2017 Chef Software Inc
8+
#
9+
# Licensed under the Apache License, Version 2.0 (the "License");
10+
# you may not use this file except in compliance with the License.
11+
# You may obtain a copy of the License at
12+
#
13+
# http://www.apache.org/licenses/LICENSE-2.0
14+
#
15+
# Unless required by applicable law or agreed to in writing, software
16+
# distributed under the License is distributed on an "AS IS" BASIS,
17+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
18+
# See the License for the specific language governing permissions and
19+
# limitations under the License.
20+
21+
resource_name 'chef_backend'
22+
default_action :create
23+
24+
property :name, String, name_property: true
25+
property :channel, Symbol, default: :stable
26+
property :version, [String, Symbol], default: :latest
27+
property :config, String, default: ''
28+
property :accept_license, [TrueClass, FalseClass], default: false
29+
property :peers, [String, Array], required: true
30+
property :publish_address, String, default: node['ipaddress']
31+
property :chef_backend_secrets, String, default: ''
32+
property :platform, String
33+
property :platform_version, String
34+
35+
alias :bootstrap_node :peers
36+
37+
load_current_value do
38+
# node.run_state['chef-users'] ||= Mixlib::ShellOut.new('chef-server-ctl user-list').run_command.stdout
39+
# current_value_does_not_exist! unless node.run_state['chef-users'].index(/^#{username}$/)
40+
end
41+
42+
action :create do
43+
raise 'Must accept the Chef License agreement before continuing.' unless new_resource.accept_license
44+
45+
new_resource.config = ensurekv(new_resource.config, publish_address: new_resource.publish_address)
46+
chef_ingredient 'chef-backend' do
47+
action :upgrade
48+
channel new_resource.channel
49+
version new_resource.version
50+
config new_resource.config # TODO: Figure out why this isn't working in chef-ingredient
51+
accept_license new_resource.accept_license
52+
platform new_resource.platform if new_resource.platform
53+
platform_version new_resource.platform_version if new_resource.platform_version
54+
end
55+
56+
file '/etc/chef-backend/chef-backend.rb' do
57+
content new_resource.config
58+
end
59+
60+
chef_file '/etc/chef-backend/chef-backend-secrets.json' do
61+
source new_resource.chef_backend_secrets
62+
user 'root'
63+
group 'root'
64+
mode '0600'
65+
not_if { new_resource.chef_backend_secrets.empty? }
66+
end
67+
68+
http_retry_count = Chef::Config['http_retry_count']
69+
Chef::Config['http_retry_count'] = 0
70+
existing_peer = false
71+
peers = (new_resource.peers.is_a?(Array) ? new_resource.peers : [new_resource.peers])
72+
73+
peers.each do |peer|
74+
begin
75+
Chef::HTTP.new("http://#{peer}:2379").get('/version')
76+
existing_peer = peer
77+
break
78+
rescue
79+
next
80+
end
81+
end
82+
Chef::Config['http_retry_count'] = http_retry_count
83+
84+
execute 'chef-backend-ctl create-cluster --accept-license --yes' do
85+
not_if 'chef-backend-ctl cluster-status &> /dev/null'
86+
not_if { existing_peer }
87+
end
88+
89+
execute "chef-backend-ctl join-cluster #{existing_peer} --accept-license --yes" do
90+
not_if 'chef-backend-ctl cluster-status &> /dev/null'
91+
only_if { existing_peer }
92+
end
93+
end

0 commit comments

Comments
 (0)