How to install Cumin
Jump to navigation
Jump to search
Cumin is open Source program written in Python that allows you to execute multiple commands on multiple hosts in parallel. For more information on Cumin please see the references section.
Prerequisites
For this tutorial, you need:
- pupetmaster node
- puppetdb node
- cumin node
- some nodes for testing
Environment
Nodes names
All nodes in this tutorial are running Debian Stretch with Puppet 6
- Puppetmaster1002 : 10.192.32.4 Puppet master node
- Puppetdb1002 : IP address 10.192.32.10 PuppetDb node
- cumin1001 : IP address 10.192.48.13 Running cumin
- lab1001: Test node
Installation
One of the required package for Cumin 3.0.2 is python3-clustershell. Debian Stretch doesn't have that package so you will need to download it manually on your puppetmaster and the cumin class will install it. ( See cumin class below)
ppaul@cumin1001:~$ sudo apt-get install python3-clustershell Reading package lists... Done Building dependency tree Reading state information... Done E: Unable to locate package python3-clustershell
You can also install Debian Buster if you want on the cumin node since Debian Buster has the package.
- Test from a node running Debian Buster
ppaul@lab1002:~$ sudo apt-get install python3-clustershell Reading package lists... Done Building dependency tree Reading state information... Done Suggested packages: vim-addon-manager The following NEW packages will be installed: python3-clustershell 0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded. Need to get 126 kB of archives. After this operation, 625 kB of additional disk space will be used. Get:1 http://apt1001.dfw.labnet/debian buster/main amd64 python3-clustershell all 1.8.1-1 [126 kB] Fetched 126 kB in 1s (249 kB/s) Selecting previously unselected package python3-clustershell. (Reading database ... 57033 files and directories currently installed.) Preparing to unpack .../python3-clustershell_1.8.1-1_all.deb ... Unpacking python3-clustershell (1.8.1-1) ... Setting up python3-clustershell (1.8.1-1) ...
cumin Class
On your puppetmaster create the cumin class
cd /etc/puppetlabs/code/environments/production/modules sudo mkdir cumin cd cumin sudo mkdir manifests sudo mkdir templates cd manifests
Create the master.pp file, copy and paste the contain below
class profile::cumin::master (
$puppetdb_host = hiera('puppetdb_host'),
) {
$cumin_log_path = '/var/log/cumin'
require_package([
'python3-dnspython',
'python3-colorama',
'python3-pyparsing',
'python3-tqdm',
'python3-requests',
'python3-yaml',
])
file { '/srv/cumin_3.0.2-1_amd64.deb':
ensure => present,
owner => 'root',
group => 'root',
source => 'puppet:///modules/cumin/cumin_3.0.2-1_amd64.deb',
}
file { '/srv/python3-clustershell_1.8.1-2_all.deb':
ensure => present,
owner => 'root',
group => 'root',
source => 'puppet:///modules/cumin/python3-clustershell_1.8.1-2_all.deb',
}
package { 'python3-clusterhsell':
ensure => installed,
provider => dpkg,
source => '/srv/python3-clustershell_1.8.1-2_all.deb',
require => File['/srv/python3-clustershell_1.8.1-2_all.deb'],
}
package { 'cumin':
ensure => installed,
provider => dpkg,
source => '/srv/cumin_3.0.2-1_amd64.deb',
require => File['/srv/cumin_3.0.2-1_amd64.deb'],
}
file { $cumin_log_path:
ensure => directory,
owner => 'root',
group => 'root',
mode => '0750',
}
file { '/etc/cumin':
ensure => directory,
owner => 'root',
group => 'root',
mode => '0755',
}
file { '/etc/cumin/config.yaml':
ensure => present,
owner => 'root',
group => 'root',
mode => '0640',
content => template('cumin/config.yaml.erb'),
require => File['/etc/cumin'],
}
file { '/etc/cumin/config-installer.yaml':
ensure => present,
owner => 'root',
group => 'root',
mode => '0640',
content => template('cumin/config-installer.yaml.erb'),
require => File['/etc/cumin'],
}
file { '/etc/cumin/aliases.yaml':
ensure => present,
owner => 'root',
group => 'root',
mode => '0644',
content => template('cumin/aliases.yaml.erb'),
require => File['/etc/cumin'],
}
if os_version('debian == stretch') {
$python_version = '3.5'
} else {
$python_version = '3.6'
}
}
File needed
Download cumin_3.0.2-1_amd64.deb and python3-clustershell_1.8.1-2_all.deb and save those files in /etc/puppetlabs/code/environments/production/modules/cumin/files
http://ftp.us.debian.org/debian/pool/main/c/clustershell/python3-clustershell_1.8.1-2_all.deb https://github.com/wikimedia/cumin/releases/download/v3.0.2/cumin_3.0.2-1_amd64.deb
Create aliases.yaml.erb, config-installer.yaml.erb and config.yaml.erb in /etc/puppetlabs/code/environments/production/modules/cumin/templates For now we are going just to focus on the config.yaml.erb file. Open the config.yaml.erb and copy and paste the contain below. Leave the other files empty for now.
transport: clustershell
log_file: <%= @cumin_log_path %>/cumin.log
default_backend: puppetdb
environment:
SSH_AUTH_SOCK: /run/keyholder/proxy.sock
puppetdb:
host: <%= @puppetdb_host %>
port: 443
api_version: 4
<% if scope.function_os_version(['debian >= stretch']) -%>
urllib3_disable_warnings:
- SubjectAltNameWarning
<% end -%>
knownhosts:
files:
- /etc/ssh/ssh_known_hosts
clustershell:
ssh_options:
- '-o BatchMode=yes'
- '-o ConnectTimeout=2'
- '-o StrictHostKeyChecking=yes'
Configure hiera
-Add puppetdb_host to cumin1001.yaml in hiera
puppetdb_host: puppetdb1002.dfw.labnet
-Add cumin IP address to common.yaml in hiera
cumin_masters: - 10.192.48.13
Apply the cumin class
Open your site.pp file and apply the cumin class to your cumin node
node 'cumin1001.dfw.labnet' {
include cumin::master
}
Install Cumin
On the cumin node run
sudo puppet agent -t
verification
navigate to /usr/lib/python3/dist-packages/ and make sure that the cumin folder is there
ppaul@cumin1001:/usr/lib/python3/dist-packages/cumin$ ls -l total 60 drwxr-xr-x 3 root root 4096 Oct 5 12:56 backends -rw-r--r-- 1 root root 19656 Jul 30 2018 cli.py -rw-r--r-- 1 root root 6051 Jul 30 2018 grammar.py -rw-r--r-- 1 root root 4852 Jul 30 2018 __init__.py drwxr-xr-x 2 root root 4096 Oct 5 12:56 __pycache__ -rw-r--r-- 1 root root 6877 Jul 30 2018 query.py -rw-r--r-- 1 root root 2068 Jul 30 2018 transport.py drwxr-xr-x 3 root root 4096 Oct 5 12:56 transports
We have now Cumin installed on our node. The next setup is to configure our cumin node to talk to the puppetdb node
Configuration
Firewall
On the puppetdb node puppetdb1002 only the puppetmaster node and the icinga node are allowed to talk to the puppetdb node.
8081/tcp ALLOW 10.192.16.12 #allow icinga1002.dfw.labnet 8081/tcp ALLOW 10.192.32.4 #allow puppetmaster1002.dfw.labnet
We are going to allow cumin1001.dfw.labnet to talk to the puppetdb node
8081/tcp ALLOW 10.192.48.13
-Testing
ppaul@cumin1001:/srv$ telnet puppetdb1002.dfw.labnet 8081 Trying 10.192.32.10... Connected to puppetdb1002.dfw.labnet. Escape character is '^]'.
We see that our cumin is allow to connect on port 8081 on the puppetdb node
Puppetdb 6 and Cumin ssl issue
Right now we have cumin node is allow to connect to the puppetdb node. Let try to see if we can run a cumin command
sudo cumin -b 1 -s 5 'lab1001.dfw.labnet' 'run-puppet-agent'
Result
Caught SSLError exception: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:720)
Let us try to curl from our cumin node to the puppetdb node
ppaul@cumin1001:~$ sudo curl -vG 'https://puppetdb1002.dfw.labnet:8081/pdb/query/v4/facts' * Trying 10.192.32.10... * TCP_NODELAY set * Connected to puppetdb1002.dfw.labnet (10.192.32.10) port 8081 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH * successfully set certificate verify locations: * CAfile: /etc/ssl/certs/ca-certificates.crt CApath: /etc/ssl/certs * TLSv1.2 (OUT), TLS header, Certificate Status (22): * TLSv1.2 (OUT), TLS handshake, Client hello (1): * TLSv1.2 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (OUT), TLS alert, Server hello (2): * SSL certificate problem: unable to get local issuer certificate * Curl_http_done: called premature == 1 * stopped the pause stream! * Closing connection 0 curl: (60) SSL certificate problem: unable to get local issuer certificate More details here: https://curl.haxx.se/docs/sslcerts.html
We can see that the puppetdb node is not allowing any connection without a certificate.
Let us try to run the same command now by bypassing the certificate. just add -k to the previous command
ppaul@cumin1001:~$ sudo curl -vkG 'https://puppetdb1002.dfw.labnet:8081/pdb/query/v4/facts' * Trying 10.192.32.10... * TCP_NODELAY set * Connected to puppetdb1002.dfw.labnet (10.192.32.10) port 8081 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH * successfully set certificate verify locations: * CAfile: /etc/ssl/certs/ca-certificates.crt CApath: /etc/ssl/certs * TLSv1.2 (OUT), TLS header, Certificate Status (22): * TLSv1.2 (OUT), TLS handshake, Client hello (1): * TLSv1.2 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Request CERT (13): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Certificate (11): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS change cipher, Client hello (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS alert, Server hello (2): * error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate * Curl_http_done: called premature == 1 * stopped the pause stream! * Closing connection 0 curl: (35) error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate
Same problem we can not bypass the certificate verification
According to the PuppetDB documentation in the link below, to make secured requests from other hosts, you will need to supply
--cacert --cert --key
https://puppet.com/docs/puppetdb/5.2/api/query/curl.html
Let us try to curl again but this time we we follow the documentation. create the bash script below
#!/bin/bash curl -vG 'https://puppetdb1002.dfw.labnet:8081/pdb/query/v4/facts' \ --tlsv1 \ --cacert /etc/puppetlabs/puppet/ssl/certs/ca.pem \ --cert /etc/puppetlabs/puppet/ssl/certs/cumin1001.dfw.labnet.pem \ --key /etc/puppetlabs/puppet/ssl/private_keys/cumin1001.dfw.labnet.pem
make it excutable and run it
ppaul@cumin1001:~$ sudo ./puppetdbquery * Trying 10.192.32.10... * TCP_NODELAY set * Connected to puppetdb1002.dfw.labnet (10.192.32.10) port 8081 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH * successfully set certificate verify locations: * CAfile: /etc/puppetlabs/puppet/ssl/certs/ca.pem CApath: /etc/ssl/certs * TLSv1.2 (OUT), TLS header, Certificate Status (22): * TLSv1.2 (OUT), TLS handshake, Client hello (1): * TLSv1.2 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Request CERT (13): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Certificate (11): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS handshake, CERT verify (15): * TLSv1.2 (OUT), TLS change cipher, Client hello (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS change cipher, Client hello (1): * TLSv1.2 (IN), TLS handshake, Finished (20): * SSL connection using TLSv1.2 / DHE-RSA-AES128-GCM-SHA256 * ALPN, server did not agree to a protocol * Server certificate: * subject: CN=puppetdb1002.dfw.labnet * start date: Sep 30 00:52:53 2019 GMT * expire date: Sep 29 00:52:53 2024 GMT * common name: puppetdb1002.dfw.labnet (matched) * issuer: CN=Puppet CA: puppetmaster1002.dfw.labnet * SSL certificate verify ok. > GET /pdb/query/v4/facts HTTP/1.1 > Host: puppetdb1002.dfw.labnet:8081 > User-Agent: curl/7.52.1 > Accept: */* > < HTTP/1.1 200 OK < Date: Fri, 04 Oct 2019 02:33:28 GMT < Content-Type: application/json;charset=utf-8 < Vary: Accept-Encoding, User-Agent < Transfer-Encoding: chunked
Now we are able to connect to our puppetdb node with out a problem
What is the fix for now?
The fix for now is to install a SSL proxy using Apache2 or Nginx on the puppetdb node. I decided to use Nginx in my case.
I am not going to discuss about how to install and configure Nginx here but only provide the Nginx config file if that will help
- Nginx config file
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name <%= @server_name %>;
root /usr/share/nginx/www;
ssl_certificate /etc/ssl/certs/wildcard.tx.labnet.crt;
ssl_certificate_key /etc/ssl/private/wildcard.tx.labnet.key;
add_header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload";
location / {
proxy_pass http://localhost:8080/;
}
}
Replace <%= @server_name %>; with your puppetdb node name Note: on the puppetdb node allow the cumin node to connect to port 443
443/tcp ALLOW 10.192.48.13
We are going to run again the same curl command from the cumin node by bypassing the certificate verification
ppaul@cumin1001:~$ curl -vkG 'https://puppetdb1002.dfw.labnet:443/pdb/query/v4/facts' * Trying 10.192.32.10... * TCP_NODELAY set * Connected to puppetdb1002.dfw.labnet (10.192.32.10) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH * successfully set certificate verify locations: * CAfile: /etc/ssl/certs/ca-certificates.crt CApath: /etc/ssl/certs * TLSv1.2 (OUT), TLS header, Certificate Status (22): * TLSv1.2 (OUT), TLS handshake, Client hello (1): * TLSv1.2 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS change cipher, Client hello (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS change cipher, Client hello (1): * TLSv1.2 (IN), TLS handshake, Finished (20): * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 * ALPN, server accepted to use h2 * Server certificate: * subject: C=US; ST=TX; L=DAllas; O=IT * start date: Feb 6 03:18:24 2019 GMT * expire date: Feb 6 03:18:24 2020 GMT * issuer: C=US; ST=TX; L=DAllas; O=IT * SSL certificate verify result: self signed certificate (18), continuing anyway. * Using HTTP2, server supports multi-use * Connection state changed (HTTP/2 confirmed) * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0 * Using Stream ID: 1 (easy handle 0x5636b9310d00) > GET /pdb/query/v4/facts HTTP/1.1 > Host: puppetdb1002.dfw.labnet > User-Agent: curl/7.52.1 > Accept: */* > * Connection state changed (MAX_CONCURRENT_STREAMS updated)! < HTTP/2 200 < server: nginx/1.17.4 < date: Thu, 10 Oct 2019 02:09:41 GMT < content-type: application/json;charset=utf-8 < vary: Accept-Encoding, User-Agent < strict-transport-security: max-age=15768000; includeSubDomains; preload
Testing
Run the same cumin command
sudo cumin -b 1 -s 5 'lab1001.dfw.labnet' 'puppet agent -t'
Result
Caught SSLError exception: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:720)
Disable certificate verification in cumin
open
/usr/lib/python3/dist-packages/cumin/backends/pupeptdb.py$
change line 503 from
503 resources = requests.get(self.url + self.endpoint, params={'q uery': query}, verify=True)
to
503 resources = requests.get(self.url + self.endpoint, params={'q uery': query}, verify=False)
change line 505 from
505 resources = requests.post(self.url + self.endpoint, json={'qu ery': query}, verify=True)
to
505 resources = requests.post(self.url + self.endpoint, json={'qu ery': query}, verify=False)
Run the command again
sudo cumin -b 1 -s 5 'lab1001.dfw.labnet' 'puppet agent -t'
Result
ppaul@cumin1001:~$ sudo cumin -b 1 -s 5 'lab1001.dfw.labnet' 'puppet agent -t' /usr/lib/python3/dist-packages/urllib3/connectionpool.py:845: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 1 hosts will be targeted: lab1001.dfw.labnet Confirm to continue [y/n]? y ----- OUTPUT of 'run-puppet-agent -q' ----- No ECDSA host key is known for lab1001.dfw.labnet and you have requested strict checking. Host key verification failed. ================ PASS: | | 0% (0/1) [00:05<?, ?hosts/s] FAIL: |████████████████████████| 100% (1/1) [00:05<00:00, 7.74hosts/s] 100.0% (1/1) of nodes failed to execute command 'run-puppet-agent -q': lab1001.dfw.labnet 0.0% (0/1) success ratio (< 100.0% threshold) for command: 'run-puppet-agent -q'. Aborting. 0.0% (0/1) success ratio (< 100.0% threshold) of nodes successfully executed all commands. Aborting.
How to fix 1?
The ECDSA error is very easy and can be fix quickly by just running
sudo su ssh lab1001.dfw.labnet
from the cumin node and answering yes to add the ECDSA key fingerprint
Run the command again
ppaul@cumin1001:~$ sudo cumin -b 1 -s 5 'lab1001.dfw.labnet' 'puppet agent -t' /usr/lib/python3/dist-packages/urllib3/connectionpool.py:845: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 1 hosts will be targeted: lab1001.dfw.labnet Confirm to continue [y/n]? y ----- OUTPUT of 'run-puppet-agent -q' ----- Permission denied (publickey). ================ PASS: | | 0% (0/1) [00:05<?, ?hosts/s] FAIL: |████████████████████████| 100% (1/1) [00:05<00:00, 9.26hosts/s] 100.0% (1/1) of nodes failed to execute command 'run-puppet-agent -q': lab1001.dfw.labnet 0.0% (0/1) success ratio (< 100.0% threshold) for command: 'run-puppet-agent -q'. Aborting. 0.0% (0/1) success ratio (< 100.0% threshold) of nodes successfully executed all commands. Aborting.
How to fix 2?
-Fix 1: The faster way to fix this is to comment or remove line 25 in /etc/cumin/config.yaml
21 clustershell:
22 ssh_options:
23 - '-o BatchMode=yes'
24 - '-o ConnectTimeout=2'
25 # - '-o StrictHostKeyChecking=yes'
- Fix 2: Create a ssk-key for the user root and copy the public key to the remote host (lab1001.dfw.labent) This method really will not work well if you have a lot of nodes.
- Fix 3: The way I fixed this is to use the class target.pp from https://github.com/wikimedia/puppet/blob/production/modules/profile/manifests/cumin/target.pp
As you can see in the target.pp class on line 32 you have
ssh::userkey
so you will need also the ssh userkey.pp class from https://github.com/wikimedia/puppet/blob/production/modules/ssh/manifests/userkey.pp
Run the command again
ppaul@cumin1001:~$ sudo cumin -b 1 -s 5 'lab1001.dfw.labnet' 'puppet agent -t' /usr/lib/python3/dist-packages/urllib3/connectionpool.py:845: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 1 hosts will be targeted: lab1001.dfw.labnet Confirm to continue [y/n]? y ----- OUTPUT of 'puppet agent -t' ----- Info: Using configured environment 'production' Info: Retrieving pluginfacts Info: Retrieving plugin Info: Retrieving locales Info: Loading facts Info: Caching catalog for lab1001.dfw.labnet Info: Applying configuration version '1570416225' Notice: Applied catalog in 4.76 seconds ================ PASS: |████████████████████████| 100% (1/1) [00:19<00:00, 14.63s/hosts] FAIL: | | 0% (0/1) [00:19<?, ?hosts/s] 100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'puppet agent -t'. 100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
Running commands
- Run apt-get update on node lab1001.dfw.labnet
ppaul@cumin1001:~$ sudo cumin -b 1 -s 5 'lab1001.dfw.labnet' 'apt-get update' /usr/lib/python3/dist-packages/urllib3/connectionpool.py:845: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 1 hosts will be targeted: lab1001.dfw.labnet Confirm to continue [y/n]? y ----- OUTPUT of 'apt-get update' ----- Ign:1 http://apt1001.dfw.labnet/debian stretch InRelease Hit:2 http://apt1001.dfw.labnet/debian stretch Release Hit:3 http://deb.debian.org/debian stretch-backports InRelease Hit:4 http://security.debian.org/debian-security stretch/updates InRelease Hit:5 http://security.debian.org stretch/updates InRelease Hit:7 http://ams2.mirrors.digitalocean.com/mariadb/repo/10.2/debian stretch InRelease Hit:8 http://apt.puppetlabs.com stretch InRelease Hit:9 https://packages.sury.org/php stretch InRelease Reading package lists... ================ PASS: |████████████████████████| 100% (1/1) [00:07<00:00, 2.93s/hosts] FAIL: | | 0% (0/1) [00:07<?, ?hosts/s] 100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'apt-get update'. 100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
- Run apt-get update on all node with host name starting with db1
ppaul@cumin1001:~$ sudo cumin -b 1 -s 5 'db1*.dfw.labnet' 'apt-get update' /usr/lib/python3/dist-packages/urllib3/connectionpool.py:845: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 7 hosts will be targeted: db[1001-1007].dfw.labnet Confirm to continue [y/n]? y ===== NODE GROUP ===== (1) db1007.dfw.labnet ----- OUTPUT of 'apt-get update' ----- Hit:1 http://apt1001.dfw.labnet/debian buster InRelease Hit:2 http://security.debian.org/debian-security buster/updates InRelease Hit:3 http://apt.puppetlabs.com stretch InRelease Hit:4 http://security.debian.org buster/updates InRelease Hit:5 http://apt.puppetlabs.com buster InRelease Reading package lists... ===== NODE GROUP ===== (1) db1004.dfw.labnet ----- OUTPUT of 'apt-get update' ----- Ign:1 http://apt1001.dfw.labnet/debian stretch InRelease Hit:2 http://apt1001.dfw.labnet/debian stretch Release Hit:3 http://security.debian.org/debian-security stretch/updates InRelease Hit:4 http://deb.debian.org/debian stretch-backports InRelease Hit:5 http://security.debian.org stretch/updates InRelease Hit:6 http://repo.saltstack.com/py3/debian/9/amd64/2018.3 stretch InRelease Hit:7 http://apt.puppetlabs.com stretch InRelease Reading package lists... ===== NODE GROUP ===== (1) db1005.dfw.labnet ----- OUTPUT of 'apt-get update' ----- Ign:1 http://apt1001.dfw.labnet/debian stretch InRelease Hit:2 http://apt1001.dfw.labnet/debian stretch Release Hit:3 http://security.debian.org/debian-security stretch/updates InRelease Hit:4 http://deb.debian.org/debian stretch-backports InRelease Hit:5 http://security.debian.org stretch/updates InRelease Hit:6 http://apt.puppetlabs.com stretch InRelease Reading package lists... ===== NODE GROUP ===== (1) db1001.dfw.labnet ----- OUTPUT of 'apt-get update' ----- Ign:1 http://apt1001.dfw.labnet/debian stretch InRelease Hit:2 http://apt1001.dfw.labnet/debian stretch Release Hit:3 http://repo.saltstack.com/py3/debian/9/amd64/2018.3 stretch InRelease Hit:4 http://apt.puppetlabs.com stretch InRelease Hit:6 http://security.debian.org/debian-security stretch/updates InRelease Hit:7 http://security.debian.org stretch/updates InRelease Hit:8 http://deb.debian.org/debian stretch-backports InRelease Reading package lists... ===== NODE GROUP ===== (1) db1002.dfw.labnet ----- OUTPUT of 'apt-get update' ----- Ign:1 http://apt1001.dfw.labnet/debian stretch InRelease Hit:2 http://apt1001.dfw.labnet/debian stretch Release Hit:3 http://deb.debian.org/debian stretch-backports InRelease Hit:4 http://security.debian.org/debian-security stretch/updates InRelease Hit:5 http://security.debian.org stretch/updates InRelease Hit:7 http://apt.puppetlabs.com stretch InRelease Reading package lists... ===== NODE GROUP ===== (2) db[1003,1006].dfw.labnet ----- OUTPUT of 'apt-get update' ----- Ign:1 http://apt1001.dfw.labnet/debian stretch InRelease Hit:2 http://apt1001.dfw.labnet/debian stretch Release Hit:3 http://deb.debian.org/debian stretch-backports InRelease Hit:4 http://security.debian.org/debian-security stretch/updates InRelease Hit:5 http://security.debian.org stretch/updates InRelease Hit:6 http://repo.saltstack.com/py3/debian/9/amd64/2018.3 stretch InRelease Hit:7 http://apt.puppetlabs.com stretch InRelease Reading package lists... ================ PASS: |████████████████████████| 100% (7/7) [00:53<00:00, 7.06s/hosts] FAIL: | | 0% (0/7) [00:53<?, ?hosts/s] 100.0% (7/7) success ratio (>= 100.0% threshold) for command: 'apt-get update'. 100.0% (7/7) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.
References
https://doc.wikimedia.org/cumin/master/installation.html
https://puppet.com/docs/puppetdb/5.2/api/query/curl.html
https://github.com/wikimedia/puppet/blob/production/modules/ssh/manifests/userkey.pp