<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en-US">
 <title type="text">Stefan Schlesinger</title>
 <link rel="self" type="application/atom+xml" href="http://sts.ono.at/atom.xml" />
 <link rel="alternate" type="text/html" href="http://sts.ono.at/" />
 <id>http://sts.ono.at/</id>
 <author>
     <name>Stefan Schlesinger</name>
     <email>sts@ono.at</email>
     <uri>http://sts.ono.at</uri>
 </author>
 <updated>2012-02-02T13:15:15+01:00</updated>

 
 <entry xmlns="http://www.w3.org/2005/Atom">
   <title>A Systems Policy</title>
   
     <category term="Infrastructure" label="Infrastructure" />
   
     <category term="Operations" label="Operations" />
   
     <category term="Policy" label="Policy" />
   
   <link href="http://sts.ono.at/blog/2012/02/01/a-systems-policy"/>
   <updated>2012-02-01T09:12:00+01:00</updated>
   <id>http://sts.ono.at/blog/2012/02/01/a-systems-policy</id>
   <content type="html">&lt;p&gt;Recently I talked to a couple of friends, which all wailed quite a bit about their operations or internal IT departments.&lt;/p&gt;

&lt;p&gt;Most of these teams had to fight with some very basic things. They lacked a decent monitoring system or monitoring at all. They didn&amp;#8217;t deploy systems, they installed it by hand. Systems where not documented etc.&lt;/p&gt;

&lt;p&gt;So here are some guidelines, I try to aspire with my team. This is by far not a complete list of things you need to run successful operations but it should give you a fair hint about what it takes.&lt;/p&gt;

&lt;p&gt;Also please note that you might want to adapt your own policy a bit to fit your needs. I&amp;#8217;m coming from the web industry, but we still run our own hardware, so this might especially not fit a typical cloud based infrastructure.&lt;/p&gt;

&lt;h2 id='systems'&gt;Systems&lt;/h2&gt;

&lt;blockquote&gt;
&lt;p&gt;A System is considered the lowest part of our infrastructure and services. All rules defined here, should be considered in all other policies.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;A system&amp;#8230;.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;is documented at a central location.&lt;/li&gt;

&lt;li&gt;is monitored and being graphed.&lt;/li&gt;

&lt;li&gt;is being backuped.&lt;/li&gt;

&lt;li&gt;is updated regularly.&lt;/li&gt;

&lt;li&gt;has a defined production level. (spare, pre-production, production)&lt;/li&gt;

&lt;li&gt;has a defined owner and maintainer.&lt;/li&gt;

&lt;li&gt;has a predefined maintenance level.&lt;/li&gt;

&lt;li&gt;has a predefined availability.&lt;/li&gt;

&lt;li&gt;has a physical location.&lt;/li&gt;

&lt;li&gt;has a unique name, which is resolvable by DNS.&lt;/li&gt;

&lt;li&gt;has only required software installed.&lt;/li&gt;

&lt;li&gt;was installed with all currently available updates.&lt;/li&gt;

&lt;li&gt;was inspected and approved by a second man before being released to production.&lt;/li&gt;

&lt;li&gt;All parts are functional at any time. All Faults get documented RFN and repaired as soon as possible.&lt;/li&gt;

&lt;li&gt;There are always 2+ people informed about it.&lt;/li&gt;

&lt;li&gt;Network access vectors are defined.&lt;/li&gt;

&lt;li&gt;Configurations are not only available locally (including scripts).&lt;/li&gt;

&lt;li&gt;Sensible data gets protected.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id='hardware'&gt;Hardware&lt;/h2&gt;

&lt;blockquote&gt;
&lt;p&gt;A piece of hardware can be anything from a big server to a small temperature sensor in your server room.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;A piece of hardware&amp;#8230;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;has a maintenance contract or spare hardware available.&lt;/li&gt;

&lt;li&gt;has got an inventory number.&lt;/li&gt;

&lt;li&gt;is labeled (hostname + inventory).&lt;/li&gt;

&lt;li&gt;is physically secure (environmental! and mechanical access control).&lt;/li&gt;

&lt;li&gt;has got a bill, which is documented at a central location.&lt;/li&gt;

&lt;li&gt;should have redundant power supplies.&lt;/li&gt;

&lt;li&gt;should have some kind of out of band management solution (OOB).&lt;/li&gt;

&lt;li&gt;has at least one power circuit connected to an electronic circuit protected by an uninterruptible power supply (USV).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;All tools needed to open and repair any part of the system are available.&lt;/p&gt;

&lt;h2 id='servers'&gt;Servers&lt;/h2&gt;

&lt;p&gt;A server&amp;#8230;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;has at least two disks configured with RAID &amp;gt;= 1.&lt;/li&gt;

&lt;li&gt;has at least two separate network interface cards (NICs).&lt;/li&gt;

&lt;li&gt;has all RAID controllers backed with battery backed write caches (BBWC).&lt;/li&gt;

&lt;li&gt;was dimensioned with adequate future-proof hardware.&lt;/li&gt;

&lt;li&gt;has a lifetime of 2+ years.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id='switches'&gt;Switches&lt;/h2&gt;

&lt;p&gt;A switch&amp;#8230;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;is manage- or configurable.&lt;/li&gt;

&lt;li&gt;is supported by the configuration backup software in use (e.g. RANCID)&lt;/li&gt;

&lt;li&gt;provides the following protocols: STP, SNMP, IPv6 support (mgmt+multicast), RADIUS for AAA&lt;/li&gt;

&lt;li&gt;does not forward the default VLAN (1) on it&amp;#8217;s uplink/trunk ports.&lt;/li&gt;

&lt;li&gt;does have a description for every port in use (including hostname and interface, e.g.: server01#eth0, server01#oob, switch03#24)&lt;/li&gt;

&lt;li&gt;does not have any enabled, unused ports: set them to disabled and remove any other configuration for this port.&lt;/li&gt;

&lt;li&gt;blocks or does not forward any discovery protocols on it&amp;#8217;s user ports.&lt;/li&gt;

&lt;li&gt;is using AAA for authenticating users.&lt;/li&gt;

&lt;li&gt;logs to a central syslog server.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id='operating_systems'&gt;Operating Systems&lt;/h2&gt;

&lt;blockquote&gt;
&lt;p&gt;An operating system (OS) is considered as everything running on a server or instance, to support a service or an application.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;An Operating System&amp;#8230;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;uses &lt;strong&gt;OS-CHOICE-HERE/stable&lt;/strong&gt; as default distribution on servers.&lt;/li&gt;

&lt;li&gt;uses &lt;strong&gt;OS-CHOICE-HERE&lt;/strong&gt; as default on clients.&lt;/li&gt;

&lt;li&gt;is rebooting without any manual interventions.&lt;/li&gt;

&lt;li&gt;provides access by SSH.&lt;/li&gt;

&lt;li&gt;does not permit root login via SSH.&lt;/li&gt;

&lt;li&gt;has a root password set.&lt;/li&gt;

&lt;li&gt;has the current time, synchronized with a time server and uses &lt;strong&gt;TIMEZONE-CHOICE-HERE&lt;/strong&gt; as time zone.&lt;/li&gt;

&lt;li&gt;can resolve internal and internet names via DNS.&lt;/li&gt;

&lt;li&gt;installs software by packages.&lt;/li&gt;

&lt;li&gt;installs packages from a central internal repository and the official distribution repositories.&lt;/li&gt;

&lt;li&gt;software installed by packages should conform to the FHS.&lt;/li&gt;

&lt;li&gt;software not installed by packages should be installed by a reproducible deployment process.&lt;/li&gt;

&lt;li&gt;has sane defaults set, for user and process environments (locales, shells, screen, got some handy tools, etc.).&lt;/li&gt;

&lt;li&gt;should not provide typical compiler tools (gcc, build-essential).&lt;/li&gt;

&lt;li&gt;provides a manageable AAA concept (e.g. automated provisioning and de-provisioning of staff users).&lt;/li&gt;

&lt;li&gt;sends mails destinated for root to a central location.&lt;/li&gt;

&lt;li&gt;provides a local mailer.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id='hostnames'&gt;Hostnames&lt;/h2&gt;

&lt;blockquote&gt;
&lt;p&gt;Hostnames exist to identify every part of your infrastructure uniquely. They are used to refer to systems in your configurations and in discussions. You should think about a naming convention, but here are some rough guidelines.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Hostnames &amp;#8230;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;have to be unique.&lt;/li&gt;

&lt;li&gt;have to end with a number, which should never be reused and always be incremented.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id='services'&gt;Services&lt;/h2&gt;

&lt;blockquote&gt;
&lt;p&gt;A service is considered as everything running on a server&amp;#8217;s operating system, to provide continuous functionality (e.g. a script or an application).&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;A service&amp;#8230;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;does only log errors and auditing information. Application services may as well log more information (e.g. Apache access log).&lt;/li&gt;

&lt;li&gt;has defined log retention times.&lt;/li&gt;

&lt;li&gt;logs to syslog unless it&amp;#8217;s not possible.&lt;/li&gt;

&lt;li&gt;is authenticating only on secure connections.&lt;/li&gt;

&lt;li&gt;has an adequate and future-proof dimensioned datastore.&lt;/li&gt;

&lt;li&gt;was deployed in a reproducible way.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id='networks'&gt;Networks&lt;/h2&gt;

&lt;blockquote&gt;
&lt;p&gt;A network is considered any part of infrastructure, which is used to interconnect servers or systems. (Layer 1,2,3,4,&amp;#8230;)&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;A Network&amp;#8230;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;has clear entry and routing points.&lt;/li&gt;

&lt;li&gt;has a diagram which describes access vectors, the logical and physical setup.&lt;/li&gt;

&lt;li&gt;is deployed in adequate and future-proof dimensions (vlans, ip addresses, bandwidth).&lt;/li&gt;

&lt;li&gt;uses structured cabling.&lt;/li&gt;

&lt;li&gt;there is no cross-cabling, except for very rare situations (e.g. HA cabling).&lt;/li&gt;

&lt;li&gt;should not be used for multiple purposes at least not share one of the following classifications. &lt;table&gt;
  &lt;tr&gt;&lt;th&gt;Class&lt;/th&gt;&lt;th&gt;Description&lt;/th&gt;&lt;/tr&gt;
  &lt;tr&gt;&lt;td&gt;net     &lt;/td&gt;&lt;td&gt; Internet/upstream network &lt;/td&gt;&lt;/tr&gt;
  &lt;tr&gt;&lt;td&gt;mgmt    &lt;/td&gt;&lt;td&gt; Management network (monitoring, remote access)&lt;/td&gt;&lt;/tr&gt;
  &lt;tr&gt;&lt;td&gt;traffic &lt;/td&gt;&lt;td&gt; Site local traffic network&lt;/td&gt;&lt;/tr&gt;
  &lt;tr&gt;&lt;td&gt;backup  &lt;/td&gt;&lt;td&gt; Traffic network for backups&lt;/td&gt;&lt;/tr&gt;
  &lt;tr&gt;&lt;td&gt;voip    &lt;/td&gt;&lt;td&gt; Voip Telephony network&lt;/td&gt;&lt;/tr&gt;
  &lt;tr&gt;&lt;td&gt;clients &lt;/td&gt;&lt;td&gt; A network with client workstations.&lt;/td&gt;&lt;/tr&gt;
  &lt;tr&gt;&lt;td&gt;devel   &lt;/td&gt;&lt;td&gt; A network with development machines.&lt;/td&gt;&lt;/tr&gt;
  &lt;tr&gt;&lt;td&gt;staging &lt;/td&gt;&lt;td&gt; A network with staging equipment.&lt;/td&gt;&lt;/tr&gt;
 &lt;/table&gt; &lt;br /&gt; &lt;br /&gt;&lt;/li&gt;

&lt;li&gt;OOBs are easy to reach, even in case of an outage.&lt;/li&gt;

&lt;li&gt;VLAN-IDs are considered global, create a list.&lt;/li&gt;

&lt;li&gt;All VLAN-IDs below 99 are switch-local.&lt;/li&gt;

&lt;li&gt;VLANs have a name and a location.&lt;/li&gt;

&lt;li&gt;All address space is considered global (vlans, ip- and mac addresses, including RFC1918)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;To round up my article, here is a example checklist we use to peer review new systems:&lt;/p&gt;

&lt;h2 id='example_review_checklist'&gt;Example Review Checklist&lt;/h2&gt;

&lt;p&gt;Every newly deployed host or instance should undergo a peer-review process. The checklist below will provide you with a couple of base acceptance criteria and is going to ensure a certain level of quality. Give it to any other sysadmin and ask him or her to check the system, before it&amp;#8217;s put into production.&lt;/p&gt;
&lt;pre&gt;
* DNS works (including reverse dns)               :
* SSH login works                                 :
* Host+services monitored                         :
* Host+services graphed                           :
* All Filesystems backuped                        :
* Database dumps                                  :
* All Updates installed                           :
* Host in HostDoc                                 : 
* Puppet works                                    :
* Time is accurate                                :
* Root mails are being delivered                  :
* Firewall is active                              :
* No unneeded services are reachable (nmap)       :
* Network configuration works (+ipv6)             :
* Syslog/dmesg/oob logs are clean of errors       :

-- Physical Host --

* Root password documented                        :
* Root login works                                :
* OOB password documented                         :
* OOB login works                                 :
* OOB monitored                                   :
* Switch ports are labeled (+ documented)         :
* Hardware is labeled (+ documented in rack docu) :
* Firmware up to date                             :
* RAID level is &gt; 1 and all disks OK              :
&lt;/pre&gt;</content>
 </entry>
 
 <entry xmlns="http://www.w3.org/2005/Atom">
   <title>OpenVZ API</title>
   
   <link href="http://sts.ono.at/blog/2011/10/26/openvz-api"/>
   <updated>2011-10-26T13:10:00+02:00</updated>
   <id>http://sts.ono.at/blog/2011/10/26/openvz-api</id>
   <content type="html">&lt;p&gt;I love OpenVZ, I think its one of the easiest to use virtualisation technologies on the market and it adds almost no overhead compared to other technologies.&lt;/p&gt;

&lt;p&gt;I&amp;#8217;ve been using it for a couple of years now and I always wanted to have a nicer way to automate container creation, configuration or actions than writing shell scripts. There are already a couple of webinterfaces around, but none of them I liked.&lt;/p&gt;

&lt;p&gt;Another possibility would be to use libvirt - but libvirt always felt a bit too complex, since its a general API implementation for several hypervisors.&lt;/p&gt;

&lt;p&gt;So I started to implement my own API, which should rather be a simple and minimalistic approach. The project is hosted on GitHub but you can as well install it by Rubygems.&lt;/p&gt;
&lt;a href='https://github.com/sts/ruby-openvz' class='bigbutton btn-download'&gt;&lt;span&gt;&lt;span class='icon'&gt; &lt;/span&gt;Get ruby-openvz&lt;/span&gt;&lt;/a&gt;&lt;h2&gt;Installing&lt;/h2&gt;&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='bash'&gt; &lt;span class='nv'&gt;$ &lt;/span&gt;gem install openvz
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;&lt;h2&gt;Example: Restart&lt;/h2&gt;&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='ruby'&gt; &lt;span class='nb'&gt;require&lt;/span&gt; &lt;span class='s1'&gt;&amp;#39;rubygems&amp;#39;&lt;/span&gt;
 &lt;span class='nb'&gt;require&lt;/span&gt; &lt;span class='s1'&gt;&amp;#39;openvz&amp;#39;&lt;/span&gt;

 &lt;span class='n'&gt;container&lt;/span&gt; &lt;span class='o'&gt;=&lt;/span&gt; &lt;span class='no'&gt;OpenVZ&lt;/span&gt;&lt;span class='o'&gt;::&lt;/span&gt;&lt;span class='no'&gt;Container&lt;/span&gt;&lt;span class='o'&gt;.&lt;/span&gt;&lt;span class='n'&gt;new&lt;/span&gt;&lt;span class='p'&gt;(&lt;/span&gt;&lt;span class='s1'&gt;&amp;#39;109&amp;#39;&lt;/span&gt;&lt;span class='p'&gt;)&lt;/span&gt;
 &lt;span class='n'&gt;container&lt;/span&gt;&lt;span class='o'&gt;.&lt;/span&gt;&lt;span class='n'&gt;restart&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;&lt;h2&gt;Example: Provisioning&lt;/h2&gt;
&lt;p&gt;Here is the example of how a whole provisioning for a new container could look like:&lt;/p&gt;
&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='ruby'&gt; &lt;span class='nb'&gt;require&lt;/span&gt; &lt;span class='s1'&gt;&amp;#39;rubygems&amp;#39;&lt;/span&gt;
 &lt;span class='nb'&gt;require&lt;/span&gt; &lt;span class='s1'&gt;&amp;#39;openvz&amp;#39;&lt;/span&gt;

 &lt;span class='n'&gt;container&lt;/span&gt; &lt;span class='o'&gt;=&lt;/span&gt; &lt;span class='no'&gt;OpenVZ&lt;/span&gt;&lt;span class='o'&gt;::&lt;/span&gt;&lt;span class='no'&gt;Container&lt;/span&gt;&lt;span class='o'&gt;.&lt;/span&gt;&lt;span class='n'&gt;new&lt;/span&gt;&lt;span class='p'&gt;(&lt;/span&gt;&lt;span class='s1'&gt;&amp;#39;110&amp;#39;&lt;/span&gt;&lt;span class='p'&gt;)&lt;/span&gt;

 &lt;span class='n'&gt;container&lt;/span&gt;&lt;span class='o'&gt;.&lt;/span&gt;&lt;span class='n'&gt;create&lt;/span&gt;&lt;span class='p'&gt;(&lt;/span&gt; &lt;span class='ss'&gt;:ostemplate&lt;/span&gt; &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='s1'&gt;&amp;#39;debain-6.0-boostrap&amp;#39;&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;
                   &lt;span class='ss'&gt;:config&lt;/span&gt;     &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='s1'&gt;&amp;#39;vps.basic&amp;#39;&lt;/span&gt; &lt;span class='p'&gt;)&lt;/span&gt;

 &lt;span class='n'&gt;container&lt;/span&gt;&lt;span class='o'&gt;.&lt;/span&gt;&lt;span class='n'&gt;deboostrap&lt;/span&gt;&lt;span class='p'&gt;(&lt;/span&gt; &lt;span class='ss'&gt;:dist&lt;/span&gt;   &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='s1'&gt;&amp;#39;squeeze&amp;#39;&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;
                       &lt;span class='ss'&gt;:mirror&lt;/span&gt; &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='s1'&gt;&amp;#39;http://cdn.debian.net/debian&amp;#39;&lt;/span&gt; &lt;span class='p'&gt;)&lt;/span&gt;

 &lt;span class='n'&gt;container&lt;/span&gt;&lt;span class='o'&gt;.&lt;/span&gt;&lt;span class='n'&gt;set&lt;/span&gt;&lt;span class='p'&gt;(&lt;/span&gt; &lt;span class='ss'&gt;:nameserver&lt;/span&gt; &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='s1'&gt;&amp;#39;8.8.8.8&amp;#39;&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;
                &lt;span class='ss'&gt;:ipadd&lt;/span&gt;      &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='s1'&gt;&amp;#39;10.0.0.2&amp;#39;&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;
                &lt;span class='ss'&gt;:hostname&lt;/span&gt;   &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='s1'&gt;&amp;#39;foo.ono.at&amp;#39;&lt;/span&gt; &lt;span class='p'&gt;)&lt;/span&gt;

 &lt;span class='n'&gt;container&lt;/span&gt;&lt;span class='o'&gt;.&lt;/span&gt;&lt;span class='n'&gt;start&lt;/span&gt;

 &lt;span class='c1'&gt;# Update the system&lt;/span&gt;
 &lt;span class='n'&gt;container&lt;/span&gt;&lt;span class='o'&gt;.&lt;/span&gt;&lt;span class='n'&gt;command&lt;/span&gt;&lt;span class='p'&gt;(&lt;/span&gt;&lt;span class='s1'&gt;&amp;#39;aptitude update ; aptitude -y upgrade ; apt-key update&amp;#39;&lt;/span&gt;&lt;span class='p'&gt;)&lt;/span&gt;

 &lt;span class='c1'&gt;# Install puppet&lt;/span&gt;
 &lt;span class='n'&gt;container&lt;/span&gt;&lt;span class='o'&gt;.&lt;/span&gt;&lt;span class='n'&gt;command&lt;/span&gt;&lt;span class='p'&gt;(&lt;/span&gt;&lt;span class='s1'&gt;&amp;#39;aptitude -o Aptitude::Cmdline::ignore-trust-violations=true -y install puppet&amp;#39;&lt;/span&gt;&lt;span class='p'&gt;)&lt;/span&gt;

 &lt;span class='c1'&gt;# Install puppet&lt;/span&gt;
 &lt;span class='n'&gt;container&lt;/span&gt;&lt;span class='o'&gt;.&lt;/span&gt;&lt;span class='n'&gt;command&lt;/span&gt;&lt;span class='p'&gt;(&lt;/span&gt;&lt;span class='s1'&gt;&amp;#39;puppetd -t --server=puppet.ono.at&amp;#39;&lt;/span&gt;&lt;span class='p'&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;p&gt;The first version is basically able to replace the provisioning scripts I&amp;#8217;ve been using for a while. In future versions I&amp;#8217;d probably like to integrate this into mcollective and puppet to orchestrate my machines. :-)&lt;/p&gt;</content>
 </entry>
 
 <entry xmlns="http://www.w3.org/2005/Atom">
   <title>Upgrading HP Firmware</title>
   
   <link href="http://sts.ono.at/blog/2011/03/03/upgrading-hp-firmware"/>
   <updated>2011-03-03T08:12:00+01:00</updated>
   <id>http://sts.ono.at/blog/2011/03/03/upgrading-hp-firmware</id>
   <content type="html">&lt;p&gt;Lately we bought a new HP blade chassis to replace a customer&amp;#8217;s old database server. All it&amp;#8217;s services run on ~15 blades, splitted cross two HP C7000 chassis.&lt;/p&gt;

&lt;p&gt;The Proliant BL460 G6 we bought came with much newer firmware revisions than all the existing G1 &amp;#8211; part of the infrastructure didn&amp;#8217;t receive much sysadmin love over quite some time. :-)&lt;/p&gt;

&lt;p&gt;Blades, ILO, chassis and controllers where all running way outdated firmware and upgrading was highly recommended. The arising firmware combinations haven&amp;#8217;t been tested and the new blade wouldn&amp;#8217;t even be detected, so HP. They offered us an upgrade for about $2000 and 6 hours of downtime per chassis.&lt;/p&gt;

&lt;p&gt;Here are some handsome findings, to do the upgrade on your own:&lt;/p&gt;
&lt;h2&gt;HP Firmware Comapatibility Matrix&lt;/h2&gt;
&lt;p&gt;HP tested certain sets of firmware for compatibility. Take a look at their compatibility matrix and try to stay within the tested boundaries. This could mean to upgrade in more than one step, if you are running an older release.&lt;/p&gt;

&lt;p&gt;(http://h18004.www1.hp.com/products/blades/components/c-class.html)&lt;/p&gt;
&lt;h2&gt;Hp-Firmware-Catalog&lt;/h2&gt;
&lt;p&gt;There is &lt;a href='http://christian.hofstaedtler.name/'&gt;Christian Hofstedtlers&lt;/a&gt; great firmware upgrade script, which automatically downloads the latest and greatest HP firmware installation packages. Its even creating softlinks, to reference cryptic firmware package names to their corresponding hardware components.&lt;/p&gt;

&lt;p&gt;(https://github.com/zeha/hp-firmware-catalog)&lt;/p&gt;

&lt;p&gt;You can run them from your OS as an online upgrade. Certain components still might require rebooting, to finish the &amp;#8220;delayed upgrade&amp;#8221;.&lt;/p&gt;

&lt;p&gt;I would love to see HP maintaining this, since the approach provides a good example of providing customers with a modern and automated way to upgrade and monitor firmware for more recent releases.&lt;/p&gt;
&lt;h2&gt;ILO Shell&lt;/h2&gt;
&lt;p&gt;When upgrading many machines it will save you a lot of time, if you just use the SSH shell for configuring a boot device and rebooting the server.&lt;/p&gt;
&lt;h3&gt;Connect to ILO by SSH&lt;/h3&gt;
&lt;p&gt;Make sure you send the right username, AFAIK it&amp;#8217;s case sensitive on the ILO:&lt;/p&gt;
&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='bash'&gt;     ssh phx-vnode03.oob.ono.at -l Administrator
   
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;&lt;h3&gt;Set an ILO Advanced Licence key&lt;/h3&gt;&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='bash'&gt;     &lt;span class='nb'&gt;cd&lt;/span&gt; /map1
     &lt;span class='nb'&gt;set &lt;/span&gt;&lt;span class='nv'&gt;license&lt;/span&gt;&lt;span class='o'&gt;=&lt;/span&gt;YOUR-LICENCE-KEY
   
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;p&gt;The advanced licence key is required to enable virtual device firmware features. Eg. to make use of the remote console or a virtual disk boot drive.&lt;/p&gt;
&lt;h3&gt;Mount and configure a network hosted ISO image as boot device&lt;/h3&gt;&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='bash'&gt;     &lt;span class='nb'&gt;cd&lt;/span&gt; /map1
     vm cdrom insert http://10.0.10.21/FW920B.2010_1129.2.iso
     vm cdrom &lt;span class='nb'&gt;set &lt;/span&gt;boot_always
   
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;p&gt;&amp;#8230;be it a firmware upgrade or an OS installation disk. Make sure you run the following command to &amp;#8220;eject&amp;#8221; it again:&lt;/p&gt;
&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='bash'&gt;     &lt;span class='nb'&gt;cd&lt;/span&gt; /map1
     vm cdrom eject
   
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;&lt;h2&gt;Monitoring&lt;/h2&gt;
&lt;p&gt;To please your monitoring system as well, check out check_mk. They wrote a couple of good &lt;a href='http://mathias-kettner.de/checkmk_checks.html'&gt;SNMP checks&lt;/a&gt; for your HP or IBM bladecenter.&lt;/p&gt;

&lt;p&gt;In the end I can highly recommend to keep your hardware firmware up to date. At least HP, my vendor of choice, they add a lot of useful bug fixes.&lt;/p&gt;

&lt;p&gt;HP currently informs customers by a e-mail newsletter about updates, I would love to see this in my monitoring system too, like all the other security upgrades.&lt;/p&gt;

&lt;p&gt;Try to plan the upgrade a bit or use existing downtimes to boot the HP Firmware Maintenance image.&lt;/p&gt;</content>
 </entry>
 
 <entry xmlns="http://www.w3.org/2005/Atom">
   <title>munin-host-rename</title>
   
   <link href="http://sts.ono.at/blog/2011/02/16/rename-munin-nodes"/>
   <updated>2011-02-16T10:32:00+01:00</updated>
   <id>http://sts.ono.at/blog/2011/02/16/rename-munin-nodes</id>
   <content type="html">&lt;p&gt;Recently we decided upon a new host naming convention for our infrastructure at my $dayjob. So I will soon be renaming a few hosts.&lt;/p&gt;

&lt;p&gt;I noticed that &lt;a href='http://munin-monitoring.org/'&gt;munin&lt;/a&gt; still doesn&amp;#8217;t provide an easy way, to rename an existing node without losing historical data. So I wrote a small shell script which does that for me and might be of use to everybody else in the same situation.&lt;/p&gt;
&lt;a href='https://github.com/sts/munin-host-rename' class='bigbutton btn-download'&gt;&lt;span&gt;&lt;span class='icon'&gt; &lt;/span&gt;Download munin-host-rename&lt;/span&gt;&lt;/a&gt;</content>
 </entry>
 
 <entry xmlns="http://www.w3.org/2005/Atom">
   <title>Synchronize Puppet with Git</title>
   
     <category term="Puppet" label="Puppet" />
   
     <category term="Git" label="Git" />
   
   <link href="http://sts.ono.at/blog/2010/12/22/synchronize-puppet-with-git"/>
   <updated>2010-12-22T18:29:00+01:00</updated>
   <id>http://sts.ono.at/blog/2010/12/22/synchronize-puppet-with-git</id>
   <content type="html">&lt;a href='https://github.com/sts/puppet-sync' class='bigbutton btn-download'&gt;&lt;span&gt;&lt;span class='icon'&gt; &lt;/span&gt;Download Puppet-Sync&lt;/span&gt;&lt;/a&gt;
&lt;p&gt;Puppet really shines at automating infrastructures. You will notice a sudden change of working methodology, once you manage the first systems with it.&lt;/p&gt;

&lt;p&gt;Instead of manually logging on to each single system for updating a certain part of configuration by issuing shell commands, you will stop to repeat yourself and just update a single piece of code, which describes the desired config state for all systems.&lt;/p&gt;

&lt;p&gt;&lt;a href='http://projects.puppetlabs.com/projects/1/wiki/Advanced_Puppet_Pattern'&gt;As recommended in the Puppet documentation&lt;/a&gt; you are well advised to keep your Puppet manifests under revision control.&lt;/p&gt;

&lt;p&gt;I wrote a small script which will come in handy, to ease your life with keeping your repository and the manifests on the master in sync and should fit to most of the environments out there.&lt;/p&gt;

&lt;p&gt;Once installed, you can store the manifests for each Puppet environment in its own GIT branch and every time you commit a new version to one of your branches, it will automatically sync the most recent version and inform the Puppet master process.&lt;/p&gt;

&lt;p&gt;BTW. this could also be used to keep the manifests on multiple Puppet instances in sync.&lt;/p&gt;
&lt;h2&gt;Puppet-Sync&lt;/h2&gt;
&lt;p&gt;Puppet-sync is a Ruby based command line tool to synchronize every commit from a central GIT repository to your Puppet master instance. You should install it on your Puppet master and configure a GIT hook which calls the script over ssh.&lt;/p&gt;

&lt;p&gt;Puppet-sync takes some parameters to specify how the environment on the master looks like. Run it with &amp;#8217;&amp;#8211;help&amp;#8217; to get a list of available options. Here is an example:&lt;/p&gt;
&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='bash'&gt;puppet-sync --branch master &lt;span class='se'&gt;\&lt;/span&gt;
            --passenger     &lt;span class='se'&gt;\&lt;/span&gt;
            --destination /etc/puppet/environments &lt;span class='se'&gt;\&lt;/span&gt;
            --repository ssh+git://git.ono.at/srv/git/puppet.git
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;p&gt;By running the command above, the script connects to the git repository, fetches the manifests from the master branch and puts it into /etc/puppet/environments/production. Since we use the master branch for production, I added the logic to translate &amp;#8220;master&amp;#8221;-branch to &amp;#8220;production&amp;#8221;-environment.&lt;/p&gt;
&lt;h2&gt;Installation on the master&lt;/h2&gt;
&lt;p&gt;Instead of listing each and every shell command needed to install the environment for the script, I&amp;#8217;d like to provide a simple manifest instead. I think this is more readable and you can either use it or figure out the appropriate shell commands. ;-)&lt;/p&gt;
&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='ruby'&gt;&lt;span class='n'&gt;file&lt;/span&gt; &lt;span class='p'&gt;{&lt;/span&gt; &lt;span class='s2'&gt;&amp;quot;/usr/local/bin/pupept-sync&amp;quot;&lt;/span&gt;&lt;span class='p'&gt;:&lt;/span&gt;
    &lt;span class='k'&gt;ensure&lt;/span&gt;  &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='n'&gt;present&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;
    &lt;span class='n'&gt;source&lt;/span&gt;  &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='s2'&gt;&amp;quot;file:///puppet-sync&amp;quot;&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;
&lt;span class='p'&gt;}&lt;/span&gt;

&lt;span class='n'&gt;file&lt;/span&gt; &lt;span class='p'&gt;{&lt;/span&gt; &lt;span class='s2'&gt;&amp;quot;/home/psync/.ssh&amp;quot;&lt;/span&gt;&lt;span class='p'&gt;:&lt;/span&gt;
    &lt;span class='k'&gt;ensure&lt;/span&gt;  &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='n'&gt;directory&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;
    &lt;span class='n'&gt;owner&lt;/span&gt;   &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='s2'&gt;&amp;quot;psync&amp;quot;&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;
    &lt;span class='n'&gt;mode&lt;/span&gt;    &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='mi'&gt;700&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;
    &lt;span class='nb'&gt;require&lt;/span&gt; &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='no'&gt;User&lt;/span&gt;&lt;span class='o'&gt;[&lt;/span&gt;&lt;span class='s2'&gt;&amp;quot;psync&amp;quot;&lt;/span&gt;&lt;span class='o'&gt;]&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;
&lt;span class='p'&gt;}&lt;/span&gt;

&lt;span class='n'&gt;file&lt;/span&gt; &lt;span class='p'&gt;{&lt;/span&gt; &lt;span class='s2'&gt;&amp;quot;/etc/puppet/environments&amp;quot;&lt;/span&gt;&lt;span class='p'&gt;:&lt;/span&gt;
    &lt;span class='k'&gt;ensure&lt;/span&gt;  &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='n'&gt;directory&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;
    &lt;span class='n'&gt;owner&lt;/span&gt;   &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='s2'&gt;&amp;quot;psync&amp;quot;&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;
    &lt;span class='n'&gt;mode&lt;/span&gt;    &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='mi'&gt;775&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;
    &lt;span class='nb'&gt;require&lt;/span&gt; &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='no'&gt;User&lt;/span&gt;&lt;span class='o'&gt;[&lt;/span&gt;&lt;span class='s2'&gt;&amp;quot;psync&amp;quot;&lt;/span&gt;&lt;span class='o'&gt;]&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;
&lt;span class='p'&gt;}&lt;/span&gt;

&lt;span class='n'&gt;user&lt;/span&gt; &lt;span class='p'&gt;{&lt;/span&gt; &lt;span class='s2'&gt;&amp;quot;psync&amp;quot;&lt;/span&gt;&lt;span class='p'&gt;:&lt;/span&gt;
    &lt;span class='k'&gt;ensure&lt;/span&gt;     &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='n'&gt;present&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;
    &lt;span class='n'&gt;home&lt;/span&gt;       &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='s2'&gt;&amp;quot;/home/psync&amp;quot;&lt;/span&gt;
    &lt;span class='n'&gt;managehome&lt;/span&gt; &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='kp'&gt;true&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;
&lt;span class='p'&gt;}&lt;/span&gt;

&lt;span class='n'&gt;ssh_authorized_key&lt;/span&gt; &lt;span class='p'&gt;{&lt;/span&gt; &lt;span class='s2'&gt;&amp;quot;puppet-sync-ssh-key&amp;quot;&lt;/span&gt;&lt;span class='p'&gt;:&lt;/span&gt;
    &lt;span class='k'&gt;ensure&lt;/span&gt;  &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='n'&gt;present&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;
    &lt;span class='n'&gt;key&lt;/span&gt;     &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='s2'&gt;&amp;quot;AAAAB3.....lVBp0nPLNcs=&amp;quot;&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;
    &lt;span class='n'&gt;type&lt;/span&gt;    &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='s2'&gt;&amp;quot;ssh-rsa&amp;quot;&lt;/span&gt;
    &lt;span class='n'&gt;user&lt;/span&gt;    &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='s2'&gt;&amp;quot;psync&amp;quot;&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;
    &lt;span class='nb'&gt;require&lt;/span&gt; &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='no'&gt;File&lt;/span&gt;&lt;span class='o'&gt;[&lt;/span&gt;&lt;span class='s2'&gt;&amp;quot;${homeroot}/$name/.ssh&amp;quot;&lt;/span&gt;&lt;span class='o'&gt;]&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;
&lt;span class='p'&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;p&gt;I have the following configuration in my puppet.conf to make puppet aware of each of the directories in /etc/environments:&lt;/p&gt;
&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='ini'&gt;&lt;span class='err'&gt;...&lt;/span&gt;
&lt;span class='k'&gt;[master]&lt;/span&gt;
&lt;span class='err'&gt;.....&lt;/span&gt;
&lt;span class='na'&gt;templatedir&lt;/span&gt; &lt;span class='o'&gt;=&lt;/span&gt; &lt;span class='s'&gt;/etc/puppet/environments/$environment/&lt;/span&gt;
&lt;span class='na'&gt;modulepath&lt;/span&gt;  &lt;span class='o'&gt;=&lt;/span&gt; &lt;span class='s'&gt;/etc/puppet/environments/$environment/modules/&lt;/span&gt;
&lt;span class='na'&gt;manifest&lt;/span&gt;    &lt;span class='o'&gt;=&lt;/span&gt; &lt;span class='s'&gt;/etc/puppet/environments/$environment/manifests/site.pp&lt;/span&gt;
&lt;span class='na'&gt;manifestdir&lt;/span&gt; &lt;span class='o'&gt;=&lt;/span&gt; &lt;span class='s'&gt;/etc/puppet/environments/$environment/manifests&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;&lt;h2&gt;Git Hook&lt;/h2&gt;
&lt;p&gt;The only thing left is to create a Git Hook in your repository. Here is the one i use. I also created a psync user on the master, so i just need to store the private ssh key in psync&amp;#8217;s home directory.&lt;/p&gt;
&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='bash'&gt;&lt;span class='c'&gt;#!/bin/sh&lt;/span&gt;
&lt;span class='c'&gt;#&lt;/span&gt;
&lt;span class='c'&gt;# An example hook script to prepare a packed repository for use over&lt;/span&gt;
&lt;span class='c'&gt;# dumb transports.&lt;/span&gt;
&lt;span class='c'&gt;#&lt;/span&gt;
&lt;span class='c'&gt;# To enable this hook, make this file executable by &amp;quot;chmod +x post-update&amp;quot;.&lt;/span&gt;

&lt;span class='nv'&gt;branch&lt;/span&gt;&lt;span class='o'&gt;=&lt;/span&gt;&lt;span class='sb'&gt;`&lt;/span&gt;&lt;span class='nb'&gt;echo&lt;/span&gt; &lt;span class='nv'&gt;$1&lt;/span&gt; | awk -F/ &lt;span class='o'&gt;{&lt;/span&gt;&lt;span class='s1'&gt;&amp;#39;print $3&amp;#39;&lt;/span&gt;&lt;span class='o'&gt;}&lt;/span&gt;&lt;span class='sb'&gt;`&lt;/span&gt;

sudo -u psync ssh puppet.ono.at /usr/local/bin/puppet-sync &lt;span class='se'&gt;\&lt;/span&gt;
                                       --passenger         &lt;span class='se'&gt;\&lt;/span&gt;
                                       --branch &lt;span class='nv'&gt;$branch&lt;/span&gt;

&lt;span class='nb'&gt;exec &lt;/span&gt;git-update-server-info
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;p&gt;Perfect its installed and you are ready to use it. Go ahead an try to commit a new version.&lt;/p&gt;</content>
 </entry>
 
 <entry xmlns="http://www.w3.org/2005/Atom">
   <title>Monitoring Puppet and Bacula</title>
   
     <category term="Monitoring" label="Monitoring" />
   
     <category term="CheckMK" label="CheckMK" />
   
   <link href="http://sts.ono.at/blog/2010/12/07/checkmk-update"/>
   <updated>2010-12-07T17:51:00+01:00</updated>
   <id>http://sts.ono.at/blog/2010/12/07/checkmk-update</id>
   <content type="html">&lt;a href='https://github.com/sts/checkmk' class='bigbutton btn-download'&gt;&lt;span&gt;&lt;span class='icon'&gt; &lt;/span&gt;Download CheckMK Plugins&lt;/span&gt;&lt;/a&gt;
&lt;p&gt;Rescently I blogged about my &lt;a href='/blog/2010/10/10/checkmk-apt'&gt;CheckMK APT plugin&lt;/a&gt;, capable of checking for upgradeable packages.&lt;/p&gt;

&lt;p&gt;Meanwhile I wrote two more plugins, both adopting existing checks and migrated all plugins into one single &lt;a href='https://github.com/sts/checkmk'&gt;&amp;#8216;checkmk&amp;#8217; repository&lt;/a&gt; over at GitHub.&lt;/p&gt;
&lt;h2&gt;bacula&lt;/h2&gt;
&lt;p&gt;This check adapts the idea of &lt;a href='https://github.com/bmiklautz/bacula-utils'&gt;Bernhard Miklautz bacula-utils&lt;/a&gt;. The plugin will define four different services to be monitored on each Bacula server:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;bacula.errorvols&lt;/p&gt;

&lt;p&gt;Checks the state of available backup volumes and will report critical, if errounos volumes are found. In case of an error, it will as well return the names of these volumes.&lt;/p&gt;

&lt;p&gt;Use the bacula-clear-errvols script to resolve these issues.&lt;/p&gt;
&lt;/li&gt;

&lt;li&gt;
&lt;p&gt;bacula.freshness&lt;/p&gt;

&lt;p&gt;Checks whether all clients have an associated backup within the last 30 hours.&lt;/p&gt;
&lt;/li&gt;

&lt;li&gt;
&lt;p&gt;bacula.fullbackups&lt;/p&gt;

&lt;p&gt;Checks whether all clients have an associated full-backup.&lt;/p&gt;
&lt;/li&gt;

&lt;li&gt;
&lt;p&gt;bacula.fullbackupspool&lt;/p&gt;

&lt;p&gt;Checks whether any volumes used for full-backups come from a full-backup pool.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;puppet&lt;/h2&gt;
&lt;p&gt;The second plugin will monitor the status of the puppet agent. It integrates nicely with puppetstatus.py. This script was initially written by TMZ from the Fedora Infrastructure Team and can be used to enable or disable the puppet agent on a specific host.&lt;/p&gt;

&lt;p&gt;You are able to disable Puppet on a server, by running the following command:&lt;/p&gt;
&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='bash'&gt; sudo puppetstatus -d &lt;span class='s2'&gt;&amp;quot;Interesting things are about to happen.&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;p&gt;The monitoring system will change the state of the Puppet check to WARNING and will display this message and the username of the person who was disabling the agent.&lt;/p&gt;

&lt;p&gt;When the agent is not disabled on a host, the check will just change to WARNING after 3 and to CRITICAL after 4 hours.&lt;/p&gt;

&lt;p&gt;I&amp;#8217;d be glad to get some feedback on my plugins, please report any bugs by sending me an e-mail or leave a comment below.&lt;/p&gt;</content>
 </entry>
 
 <entry xmlns="http://www.w3.org/2005/Atom">
   <title>Puppet+Capistrano</title>
   
     <category term="Puppet" label="Puppet" />
   
     <category term="Capistrano" label="Capistrano" />
   
   <link href="http://sts.ono.at/blog/2010/11/25/puppet-capistrano"/>
   <updated>2010-11-25T19:31:00+01:00</updated>
   <id>http://sts.ono.at/blog/2010/11/25/puppet-capistrano</id>
   <content type="html">&lt;p&gt;I&amp;#8217;m currently working on our upgrade to Puppet 2.6. One of the ideas I came up with was to deploy our puppet manifests (which we already store in a GIT repository) using &lt;a href='http://www.capify.org/'&gt;Capistrano&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;For those who don&amp;#8217;t know Capistrano already: Capistrano is a Ruby based deployment tool, used to get application code from a source control system to your application servers. Initially Capistrano was written to deploy Ruby on Rails applications. Meanwhile they also provide a &amp;#8216;railsless-deploy&amp;#8217; module, which can be used to deploy any other code or framework.&lt;/p&gt;

&lt;p&gt;What I like most about Capistrano, is that you don&amp;#8217;t have to install it on your servers. It&amp;#8217;s designed to read a couple of instructions from a &amp;#8220;Capfile&amp;#8221; which resides in the top directory of your application code. Once you trigger the deployment of a release, it will connect to your servers using SSH and trigger commands to deploy your Puppet manifests.&lt;/p&gt;

&lt;p&gt;Important for all of this to work propperly is a SSH setup which uses public keys for authentication.&lt;/p&gt;

&lt;p&gt;&lt;img src='/images/post-2010-11-25-puppet-capistrano.png' alt='ssh process' /&gt;&lt;/p&gt;

&lt;p&gt;Capistrano will connect to the Puppet master and issue GIT commands against your GIT server. So not only password-less authentication to the puppet server must work, but also connecting from a shell on the Puppet server to the GIT server.&lt;/p&gt;
&lt;h2&gt;Installation&lt;/h2&gt;
&lt;p&gt;Capistrano is distrubuted as a ruby gem. If your distribution does already provide a package, use it. Otherwise you can simply use rubygems.&lt;/p&gt;
&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='bash'&gt;  gem install capistrano
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;&lt;h2&gt;Capify&lt;/h2&gt;
&lt;p&gt;Next up copy the Capfile below, to your Puppet top level directory. Here is how it should look like:&lt;/p&gt;
&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='bash'&gt;  ls -l puppet/
  -rw-r--r--   1 sts sts  1499 Nov 26 10:33 Capfile
  -rw-r--r--   1 sts sts  1464 Nov  9 20:02 README
  -rw-r--r--   1 sts sts   907 Nov  9 20:04 TODO
  drwxr-xr-x   6 sts sts   204 Nov  9 20:02 manifests
  drwxr-xr-x  42 sts sts  1428 Nov 22 11:04 modules
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;p&gt;There are some variables you need to change in order to make it work with your setup. Basically look through all lines which start with &amp;#8220;set&amp;#8221; and ajust them according to your needs.&lt;/p&gt;

&lt;p&gt;Basically Capistrano is going to do the following:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Connect to your puppet server.&lt;/li&gt;

&lt;li&gt;Create a directory for the release eg. /etc/puppet/git/#{:branch}/releases/20101125203021.&lt;/li&gt;

&lt;li&gt;Checkout the HEAD of the specified GIT branch to this directory, by connecting from your Puppet server, to your GIT server using SSH.&lt;/li&gt;

&lt;li&gt;Ajust the symlink from /etc/puppet/git/#{:branch}/current to point to the new release.&lt;/li&gt;

&lt;li&gt;Tell Fusion Passenger to reload the application code by creating the following file: /usr/share/puppet/rack/puppetmasterd/tmp/restart.txt&lt;/li&gt;
&lt;/ul&gt;
&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='ruby'&gt;&lt;span class='nb'&gt;require&lt;/span&gt; &lt;span class='s1'&gt;&amp;#39;railsless-deploy&amp;#39;&lt;/span&gt;
&lt;span class='nb'&gt;require&lt;/span&gt; &lt;span class='s1'&gt;&amp;#39;net/smtp&amp;#39;&lt;/span&gt;

&lt;span class='n'&gt;set&lt;/span&gt; &lt;span class='ss'&gt;:application&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;     &lt;span class='s2'&gt;&amp;quot;puppet&amp;quot;&lt;/span&gt;
&lt;span class='n'&gt;set&lt;/span&gt; &lt;span class='ss'&gt;:user&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;            &lt;span class='s2'&gt;&amp;quot;root&amp;quot;&lt;/span&gt;
&lt;span class='n'&gt;set&lt;/span&gt; &lt;span class='ss'&gt;:use_sudo&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;        &lt;span class='kp'&gt;true&lt;/span&gt;
&lt;span class='n'&gt;set&lt;/span&gt; &lt;span class='ss'&gt;:group_writebale&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt; &lt;span class='kp'&gt;false&lt;/span&gt;
&lt;span class='n'&gt;set&lt;/span&gt; &lt;span class='ss'&gt;:deploy_to&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;       &lt;span class='s2'&gt;&amp;quot;/etc/puppet/git/&lt;/span&gt;&lt;span class='si'&gt;#{&lt;/span&gt;&lt;span class='ss'&gt;:branch&lt;/span&gt;&lt;span class='si'&gt;}&lt;/span&gt;&lt;span class='s2'&gt;&amp;quot;&lt;/span&gt;
&lt;span class='n'&gt;set&lt;/span&gt; &lt;span class='ss'&gt;:keep_releases&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;   &lt;span class='mi'&gt;2&lt;/span&gt;

&lt;span class='n'&gt;set&lt;/span&gt; &lt;span class='ss'&gt;:scm&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;         &lt;span class='s2'&gt;&amp;quot;git&amp;quot;&lt;/span&gt;
&lt;span class='n'&gt;set&lt;/span&gt; &lt;span class='ss'&gt;:repository&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;  &lt;span class='s2'&gt;&amp;quot;git+ssh://sts@git.ono.at/srv/git/&lt;/span&gt;&lt;span class='si'&gt;#{&lt;/span&gt;&lt;span class='n'&gt;application&lt;/span&gt;&lt;span class='si'&gt;}&lt;/span&gt;&lt;span class='s2'&gt;.git&amp;quot;&lt;/span&gt;
&lt;span class='n'&gt;set&lt;/span&gt; &lt;span class='ss'&gt;:branch&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt;      &lt;span class='s2'&gt;&amp;quot;master&amp;quot;&lt;/span&gt;

&lt;span class='n'&gt;role&lt;/span&gt; &lt;span class='ss'&gt;:app&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt; &lt;span class='s2'&gt;&amp;quot;puppet.ono.at&amp;quot;&lt;/span&gt;

&lt;span class='n'&gt;namespace&lt;/span&gt; &lt;span class='ss'&gt;:deploy&lt;/span&gt; &lt;span class='k'&gt;do&lt;/span&gt;

    &lt;span class='n'&gt;task&lt;/span&gt; &lt;span class='ss'&gt;:default&lt;/span&gt; &lt;span class='k'&gt;do&lt;/span&gt;
        &lt;span class='n'&gt;update&lt;/span&gt;
        &lt;span class='n'&gt;cleanup&lt;/span&gt;
    &lt;span class='k'&gt;end&lt;/span&gt;

    &lt;span class='n'&gt;task&lt;/span&gt; &lt;span class='ss'&gt;:notify&lt;/span&gt; &lt;span class='k'&gt;do&lt;/span&gt;
        &lt;span class='n'&gt;changes_list&lt;/span&gt; &lt;span class='o'&gt;=&lt;/span&gt; &lt;span class='s2'&gt;&amp;quot;puppet-changes@ono.at&amp;quot;&lt;/span&gt;
        &lt;span class='n'&gt;user_name&lt;/span&gt;    &lt;span class='o'&gt;=&lt;/span&gt; &lt;span class='sb'&gt;`git config user.name`&lt;/span&gt;&lt;span class='o'&gt;.&lt;/span&gt;&lt;span class='n'&gt;strip&lt;/span&gt;
        &lt;span class='n'&gt;user_email&lt;/span&gt;   &lt;span class='o'&gt;=&lt;/span&gt; &lt;span class='sb'&gt;`git config user.email`&lt;/span&gt;&lt;span class='o'&gt;.&lt;/span&gt;&lt;span class='n'&gt;strip&lt;/span&gt;
        &lt;span class='n'&gt;commits&lt;/span&gt;      &lt;span class='o'&gt;=&lt;/span&gt; &lt;span class='sb'&gt;`git log --pretty=oneline --abbrev-commit &lt;/span&gt;&lt;span class='si'&gt;#{&lt;/span&gt;&lt;span class='n'&gt;previous_revision&lt;/span&gt;&lt;span class='si'&gt;}&lt;/span&gt;&lt;span class='sb'&gt;..&lt;/span&gt;&lt;span class='si'&gt;#{&lt;/span&gt;&lt;span class='n'&gt;current_revision&lt;/span&gt;&lt;span class='si'&gt;}&lt;/span&gt;&lt;span class='sb'&gt;`&lt;/span&gt;&lt;span class='o'&gt;.&lt;/span&gt;&lt;span class='n'&gt;strip&lt;/span&gt;

        &lt;span class='n'&gt;msg&lt;/span&gt;          &lt;span class='o'&gt;=&lt;/span&gt; &lt;span class='o'&gt;&amp;lt;&amp;lt;&lt;/span&gt;&lt;span class='no'&gt;EOM&lt;/span&gt;
&lt;span class='sh'&gt;From: &amp;quot;#{user_name}&amp;quot; &amp;lt;#{user_email}&amp;gt;&lt;/span&gt;
&lt;span class='sh'&gt;To: #{changes_list}&lt;/span&gt;
&lt;span class='sh'&gt;Subject: [puppet] deployed #{release_name}.&lt;/span&gt;

&lt;span class='sh'&gt;The following changes where just pushed to puppet by #{user_name}:&lt;/span&gt;

&lt;span class='sh'&gt;  Branch  : #{:branch}&lt;/span&gt;
&lt;span class='sh'&gt;  Release : #{release}&lt;/span&gt;
&lt;span class='sh'&gt;  Server  : #{:app}&lt;/span&gt;

&lt;span class='sh'&gt;Applied Commits /////////////////////////////////////////////// ///////&lt;/span&gt;

&lt;span class='sh'&gt;#{commits}&lt;/span&gt;

&lt;span class='sh'&gt;If you want to revert this release, please run:&lt;/span&gt;

&lt;span class='sh'&gt; $ cap deploy:rollback&lt;/span&gt;

&lt;span class='sh'&gt;Cheers,&lt;/span&gt;
&lt;span class='sh'&gt;Capistrano.&lt;/span&gt;
&lt;span class='no'&gt;EOM&lt;/span&gt;

        &lt;span class='no'&gt;Net&lt;/span&gt;&lt;span class='o'&gt;::&lt;/span&gt;&lt;span class='no'&gt;SMTP&lt;/span&gt;&lt;span class='o'&gt;.&lt;/span&gt;&lt;span class='n'&gt;start&lt;/span&gt;&lt;span class='p'&gt;(&lt;/span&gt;&lt;span class='s1'&gt;&amp;#39;localhost&amp;#39;&lt;/span&gt;&lt;span class='p'&gt;)&lt;/span&gt; &lt;span class='k'&gt;do&lt;/span&gt; &lt;span class='o'&gt;|&lt;/span&gt;&lt;span class='n'&gt;smtp&lt;/span&gt;&lt;span class='o'&gt;|&lt;/span&gt;
             &lt;span class='n'&gt;smtp&lt;/span&gt;&lt;span class='o'&gt;.&lt;/span&gt;&lt;span class='n'&gt;send_message&lt;/span&gt; &lt;span class='n'&gt;msg&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt; &lt;span class='n'&gt;user_email&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt; &lt;span class='n'&gt;changes_list&lt;/span&gt;
        &lt;span class='k'&gt;end&lt;/span&gt;
    &lt;span class='k'&gt;end&lt;/span&gt;

    &lt;span class='n'&gt;dec&lt;/span&gt; &lt;span class='s2'&gt;&amp;quot;Restart Puppetmaster&amp;quot;&lt;/span&gt;
    &lt;span class='n'&gt;task&lt;/span&gt; &lt;span class='ss'&gt;:restart&lt;/span&gt;&lt;span class='p'&gt;,&lt;/span&gt; &lt;span class='ss'&gt;:roles&lt;/span&gt; &lt;span class='o'&gt;=&amp;gt;&lt;/span&gt; &lt;span class='ss'&gt;:app&lt;/span&gt; &lt;span class='k'&gt;do&lt;/span&gt;
        &lt;span class='n'&gt;run&lt;/span&gt; &lt;span class='s2'&gt;&amp;quot;touch /usr/share/puppet/rack/puppetmasterd/tmp/restart.txt&amp;quot;&lt;/span&gt;
    &lt;span class='k'&gt;end&lt;/span&gt;
&lt;span class='k'&gt;end&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;&lt;h2&gt;Configure GIT&lt;/h2&gt;
&lt;p&gt;Please also make sure you configure GIT with your name and e-mail address. Capistrano will read these from your git configuration to send out notification e-mails.&lt;/p&gt;
&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='bash'&gt;    git config --global --add user.name &lt;span class='s2'&gt;&amp;quot;CHANGE-ME&amp;quot;&lt;/span&gt;
    git config --glibal --add user.email &lt;span class='s2'&gt;&amp;quot;CHANGE-ME&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;&lt;h2&gt;Go&lt;/h2&gt;
&lt;p&gt;You should be able to deploy updates to your Puppet servers using the following command:&lt;/p&gt;
&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='bash'&gt;  cap deploy
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;&lt;h2&gt;Future enhancements&lt;/h2&gt;
&lt;p&gt;If you&amp;#8217;ve got any ideas or see a way to enhance this process, I&amp;#8217;d be glad if you leave me a comment. :-)&lt;/p&gt;</content>
 </entry>
 
 <entry xmlns="http://www.w3.org/2005/Atom">
   <title>APT Plugin for CHECK_MK</title>
   
     <category term="Debian" label="Debian" />
   
     <category term="Monitoring" label="Monitoring" />
   
     <category term="CheckMK" label="CheckMK" />
   
   <link href="http://sts.ono.at/blog/2010/10/10/checkmk-apt"/>
   <updated>2010-10-10T10:10:00+02:00</updated>
   <id>http://sts.ono.at/blog/2010/10/10/checkmk-apt</id>
   <content type="html">&lt;p&gt;We are using &lt;a href='http://mathias-kettner.de/check_mk.html'&gt;Check_MK&lt;/a&gt; for monitoring at work. It features a quite nice replacement for NRPE agents and automatic Nagios configuration generation.&lt;/p&gt;

&lt;p&gt;I wrote an APT Plugin which will refresh the package cache on every agent, every 60 minutes and check for new Debian upgrades or security updates. Depending on the severity it will return different Nagios status codes:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;OK - No upgrades are available.&lt;/li&gt;

&lt;li&gt;WARNING - Only non-security upgrades are available.&lt;/li&gt;

&lt;li&gt;CRITICAL - Security upgrades are available (might also involve normal upgrades).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;its hosted at &lt;a href='https://github.com/sts/checkmk-apt'&gt;GitHub&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Agent installation&lt;/h2&gt;
&lt;p&gt;Install the python-apt package and copy plugins/apt to your servers. You would properly want to add this to puppet.&lt;/p&gt;
&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='bash'&gt;  aptitude install python-apt
  git clone git://github.com/sts/checkmk-apt.git
  checkmk-apt/plugins/apt /usr/lib/check_mk_agent/plugins/apt
  chmod a+x /usr/lib/check_mk_agent/plugins/apt
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;&lt;h2&gt;Installing on your Nagios Server&lt;/h2&gt;
&lt;p&gt;Copy checks/apt to your checks directory and run a Check_MK inventarize and config update.&lt;/p&gt;
&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='bash'&gt;  git clone git://github.com/sts/checkmk-apt.git
  cp  checkmk-apt/checks/apt /usr/local/share/check_mk/scripts/
  chmod a+x /usr/local/share/check_mk/scripts/apt

  &lt;span class='c'&gt;# Check_MK Inventory+Generate Nagios Configuration&lt;/span&gt;
  check_mk -I alltcp
  check_mk -U
  invoke-rc.d nagios3 restart
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;</content>
 </entry>
 
 <entry xmlns="http://www.w3.org/2005/Atom">
   <title>Puppet+Passenger</title>
   
     <category term="Puppet" label="Puppet" />
   
   <link href="http://sts.ono.at/blog/2010/08/31/debian-puppet-passenger"/>
   <updated>2010-08-31T21:09:00+02:00</updated>
   <id>http://sts.ono.at/blog/2010/08/31/debian-puppet-passenger</id>
   <content type="html">&lt;p&gt;Puppet is a configuration management tool, its been under heavy development for almost five years now. It became a major open source project in the last few years, surrounded by a large community.&lt;/p&gt;

&lt;p&gt;In most of the current environments Puppet Masters will either run on webrick or for the larger environments mongrel was quite standard for a while now.&lt;/p&gt;

&lt;p&gt;But Puppet is as well able to be run from within Apache or Nginx. Then you would be using mod_rails (aka. &lt;a href='http://www.modrails.com/'&gt;Phusion
Passenger&lt;/a&gt;). This solution is known to scale best, but was always a bit bulky to install.&lt;/p&gt;

&lt;p&gt;&lt;img src='/images/post-2010-08-31-debian-puppet-passenger.png' alt='puppet-passenger-diagram' /&gt;&lt;/p&gt;

&lt;p&gt;Debian&amp;#8217;s Puppet package maintainers have prepared the puppetmaster package in Squeeze for an easy installation with mod_rails, so I think this could get the new standard way to install the puppet server.&lt;/p&gt;

&lt;p&gt;&lt;a href='http://lists.debian.org/debian-announce/2010/msg00009.html'&gt;Rescently&lt;/a&gt; the Debian project announced the freeze on its testing branch (codename &amp;#8220;Squeeze&amp;#8221;) which in consequence means that: no more new features will be added and all work will be consentrated on polishing testing up to production level.&lt;/p&gt;

&lt;p&gt;I thought it to be a good time then to prepare the &amp;#8220;Squeeze&amp;#8221; upgrade of my puppet servers and write a short article about it.&lt;/p&gt;
&lt;h2&gt;Install the packages&lt;/h2&gt;&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='bash'&gt;  aptitude install puppetmaster libapache2-mod-passenger
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;&lt;h2&gt;Enable the Apache Modules&lt;/h2&gt;&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='bash'&gt;  a2enmod headers
  a2enmod ssl
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;&lt;h2&gt;Manually change config.ru&lt;/h2&gt;&lt;p class='note'&gt;Update: Meanwhile you can skip this step.&lt;/p&gt;
&lt;p&gt;I had to manually fix the Rackup-file which comes with the squeeze puppetmaster package. This file contains logic for initializing puppetmaster as a rack application. It still tries to initialize &lt;em&gt;puppetmaster&lt;/em&gt;, although the puppet server component was renamed to &lt;em&gt;master&lt;/em&gt;. &lt;br /&gt; There is already a open Debian bug for this, and should get fixed until the final release. &lt;a href='http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=593557'&gt;See bug
 #593557&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Now change /usr/share/puppet/rack/puppetmasterd/config.ru to:&lt;/p&gt;
&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='ruby'&gt;  &lt;span class='c1'&gt;# a config.ru, for use with every rack-compatible webserver.&lt;/span&gt;
  &lt;span class='c1'&gt;# SSL needs to be handled outside this, though.&lt;/span&gt;
  
  &lt;span class='c1'&gt;# if puppet is not in your RUBYLIB:&lt;/span&gt;
  &lt;span class='c1'&gt;# $:.unshift(&amp;#39;/opt/puppet/lib&amp;#39;)&lt;/span&gt;
  
  &lt;span class='vg'&gt;$0&lt;/span&gt; &lt;span class='o'&gt;=&lt;/span&gt; &lt;span class='s2'&gt;&amp;quot;puppetmasterd&amp;quot;&lt;/span&gt;
  &lt;span class='nb'&gt;require&lt;/span&gt; &lt;span class='s1'&gt;&amp;#39;puppet&amp;#39;&lt;/span&gt;
  
  &lt;span class='c1'&gt;# if you want debugging:&lt;/span&gt;
  &lt;span class='c1'&gt;# ARGV &amp;amp;lt;&amp;amp;lt; &amp;quot;--debug&amp;quot;&lt;/span&gt;
  
  &lt;span class='no'&gt;ARGV&lt;/span&gt; &lt;span class='o'&gt;&amp;amp;&lt;/span&gt;&lt;span class='n'&gt;lt&lt;/span&gt;&lt;span class='p'&gt;;&lt;/span&gt;&lt;span class='o'&gt;&amp;amp;&lt;/span&gt;&lt;span class='n'&gt;lt&lt;/span&gt;&lt;span class='p'&gt;;&lt;/span&gt; &lt;span class='s2'&gt;&amp;quot;--rack&amp;quot;&lt;/span&gt;
  &lt;span class='nb'&gt;require&lt;/span&gt; &lt;span class='s1'&gt;&amp;#39;puppet/application/master&amp;#39;&lt;/span&gt;
  &lt;span class='c1'&gt;# we&amp;#39;re usually running inside a Rack::Builder.new {} block,&lt;/span&gt;
  &lt;span class='c1'&gt;# therefore we need to call run *here*.&lt;/span&gt;
  &lt;span class='n'&gt;run&lt;/span&gt; &lt;span class='no'&gt;Puppet&lt;/span&gt;&lt;span class='o'&gt;::&lt;/span&gt;&lt;span class='no'&gt;Application&lt;/span&gt;&lt;span class='o'&gt;[&lt;/span&gt;&lt;span class='ss'&gt;:master&lt;/span&gt;&lt;span class='o'&gt;].&lt;/span&gt;&lt;span class='n'&gt;run&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;&lt;h2&gt;Configure Puppet&lt;/h2&gt;
&lt;p&gt;When you lounch Puppet the first time, it will generate the SSL certificates. Make sure you configured the puppet &lt;em&gt;certname&lt;/em&gt; option to fit your hostname.&lt;/p&gt;

&lt;p&gt;If puppetmaster started before you configured it, you can simply delete the ssl directory (&lt;em&gt;/var/lib/puppet/ssl&lt;/em&gt;) and restart the puppet master. It will regenerate the directory automatically.&lt;/p&gt;

&lt;p&gt;The following represents my puppet configuration in &lt;em&gt;/etc/puppet/puppet.conf&lt;/em&gt;.&lt;/p&gt;
&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='ini'&gt;  &lt;span class='k'&gt;[main]&lt;/span&gt;
  &lt;span class='na'&gt;logdir&lt;/span&gt;&lt;span class='o'&gt;=&lt;/span&gt;&lt;span class='s'&gt;/var/log/puppet&lt;/span&gt;
&lt;span class='s'&gt;  vardir=/var/lib/puppet&lt;/span&gt;
&lt;span class='s'&gt;  ssldir=/var/lib/puppet/ssl&lt;/span&gt;
&lt;span class='s'&gt;  rundir=/var/run/puppet&lt;/span&gt;
&lt;span class='s'&gt;  factpath=$vardir/lib/facter&lt;/span&gt;
&lt;span class='s'&gt;  templatedir=$confdir/templates&lt;/span&gt;
&lt;span class='s'&gt;  &lt;/span&gt;
&lt;span class='s'&gt;  [master]&lt;/span&gt;
&lt;span class='s'&gt;  certname=puppet.ono.at&lt;/span&gt;
&lt;span class='s'&gt;  ssl_client_header=SSL_CLIENT_S_DN&lt;/span&gt;
&lt;span class='s'&gt;  ssl_client_verify_header=SSL_CLIENT_VERIFY      &lt;/span&gt;

  &lt;span class='k'&gt;[agent]&lt;/span&gt;
  &lt;span class='na'&gt;server&lt;/span&gt;&lt;span class='o'&gt;=&lt;/span&gt;&lt;span class='s'&gt;puppet.ono.at&lt;/span&gt;

  &lt;span class='k'&gt;[cert]&lt;/span&gt;
  &lt;span class='na'&gt;autosign&lt;/span&gt;&lt;span class='o'&gt;=&lt;/span&gt;&lt;span class='s'&gt;false&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;p&gt;After the certificates are generated, you should disable the puppetmaster daemon in &lt;em&gt;/etc/default/puppetmaster&lt;/em&gt; by setting $START from yes to &lt;em&gt;no&lt;/em&gt;.&lt;/p&gt;
&lt;h2&gt;Apache Configuration&lt;/h2&gt;
&lt;p&gt;Finally configure an Apache virtual host to listen on port &lt;em&gt;8140&lt;/em&gt; and point it to the ssl certificates generated by puppet. Put the following configuration in &lt;em&gt;/etc/apache2/sites-available/puppetmaster&lt;/em&gt;:&lt;/p&gt;
&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='apache'&gt;  &lt;span class='c'&gt;## Puppetmaster Apache Vhost Configuration&lt;/span&gt;
  
  &lt;span class='c'&gt;## Passenger Limits&lt;/span&gt;
  &lt;span class='nb'&gt;PassengerHighPerformance&lt;/span&gt;   &lt;span class='k'&gt;on&lt;/span&gt;
  &lt;span class='nb'&gt;PassengerMaxPoolSize&lt;/span&gt;       &lt;span class='m'&gt;12&lt;/span&gt;
  &lt;span class='nb'&gt;PassengerPoolIdleTime&lt;/span&gt;    &lt;span class='m'&gt;1500&lt;/span&gt;
  &lt;span class='c'&gt;# PassengerMaxRequests   1000&lt;/span&gt;
  &lt;span class='nb'&gt;PassengerStatThrottleRate&lt;/span&gt; &lt;span class='m'&gt;120&lt;/span&gt;
  &lt;span class='nb'&gt;RackAutoDetect&lt;/span&gt;            &lt;span class='k'&gt;Off&lt;/span&gt;
  &lt;span class='nb'&gt;RailsAutoDetect&lt;/span&gt;           &lt;span class='k'&gt;Off&lt;/span&gt;
  
  &lt;span class='nb'&gt;Listen&lt;/span&gt; &lt;span class='m'&gt;8140&lt;/span&gt;
  
  &lt;span class='nt'&gt;&amp;lt;VirtualHost&lt;/span&gt; &lt;span class='s'&gt;*:8140&lt;/span&gt;&lt;span class='nt'&gt;&amp;gt;&lt;/span&gt;
     &lt;span class='nb'&gt;ServerName&lt;/span&gt; puppet.ono.at
  
     &lt;span class='nb'&gt;SSLEngine&lt;/span&gt; &lt;span class='k'&gt;on&lt;/span&gt;
     &lt;span class='nb'&gt;SSLCipherSuite&lt;/span&gt; SSLv2:-LOW:-EXPORT:RC4+RSA
  
     &lt;span class='nb'&gt;SSLCertificateFile&lt;/span&gt;      &lt;span class='sx'&gt;/var/lib/puppet/ssl/certs/puppet.ono.at.pem&lt;/span&gt;
     &lt;span class='nb'&gt;SSLCertificateKeyFile&lt;/span&gt;   &lt;span class='sx'&gt;/var/lib/puppet/ssl/private_keys/puppet.ono.at.pem&lt;/span&gt;
     &lt;span class='nb'&gt;SSLCertificateChainFile&lt;/span&gt; &lt;span class='sx'&gt;/var/lib/puppet/ssl/ca/ca_crt.pem&lt;/span&gt;
     &lt;span class='nb'&gt;SSLCACertificateFile&lt;/span&gt;    &lt;span class='sx'&gt;/var/lib/puppet/ssl/ca/ca_crt.pem&lt;/span&gt;
  
     &lt;span class='c'&gt;## CRL checking should be enabled; if you have problems with&lt;/span&gt;
     &lt;span class='c'&gt;## Apache complaining about the CRL, disable the next line&lt;/span&gt;
     &lt;span class='nb'&gt;SSLCARevocationFile&lt;/span&gt;     &lt;span class='sx'&gt;/var/lib/puppet/ssl/ca/ca_crl.pem&lt;/span&gt;
     &lt;span class='nb'&gt;SSLVerifyClient&lt;/span&gt;         optional
     &lt;span class='nb'&gt;SSLVerifyDepth&lt;/span&gt;          &lt;span class='m'&gt;1&lt;/span&gt;
     &lt;span class='nb'&gt;SSLOptions&lt;/span&gt;              +StdEnvVars
  
     &lt;span class='c'&gt;## The following client headers allow the same configuration&lt;/span&gt;
     &lt;span class='c'&gt;## to work with Pound.&lt;/span&gt;
     &lt;span class='nb'&gt;RequestHeader&lt;/span&gt; set X-SSL-Subject %{SSL_CLIENT_S_DN}e
     &lt;span class='nb'&gt;RequestHeader&lt;/span&gt; set X-Client-DN %{SSL_CLIENT_S_DN}e
     &lt;span class='nb'&gt;RequestHeader&lt;/span&gt; set X-Client-Verify %{SSL_CLIENT_VERIFY}e
  
     &lt;span class='nb'&gt;RackAutoDetect&lt;/span&gt; &lt;span class='k'&gt;On&lt;/span&gt;
  
     &lt;span class='nb'&gt;DocumentRoot&lt;/span&gt; &lt;span class='sx'&gt;/usr/share/puppet/rack/puppetmasterd/public&lt;/span&gt;
  
     &lt;span class='nt'&gt;&amp;lt;Directory&lt;/span&gt; &lt;span class='s'&gt;&amp;quot;/usr/share/puppet/rack/puppetmasterd&amp;quot;&lt;/span&gt;&lt;span class='nt'&gt;&amp;gt;&lt;/span&gt;
         &lt;span class='nb'&gt;Options&lt;/span&gt; &lt;span class='k'&gt;None&lt;/span&gt;
         &lt;span class='nb'&gt;AllowOverride&lt;/span&gt; &lt;span class='k'&gt;None&lt;/span&gt;
         &lt;span class='nb'&gt;Order&lt;/span&gt; allow,deny
         &lt;span class='nb'&gt;allow&lt;/span&gt; from &lt;span class='k'&gt;all&lt;/span&gt;
     &lt;span class='nt'&gt;&amp;lt;/Directory&amp;gt;&lt;/span&gt;
  &lt;span class='nt'&gt;&amp;lt;/VirtualHost&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;p&gt;Now enable the virtual host configuration, enable all required modules and restart the Apache daemon:&lt;/p&gt;
&lt;div class='highlight'&gt;&lt;pre&gt;&lt;code class='bash'&gt;  a2ensite puppetmaster
  a2enmod header
  a2enmode passenger
  apache2ctl configtest
  apache2ctl restart
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;&lt;h2&gt;Final Step - Test&lt;/h2&gt;
&lt;p&gt;After everything is in place, please test your setup. Open up a web browser and point it to the following Url [adapt the hostname]:&lt;/p&gt;

&lt;p&gt;https://puppet.ono.at:8140&lt;/p&gt;

&lt;p&gt;You should see a line stating:&lt;/p&gt;

&lt;p&gt;&amp;#8220;The environment must be purely alphanumeric, not &amp;#8221;&amp;#8221;&lt;/p&gt;</content>
 </entry>
 
 <entry xmlns="http://www.w3.org/2005/Atom">
   <title>Happy Birthday!</title>
   
   <link href="http://sts.ono.at/blog/2010/08/16/happy-birthday"/>
   <updated>2010-08-16T15:12:00+02:00</updated>
   <id>http://sts.ono.at/blog/2010/08/16/happy-birthday</id>
   <content type="html">&lt;p&gt;I was a bit surprised when I opened my inbox today. Today is my birthday, but today is also Debian&amp;#8217;s Birthday! &lt;a href='http://thank.debian.net/'&gt;thank.debian.net&lt;/a&gt; &lt;br /&gt; Happy Birthday Debian!&lt;/p&gt;</content>
 </entry>
 
 
</feed>

