Opened 5 years ago

Last modified 4 years ago

#3638 closed change

Refactor hiera('servers') into Puppet class adblockplus::host — at Initial Version

Reported by: matze Assignee: matze
Priority: P2 Milestone:
Module: Infrastructure Keywords:
Cc: fred, fhd Blocked By:
Blocking: Platform: Unknown / Cross platform
Ready: yes Confidential: no
Tester: Unknown Verified working: no
Review URL(s):

https://codereview.adblockplus.org/29338046/
https://codereview.adblockplus.org/29338054/
https://codereview.adblockplus.org/29340915/
https://codereview.adblockplus.org/29340979/
https://codereview.adblockplus.org/29340973/
https://codereview.adblockplus.org/29367240/

Description

The current approach to feed Puppet information about our servers has proven to be quite impractical. Introduced together with the migration to our YAML-based, combined Hiera/ENC solution last year, the use of the hiera("servers") hash setup in base.yaml is quite repetitive, see i.e.:

In addition, this approach does not align well with the Puppet language, especially considering the limitations of the 2.7* version family, but that is an issue for another day.

Background

While managing these in a named hash, available as e.g. hiera("servers"), seems to be a simple way to access the desired information, the common approach using create_resources() to process the returned hash (one of the few tools available in Puppet 2.7) shows various disadvantages:

  • Each use requires an individual Puppet type to be created, and passed on to the create_resources() function, along with the $servers hash. The code and documentation overhead is relatively large, though the results are usually quite practical.
  • The Puppet type must recognize every parameter, even those which are irrelevant to its domain, otherwise the resource creation will result in a Puppet error.
  • The hash key or $title of any hash item has a special meaning as a Puppet namevar, but at the same type any "name" key has no effect when create_resources() is invoked. Instead, the default behavior using $title is applied, which leads to inconsistent results with other approaches (see below).
  • The $dns parameter is optional, by default accumulated by $name and some $domain or $authority value. This is never documented, but implemented in various manifests, templates, and scripts (see below).
  • The $domain or $authority value is not always taken from the same source.

There are other code fragments examining the servers hash, i.e.:

  • Puppet ERB templates
  • Possible hacks with inline templates
  • Ruby functions (not used yet)

For now, the most problematic, most often occurring use-case is a setup where host M needs to recognize each host C[x] of a list of clients C[0..n], like in the examples above, or any other master-slave relationship. More advanced examples include setups where any client C must recognize properties of the master, i.e. the TLS certificate of M (like Puppet itself, or #6 Bacula).

Todo

The improvement with the most effect for now seems to be the introduction of a conventional (meta-) resource any other code fragment may access. This allows for both keeping the current schema for now, whilst establishing a single official resource, featuring commonly accumulated properties:

  • Introduce a Puppet type adblockplus::host aligned with base::explicit_host_record()
  • Introduce parameter $adblockplus::hosts to setup hosts based on hiera_hash() and create_resources()
  • Migrate either of the above workarounds to access the newly created resources instead
  • Remove obsolete code fragments

Finally, we should refactor our puppet_node_classifier.rb script to not read the server information from hosts.yaml, but instead require each host to be present as an individual hosts/$HOSTNAME.yaml file. In practice most hosts require individual configuration anyway, i.e. for persistent encryption keys and monitoring quota, and maintaining host information would become more intuitive.

Change History (0)

Note: See TracTickets for help on using tickets.