clusterinfo

1. Module Description
2. Setup
   1. Requirements
   2. Config File Layout
   3. Required Top Level Variables
3. Usage
   1. Keys in clusterinfo hiera and their sources
4. Limitations
5. Development

Module Description

clusterinfo supplies a central source of information for several modules developed at ZIT-RLP. Configuration files are grouped by customer, application, operation mode and cluster. The module supplies a custom hiera backend supplying nodes with their respective config, e.g. a database server with all database parameters of all clusters in the DB server's operation mode.

A cluster in this context is a collection of nodes which provide different internal services in order to provide one user visible service, e.g. a web server, a database server and several application servers providing one web accessible application.

The module was originally developed to support HIS QIS and HISinOne.

Setup

Requirements

clusterinfo needs those modules:

• puppetlabs-stdlib
• zitrlp-netinfo (similar module concerning network config) The module accesses hiera data of other modules. The modules don't have to be installed, but the keys named in the corresponding clusterinfo key must be filled in order to get data from clusterinfo.

Config File Layout

Config files are stored in ${CODEDIR}/environments/${ENVIRONMENT}/data/, e.g. /etc/puppetlabs/code/environments/productions/data. Inside this directory, the following structure is expected:

+-- nodes
|   +-- node1.yaml
|   +-- node2.yaml
+-- cluster
    +-- CUST
	|   +-- CUST_hs.yaml
	|   +-- some_app
	|   |   +-- some_app_application.yaml
	|   |   +-- NONPROD
	|   |   |   +-- CUST_opmode.yaml
	|   |   |   +-- CUSTTEST01.yaml
	|   |   |   +-- CUSTDEVL01.yaml
	|   |   +-- PROD
	|   |       +-- CUST_opmode.yaml
	|   |       +-- CUSTPROD01.ymal
    |   +-- some_other_app
	|   +-- [...]
	+-- OCST
	    +-- [...]

In this structure, these naming restrictions apply:

• Customers names are four upper case characters, e.g. CUST.
• In the customer's directory, ${customer}_hs.yaml, e.g. CUST_hs.yaml, customer specific config is stored.
• Application directories are stored within the customer directories. There is no restriction for the application name.
• Within the application directory, ${customer}_application.yaml, e.g. CUST_application.yaml contains application specific config.
• Operation mode (opmode) config is stored in the respective application directory. Valid opmodes are PROD and NONPROD.
• In the opmode directory, ${customer}_opmode.yalm, e.g. CUST_opmode.yaml, contains opmode specific config.
• Cluster configs are stored in their respective opmode directory. Their name starts with the customer name, followed by four upper case characters ('stage') and two numbers ('nodenum'), e.g. CUSTTEST01.yaml. Node names must follow this scheme for some keys:
• Application Servers: #{customer}-#{application}-#{stage}-app-#{nodenum}.yaml, all lower case
• Required for
  • clusterinfo::web::node_clusternames
  • clusterinfo::web::worker_node_by_cluster If these keys are not to be used, node naming is not restricted.

Required Top Level Variables

In order to select the cluster for a node, some variables must be supplied, e.g. by an external node classifier (ENC):

• hs (customer)
• application
• opmode
• nodenum

Usage

clusterinfo works by collecting keys in all layers up to the cluster itself and constructing keys in the clusterinfo namespace from them.

Keys

clusterinfo::bi_sql_tasks

@todo mha

Data Structure: Hash

{ 
	$cluster => {
		pre_sql_cron_term,
		pre_sql_steps,
		post_sql_cron_term,
		post_sql_steps,
		hiodb=>{
			database,
			user
		},
		eduetldb=>{
			database,
			user
		}
	}
}

Input Keys:

• hisinone::bi_update
• qisserver::dbconnections

Transformation:

clusterinfo::cluster_list

List of clusters in the current customer/application/opmode hierarchy.

Input Keys: none

Transformation: On iterating over the config directory, add each processed cluster to this list.

clusterinfo::database_list

List of databases.

Data Structure: Array of database names

Input Keys:

• zit_interface::dbconnections
• qisserver::dbconnections

Transformation: Merge of input keys, iteration over the result with $input for each DB connection: Put $input['database'] in Array, i.e. same as keys of clusterinfo::databases.

clusterinfo::databases

Databases with their corresponding owner.

Data Structure: Hash {database_name => owner, ...}

Input Keys:

• zit_interface::dbconnections
• qisserver::dbconnections

Transformation: Merge of input keys, iteration over the result with $input for each DB connection: $input['database'] is used as key, $input['user'] as value.

clusterinfo::database_with_schemas

Mapping of databases to their default schema, owner and connection name.

Data Structure: Hash { database_name => {schema, owner, connection_name}}

Input Keys:

• zit_interface::dbconnections
• qisserver::dbconnections

Transformation: Merge of input keys, iteration over the result with $input for each DB connection and $name for DB connection's name: Use $input['database'] as key, $input['user'] as owner and $name as connection_name. Map connection_name to schema:

clusterinfo::db_owner_roles

Users which own at least one database.

Data Structure: User resource: Hash { $username => { ensure => present, update_password => true, password_hash => $password, username => $username }}

Input Keys:

• zit_interface::dbconnections
• qisserver::dbconnections

Transformation: Merge of input keys, iteration over the result with $input for each DB connection: $input['user'] is used as $username, $input['password'] as $password. password_hash is generated as 'md5'+MD5_hex_digest(password+$user).

Assertions:

• Fail if a user occurs more than once with non-matching passwords.

clusterinfo::postgresql_admins

List of PostgreSQL admin users.

Data Structure: Hash, merge of customer, opmode, application and each cluster's zit_postgresql::admins.

Input Keys:

• zit_postgresql::admins

Transformation: Iterate over applying customer, application, opmode and contained clusters and merge each occurrence of zit_postgreqsl::admins to result.

clusterinfo::postgresql_users

Data Structure: Hash, merge of customer, opmode, application and each cluster's zit_postgresql::users with sanitized permissions.

Input Keys:

• zit_postgresql::users

Transformation: Iterate over applying customer, application, opmode and contained clusters and merge each occurrence of zit_postgreqsl::users to result. Additionally check if there is a permissions key for each user. If so, apply the according permission and remove key permissions from the user:

assertions:

• if permissions is set to an invalid value, fail with error.

clusterinfo::web::access_rules_services2

data structure: hash:

{
	$cluster  => "content of qisserver::access_rules_services2"
	$cluster+$nodetype => "content of qisserver::access_rules_services2::$nodetype"
}

input keys:

• qisserver::access_rules_services2

transformation: merge qisserver::access_rules_services2 from customer/application/opmode and all applying clusters. If entries for additional node types exist, they are merged into the corresponding node type entry. Otherwise, the node type entry is a copy of the cluster entry.

clusterinfo::web::allow_ldap_groups

data structure: hash:

{
	$cluster  => "content of qisserver::access_rules_services2"
	$cluster+$nodetype => "content of qisserver::access_rules_services2::$nodetype"
}

input keys:

• qisserver::access_rules_services2

transformation: merge qisserver::access_rules_services2 from customer/application/opmode and all applying clusters. If entries for additional node types exist, they are merged into the corresponding node type entry. Otherwise, the node type entry is a copy of the cluster entry.

clusterinfo::web::allow_nets

data structure: hash:

{
	$cluster  => "content of qisserver::allow_nets"
	$cluster+$nodetype => "content of qisserver::allow_nets::$nodetype"
}

input keys:

• qisserver::allow_nets

transformation: merge qisserver::allow_nets from customer/application/opmode and all applying clusters. If entries for additional node types exist, they are merged into the corresponding node type entry. Otherwise, the node type entry is a copy of the cluster entry.

clusterinfo::web::allow_nets_regex

data structure: hash:

{
	$cluster  => "content of qisserver::allow_nets_regex"
	$cluster+$nodetype => "content of qisserver::allow_nets_regex::$nodetype"
}

input keys:

• qisserver::allow_nets_regex

transformation: merge qisserver::allow_nets_regex from customer/application/opmode and all applying clusters. If entries for additional node types exist, they are merged into the corresponding node type entry. Otherwise, the node type entry is a copy of the cluster entry.

clusterinfo::web::cluster_url_host_overrides

data structure: hash:

{
	$cluster  => "content of qisserver::url_hostparts"
}

input keys:

• qisserver::url_hostparts

transformation: merge qisserver::url_hostparts from customer/application/opmode and all applying clusters.

clusterinfo::web::domains_by_cluster

data structure: hash:

{
	$cluster  => "content of qisserver::domains"
}

input keys:

• qisserver::domains

transformation: merge qisserver::domains from customer/application/opmode and all applying clusters.

clusterinfo::web::ignore_maintenance_nets

data structure: hash:

{
	$cluster  => "content of qisserver::ignore_maintenance_nets"
	$cluster+$nodetype => "content of qisserver::ignore_maintenance_nets::$nodetype"
}

input keys:

• qisserver::ignore_maintenance_nets

transformation: merge qisserver::ignore_maintenance_nets from customer/application/opmode and all applying clusters. If entries for additional node types exist, they are merged into the corresponding node type entry. Otherwise, the node type entry is a copy of the cluster entry.

clusterinfo::web::ignore_maintenance_nets_regex

data structure: hash:

{
	$cluster  => "content of qisserver::ignore_maintenance_nets_regex"
	$cluster+$nodetype => "content of qisserver::ignore_maintenance_nets_regex::$nodetype"
}

input keys:

• qisserver::ignore_maintenance_nets_regex

transformation: merge qisserver::ignore_maintenance_nets_regex from customer/application/opmode and all applying clusters. If entries for additional node types exist, they are merged into the corresponding node type entry. Otherwise, the node type entry is a copy of the cluster entry.

clusterinfo::web::node_clusternames

Data Structure: Hash:

{
	nodename => cluster
}

Input Keys:

• qisserver::hsclustermap

Transformation: Iterate over all known clusters for this customer/application/opmode and build expected host name. Iterate over all node configs and add existing nodes to list.

clusterinfo::web::repos

Data Structure: Hash,

{
	$cluster => $copy_of_zit_repo::branches,
	$cluster+$nodetype => $copy_of_zit_repo::branches_of_cluster,
}

Input Keys:

• zit_repos::branches

Transformation: zit_repo::branches is copied to this key for each cluster. If a non NODE-type node is found, its cluster's zit_repo::branches is duplicated to the key $nodetype.

clusterinfo::web::shibboleth

Data Structure: Hash

{
	default => config_hash,
	cluster => {
		$cluster1 => config_hash,
		$cluster2 => config_hash,
		...
	},
	odbc_used => (true|false),
	defaultcluster => $cluster_name_of_default,
	metadata_provider_config_files => [file1, file2,...],
	metadata_provider_cert_files => [file1, file2,...]
}

`config_hash` contains these keys:
* entity_id
* application_id
* name
* key
* cert
and may contain:
* `entity_id_sso`
* `metadata_provider_uri`
* `metadata_provider_file_path`
* `metadata_provider_valid_until`
* `metadata_provider_cert_file`
* `default_application_id`

Input Keys:

• shibboleth
• clusterinfo::web::urls

Transformation:

• The default config is set to the merge of customer/application/opmode-lookups of key shibboleth. If there are any options which are also set by all clusters, they are removed from the default config.
• To fill the clusters key, all clusters for this customer/application/opmode are iterated:
  • $fqdn is either set to shibboleth/primary_fqdn of present or to the first system entry of the current cluster's URL list (clusterinfo::web::urls).
  • if is_default is true, application_id is set to 'default', otherwise application_id is set to $fqdn
  • cert is set to ${fqdn}.cert.pem
  • entity_id is set to https://${fqdn}/shibbo
  • key is set to ${fqdn}.key.pem
  • name is set to $fqdn
  • For each optional key listed above, the key is added to the cluster if it deviates from the default config, otherwise it is omitted. If metadata_provider_file_path or metadata_provider_cert_file are encountered, the values are also added to metadata_provider_config_files or metadata_provider_cert_files, respectively.
• If there is a cluster with is_default set, its name is stored in defaultcluster.
• If there is any cluster with shibboleth/odbc_store set, odbc_used is set to true and the cluster's shibbolet/odbc_store is copied to the cluster's config in the result of this lookup.
• If shibbolet/additional_nodetypes is set, the cluster's base config is copied to cluster/${cluster}${nodetype}. In this copy, $fqdn` as used above is replaced by the additional node type's value. Assertions
• If there are two clusters with is_default==true, the module fails with an according error.

clusterinfo::web::untrusted_domains_by_cluster

data structure: hash:

{
	$cluster  => "content of qisserver::untrusted_domains"
}

input keys:

• qisserver::untrusted_domains

transformation: merge qisserver::untrusted_domains from customer/application/opmode and all applying clusters.

clusterinfo::web::urls

Data Structure: Hash

{
	$cluster => {
		'system'   => [url1, url2,...],
		'untrusted'=> [url1, url2,...]
	}
}

Input Keys:

• clusterinfo::web::worker_node_by_cluster
• qisserver::domains
• qisserver::nodetype_host_map
• qisserver::sysnameurlmap
• qisserver::untrusted_domains
• qisserver::url_hostparts

Transformation:

• If qisserver::url_hostparts contains values, use it as list of host-parts. Otherwise, use
  information from qisserver::sysnameurlmap to build host parts.
• Iterate over all domains in qisserver::domains and prepend host parts. If qisserver::sysnameurlmap has basedomain_url==true or qisserver::url_hostparts contains GENERATE_BASEDOMAIN, also generate a URL for the domain without a host part. If clusterinfo::web::worker_node_by_cluster contains additional workers, i.e. application servers of Type ~= 'node', also generate URLs based on qisserver::nodetype_host_map. URLs generated in this step are put into the 'system' part of the cluster's URL hash.
• To generate the 'untrusted' part, use the same host parts generated above and create an entry for each domain in qisserver::untrusted_domains. The list of untrusted domain for non-default node types is empty.

clusterinfo::web::worker_node_by_cluster

Map of load balance worker to worker and hostname for apache's mod_ajp.

Data Structure: Hash

{
	lb_workername => {
		worker => hostname
	}
}

Input Keys:

• clusterinfo::web::node_clusternames

Transformation: Iterade over clusterinfo::web::node_clusternames, if node type is node, the cluster name is used as lb_workername otherwise, lb_workername is set to ${clustername}${nodetype}. Workers are added as ${clustername}${nodetype}${nodenum}.

Limitations

• The module is only tested agains ZIT-RLP's ENC, available at https://software.service.zit-rlp.de.
• Parts of the module may work even without strictly following the naming scheme, for full functuallity, the naming scheme must be applied.