  <chapter id="ch-cluster-options">
    <title>Cluster Options</title>
    <section id="s-options-special">
      <title>Special Options</title>
      <para>
	The reason for these fields to be placed at the top level instead of with the rest of cluster options is simply a matter of parsing.
	These options are used by the configuration database which is, by design, mostly ignorant of the content it holds.
	So the decision was made to place them in an easy to find location.
      </para>
      <section id="s-options-version">
	<title>Configuration Version</title>
	<para>
	  When a node joins the cluster, the cluster will perform a check to see who has the best configuration based on the fields below.
	  It then asks the node with the highest (<varname>admin_epoch, epoch, num_updates</varname>) tuple to replace the configuration on all the nodes - which makes setting them and setting them correctly very important.
	</para>
	<table frame="all">
	  <title>Configuration Version Properties</title>
	  <tgroup cols="2">
	    <colspec colwidth="20em"/>
            <thead>
              <row>
		<entry>Field</entry>
		<entry>Description</entry>
              </row>
	    </thead>
	    <tbody>
	      <row>
		<entry>admin_epoch</entry>
		<entry>
		  <para>Never modified by the cluster. Use this to make the configurations on any inactive nodes obsolete.</para>
		  <para><emphasis>Never set this value to zero</emphasis>, in such cases the cluster cannot tell the difference between your configuration and the "empty" one used when nothing is found on disk. </para>
		</entry>
              </row>
              <row>
		<entry>epoch</entry>
		<entry>Incremented every time the configuration is updated (usually by the admin)</entry>
              </row>
              <row>
		<entry>num_updates</entry>
		<entry>Incremented every time the configuration or status is updated (usually by the cluster)</entry>
              </row>
	    </tbody>
	  </tgroup>
      </table>
    </section>
    <section id="s-options-validation">
      <title>Other Fields</title>
	<table frame="all">
	  <title>Properties Controling Validation</title>
	  <tgroup cols="2">
            <thead>
              <row>
		<entry>Field</entry>
		<entry>Description</entry>
              </row>
            </thead>
	    <tbody>
	      <row>
		<entry>validate-with</entry>
		<entry>
		  Determines the type of validation being done on the configuration.
		  If set to "none", the cluster will not verify that updates conform the the DTD (nor reject ones that don't). This option can be useful when operating a mixed version cluster during an upgrade.
		</entry>
              </row>
	    </tbody>
	  </tgroup>
      </table>
      </section>
      <section id="s-options-read-only">
	<title>Fields Maintained by the Cluster</title>
	<table frame="all">
	  <title>Properties Maintained by the Cluster</title>
	  <tgroup cols="2">
            <thead>
              <row>
		<entry>Field</entry>
		<entry>Description</entry>
              </row>
            </thead>
	    <tbody>
	      <row>
		<entry>crm-debug-origin</entry>
		<entry>Indicates where the last update came from. Informational purposes only.</entry>
              </row>
              <row>
		<entry>cib-last-written</entry>
		<entry>Indicates when the configuration was last written to disk. Informational purposes only.</entry>
              </row>
              <row>
		<entry>dc-uuid</entry>
		<entry>Indicates which cluster node is the current leader. Used by the cluster when placing resources and determining the order of some events.</entry>
              </row>
              <row>
		<entry>have-quorum</entry>
		<entry>Indicates if the cluster has quorum. If false, this may mean that the cluster cannot start resources or fence other nodes. See no-quorum-policy below.</entry>
              </row>
	    </tbody>
	  </tgroup>
	</table>
	<para>
	  Note that although these fields can be written to by the admin, in most cases the cluster will overwrite any values specified by the admin with the "correct" ones.
	  To change the <varname>admin_epoch</varname>, for example, one would use:
	</para>
	<para><command>cibadmin --modify --crm_xml ‘&lt;cib admin_epoch="42"/>'</command></para>
	<para>A complete set of fields will look something like this:</para>
	<example>
	  <title>An example of the fields set for a cib object</title>
	  <programlisting>
<![CDATA[
 <cib have-quorum="true" validate-with="pacemaker-1.0" admin_epoch="1" epoch="12" num_updates="65"
    dc-uuid="ea7d39f4-3b94-4cfa-ba7a-952956daabee">
]]>
	  </programlisting>
	</example>
      </section>
    </section>
    <section id="s-cluster-options">
      <title>Cluster Options</title>
      <para>Cluster options, as you'd expect, control how the cluster behaves when confronted with certain situations.</para>
      <para>They are grouped into sets and, in advanced configurations, there may be more than one.<footnote>
          <para>This will be described later in the section on <xref linkend="ch-rules"/> where we will show how to have the cluster use different sets of options during working hours (when downtime is usually to be avoided at all costs) than it does during the weekends (when resources can be moved to the their preferred hosts without bothering end users)</para>
	</footnote> For now we will describe the simple case where each option is present at most once.</para>
      <section>
	<title>Available Cluster Options</title>
	<table frame="all">
	  <title>Cluster Options</title>
	  <tgroup cols="3">
            <thead>
              <row>
		<entry>Option</entry>
		<entry>Default</entry>
		<entry>Description</entry>
              </row>
            </thead>
	    <tbody>
	      <row>
		<entry>batch-limit</entry>
		<entry>30</entry>
		<entry>The number of jobs that the TE is allowed to execute in parallel. The "correct" value will depend on the speed and load of your network and cluster nodes.</entry>
              </row>
              <row>
		<entry>no-quorum-policy</entry>
		<entry>stop</entry>
		<entry>
		  What to do when the cluster does not have quorum.
		  Allowed values:
		  <itemizedlist>
		    <listitem><para>ignore - continue all resource management</para></listitem>
		    <listitem><para>freeze - continue resource management, but don't recover resources from nodes not in the affected partition</para></listitem>
		    <listitem><para>stop - stop all resources in the affected cluster parition</para></listitem>
		    <listitem><para>suicide - fence all nodes in the affected cluster partition</para></listitem>
		  </itemizedlist>
		</entry>
              </row>
              <row>
		<entry>symmetric-cluster</entry>
		<entry>TRUE</entry>
		<entry>Can all resources run on any node by default?</entry>
              </row>
              <row>
		<entry>stonith-enabled</entry>
		<entry>TRUE</entry>
		<entry>
		  <para>Should failed nodes and nodes with resources that can't be stopped be shot? If you value your data, set up a STONITH device and enable this.</para>
		  <para>If true, or unset, the cluster will refuse to start resources unless one or more STONITH resources have been configured also.</para>
		</entry>
              </row>
              <row>
		<entry>stonith-action</entry>
		<entry>reboot</entry>
		<entry>Action to send to STONITH device. Allowed values: reboot, poweroff.</entry>
              </row>
              <row>
		<entry>cluster-delay</entry>
		<entry>60s</entry>
		<entry>Round trip delay over the network (excluding action execution). The "correct" value will depend on the speed and load of your network and cluster nodes.</entry>
              </row>
              <row>
		<entry>stop-orphan-resources</entry>
		<entry>TRUE</entry>
		<entry>Should deleted resources be stopped</entry>
              </row>
              <row>
		<entry>stop-orphan-actions</entry>
		<entry>TRUE</entry>
		<entry>Should deleted actions be cancelled</entry>
              </row>
              <row>
		<entry>start-failure-is-fatal</entry>
		<entry>TRUE</entry>
		<entry>When set to FALSE, the cluster will instead use the resource's failcount and value for resource-failure-stickiness</entry>
              </row>
              <row>
		<entry>pe-error-series-max</entry>
		<entry>-1 (all)</entry>
		<entry>The number of PE inputs resulting in ERRORs to save. Used when reporting problems.</entry>
              </row>
              <row>
		<entry>pe-warn-series-max</entry>
		<entry>-1 (all)</entry>
		<entry>The number of PE inputs resulting in WARNINGs to save. Used when reporting problems.</entry>
              </row>
              <row>
		<entry>pe-input-series-max</entry>
		<entry>-1 (all)</entry>
		<entry>The number of "normal" PE inputs to save. Used when reporting problems.</entry>
              </row>
	    </tbody>
	  </tgroup>
	</table>
	<para>You can always obtain an up-to-date list of cluster options, including their default values by running the pengine metadata command.</para>
      </section>
      <section>
	<title>Querying and Setting Cluster Options</title>
	<para>
	  Cluster options can be queried and modified using the crm_attribute tool.
	  To get the current value of cluster-delay, simply use:
	</para>
	<para><command>crm_attribute --attr-name cluster-delay --get-value</command></para>
	<para>which is more simply written as</para>
	<para><command>crm_attribute --get-value -n cluster-delay</command></para>
	<para>If a value is found, the you'll see a result such as this</para>
	<screen>
 <command> # crm_attribute --get-value -n cluster-delay</command>
 name=cluster-delay value=60s</screen>
	<para>However if no value is found, the tool will display an error:</para>
	<screen>
 <command># crm_attribute --get-value -n clusta-deway</command>
 name=clusta-deway value=(null)
 Error performing operation: The object/attribute does not exist</screen>
	<para>To use a different value, eg. 30s, simply run:</para>
	<para><command>crm_attribute --attr-name cluster-delay --attr-value 30s</command></para>
	<para>To go back to the cluster's default value, you can then delete the value with:</para>
	<para><command>crm_attribute --attr-name cluster-delay --delete-attr</command></para>
      </section>
      <section>
	<title>When Options are Listed More Than Once</title>
	<para>If you ever see something like the following, it means that the option you're modifying is present more than once.</para>
	<example>
	  <title>Deleting an option that is listed twice</title>
	  <screen>
 <command># crm_attribute --attr-name batch-limit --delete-attr</command>
 Multiple attributes match name=batch-limit in crm_config:
 Value: 50          (set=cib-bootstrap-options, id=cib-bootstrap-options-batch-limit)
 Value: 100         (set=custom, id=custom-batch-limit)
 Please choose from one of the matches above and supply the 'id' with --attr-id</screen>
	</example>
	<para>In such cases follow the on-screen instructions to perform the requested action.
To determine which value is currently being used by the cluster, please refer to the the section on <xref linkend="ch-rules"/>.</para>
      </section>
    </section>
  </chapter>
