Target audience: Oracle SuperCluster customers

The primary objective of this blog post is to provide some related information on this obscure tool to inquisitive users/customers as they might have noticed the osc-interdom service and the namesake package and wondered what is it for at some point.

SuperCluster InterDomain Communcation Tool, osc-interdom, is an infrastructure framework and a service that runs on Oracle SuperCluster products to provide flexible monitoring and management capabilities across SuperCluster domains. It provides means to inspect and enumerate the components of a SuperCluster so that other components can fulfill their roles in managing the SuperCluster. The framework also allows commands to be executed from a control domain to take effect across all domains on the server node (eg., a PDom on M8) and, optionally, across all servers (eg., other M8 PDoms in the cluster) on the system.

SuperCluster Virtual Assistant (SVA), ssctuner, exachk and Oracle Enterprise Manager (EM) are some of the consumers of the osc-interdom framework.

Installation and Configuration

Interdom framework requires osc-interdom package from exa-family repository be installed and enabled on all types of domains in the SuperCluster.

In order to enable communication between domains in the SuperCluster, interdom must be configured on all domains that need to be part of the inter-domain communication channel. In other words, it is not a requirement for all domains in the cluster to be part of the osc-interdom configuration. It is possible to exclude some domains from the comprehensive interdom directory either during initial configuration or at a later time. Also once the interdom directory configuration was built, it can be refreshed or rebuilt any time at will.

Since installing and configuring osc-interdom was automated and made part of SuperCluster installation and configuration processes, it is unlikely that anyone from the customer site need to know or to perform those tasks manually.

# svcs osc-interdom
STATE          STIME    FMRI
online         22:24:13 svc:/site/application/sysadmin/osc-interdom:default

Domain Registry and Command Line Interface (CLI)

Configuring interdom results in a Domain Registry. The purpose of the registry is to provide an accurate and up-to-date database of all SuperCluster domains and their characteristics.

oidcli is a simple command line interface for the domain registry. The oidcli command line utility is located in /opt/oracle.supercluster/bin directory.

oidcli utility can be used to query interdom domain registry for data that is associated with different components in the SuperCluster. Each component maps to a domain in the SuperCluster; and each component is uniquely identified by a UUID.

The SuperCluster Domain Registry is stored on Master Control Domain (MCD). The "master" is usually the first control domain in the SuperCluster. Since the domain registry is on the master control domain, it is expected to run oidcli on MCD to query the data. When running from other domains, option -a must be specified along with the management IP address of the master control domain.

Keep in mind that the data returned by oidcli is meant for other SuperCluster tools that have the ability to interpret the data correctly and coherently. Therefore humans who are looking at the same data may need some extra effort to digest and understand.

eg.,

# cd /opt/oracle.supercluster/bin

# ./oidcli -h
Usage: oidcli [options] dir |  [options]   [...]
           invalidate|get_data|get_value
            , other component ID or 'all'
          (e.g. 'hostname', 'control_uuid') or 'all'
  NOTE: get_value must request single 

Options:
  -h, --help  show this help message and exit
  -p          Output in PrettyPrinter format
  -a ADDR     TCP address/hostname (and optional ',') for connection
  -d          Enable debugging output
  -w W        Re-try for up to  seconds for success. Use 0 for no wait.
              Default: 1801.0.

List all components (domains)

# ./oidcli -p dir
[   ['db8c979d-4149-452f-8737-c857e0dc9eb0', True],
    ['4651ac93-924e-4990-8cf9-83be556eb667', True],
 ..
    ['945696fb-97f1-48e3-aa20-8c8baf198ea8', True],
    ['4026d670-61db-425e-834a-dfc45ff9a533', True]]

List the hostname of all domains

# ./oidcli -p get_data all hostname
db8c979d-4149-452f-8737-c857e0dc9eb0:
{   'hostname': {   'mtime': 1538089861, 'name': 'hostname', 'value': 'alpha'}}
3cfc9039-2157-4b62-ac69-ea3d85f2a19f:
{   'hostname': {   'mtime': 1538174309,
                    'name': 'hostname',
                    'value': 'beta'}}
...

List all available properties for all domains

# ./oidcli -p get_data all all
db8c979d-4149-452f-8737-c857e0dc9eb0:
{   'banner_name': {   'mtime': 1538195164,
                       'name': 'banner_name',
                       'value': 'SPARC M7-4'},
    'comptype': {   'mtime': 1538195164, 'name': 'comptype', 'value': 'LDom'},
    'control_uuid': {   'mtime': 1538195164,
                        'name': 'control_uuid',
                        'value': 'Unknown'},
    'guests': {   'mtime': 1538195164, 'name': 'guests', 'value': None},
    'host_domain_chassis': {   'mtime': 1538195164,
                               'name': 'host_domain_chassis',
                               'value': 'AK00251676'},
    'host_domain_name': {   'mtime': 1538284541,
                            'name': 'host_domain_name',
                            'value': 'ssccn1-io-alpha'},
 ...

Query a specific property from a specific domain

# ./oidcli -p get_data 4651ac93-924e-4990-8cf9-83be556eb667 mgmt_ipaddr
mgmt_ipaddr:
{   'mtime': 1538143865,
    'name': 'mgmt_ipaddr',
    'value': ['xx.xxx.xxx.xxx', 20, 'scm_ipmp0']}

The domain registry is persistent and updated hourly. When accurate and up-to-date is needed, it is recommended to query the registry with --no-cache option.

eg.,

# ./oidcli -p get_data --no-cache 4651ac93-924e-4990-8cf9-83be556eb667 load_average
load_average:
{   'mtime': 1538285043,
    'name': 'load_average',
    'value': [0.01171875, 0.0078125, 0.0078125]}

The mtime attribute in all examples above represent the UNIX timestamp.

Debug Mode

By default, osc-interdom service runs in non-debug mode. Running the service in debug mode enables logging more details to osc-interdom service log.

In general, if osc-interdom service is transitioning to maintenance state, switching to the debug mode may provide few additional clues.

To check if debug mode was enabled, run:

svcprop -c -p config osc-interdom | grep debug

To enable debug mode, run:

svccfg -s sysadmin/osc-interdom:default setprop config/oidd_debug '=' true
svcadm restart osc-interdom

Finally check the service log for debug messages. svcs -L osc-interdom command output points to the location of osc-interdom service log.

Documentation

Similar to SuperCluster resource allocation engine, osc-resalloc, interdom framework is mostly meant for automated tools with little or no human interaction. Consequently there are no references to osc-interdom in SuperCluster Documentation Set.

Related:

• Oracle SuperCluster: A Brief Introduction to osc-resalloc

• Oracle SuperCluster: A Brief Introduction to osc-setcoremem

• Non-Interactive osc-setcoremem

• Oracle Solaris Tools for Locality Observability

Acknowledgments/Credit:

  Tim Cook

Labels: Oracle SuperCluster Solaris LDoms osc-interdom interdom Guest Domains VM

Posted by: Giri Mandalika.    # 11:22 PM Tweet