Provide access to resource usage for processes and nodes

Define two attributes by which applications and/or tools can request
resource usage of processes and nodes. Define a structure for each
case to contain the information, and associated functions for constructing
and destructing those structures.

Signed-off-by: Ralph Castain <[email protected]>

Author: rhc54

Chap_API_Job_Mgmt.tex

@@ -605,7 +605,6 @@ \subsection{Job control attributes}
 When recursively cleaning subdirectories, do not remove the top-level directory (the one given in the cleanup request).
 }
 
-
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 \section{Process and Job Monitoring}

Chap_API_Query.tex

@@ -6,18 +6,18 @@ \chapter{Query Operations}
 
 This chapter presents mechanisms for generalized queries that
 access information about the host environment and the system in general.
-The chapter presents the concept of a query followed by a detailed explanation 
+The chapter presents the concept of a query followed by a detailed explanation
 of the query \acp{API} provided.  The chapter compares the use of these \acp{API} with \refapi{PMIx_Get}.  The chapter concludes with detailed information about how to use
 the query interface to access information about what \ac{PMIx} \acp{API} an implementation supports as well as what attributes each supported \ac{API} supports.
 
 \section{PMIx_Query_info}
 As the level of interaction between applications and the host \ac{SMS} grows, so too does the need for the application to query the \ac{SMS} regarding its capabilities and state information. \ac{PMIx} provides a generalized query interface for this purpose, along with a set of standardized attribute keys to support a range of requests. This includes requests to determine the status of scheduling queues and active allocations, the scope of \ac{API} and attribute support offered by the \ac{SMS}, namespaces of active jobs, location and information about a job's processes, and information regarding available resources.
 
-An example use-case for the \refapi{PMIx_Query_info_nb} \ac{API} is to ensure clean job completion. Time-shared systems frequently impose maximum run times when assigning jobs to resource allocations. To shut down gracefully (e.g., to write a checkpoint before termination) it is necessary for an application to periodically query the resource manager for the time remaining in its allocation. This is especially true on systems for which allocation times may be shortened or lengthened from the original time limit. Many resource managers provide \acp{API} to dynamically obtain this information, but each \ac{API} is specific to the resource manager.  
-\ac{PMIx} supports this use-case by defining an attribute key (\refattr{PMIX_TIME_REMAINING}) that can be used with the \refapi{PMIx_Query_info_nb} interface to obtain the number of seconds remaining in the current job allocation. 
+An example use-case for the \refapi{PMIx_Query_info_nb} \ac{API} is to ensure clean job completion. Time-shared systems frequently impose maximum run times when assigning jobs to resource allocations. To shut down gracefully (e.g., to write a checkpoint before termination) it is necessary for an application to periodically query the resource manager for the time remaining in its allocation. This is especially true on systems for which allocation times may be shortened or lengthened from the original time limit. Many resource managers provide \acp{API} to dynamically obtain this information, but each \ac{API} is specific to the resource manager.
+\ac{PMIx} supports this use-case by defining an attribute key (\refattr{PMIX_TIME_REMAINING}) that can be used with the \refapi{PMIx_Query_info_nb} interface to obtain the number of seconds remaining in the current job allocation.
 
-\ac{PMIx} sometimes provides multiple methods by which an application can obtain information or services.  For this example, 
-note that one could alternatively use the \refapi{PMIx_Register_event_handler} \ac{API} to register for an event indicating incipient job termination, and then use the \refapi{PMIx_Job_control_nb} \ac{API} to request that the host \ac{SMS} generate an event a specified amount of time prior to reaching the maximum run time. 
+\ac{PMIx} sometimes provides multiple methods by which an application can obtain information or services.  For this example,
+note that one could alternatively use the \refapi{PMIx_Register_event_handler} \ac{API} to register for an event indicating incipient job termination, and then use the \refapi{PMIx_Job_control_nb} \ac{API} to request that the host \ac{SMS} generate an event a specified amount of time prior to reaching the maximum run time.
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 \subsection{Query Structure}
@@ -105,7 +105,7 @@ \subsection{\code{PMIx_Query_info}}
 \pasteAttributeItem{PMIX_HOST_ATTRIBUTES}
 \pasteAttributeItem{PMIX_TOOL_ATTRIBUTES}
 
-Note that inclusion of both the \refattr{PMIX_PROCID} directive and either the \refattr{PMIX_NSPACE} or the \refattr{PMIX_RANK} attribute will return a \refconst{PMIX_ERR_BAD_PARAM} result, and that the inclusion of a process identifier must apply to all keys in that \refstruct{pmix_query_t}. Queries for information on multiple specific processes therefore requires submitting multiple \refstruct{pmix_query_t} structures, each referencing one process.  Directives which are not applicable to a key are ignored. 
+Note that inclusion of both the \refattr{PMIX_PROCID} directive and either the \refattr{PMIX_NSPACE} or the \refattr{PMIX_RANK} attribute will return a \refconst{PMIX_ERR_BAD_PARAM} result, and that the inclusion of a process identifier must apply to all keys in that \refstruct{pmix_query_t}. Queries for information on multiple specific processes therefore requires submitting multiple \refstruct{pmix_query_t} structures, each referencing one process.  Directives which are not applicable to a key are ignored.
 
 % Use of pmix_server_query_fn is covered in server interfaces chapter
 \reqattrend
@@ -134,6 +134,8 @@ \subsection{\code{PMIx_Query_info}}
 \pasteAttributeItem{PMIX_QUERY_AUTHORIZATIONS}
 \pasteAttributeItem{PMIX_PROC_PID}
 \pasteAttributeItem{PMIX_PROC_STATE_STATUS}
+\pasteAttributeItem{PMIX_QUERY_PROC_RESOURCE_USAGE}
+\pasteAttributeItem{PMIX_QUERY_NODE_RESOURCE_USAGE}
 
 %%%%
 \descr
@@ -228,7 +230,7 @@ \subsection{\code{PMIx_Query_info_nb}}
 \pasteAttributeItem{PMIX_HOST_ATTRIBUTES}
 \pasteAttributeItem{PMIX_TOOL_ATTRIBUTES}
 
-Note that inclusion of both the \refattr{PMIX_PROCID} directive and either the \refattr{PMIX_NSPACE} or the \refattr{PMIX_RANK} attribute will return a \refconst{PMIX_ERR_BAD_PARAM} result, and that the inclusion of a process identifier must apply to all keys in that \refstruct{pmix_query_t}. Queries for information on multiple specific processes therefore requires submitting multiple \refstruct{pmix_query_t} structures, each referencing one process.  Directives which are not applicable to a key are ignored. 
+Note that inclusion of both the \refattr{PMIX_PROCID} directive and either the \refattr{PMIX_NSPACE} or the \refattr{PMIX_RANK} attribute will return a \refconst{PMIX_ERR_BAD_PARAM} result, and that the inclusion of a process identifier must apply to all keys in that \refstruct{pmix_query_t}. Queries for information on multiple specific processes therefore requires submitting multiple \refstruct{pmix_query_t} structures, each referencing one process.  Directives which are not applicable to a key are ignored.
 
 % Use of pmix_server_query_fn is covered in server interfaces chapter
 \reqattrend
@@ -257,6 +259,8 @@ \subsection{\code{PMIx_Query_info_nb}}
 \pasteAttributeItem{PMIX_QUERY_AUTHORIZATIONS}
 \pasteAttributeItem{PMIX_PROC_PID}
 \pasteAttributeItem{PMIX_PROC_STATE_STATUS}
+\pasteAttributeItem{PMIX_QUERY_PROC_RESOURCE_USAGE}
+\pasteAttributeItem{PMIX_QUERY_NODE_RESOURCE_USAGE}
 
 
 %%%%
@@ -267,7 +271,7 @@ \subsection{\code{PMIx_Query_info_nb}}
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 %% NOTE: This is not used anywhere.  If this is supposed to be returned by
-%% the query API's, it was never indicated.  They currently return 
+%% the query API's, it was never indicated.  They currently return
 %% PMIX_ERR_PARTIAL_SUCCESS
 %%\subsection{Query-specific constants}
 %%\label{api:struct:constants:query}
@@ -350,6 +354,24 @@ \subsection{Query keys}
 %
 \pasteAttributeItem{PMIX_QUERY_PSET_MEMBERSHIP}
 %
+\declareAttributeProvisional{PMIX_QUERY_PROC_RESOURCE_USAGE}{"pmix.qry.pres"}{pmix_proc_t}{
+Return the resource usage statistics for the specified process in a \refstruct{pmix_proc_stats_t} structure.
+If a namespace is combined with \refconst{PMIX_RANK_WILDCARD}, then results for all processes in the given
+job shall be returned in a \refstruct{pmix_data_array_t} of \refstruct{pmix_proc_stats_t} structures.
+OPTIONAL QUALIFIERS: \refattr{PMIX_SESSION_ID} to identify the session of the namespace whose statistics
+are being requested; \refattr{PMIX_NODEID} or \refattr{PMIX_HOSTNAME} to restrict a wildcard request to
+processes on a given node.
+}
+%
+\declareAttributeProvisional{PMIX_QUERY_NODE_RESOURCE_USAGE}{"pmix.qry.nres"}{char*}{
+Return the resource usage statistics for the specified node in a \refstruct{pmix_node_stats_t} structure.
+If no node is specified, then results for all nodes hosting processes with the job of the requestor will be
+returned in a \refstruct{pmix_data_array_t} of \refstruct{pmix_node_stats_t} structures.
+OPTIONAL QUALIFIERS: \refattr{PMIX_SESSION_ID} to identify the session of the job whose node statistics
+are being requested; \refattr{PMIX_NSPACE} or \refattr{PMIX_JOBID} to identify the job whose node usage
+is being requested (if other than the job of the requestor).
+}
+%
 \declareAttribute{PMIX_QUERY_AVAIL_SERVERS}{"pmix.qry.asrvrs"}{pmix_data_array_t*}{
 Return an array of \refstruct{pmix_info_t}, each element itself containing a \refattr{PMIX_SERVER_INFO_ARRAY} entry holding all available data for a server on this node to which the caller might be able to connect.
 }
@@ -384,7 +406,7 @@ \subsection{Query keys}
 \subsection{Query attributes}
 \label{api:struct:attributes:query}
 
-Attributes used to direct behavior of the 
+Attributes used to direct behavior of the
 \refapi{PMIx_Query_info} and \refapi{PMIx_Query_info_nb} \acp{API}:
 
 \declareAttribute{PMIX_QUERY_RESULTS}{"pmix.qry.res"}{pmix_data_array_t}{