Node Acceptance and Initialization Guide

This document will provide details about the process of accepting and initializing nodes via Skyring REST API.

####Storing Un Managed Node:# The prerequisite for accepting/initializing node is salt-minion(node) has to be pointed to salt-master(Skyring-Server). This can be achieved with the help of ceph installer. Use the following command to achieve this :

curl http://<server hostname>:8181/setup/ | bash

When salt-master receives the salt-minion intimation then it will notify the Skyring – Server. When Skyring – Server Receives the intimation it will store the hostname and fingerprint of the corresponding minion in db and state of the node is marked as unknown. This node is also called as un-managed node.

REST API  for displaying un-managed node is :
	GET – http://server-ip/api/v1/unmanaged_nodes

(In Skyring unaccepted nodes are also called as un-managed nodes / discovered nodes)

####Accepting Node :# For accepting node hostname and fingerprint of the node is required.

REST API  for accept node is :
	POST – http://server-ip/api/v1/unmanaged_nodes/{hostname}/accept
	BODY - { "hostname": "dhcpXX-YYY.lab.eng.blr.redhat.com",
			     "fingerprint" : "nodes-finger-print" }		

When accept request is sent, The Skyring – Server will receive the request and invoke the corresponding function, The function will create asynchronous task for accepting the node. Because of asynchronous task that accepting nodes process happens in parallel. Accepting node happens locally (in server), it does not involve any interaction with the minions(nodes).

(Note : Acceptance is possible even when the node (also called minion) is in down state)

If the node is accepted then it is moved from unknown state to initializing state.

REST API  for displaying accepted nodes is :
	GET – http://server-ip/api/v1/nodes

Skyring - Server handles the job of accepting node as a task. The status of task is marked accordingly depending on how it ended.

After the node is moved to initializing state, every 30 seconds once Skyring-Server will monitor whether initialization is done successfully for next 10 minutes. Before 10 minutes If initialization is successful then the node is moved from initializing state to active state or else Skyring – Server will trigger the initialization process once again and check whether node is initialized, if it is yes then move the node from initializing state to active otherwise it moves to failed state.

####Initializing Node :# After the node is accepted successfully, salt-minion will send a signal to server saying it is ready. This signal will trigger the initialization process.

( Note : Minion will send that particular signal only once. Sometimes salt-minion can't send the signal because of some network issue or change of minion-id or minion is down, Because of this reason after acceptance initialization may not happen)

#####Steps Involved In Initialization Process :#

1. The Skyring – server will try to ping the corresponding node, if the ping is successful then go further otherwise the node is moves to failed state.
2. Execute salt-commands successfully and store all details about the nodes in db successfully.
3. Monitoring related configurations are made on the node.
4. Storaged and skynet service are started.
5. Move node state from Initializing to Active.

#####Possible Causes for Initialization Not Happening :#

1. Network Connection Failure : During initialization if the network connection fails then Skyring - Server cannot communicate with minion, it will leads to failed initialization process.

• Solution : If the network issue is solved before node goes to failed state then initialization will done automatically. Otherwise use re-initialization option to trigger initialization process manually.

2. Salt-minion Down : If the salt-minion is down then the Skyring - Server cannot communicate, it will lead to failed initialization process.

• Solution : Bring the salt-minion to up state using command service salt-minion start before node goes to failed state then initialization will be done automatically. Otherwise use re-initialization option to trigger initialization process manually.

3. Minion-id Changing : Sometimes minion-id may change because of rebooting nodes.

• Solution : Clear salt-minion keys and start the minion server using command "service salt-minion restart" and then try using re-initialization option.

4. Skyring-Server Restart : During initialization restarting the Skyring-server will interrupt the initialization process.

• Solution : Re-initialization is started automatically when Skyring-Server comes up.

####Re-Initializing Node :# Once the node is marked as failed then it needs user’s intervention to fix it.Because the node sends signal to initialize only once. With the help of re-initialization option we can trigger the initialization process manually any number of times when the node is in failed state.

REST API  for re-initializing nodes is :
	POST – http://server-ip/api/v1/nodes/{hostname}/actions
	BODY - {“action” : “reinitialize”}

#####Steps Involved In Initialization Process :#

1. When re-initialization request is given, Skyring – Server will try to ping the corresponding node, if the ping is successful then go further step otherwise the node is moves to failed state.
2. Execute salt-commands successfully and store all details about the nodes in db successfully.
3. Move node state from initializing to active.