Add a step to check_cname task to delete cname if already exists

[Mierdin]

This is a common failure scenario and it's something we could easily handle in the workflow.

 ❯ ssh oswaltm@10.0.4.27                                                                                                                                                                                                   [09:53:51]
Welcome to Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-1020-aws x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

28 packages can be updated.
0 updates are security updates.


*** System restart required ***
Last login: Tue Jul 18 17:13:34 2017 from 10.0.0.25
oswaltm@st2cicd007:~$ st2 login oswaltm
Password:
Logged in as oswaltm

Note: You didn't use --write-password option so the password hasn't been stored in the client config and you will need to login again in 24 hours when the auth token expires.
As an alternative, you can run st2 login command with the "--write-password" flag, but keep it mind this will cause it to store the password in plain-text in the client config file (~/.st2/config).
oswaltm@st2cicd007:~$ st2 execution get 596f82007f769004bdb02383
id: 596f82007f769004bdb02383
action.ref: st2ci.st2_pkg_e2e_test
parameters:
  distro: RHEL6
  enterprise: true
  enterprise_key: dd7b8714ed903cb58754a6860456144638ecc0ad786d6294
  hostname: st2-ent-pkg-stable-el6
  pkg_env: production
  release: stable
  version: 2.3.1
status: failed (67s elapsed)
result_task: notify_failure
result: - exit_code: 0
  result:
    channel: C02FUU8MT
    message:
      attachments:
      - color: FF0000
        fallback: '[st2ci.st2_pkg_e2e_test: FAILED]'
        id: 1
        text: "COMMIT: n/a
HOSTNAME: <http://st2-ent-pkg-stable-el6.uswest2.stackstorm.net|st2-ent-pkg-stable-el6.uswest2.stackstorm.net>
DISTRO: RHEL6
ENTERPRISE: True
RELEASE: production stable
VERSION: 2.3.1
VERSION INSTALLED:
	not installed
BOOTSTRAP: <https://raw.githubusercontent.com/StackStorm/st2-packages/master/scripts/st2_bootstrap.sh>"
        title: '[st2ci.st2_pkg_e2e_test: FAILED]'
        title_link: https://st2cicd/#/history/596f82007f769004bdb02383/general
      bot_id: B0CGVJCTV
      text: ''
      ts: '1500480031.478159'
      type: message
      user: U0CGXP6N9
    ok: true
    ts: '1500480031.478159'
  stderr: 'st2.actions.python.SlackAction: INFO     username=None&attachments=%5B%7B%22fallback%22%3A+%22%5Bst2ci.st2_pkg_e2e_test%3A+FAILED%5D%22%2C+%22title%22%3A+%22%5Bst2ci.st2_pkg_e2e_test%3A+FAILED%5D%22%2C+%22title_link%22%3A+%22https%3A%2F%2Fst2cicd%2F%23%2Fhistory%2F596f82007f769004bdb02383%2Fgeneral%22%2C+%22text%22%3A+%22COMMIT%3A+n%2Fa%5CnHOSTNAME%3A+st2-ent-pkg-stable-el6.uswest2.stackstorm.net%5CnDISTRO%3A+RHEL6%5CnENTERPRISE%3A+True%5CnRELEASE%3A+production+stable%5CnVERSION%3A+2.3.1%5CnVERSION+INSTALLED%3A%5Cn%5Ctnot+installed%5CnBOOTSTRAP%3A+https%3A%2F%2Fraw.githubusercontent.com%2FStackStorm%2Fst2-packages%2Fmaster%2Fscripts%2Fst2_bootstrap.sh%22%2C+%22color%22%3A+%22%23FF0000%22%7D%5D&unfurl_links=None&link_names=None&unfurl_media=None&parse=None&token=xoxb-12575788757-V4pFhpQvNHP9IF8eTodN5mTK&text=&icon_emoji=None&as_user=None&icon_url=None&channel=%23thunderdome

    '
  stdout: ''
- exit_code: 0
  result:
    channel: C029K4CBU
    message:
      attachments:
      - color: FF0000
        fallback: '[st2ci.st2_pkg_e2e_test: FAILED]'
        id: 1
        text: "COMMIT: n/a
HOSTNAME: <http://st2-ent-pkg-stable-el6.uswest2.stackstorm.net|st2-ent-pkg-stable-el6.uswest2.stackstorm.net>
DISTRO: RHEL6
ENTERPRISE: True
RELEASE: production stable
VERSION: 2.3.1
VERSION INSTALLED:
	not installed
BOOTSTRAP: <https://raw.githubusercontent.com/StackStorm/st2-packages/master/scripts/st2_bootstrap.sh>"
        title: '[st2ci.st2_pkg_e2e_test: FAILED]'
        title_link: https://st2cicd/#/history/596f82007f769004bdb02383/general
      bot_id: B0CGVJCTV
      text: ''
      ts: '1500480030.207967'
      type: message
      user: U0CGXP6N9
    ok: true
    ts: '1500480030.207967'
  stderr: 'st2.actions.python.SlackAction: INFO     username=None&attachments=%5B%7B%22fallback%22%3A+%22%5Bst2ci.st2_pkg_e2e_test%3A+FAILED%5D%22%2C+%22title%22%3A+%22%5Bst2ci.st2_pkg_e2e_test%3A+FAILED%5D%22%2C+%22title_link%22%3A+%22https%3A%2F%2Fst2cicd%2F%23%2Fhistory%2F596f82007f769004bdb02383%2Fgeneral%22%2C+%22text%22%3A+%22COMMIT%3A+n%2Fa%5CnHOSTNAME%3A+st2-ent-pkg-stable-el6.uswest2.stackstorm.net%5CnDISTRO%3A+RHEL6%5CnENTERPRISE%3A+True%5CnRELEASE%3A+production+stable%5CnVERSION%3A+2.3.1%5CnVERSION+INSTALLED%3A%5Cn%5Ctnot+installed%5CnBOOTSTRAP%3A+https%3A%2F%2Fraw.githubusercontent.com%2FStackStorm%2Fst2-packages%2Fmaster%2Fscripts%2Fst2_bootstrap.sh%22%2C+%22color%22%3A+%22%23FF0000%22%7D%5D&unfurl_links=None&link_names=None&unfurl_media=None&parse=None&token=xoxb-12575788757-V4pFhpQvNHP9IF8eTodN5mTK&text=&icon_emoji=None&as_user=None&icon_url=None&channel=%23stackstorm

    '
  stdout: ''
start_timestamp: 2017-07-19T16:00:00.055692Z
end_timestamp: 2017-07-19T16:01:07.076716Z
+-----------------------------+------------------------+------------------+------------------------+-------------------------------+
| id                          | status                 | task             | action                 | start_timestamp               |
+-----------------------------+------------------------+------------------+------------------------+-------------------------------+
|   596f82017f769051b0bbe8c6  | succeeded (0s elapsed) | get_webui_server | core.noop              | Wed, 19 Jul 2017 16:00:01 UTC |
| + 596f82027f769051b0bbe8ce  | failed (26s elapsed)   | create_vm        | st2cd.create_vm_role   | Wed, 19 Jul 2017 16:00:02 UTC |
|    596f82047f769051b0bbe8dd | succeeded (4s elapsed) | get_subnet_id    | st2.kv.get             | Wed, 19 Jul 2017 16:00:04 UTC |
|    596f82087f769051b0bbe8e6 | succeeded (3s elapsed) | get_ami          | st2.kv.get             | Wed, 19 Jul 2017 16:00:08 UTC |
|    596f820b7f769051b0bbe8ee | succeeded (4s elapsed) | check_cname      | aws.r53_zone_get_cname | Wed, 19 Jul 2017 16:00:11 UTC |
|   596f82027f769051b0bbe8d0  | succeeded (4s elapsed) | notify_start     | slack.chat.postMessage | Wed, 19 Jul 2017 16:00:02 UTC |
|   596f821c7f769051b0bbe8f7  | succeeded (3s elapsed) | notify_failure   | slack.chat.postMessage | Wed, 19 Jul 2017 16:00:28 UTC |
|   596f821c7f769051b0bbe8f8  | succeeded (3s elapsed) | notify_failure   | slack.chat.postMessage | Wed, 19 Jul 2017 16:00:28 UTC |
+-----------------------------+------------------------+------------------+------------------------+-------------------------------+
oswaltm@st2cicd007:~$ st2 execution get 596f820b7f769051b0bbe8ee
id: 596f820b7f769051b0bbe8ee
status: succeeded (4s elapsed)
parameters:
  name: st2-ent-pkg-stable-el6.uswest2.stackstorm.net
  zone: uswest2.stackstorm.net
result:
  exit_code: 0
  result:
  - alias_dns_name: null
    alias_evaluate_target_health: null
    alias_hosted_zone_id: null
    failover: null
    health_check: null
    identifier: null
    name: st2-ent-pkg-stable-el6.uswest2.stackstorm.net.
    region: null
    resource_records:
    - ip-10-0-3-206.us-west-2.compute.internal.
    ttl: '60'
    type: CNAME
    weight: null
  stderr: ''
  stdout: ''

[arm4b]

It's not only about CNAME, but also instance wasn't destroyed after some previously failed workflow in the middle, where cleanup didn't happen.

I was looking at the same thing before and I think this is why it wasn't destroyed: #90