Ansible container not seeing vault encrypted vars in vars_files

[s-sokolko]

ISSUE TYPE

• Bug Report

container.yml

version: "2"
settings:

  conductor:
    base: alpine:3.5
    roles_path: 
      - ../../roles

  project_name: dante
  vars_files:
    - ../../group_vars/all.yml
    - vars.yml
  vault_password_file: ../../.vault_pass
 
services:
  dante:
    from: alpine:3.5
    roles:
      - dante-container
    ports:
      - 1080:1080
    volumes:
      - /etc/sockd.conf
    entrypoint: ["dumb-init"]
    command: ["sockd"]

registries:
  docker:
    url: https://index.docker.io
    namespace: sokolko

OS / ENVIRONMENT

Ansible Container, version 0.9.2
Linux, terminator, 4.4.0-83-generic, #106-Ubuntu SMP Mon Jun 26 17:54:43 UTC 2017, x86_64
2.7.12 (default, Nov 19 2016, 06:48:10) 
[GCC 5.4.0 20160609] /home/godzilla/projects/openitsolutions/deploy/containers/.env/bin/python2
{
  "ContainersPaused": 0, 
  "Labels": null, 
  "CgroupDriver": "cgroupfs", 
  "ContainersRunning": 1, 
  "ContainerdCommit": {
    "Expected": "aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1", 
    "ID": ""
  }, 
  "InitBinary": "docker-init", 
  "NGoroutines": 27, 
  "Swarm": {
    "Managers": 0, 
    "ControlAvailable": false, 
    "NodeID": "", 
    "Cluster": {
      "Spec": {
        "TaskDefaults": {}, 
        "Orchestration": {}, 
        "EncryptionConfig": {
          "AutoLockManagers": false
        }, 
        "Raft": {
          "HeartbeatTick": 0, 
          "ElectionTick": 0
        }, 
        "CAConfig": {}, 
        "Dispatcher": {}
      }, 
      "Version": {}, 
      "ID": "", 
      "CreatedAt": "0001-01-01T00:00:00Z", 
      "UpdatedAt": "0001-01-01T00:00:00Z"
    }, 
    "Nodes": 0, 
    "Error": "", 
    "RemoteManagers": null, 
    "LocalNodeState": "inactive", 
    "NodeAddr": ""
  }, 
  "LoggingDriver": "json-file", 
  "OSType": "linux", 
  "HttpProxy": "", 
  "Runtimes": {
    "runc": {
      "path": "runc"
    }
  }, 
  "DriverStatus": [
    [
      "Root Dir", 
      "/var/lib/docker/aufs"
    ], 
    [
      "Backing Filesystem", 
      "extfs"
    ], 
    [
      "Dirs", 
      "30"
    ], 
    [
      "Dirperm1 Supported", 
      "true"
    ]
  ], 
  "OperatingSystem": "Ubuntu 16.04.2 LTS", 
  "Containers": 1, 
  "HttpsProxy": "", 
  "BridgeNfIp6tables": true, 
  "MemTotal": 8245301248, 
  "SecurityOptions": [
    "name=apparmor", 
    "name=seccomp,profile=default"
  ], 
  "Driver": "aufs", 
  "IndexServerAddress": "https://index.docker.io/v1/", 
  "ClusterStore": "", 
  "InitCommit": {
    "Expected": "949e6facb77383876aeff8a6944dde66b3089574", 
    "ID": "N/A"
  }, 
  "Isolation": "", 
  "SystemStatus": null, 
  "OomKillDisable": true, 
  "ClusterAdvertise": "", 
  "SystemTime": "2018-07-26T23:49:52.294411976+05:00", 
  "Name": "terminator", 
  "CPUSet": true, 
  "RegistryConfig": {
    "InsecureRegistryCIDRs": [
      "127.0.0.0/8"
    ], 
    "IndexConfigs": {
      "docker.io": {
        "Official": true, 
        "Name": "docker.io", 
        "Secure": true, 
        "Mirrors": null
      }
    }, 
    "Mirrors": []
  }, 
  "DefaultRuntime": "runc", 
  "ContainersStopped": 0, 
  "NCPU": 8, 
  "NFd": 21, 
  "Architecture": "x86_64", 
  "KernelMemory": true, 
  "CpuCfsQuota": true, 
  "Debug": false, 
  "ID": "4ADH:UOSB:ZSEP:5MX4:IPGK:7A3S:GNW3:7TW2:I6KO:3VXJ:U5BA:MF2V", 
  "IPv4Forwarding": true, 
  "KernelVersion": "4.4.0-83-generic", 
  "BridgeNfIptables": true, 
  "NoProxy": "", 
  "LiveRestoreEnabled": false, 
  "ServerVersion": "1.13.1", 
  "CpuCfsPeriod": true, 
  "ExperimentalBuild": false, 
  "MemoryLimit": true, 
  "SwapLimit": false, 
  "Plugins": {
    "Volume": [
      "local"
    ], 
    "Network": [
      "bridge", 
      "host", 
      "macvlan", 
      "null", 
      "overlay"
    ], 
    "Authorization": null
  }, 
  "Images": 43, 
  "DockerRootDir": "/var/lib/docker", 
  "NEventsListener": 0, 
  "CPUShares": true, 
  "RuncCommit": {
    "Expected": "9df8b306d01f59d3a8029be411de015b7304dd8f", 
    "ID": "N/A"
  }
}
{
  "KernelVersion": "4.4.0-83-generic", 
  "Arch": "amd64", 
  "BuildTime": "2017-11-02T20:40:23.484070968+00:00", 
  "ApiVersion": "1.26", 
  "Version": "1.13.1", 
  "MinAPIVersion": "1.12", 
  "GitCommit": "092cba3", 
  "Os": "linux", 
  "GoVersion": "go1.6.2"
}

SUMMARY

I've come across the situation when ansible won't include my vars_files if they are encrypted via vault even if I specify vault_password_file. In the example below everything works fine if vars.yml is not encrypted. But as soon as I encrypt vars.yml I get the error (see below). Maybe I should use some other technique to inject vault encrypted variales into my container build process?

STEPS TO REPRODUCE

ansible-container --debug  build --no-cache

EXPECTED RESULTS

Sucessfull build

ACTUAL RESULTS

2018-07-26T23:53:31.786400 Use variable file: /home/godzilla/projects/openitsolutions/deploy/group_vars/all.yml [container.config] caller_file=/home/godzilla/projects/openitsolutions/deploy/containers/.env/local/lib/python2.7/site-packages/container/config.py caller_func=_get_variables_from_file caller_line=226 file=/home/godzilla/projects/openitsolutions/deploy/group_vars/all.yml
2018-07-26T23:53:31.797446 Use variable file: /home/godzilla/projects/openitsolutions/deploy/containers/dante/vars.yml [container.config] caller_file=/home/godzilla/projects/openitsolutions/deploy/containers/.env/local/lib/python2.7/site-packages/container/config.py caller_func=_get_variables_from_file caller_line=226 file=/home/godzilla/projects/openitsolutions/deploy/containers/dante/vars.yml
2018-07-26T23:53:31.800491 Unknown exception 'str' object has no attribute 'iteritems' [container.cli] caller_file=/home/godzilla/projects/openitsolutions/deploy/containers/.env/local/lib/python2.7/site-packages/structlog/stdlib.py caller_func=exception caller_line=95
Traceback (most recent call last):
  File "/home/godzilla/projects/openitsolutions/deploy/containers/.env/local/lib/python2.7/site-packages/container/cli.py", line 299, in __call__
    getattr(core, u'hostcmd_{}'.format(args.subcommand))(**vars(args))
  File "/home/godzilla/projects/openitsolutions/deploy/containers/.env/local/lib/python2.7/site-packages/container/__init__.py", line 28, in __wrapped__
    return fn(*args, **kwargs)
  File "/home/godzilla/projects/openitsolutions/deploy/containers/.env/local/lib/python2.7/site-packages/container/core.py", line 153, in hostcmd_build
    config = get_config(base_path, vars_files=vars_files, engine_name=engine_name, project_name=project_name)
  File "/home/godzilla/projects/openitsolutions/deploy/containers/.env/local/lib/python2.7/site-packages/container/utils/__init__.py", line 49, in get_config
    project_name=project_name, vault_files=vault_files)
  File "/home/godzilla/projects/openitsolutions/deploy/containers/.env/local/lib/python2.7/site-packages/container/__init__.py", line 28, in __wrapped__
    return fn(*args, **kwargs)
  File "/home/godzilla/projects/openitsolutions/deploy/containers/.env/local/lib/python2.7/site-packages/container/config.py", line 62, in __init__
    self.set_env('prod')
  File "/home/godzilla/projects/openitsolutions/deploy/containers/.env/local/lib/python2.7/site-packages/container/docker/config.py", line 45, in set_env
    super(AnsibleContainerConfig, self).set_env(env, config=config)
  File "/home/godzilla/projects/openitsolutions/deploy/containers/.env/local/lib/python2.7/site-packages/container/config.py", line 146, in set_env
    self._resolve_defaults(config)
  File "/home/godzilla/projects/openitsolutions/deploy/containers/.env/local/lib/python2.7/site-packages/container/config.py", line 187, in _resolve_defaults
    defaults.update(self._get_variables_from_file(var_file=var_file), relax=True)
  File "/home/godzilla/projects/openitsolutions/deploy/containers/.env/local/lib/python2.7/site-packages/container/config.py", line 238, in _get_variables_from_file
    return iteritems(config)
  File "/home/godzilla/projects/openitsolutions/deploy/containers/.env/local/lib/python2.7/site-packages/six.py", line 605, in iteritems
    return d.iteritems(**kw)
AttributeError: 'str' object has no attribute 'iteritems'

[Voronenko]

@s-sokolko do you have minimal POC in form of repo ?

Documentation might be outdated, but looking at https://docs.ansible.com/ansible-container/container_yml/reference.html#supported-directives

vars_files	List of variable files to use for Jinja2 template rendering while parsing container.yml
vault_files	List of Ansible vault files, where each entry is a file path. Files are decrypted in the conductor at playbook runtime, making variables available to playbooks and roles.
vault_password_file	Path to a file containing a clear text password that can be used to decrypt any vault files.

I do not see in your example above definition for vault_files

[s-sokolko]

@Voronenko thank you for the quick reply!
I tried using vault_files option in my container.yml, but I end up having the 'variable is undefined error' as below

`fatal: [dante]: FAILED! => {"failed": true, "msg": "'proxy_users' is undefined"}
to retry, use: --limit @/tmp/tmplW_UUJ/playbook.retry

PLAY RECAP *********************************************************************
dante : ok=2 changed=1 unreachable=0 failed=1

ERROR Error applying role! engine=<container.docker.engine.Engine object at 0x7f76f0622090> exit_code=2 playbook=[{'hosts': u'dante', 'roles': ['dante-container'], 'vars': {}}]
Traceback (most recent call last):
File "/usr/bin/conductor", line 11, in
load_entry_point('ansible-container', 'console_scripts', 'conductor')()
File "/_ansible/container/init.py", line 19, in wrapped
return fn(*args, **kwargs)
File "/_ansible/container/cli.py", line 399, in conductor_commandline
**params)
File "/_ansible/container/init.py", line 19, in wrapped
return fn(*args, **kwargs)
File "/_ansible/container/core.py", line 813, in conductorcmd_build
raise RuntimeError('Build failed.')
RuntimeError: Build failed.
Conductor terminated. Cleaning up. command_rc=1 conductor_id=3e817edc3c86dd0e5b08b52c886846ee94c983550e686a27a3118df813b84291 save_container=False
ERROR Conductor exited with status 1`

As I looked through the code of ansible-containers cli it seemed to me that vault_files are not used when I run ansible-container build.
Anyway I created a repo with my failing container, you can clone in using
git clone https://github.com/s-sokolko/dante-container.git
And then perform the build command in the root of this repo to see it failing:
ansible-container build --no-cache
Any help will be truly appreciated!

[Voronenko]

If you would create minimal POC in form of repo, I can debug.

Take a look how it was done in this issue: #944
Person created small workable POC of problem - the only thing left was - debug and fix.

https://github.com/matteotanca-sardex/ac-bug-944-poc

[s-sokolko]

@Voronenko here is the repo https://github.com/s-sokolko/dante-container
I tried to make it look like the example repo provided by you

[Voronenko]

Thanks!

[s-sokolko]

@Voronenko is there any progress on the issue? Maybe I can help with some extra information?

[Voronenko]

I have it in backlog, but need to find opportunity window between my commercial work schedule.

[ITBlogger]

The docs specifically state that ansible-container only decrypts files with the run command, not build, which seems not great at all, but that probably explains your issue

[Voronenko]

That's offtopic, but I prefer hashicorp "application" roles approach to prepare images that get prepared upon run - that makes it a little more robust.