Convert provisioning shell scripts to reusable Ansible playbooks (#96) #97

BytePaul · 2025-07-30T23:03:52Z

Convert provisioning shell scripts to reusable Ansible playbooks (#96)

Summary

This pull request addresses Issue #96: Reusable provisioning by converting selected shell-based provisioning scripts into reusable, idempotent Ansible playbooks. The goal is to support reproducible and flexible provisioning across Vagrant VMs, Live USB environments (Cubic), and Apptainer containers, using a common YAML-based infrastructure.

✔️ Converted Scripts

The following shell scripts were translated into Ansible playbooks:

Original Shell Script	Ansible Playbook
`install-aste.sh`	`install-aste.yml`
`install-basics.sh`	`install-basics.yml`
`install-calculix.sh`	`install-calculix.yml`
`install-code_aster.sh`	`install-code_aster.yml`
`install-config-visualizer.sh`	`install-config-visualizer.yml`
`install-dealii.sh`	`install-dealii.yml`
`install-devel.sh`	`install-devel.yml`
`install-dune.sh`	`install-dune.yml`
`install-fenics.sh`	`install-fenics.yml`
`install-fmiprecice.sh`	`install-fmiprecice.yml`
`install-julia-bindings.sh`	`install-julia-bindings.yml`
`install-micro-manager.sh`	`install-micro-manager.yml`
`install-openfoam.sh`	`install-openfoam.yml`
`install-paraview.sh`	`install-paraview.yml`
`install-precice.sh`	`install-precice.yml`
`install-su2.sh`	`install-su2.yml`
`install-vscode.sh`	`install-vscode.yml`

🔧 Structure

ansible/
├── inventory.ini
├── playbooks/
│   ├── install-aste.yml
│   ├── install-basics.yml
│   ├── install-calculix.yml
│   ├── install-code_aster.yml
│   ├── install-config-visualizer.yml
│   ├── install-dealii.yml
│   ├── install-devel.yml
│   ├── install-dune.yml
│   ├── install-fenics.yml
│   ├── install-fmiprecice.yml
│   ├── install-julia-bindings.yml
│   ├── install-micro-manager.yml
│   ├── install-openfoam.yml
│   ├── install-paraview.yml
│   ├── install-precice.yml
│   ├── install-su2.yml
│   └── install-vscode.yml
└── README.md

inventory.ini: for host targeting (currently local via Vagrant)
README.md: describes usage, structure, and future goals

💡 Improvements Over Shell Scripts

Declarative, YAML-based structure for easier maintenance
Idempotent execution: safe to run multiple times
More portable: can be reused across VM, container, and Live USB contexts
Ready for CI integration and eventual support for tags, roles, and variables

🧪 Testing

All playbooks tested on Ubuntu 24.04 using Vagrant
Verified with --check and --diff modes
Manually validated installation outcomes for each component

✅ Issues and Fixes

Issue 1: `/root` path inaccessible

File: All playbooks
Fix: Used {{ lookup('env', 'HOME') }} to dynamically get the current user’s home directory.

Issue 2: Permission denied when creating `/root/python-venvs`

File: install-fenics.yml, install-julia-bindings.yml
Fix: Used become: false for tasks that create or access user-level directories such as virtual environments.

Issue 3: Attempt to write to `/root/.bashrc`

File: install-openfoam.yml
Fix: Ensured .bashrc modifications target the correct non-root user’s home directory using {{ user_home }}/.bashrc.

Issue 4: `install-dune.yml` tried writing to `/root/dune-dumux`

File: install-dune.yml
Fix: Used user-local directory based on {{ user_home }} and ensured proper become settings.

Issue 5: `install-fenics.yml` failed on virtualenv creation

File: install-fenics.yml
Fix: Corrected permissions by ensuring virtual environment creation uses become: false and the right user_home.

Issue 6: `install-julia-bindings.yml` failed at `/root/python-venvs`

File: install-julia-bindings.yml
Fix: Fixed path using dynamic home directory and ran venv and pip commands without become.

Issue 7: `install-openfoam.yml` failed due to the wrong tutorial directory

File: install-openfoam.yml
Fix: Updated tutorial path using {{ user_home }}/tutorials/... instead of ~/... or /root/....

Issue 8: `.bashrc` not writable by current user

File: install-openfoam.yml, install-aste.yml
Fix: Ensured tasks modifying .bashrc run without become and use the actual user’s home directory.

Issue 9: `.bashrc` edit failed silently under root context

File: install-aste.yml
Fix: Used lineinfile module with correct user path and become: false.

Issue 10: `install-code_aster.yml` has unresolved issues with installation

File: install-code_aster.yml
Fix: No Fix Yet.

Suggestions and Concerns for Ansible Playbook Refactoring

1. Use `become: false` for User-Space Actions

Concern: Tasks like creating virtual environments or modifying user .bashrc should run under the actual user context, not root.

Suggestion: Always explicitly set become: false for such tasks to avoid permission errors or modifications to /root.

2. Ensure Directories Exist Before Writing

Concern: Tasks that write to paths like ~/python-venvs/foo can fail if the base directory doesn't exist.

Suggestion: Add file tasks to ensure directories exist with correct ownership before accessing them.

3. Avoid Appending Duplicate Lines to `.bashrc`

Concern: Repeated playbook runs can duplicate export lines in .bashrc.

Suggestion: Use regexp with lineinfile to match and replace lines safely instead of blindly appending with insertafter: EOF.

4. Use `chdir` for Directory Changes

Concern: Inline cd commands in shell can fail silently or be harder to debug.

Suggestion: Use Ansible's chdir parameter to set working directories cleanly for shell or command tasks.

5. Consistent Use of `user_home` Variable

Concern: Hardcoding paths like /home/vagrant reduces portability.

Suggestion: Use lookup('env', 'HOME') or ansible_env.HOME as a variable (user_home) across playbooks for user-path operations.

6. Use `creates:` for Idempotency in Virtualenv Creation

Concern: Without creates:, a playbook may recreate or overwrite existing environments.

Suggestion: Use creates: "{{ path }}/bin/activate" to make tasks idempotent.

7. Avoid Running User Shell Configurations as Root

Concern: Tasks that append to .bashrc or clone to ~/ may default to /root/ if become: true is active.

Suggestion: Use become: false when working on user-specific files like .bashrc, Python virtual environments, or home directories.

8. Ensure Correct Permissions When Mixing `become: true` and `false`

Concern: Switching between elevated and non-elevated contexts can cause permission conflicts.

Suggestion: Always check ownership and permissions, especially when files are created with become: true and accessed without it later.

Closes #96

MakisH

This PR does not add the files themselves, but only an unresolved pointer to a submodule. There is no content to review/test here at the moment.

I also see that this PR is highly likely generated by an LLM. I would appreciate more an original description (with its expected flaws), including your viewpoint and your concerns and suggestions.

In case the playbooks are also generated by an LLM, this should be clearly stated. I am aware that ChatGPT can convert these, but what we need here is also ownership of the contribution: we need that someone has actually validated the concept and tested the system. I do not have enough evidence to feel confident of that at the moment.

Please revise.

BytePaul · 2025-07-31T23:46:07Z

Hi @MakisH,

I have created and ran the playbook on the virtual machine; hence, they have been created and validated by me.

In the updated version of PR comment, I have tried to be descriptive of the issues I faced while running it on the VM.

Also, I have fixed the accidental submodule pointer and ensured the Ansible playbooks for viewing.

Even though the initial description has been LLM generated the idea behind the comments are mine and I made use of the LLM to polish things up.

Thank you for the feedback.

MakisH · 2025-08-01T06:59:52Z

Thank you for the updates, @BytePaul, I can now see the changes, but I need some time to review and test everything.

🧪 Testing

All playbooks tested on Ubuntu 24.04 using Vagrant

Verified with --check and --diff modes

Manually validated installation outcomes for each component

Does this mean that install-code_aster.yml also works? I would expect not, but I don't see it under the list of issues.

BytePaul · 2025-08-01T11:32:52Z

It just means I tried running to the best possible outcome possible so far.

And the results have been documents and discussed in the PR

MakisH · 2025-08-01T12:15:10Z

ansible/playbooks/install-precice.yml

+        src: /usr/share/precice/examples/
+        dest: "{{ precice_path }}-examples"


This playbook currently fails at this point. The original is:

sudo apt-get install -y ./libprecice*_*.deb ... cp -r /usr/share/precice/examples/ ./precice-examples

It looks like the src and dest are swapped here.

… into feature/ansible-provisioning

MakisH · 2025-08-08T11:49:52Z

.github/workflows/ci.yml

+        uses: actions/checkout@v4
+
+      - name: Lint Ansible Playbooks 🧹
+        uses: docker://ansible/ansible-lint:latest


This image does not seem to exist on the public Docker Hub registry: https://hub.docker.com/search?q=ansible-lint Where is that recommended? Is there a common alternative?

This is the reason that the CI is currently failing.

MakisH · 2025-08-08T11:51:54Z

ansible/inventory.ini

+
+[remote]
+# Example: server1 ansible_host=192.168.1.100 ansible_user=youruser


We don't need this, right?

Suggested change

[remote]

# Example: server1 ansible_host=192.168.1.100 ansible_user=youruser

MakisH · 2025-08-08T11:53:10Z

ansible/README.md

What would be the way to run everything?
Should we define hosts that execute different playbooks?
Should we use import?

Add Ansible playbooks for reusable provisioning (precice#96)

47ba29c

MakisH requested changes Jul 31, 2025

View reviewed changes

Re-add Ansible playbooks as regular files

0c130d0

ADd GUI Installation

1d3de51

MakisH reviewed Aug 1, 2025

View reviewed changes

BytePaul and others added 9 commits August 7, 2025 21:39

CI added

bf23b84

Update README.md

822938e

Update README.md

79d686e

Merge branch 'feature/ansible-provisioning' of github.com:BytePaul/vm…

d18bd9d

… into feature/ansible-provisioning

Trigger CI

d178fd7

Trigger CI

3effe14

Modified CI

ebe1827

Basics: Add/edit comments, remove an unnecessary task

59ab775

Cleaup README.md

5959ca6

MakisH reviewed Aug 8, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Convert provisioning shell scripts to reusable Ansible playbooks (#96) #97

Convert provisioning shell scripts to reusable Ansible playbooks (#96) #97

Uh oh!

BytePaul commented Jul 30, 2025 •

edited

Loading

Uh oh!

MakisH left a comment

Uh oh!

BytePaul commented Jul 31, 2025

Uh oh!

MakisH commented Aug 1, 2025

🧪 Testing

Uh oh!

BytePaul commented Aug 1, 2025

Uh oh!

MakisH Aug 1, 2025

Uh oh!

MakisH Aug 8, 2025

Uh oh!

MakisH Aug 8, 2025

Uh oh!

MakisH Aug 8, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		src: /usr/share/precice/examples/
		dest: "{{ precice_path }}-examples"


		[remote]
		# Example: server1 ansible_host=192.168.1.100 ansible_user=youruser

Uh oh!

Convert provisioning shell scripts to reusable Ansible playbooks (#96) #97

Are you sure you want to change the base?

Convert provisioning shell scripts to reusable Ansible playbooks (#96) #97

Uh oh!

Conversation

BytePaul commented Jul 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Convert provisioning shell scripts to reusable Ansible playbooks (#96)

Summary

✔️ Converted Scripts

🔧 Structure

💡 Improvements Over Shell Scripts

🧪 Testing

✅ Issues and Fixes

Issue 1: /root path inaccessible

Issue 2: Permission denied when creating /root/python-venvs

Issue 3: Attempt to write to /root/.bashrc

Issue 4: install-dune.yml tried writing to /root/dune-dumux

Issue 5: install-fenics.yml failed on virtualenv creation

Issue 6: install-julia-bindings.yml failed at /root/python-venvs

Issue 7: install-openfoam.yml failed due to the wrong tutorial directory

Issue 8: .bashrc not writable by current user

Issue 9: .bashrc edit failed silently under root context

Issue 10: install-code_aster.yml has unresolved issues with installation

Suggestions and Concerns for Ansible Playbook Refactoring

1. Use become: false for User-Space Actions

2. Ensure Directories Exist Before Writing

3. Avoid Appending Duplicate Lines to .bashrc

4. Use chdir for Directory Changes

5. Consistent Use of user_home Variable

6. Use creates: for Idempotency in Virtualenv Creation

7. Avoid Running User Shell Configurations as Root

8. Ensure Correct Permissions When Mixing become: true and false

Uh oh!

MakisH left a comment

Choose a reason for hiding this comment

Uh oh!

BytePaul commented Jul 31, 2025

Uh oh!

MakisH commented Aug 1, 2025

🧪 Testing

Uh oh!

BytePaul commented Aug 1, 2025

Uh oh!

MakisH Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

MakisH Aug 8, 2025

Choose a reason for hiding this comment

Uh oh!

MakisH Aug 8, 2025

Choose a reason for hiding this comment

Uh oh!

MakisH Aug 8, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

BytePaul commented Jul 30, 2025 •

edited

Loading

Issue 1: `/root` path inaccessible

Issue 2: Permission denied when creating `/root/python-venvs`

Issue 3: Attempt to write to `/root/.bashrc`

Issue 4: `install-dune.yml` tried writing to `/root/dune-dumux`

Issue 5: `install-fenics.yml` failed on virtualenv creation

Issue 6: `install-julia-bindings.yml` failed at `/root/python-venvs`

Issue 7: `install-openfoam.yml` failed due to the wrong tutorial directory

Issue 8: `.bashrc` not writable by current user

Issue 9: `.bashrc` edit failed silently under root context

Issue 10: `install-code_aster.yml` has unresolved issues with installation

1. Use `become: false` for User-Space Actions

3. Avoid Appending Duplicate Lines to `.bashrc`

4. Use `chdir` for Directory Changes

5. Consistent Use of `user_home` Variable

6. Use `creates:` for Idempotency in Virtualenv Creation

8. Ensure Correct Permissions When Mixing `become: true` and `false`