Add a testing CI on the aws image #19

bidoubiwa · 2023-04-27T15:10:27Z

This PR adds a CI that is run on every pull_request and push. In the CI:

A temporary IAM with the current packer configuration is created
An EC2 instance is created based on this IAM.
It waits for the instance to be up and running. A timeout error is raised after 10 minutes
It tests is done to ensure Meilisearch is running correctly by making a request on /health and ensure the answer is {"status":"available"}.
Terraform destroy the instance and the temporary image.

We used continue-on-error: true in the steps that might fail to ensure terraform correctly removes the image

…into add_test_ci

alallema

Thanks, @bidoubiwa for this PR 🔥
So so great 🚀

alallema · 2023-05-11T08:49:41Z

.github/workflows/test.yml

+      run: |
+        wget https://releases.hashicorp.com/packer/1.7.7/packer_1.7.7_linux_amd64.zip # change version
+        unzip packer_1.7.7_linux_amd64.zip
+        sudo mv packer /usr/local/bin/


Suggested change

run: |

wget https://releases.hashicorp.com/packer/1.7.7/packer_1.7.7_linux_amd64.zip # change version

unzip packer_1.7.7_linux_amd64.zip

sudo mv packer /usr/local/bin/

uses: hashicorp/setup-packer@main

with:

version: ${{ env.PACKER_VERSION }}

Is there a reason for not using the Packer Github action directly?

I think I tried initially but it didn't work. I can try again though

I don't think it's worth spending too much time on it - if it works with the official GitHub-action, all the better, otherwise, let's keep what works for now.

.github/workflows/test.yml

alallema · 2023-06-06T07:56:21Z

.github/scripts/check_instance.sh

+while [ "$(curl -s -o /dev/null -w ''%{http_code}'' --max-time 5 http://$PUBLIC_IP)" != "200" ]; do
+  sleep 5


Maybe we can add a echo -n "." before the sleep 5, that way, we have a visual during the process. But I'm not sure it's will be visible on github-action

what about echo "Pinging the server..." ?

alallema · 2023-06-06T08:07:41Z

.github/scripts/check_instance.sh

+PUBLIC_IP=$1
+start_time=$(date +%s)
+
+while [ "$(curl -s -o /dev/null -w ''%{http_code}'' --max-time 5 http://$PUBLIC_IP)" != "200" ]; do


Why use the command -o /dev/null to capture output if you're going to send it to /dev/null? If you want to avoid having it printed in the terminal, the -s silent option is enough.

Might be a copypasta from chatgpt 😅 Good catch

alallema · 2023-06-06T08:10:51Z

.github/scripts/check_meilisearch_availability.sh

+# Check if the variable content is equal to '{"status":"available"}'
+if [ "$status" != '{"status":"available"}' ]; then
+  # If not, echo the variable value and an error message, and exit with code 1
+  echo "Error: Meilisearch is not running correctly."
+  echo "Server response: $status"
+  exit 1
+else
+  # If it is equal, echo a success message and exit with code 0


I know it's better to comment on your code, but I don't think comments here do much.

terraform-config.tf

alallema · 2023-06-06T08:17:54Z

.github/workflows/test.yml

+      run: |
+        sudo apt-get update
+        sudo apt-get install -y curl unzip
+        curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -
+        sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
+        sudo apt-get update && sudo apt-get install -y terraform


Suggested change

run: |

sudo apt-get update

sudo apt-get install -y curl unzip

curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -

sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"

sudo apt-get update && sudo apt-get install -y terraform

- uses: hashicorp/setup-terraform@v2

with:

terraform_version: 1.1.7

Same question here for Terraform? Maybe we can use both simultaneously ...

alallema · 2023-06-06T08:22:22Z

terraform-config.tf

+  id = "sg-037fd498b332442c1"
+}
+
+resource "aws_instance" "test" {


These are predefined types, otherwise, you could change test by github_action_test or CI_test or something maybe more precise WDYT?

…d_test_ci

alallema · 2023-09-05T09:27:49Z

Due to the reduced availability of the team that takes care of this repo. The team will never have time to complete this PR, so I'm closing it.

bidoubiwa force-pushed the add_test_ci branch 3 times, most recently from 5c056f9 to 324023c Compare April 27, 2023 15:24

Add a testing CI on the aws image

e523092

bidoubiwa force-pushed the add_test_ci branch 8 times, most recently from e9f4052 to e74357e Compare April 27, 2023 16:20

Fix test CI

dbdd90f

bidoubiwa force-pushed the add_test_ci branch from e74357e to dbdd90f Compare April 27, 2023 16:29

alallema linked an issue May 1, 2023 that may be closed by this pull request

Implement github-action workflow 2 - Build and Test the image #30

Open

bidoubiwa and others added 2 commits May 2, 2023 12:45

Merge branch 'main' into add_test_ci

05efef8

Add logs to understand check instance script issue

7af4160

bidoubiwa force-pushed the add_test_ci branch from 5577011 to 8f04976 Compare May 2, 2023 11:27

Merge branch 'add_test_ci' of github.com:meilisearch/cloud-providers …

3a29ebb

…into add_test_ci

bidoubiwa force-pushed the add_test_ci branch from 8f04976 to 3a29ebb Compare May 2, 2023 11:43

bidoubiwa requested a review from alallema May 2, 2023 12:17

bidoubiwa added the skip-changelog The PR will not appear in the release changelogs label May 2, 2023

bidoubiwa marked this pull request as ready for review May 2, 2023 12:17

alallema mentioned this pull request May 11, 2023

[AWS] Remove old AMI image after release #45

Open

alallema reviewed Jun 6, 2023

View reviewed changes

bidoubiwa added 4 commits June 13, 2023 14:08

Change the way packer is installed

c7e097b

Remove unecessary echo's in test CI

f4da213

Rollback github packer action

c400f4c

fix Required plugins are not installed during destroy of ami

a26449a

bidoubiwa force-pushed the add_test_ci branch from c5f11b0 to a26449a Compare June 14, 2023 09:54

bidoubiwa added 3 commits June 14, 2023 12:24

Use packer github action with latest version of packer

ed7b9cc

Merge branch 'main' of github.com:meilisearch/cloud-providers into ad…

a13e34b

…d_test_ci

Add logs on build response

ccb9255

alallema closed this Sep 5, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a testing CI on the aws image #19

Add a testing CI on the aws image #19

bidoubiwa commented Apr 27, 2023

alallema left a comment

alallema May 11, 2023

bidoubiwa Jun 13, 2023

alallema Jun 14, 2023

alallema Jun 6, 2023

bidoubiwa Jun 13, 2023

alallema Jun 6, 2023

bidoubiwa Jun 13, 2023

alallema Jun 6, 2023

alallema Jun 6, 2023

alallema Jun 6, 2023

alallema Jun 6, 2023

alallema commented Sep 5, 2023

		while [ "$(curl -s -o /dev/null -w ''%{http_code}'' --max-time 5 http://$PUBLIC_IP)" != "200" ]; do
		sleep 5

Add a testing CI on the aws image #19

Add a testing CI on the aws image #19

Conversation

bidoubiwa commented Apr 27, 2023

alallema left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alallema commented Sep 5, 2023