-
Notifications
You must be signed in to change notification settings - Fork 45
Upgrading Elastic Beanstalk from Python 3.8 to 3.12
Due to the possible complications with DNS records and SSL certificate management I recommend reading through these directions in full before proceeding, particularly section 6.
Any code snippets below assume python is directed at a Python 3 executable, depending on your Python environment configuration you may need to substitute python3, python3.12, python3.8 etc.
Python has 3.8 hit end-of-support in October 2024. This is a normal part of the Python language release cadence. The reason you need to take special system administrator action is because AWS' "Platform" shifted from Amazon Linux 2 to Amazon Linux 2023, and details in the script that configures the server at-deploy-time (beiwe-backend/.ebextensions/01.config) could not be made compatible with both platforms at the same time. Instead, the process is to update your old deployment to the most recent 3.8-compatible version of beiwe-backend, clone the environment and security groups to a new 3.12 environmnt (a step mostly automated using our launch script), deploy a Python 3.12 compatible version, and then swap your DNS configuration (most configurations will have an option that just works in a simple dropdown on the AWS online console).
- This operation consists of a new command in the cluster management launch script.
- You will need to have your credential files appropriately configured.
- You will need to run an
eb(from the "awsebcli" package) deployment command, so make sure you are configured and credentialed.- note: AWS seems to have potentially deprecated this package... I say potentially because, at least at time of writing, it continues to be updated.
- You may need to swap the Git branch you are on more than once, particularly if the main branch is substantially ahead of your servers' code version.
- (It is conceivable that you will need to update your local Python environment from Python 3.8 to 3.12 to run the launch script.)
I recommend swapping to our python3.8-legacy branch and doing an eb deploy operation on your existing Python 3.8 environment. Make sure to then swap back to the main branch after the deploy finishes as it is required to correctly deploy a Python 3.12 manager server
2. Navigate into the cluster_management folder and simply run python launch_script.py to familiarize yourself with the commands you will be running below.
This does exactly what it says.
This operation takes the credentials for the cluster you specify, re-analyzes them in case there are any issues, makes a copy of them under the new name you choose, and then deploys a new (empty) Elastic Beanstalk Environment that is configured to connect to your old database and data on S3. The old environment files in environment_configuration will not be removed, you may do so at your own convenience.
Your AWS account's DNS and SSL configuration may not be trivial to update.
Many institutions impose strict requirements on SSL certificate management, and DNS entries may use a different DNS provider than the AWS Route53 service. Depending on these factors you may need to run this step out-of-order with a first deploy operation of the Python 3.8 platform, or you may want to delay a first deployment until DNS is configured. If DNS is totally out of your control you may want to request a shortened TTL (time to live) on relevant DNS entries ahead of time (several days), and/or run this step while in direct real-time contact with an individual with DNS access privilages.
Go to the AWS EC2 online console, navigate to the Load Balancer section on the left, and identify the load balancers for your old EB Environment, and for the new one. Select the existing one and view the Listeners section at the bottom. A correct configuration of a Beiwe environment will have an entry for HTTPS (port 443). If you click on "change" you will be presented with a dialog of various options to provide the SSL certificate. You will need to replicate the old configuration, or possibly acquire and upload a new certificate depending on any organizational policies that apply.
For AWS ACM (a.k.a. "Certificate Manager"), on the new load balancer click edit, click add, select HTTPS, and forward port 443 to port 80. For the certificate click change, select "Choose a certificate from ACM", and select the certificate based on the desired domain name.
If you are using a custom cert or an existing uploaded cert select that option instead
If you are configuring last-leg encryption you will have to configure that anew for the Python 3.8 environment, and forward to port 443. (Old methods of configuring last-leg encryption are not compatible with the new server stack.)
If you are customizing the Cipher suite (also called the Security Policy) please read step 7.
If your DNS entries for both EB Environments are contained within Route53 on the same account you can use the URL swap operation on the EB online console. You can find a full direction and description here: https://docs.aws.amazon.com/whitepapers/latest/blue-green-deployments/swap-the-environment-of-an-elastic-beanstalk-application.html ("blue-green" is a DevOps phrase, you can ignore it.)
If your DNS configuration is external you will need to manually change the target of the DNS record. You can find details for this here: https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-to-beanstalk-environment.html
Updating DNS can be a little terrifying. Be aware that many institutions have their own requirements or management process for DNS configurations, so you may want to reach out to institutional support if you have it for compliance reasons, or just to get assistance with the jargon. Onnela Lab cannot provide direct support on external custom DNS configurations.
Hint: If you are using Rout53, AWS has a magic DNS hack that they call an "alias". It handles certain configuration complexities, like pointing an A-record DNS entry at a load balancer, and providing your load balancers directly in an easy-and-correct dropdown menu so you don't need to hunt for typos. Non-AWS DNS providers may or may not have an equivalent mechanism for this type of "abstracted" configuration.
Hint 2: There is also a
The main branch may be substantially ahead of the time these directions were written. For a known-working branch swap to python3.12-redeploy. This branch has identical code state to the python3.8-legacy branch, with only changes to the deployment script.
Run eb deploy my_new_environment_name_here from inside the beiwe-backend repo.
After a successful deploy of that branch you can then swap back to main and do a final fully-updated deployment operation.
It can be useful to run the excellent SSL Test at https://www.ssllabs.com/ssltest/ This takes a minute or two to complete.
Mostly I want to draw your attention to the final "Handshake Simulation" section, where you will note the versions of Android and iOS that are compatible with your server. If you edited Cipher Suite in step 6 (or if oh-perhaps-maybe AWS silently changes their default Cipher Suite again) you may need to go back and adjust them in order to allow older devices to upload data.
(A full analysis of results from this SSL test are out-of-scope for this article. SSL security is a complex and constantly evolving expert-level subject. Please do not post GitHub issues about SSL security unless you are absolutely positive that your issue will affect other deployments.)
Make sure you are on the same branch that you just deployed (e.g. main. Please.), cd into cluster_management, and run python manage.py -create-manager