Description
Describe the bug
Using the executable distribution here (and its aarch64 compatriot)
OK, I realize this is a REALLY edge case and highly niche... but we have managed to build a reasonable-sized cross platform AWS CLI V2 Lambda Layer. This relies on being able to separate common, platform independent stuff (dist/awscli and dist/base_library.zip) into a single folder and symlinking them into the X86_64 and aarch64 platform dependent trees. A top level script looks at the Lambda runtime arch and execs the proper version of the aws executable.
The distribution for 2.23.2 and prior allowed this to work very nicely
The distribution for 2.25.12 does not allow dist/awscli to be a symlink, failing to find python modules etc.
Regression Issue
- Select this option if this issue appears to be a regression.
Expected Behavior
When using a symlink to provide the awscli folder in the dist folder, it should work.
Current Behavior
The symlink to dist/base_library.zip still works
The error we get when attempting to symlink to dist/awscli (whether or not dist/base_library.zip is a symlink) is:
Traceback (most recent call last):
File "aws.py", line 19, in <module>
ModuleNotFoundError: No module named 'awscli.clidriver'
[PYI-296:ERROR] Failed to execute script 'aws' due to unhandled exception!
The difference in the python config is
2.23.2
Python path configuration:
PYTHONHOME = '/local/cli-9/aws-cli/x86_64/dist'
PYTHONPATH = (not set)
program name = '/local/cli-9/aws-cli/x86_64/dist/aws'
isolated = 0
environment = 0
user site = 0
safe_path = 0
import site = 0
is in build tree = 0
stdlib dir = ''
sys._base_executable = '/local/cli-9/aws-cli/x86_64/dist/aws'
sys.base_prefix = ''
sys.base_exec_prefix = ''
sys.platlibdir = 'lib'
sys.executable = '/local/cli-9/aws-cli/x86_64/dist/aws'
sys.prefix = ''
sys.exec_prefix = ''
sys.path = [
'/local/cli-9/aws-cli/x86_64/dist/base_library.zip',
'/local/cli-9/aws-cli/x86_64/dist/lib-dynload',
'/local/cli-9/aws-cli/x86_64/dist',
]
2.25.2
ython path configuration:
PYTHONHOME = '/opt/aws-cli/x86_64/v2/2.25.12/dist'
PYTHONPATH = (not set)
program name = '/opt/aws-cli/x86_64/v2/2.25.12/dist/aws'
isolated = 1
environment = 0
user site = 0
safe_path = 1
import site = 0
is in build tree = 0
stdlib dir = ''
sys._base_executable = '/opt/aws-cli/x86_64/v2/2.25.12/dist/aws'
sys.base_prefix = '/opt/aws-cli/x86_64/v2/2.25.12/dist'
sys.base_exec_prefix = '/opt/aws-cli/x86_64/v2/2.25.12/dist'
sys.platlibdir = 'lib'
sys.executable = '/opt/aws-cli/x86_64/v2/2.25.12/dist/aws'
sys.prefix = '/opt/aws-cli/x86_64/v2/2.25.12/dist'
sys.exec_prefix = '/opt/aws-cli/x86_64/v2/2.25.12/dist'
sys.path = [
'/opt/aws-cli/x86_64/v2/2.25.12/dist/base_library.zip',
'/opt/aws-cli/x86_64/v2/2.25.12/dist/lib-dynload',
'/opt/aws-cli/x86_64/v2/2.25.12/dist',
]
Reproduction Steps
- Install from the above link
- navigate to
v2/current/dist - move the
awsclifolder "somewhere else" - create a symlink from that location back into
v2/current/dist - attempt to execute
aws
Possible Solution
None found so far (except that it is likely to do with the changes in the python configuration and the resultant effect upon the way that python resolves modules)
I'd add that commit f486310 looks like a good place to start.
Additional Information/Context
A dual architecture layer is a very desirable thing, eliminating the need for having two different layers. This is quite important for our development practices and automated deployment pipelines, noticeably reducing complexity. So while this looks like an edge case, the ability to shuffle things around but retain the expected layout using symbolic linking is quite important to us.
CLI version used
2.25.12
Environment details (OS name and version, etc.)
Linux X86_64 and aarch64