Glue UpdateTableOptimizer Fails with Timeout

[surendragalwa]

Describe the bug

Creating and updating glue table optimizers - compaction, retention & orphan file deletion with boto3 SDK, it fails with timeout errors -

Connection was closed before we received a valid response from endpoint URL: "https://glue.us-east-2.amazonaws.com/

Does boto3 API call wait for glue to execute the optimizers? Why does it take more than 2 mins to just update the configuration? Also, I can see the glue table optimizer configuration has been updated but the API call with SDK it fails with above error.

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

Boto3 SDK should just update the configuration for glue optimizers instead of waiting for them to execute and complete.

Current Behavior

Failes with error.

Reproduction Steps

glue_client.update_table_optimizer

Possible Solution

No response

Additional Information/Context

No response

SDK version used

1.34.93

Environment details (OS name and version, etc.)

Linux

[adev-code]

Hello @surendragalwa, thank you for reaching out. The request must be sent to the service for it to be processed. Boto3 does not have information on how long the service takes to process the request and send the response back. Boto3 waits for an acknowledgment from the service, and if the read_timeout needs to be extended, it can be done by:

from botocore.config import Config
config = Config(read_timeout=120)
client = boto3.client('glue', config=config)

If the Glue table has been changed and Boto3 timed out, this suggests the service is taking longer than expected to respond. Please try adjusting the connection timeout.