Cloudwatch Query is always empty

[wilmer-suarez]

Hello, I have a similar issue to this one: #2438.
This is my code:

subprocess.Popen(["aws", "sso", "login", "--profile="+AWS_PROFILE_NAME], stdout=subprocess.PIPE).wait() # Wait until sign in is complete

...
def capture_aws_logs(serial_number):
    endTime = ...
    startTime = ...
    cloudwatch_logs_client = boto3.Session(profile_name=PROFILE_NAME, region_name='us-east-1').client('logs')
    
    print(endTime)
    print(startTime)
 
    results = []
    while not results or results['status'] == 'Running' or results['status'] == 'Scheduled':
        time.sleep(5) # Wait a bit for log to be generated
        start_query_response = cloudwatch_logs_client.start_query(
            logGroupName='AWSIotLogs',
            startTime=startTime,
            endTime=endTime,
            queryString=f'''query...''
        )
        query_id = start_query_response['queryId']
        results = cloudwatch_logs_client.get_query_results(queryId=query_id)['results']
        pprint(results)
...

def thread(args):
    ...
    capture_aws_logs(serial_number)
    ...

def callback_for_an_mqtt_conn(topic, payload, dup, qos, retain, **kwargs):
    Thread(target=thread,
           args=(args),
           daemon=True).start()

def main(args):
    while True:
        pass

if __name__ == '__main__':
    sys.exit(main(args))

The result is always empty:

{'ResponseMetadata': {'HTTPHeaders': {'content-length': '109',
                                      'content-type': 'application/x-amz-json-1.1',
                                      'date': 'Sun, 09 Apr 2023 18:53:06 GMT',
                                      'x-amzn-requestid': 'eea24274-1058-49cb-b3c2-523b215964f7'},
                      'HTTPStatusCode': 200,
                      'RequestId': 'eea24274-1058-49cb-b3c2-523b215964f7',
                      'RetryAttempts': 0},
 'results': [],
 'statistics': {'bytesScanned': 0.0,
                'recordsMatched': 0.0,
                'recordsScanned': 0.0},

If I run the exact same query in the cloudwatch web client or running the same query in a different script, I get the logs that I'm expecting. Why isn't this working?

Edit:
I should add that the timing for this very narrow. The logs are likely not ready the first query. But retrying should have the logs available. Querying manually after a few seconds give me the correct logs (as I mentioned) and within the start and end times provided.

Using boto3 v1.26.109

[tim-finnigan]

I think what you're describing is expected behavior per the response here: #2438 (comment). You could increase the sleep time and poll describe_queries to confirm when the queries are available.