Skip to content

I am getting multiple exceptions while fetching data from dynamo db suddenly. #5569

Closed
@danyal-careem

Description

@danyal-careem

Describe the bug

The issue is fetching the data from dynamo db based on user Id. which throws two exceptions. It was working fine till 20th of August suddenly getting those exceptions which getting higher day by day. One more exception we had "Error thrown from TransformingAsyncResponseHandler#onError, ignoring" which was resolved after updating SDK version.

1. java.lang.InterruptedException: null at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:386) at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) at com.careem.launcher.data.repo.ddb.DDBExpressionUserInfoRepository.loadPartition(DDBExpressionUserInfoRepository.java:225)

2. java.util.concurrent.ExecutionException: software.amazon.awssdk.core.exception.SdkClientException at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) at com.careem.launcher.data.repo.ddb.DDBExpressionUserInfoRepository.loadPartition(DDBExpressionUserInfoRepository.java:225)

JDK image is: eclipse-temurin:22-jdk
SDK version: 2.18.21 (which resolved error, thrown from TransformingAsyncResponseHandler issue)

Code snippet:

private Map<String, Object> loadPartition(Integer userId, Collection<String> subjects) {
    Map<String, Object> info = new HashMap<>();
    // building the primary key for each item to be requested
    List<Map<String, AttributeValue>> keys = subjects.stream()
        // we add the subject as an empty map entry to signify a lookup has been done
        .peek(subject -> info.put(subject, null))
        .map(subject -> Map.of(
            "user_id", AttributeValue.builder().n(Integer.toString(userId)).build(),
                SUBJECT_KEY, AttributeValue.builder().s(subject).build())
        )
        .collect(Collectors.toList());

    try {

      BatchGetItemResponse response = this.ddbClient.batchGetItem(
              BatchGetItemRequest.builder()
                      .requestItems(
                              Map.of(this.tableName, KeysAndAttributes.builder().keys(keys).build())
                      )
                      .build()
      ).get();

      Map<String, KeysAndAttributes> unprocessed;
      do {
        response.responses().get(this.tableName)
            .forEach(item -> {
              var subject = item.get(SUBJECT_KEY).s();
              item.forEach(
                  (key, attributeValue) -> {
                    if (isDynamoPayload(key)) {
                      loadDynamoItem(info, subject, key, attributeValue);
                    }
                  }
              );
            });

        // if 16MB or 100 items reached, the response will be truncated
        // the rest can be fetched by loading for unprocessed keys
        unprocessed = response.unprocessedKeys();
        if (!unprocessed.isEmpty()) {
          response = this.ddbClient.batchGetItem(
              BatchGetItemRequest.builder().requestItems(unprocessed).build()
          ).get();
        }
      }
      while (!unprocessed.isEmpty());

      return info;
    } catch (Exception e) {
      log.warn(USER_ACCESS_FAILURE, userId, e);
      circuitBreakerByUserID.put(userId, true);
      Thread.currentThread().interrupt();
      return null;
    }

Expected Behavior

it has to return data based on userId as it was working before.

No Throttle in db.

Current Behavior

Getting multiple exceptions.

Reproduction Steps

  1. Request any userId which present in dynamo db with data.
  2. Send data userId and subjects using SDK batchGetItem.
  3. It throws above exception.

Possible Solution

No response

Additional Information/Context

No response

AWS Java SDK version used

eclipse-temurin:22-jdk

JDK version used

22

Operating System and version

eclipse-temurin

Metadata

Metadata

Assignees

Labels

bugThis issue is a bug.closed-for-stalenessp2This is a standard priority issue

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions