Skip to content

outputs为es的时候,丢数据问题 #42

Open
@fl911

Description

@fl911
  • 前提是配置consistency参数为true
  • es监听中afterBulk(long arg0, BulkRequest arg1, Throwable arg2)方法:
public void afterBulk(long arg0, BulkRequest arg1, Throwable arg2) {
          logger.error("bulk got exception:", arg2);
                        
          for(ActionRequest request : arg1.requests()){
               addFailedMsg(request);
          }
                        
}

该方法中的addFailedMsg(request)传入的参数为ActionRequest,而 addFailedMsg(Object msg):

public void addFailedMsg(Object msg) {
        if (msg instanceof Map) {
            super.addFailedMsg(msg);
            return;
        }

        throw new IllegalArgumentException("addFailedMsg only accept Map instance");
    }

中实际处理时候做了msg是否为 Map 的判断,所以从这里进去的数据永远不会被添加到 failedMsgQueue,从而导致数据丢失

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions