The pipeline stops if any erroneous or invalid content is added to the queue

[sachinb80]

We checked the question set flink job and found the below exception. It looks like some input issue in the question set.

Caused by: org.sunbird.job.exception.InvalidInputException: Error while downloading file https://sunbirdsaas.comhttps://[slecosystemprod.blob.core.windows.net/content/content/do_1136691367734149121800/artifact/do](http://slecosystemprod.blob.core.windows.net/content/content/do_1136691367734149121800/artifact/do)
_1136691367734149121800_1668595797658_hq720.jpg
at org.sunbird.job.publish.helpers.ObjectBundle.$anonfun$downloadFiles$3(ObjectBundle.scala:129) ~[content-publish-1.0.0.jar:?]
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659) ~[activity-aggregate-updater-1.0.0.jar:?]
at scala.util.Success.$anonfun$map$1(Try.scala:255) ~[activity-aggregate-updater-1.0.0.jar:?]
at scala.util.Success.map(Try.scala:213) ~[activity-aggregate-updater-1.0.0.jar:?]
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292) ~[activity-aggregate-updater-1.0.0.jar:?]
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33) ~[activity-aggregate-updater-1.0.0.jar:?]
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33) ~[activity-aggregate-updater-1.0.0.jar:?]
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64) ~[activity-aggregate-updater-1.0.0.jar:?]
at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(Unknown Source) ~[?:?]
at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) ~[?:?]
at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source) ~[?:?]
at java.util.concurrent.ForkJoinPool.scan(Unknown Source) ~[?:?]
at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) ~[?:?]
at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) ~[?:?]
Caused by: java.net.UnknownHostException: sunbirdsaas.comhttps

Then,
we have skipped 2 events. Now the job is running fine.

So can anyone help me understand,
Is it the default behavior for sunbird that the pipeline stops if there is one erroneous event introduced in the queue?
Also, since we have now manually updated the offset, how do we get the missed messages to process again?

[Krishnaj20]

@pallakartheekreddy @gandham-santhosh - can you check and respond to this thread?

@vinukumar-vs

[pallakartheekreddy]

@sachinb80
Is it the default behaviour for sunbird that the pipeline stops if there is one erroneous event introduced in the queue?
Ans: Yes, It's a default behaviour.
Also, since we have now manually updated the offset, how do we get the missed messages to process again?
Ans: I hope you updated the offset because the events are invalid. Please correct the events and push them back to the same topic.

@Krishnaj20 @maheshkumargangula

[Krishnaj20]

@sachinb80 - is your query resolved?

[Krishnaj20]

@sachinb80 - Let us know if you have any follow up queries.

[Krishnaj20]

@sachinb80 - marking this as resolved since we haven't heard back from you.