data prepper cannot start with non root user for service-map-pipeline

[chhaosh]

Describe the bug
A clear and concise description of what the bug is.
If we use
securityContext:
runAsUser: 1000

we get
2021-08-31T19:36:57,064 [main] ERROR com.amazon.dataprepper.plugins.PluginFactory - Encountered exception while instantiating the plugin ServiceMapStatefulPrepper
java.lang.reflect.InvocationTargetException: null
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?]
at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
at com.amazon.dataprepper.plugins.PluginFactory.newPlugin(PluginFactory.java:24) ~[data-prepper.jar:1.0.0]
at com.amazon.dataprepper.plugins.prepper.PrepperFactory.newPreppers(PrepperFactory.java:21) ~[data-prepper.jar:1.0.0]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) [?:?]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) [?:?]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) [?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) [?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) [?:?]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) [?:?]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) [?:?]
at com.amazon.dataprepper.parser.PipelineParser.buildPipelineFromConfiguration(PipelineParser.java:92) [data-prepper.jar:1.0.0]
at com.amazon.dataprepper.parser.PipelineParser.parseConfiguration(PipelineParser.java:64) [data-prepper.jar:1.0.0]
at com.amazon.dataprepper.DataPrepper.execute(DataPrepper.java:97) [data-prepper.jar:1.0.0]
at com.amazon.dataprepper.DataPrepperExecute.main(DataPrepperExecute.java:20) [data-prepper.jar:1.0.0]
Caused by: java.lang.RuntimeException: Unable to create the directory at the provided path: service-map
at com.amazon.dataprepper.plugins.prepper.ServiceMapStatefulPrepper.createPath(ServiceMapStatefulPrepper.java:105) ~[data-prepper.jar:1.0.0]
at com.amazon.dataprepper.plugins.prepper.ServiceMapStatefulPrepper.(ServiceMapStatefulPrepper.java:83) ~[data-prepper.jar:1.0.0]
at com.amazon.dataprepper.plugins.prepper.ServiceMapStatefulPrepper.(ServiceMapStatefulPrepper.java:64) ~[data-prepper.jar:1.0.0]
... 18 more
2021-08-31T19:36:57,075 [main] ERROR com.amazon.dataprepper.parser.PipelineParser - Construction of pipeline components failed, skipping building of pipeline [service-map-pipeline] and its connected pipelines
com.amazon.dataprepper.plugins.PluginException: Encountered exception while instantiating the plugin ServiceMapStatefulPrepper
at com.amazon.dataprepper.plugins.PluginFactory.newPlugin(PluginFactory.java:34) ~[data-prepper.jar:1.0.0]
at com.amazon.dataprepper.plugins.prepper.PrepperFactory.newPreppers(PrepperFactory.java:21) ~[data-prepper.jar:1.0.0]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[?:?]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) ~[?:?]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]
at com.amazon.dataprepper.parser.PipelineParser.buildPipelineFromConfiguration(PipelineParser.java:92) [data-prepper.jar:1.0.0]
at com.amazon.dataprepper.parser.PipelineParser.parseConfiguration(PipelineParser.java:64) [data-prepper.jar:1.0.0]
at com.amazon.dataprepper.DataPrepper.execute(DataPrepper.java:97) [data-prepper.jar:1.0.0]
at com.amazon.dataprepper.DataPrepperExecute.main(DataPrepperExecute.java:20) [data-prepper.jar:1.0.0]
Caused by: java.lang.reflect.InvocationTargetException
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?]
at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
at com.amazon.dataprepper.plugins.PluginFactory.newPlugin(PluginFactory.java:24) ~[data-prepper.jar:1.0.0]
... 12 more
Caused by: java.lang.RuntimeException: Unable to create the directory at the provided path: service-map
at com.amazon.dataprepper.plugins.prepper.ServiceMapStatefulPrepper.createPath(ServiceMapStatefulPrepper.java:105) ~[data-prepper.jar:1.0.0]
at com.amazon.dataprepper.plugins.prepper.ServiceMapStatefulPrepper.(ServiceMapStatefulPrepper.java:83) ~[data-prepper.jar:1.0.0]
at com.amazon.dataprepper.plugins.prepper.ServiceMapStatefulPrepper.(ServiceMapStatefulPrepper.java:64) ~[data-prepper.jar:1.0.0]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?]
at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
at com.amazon.dataprepper.plugins.PluginFactory.newPlugin(PluginFactory.java:24) ~[data-prepper.jar:1.0.0]
... 12 more
2021-08-31T19:36:57,078 [main] ERROR com.amazon.dataprepper.DataPrepper - No valid pipeline is available for execution, exiting

and removing securityContext solve the issue, while raw-pipeline works fine with securityContext

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

We are required to not run as root so looking for a way to work around it

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

• OS: [e.g. Ubuntu 20.04 LTS]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

[dlvenable]

@chhaosh ,

This Prepper plugin currently attempts to create a directory relative to the current working directory: data/service-map/. Are you able to create this directory beforehand? If you do, then Data Prepper will not have to create it and you shouldn't have this permissions issue.

We are moving toward using the OpenSearch Data Prepper repository for code and issues. We also have proposal to deploy Data Prepper with its own directory structure. This change would help allow us to have the necessary permissions for creating this DB path in future releases.