Double index problems?

[rtfpessoa]

I am having problems running this query:

graph
  .V()
  .has("V_Foo", "prop2", "4ab25da0-3602-4f4a-bc5e-28bfefa5ca4c")
  .outE("edge1", "edge2")
  .inV()
  .toList
  .map(_.property("prop3"))

when I have an index like this:

fooClass.createIndex("V_Foo", OClass.INDEX_TYPE.UNIQUE_HASH_INDEX, "prop1", "prop2")

it seems that he is trying to convert the value of the second field of the index to the type of the first element of the index giving this exception:

Exception in thread "main" java.lang.NumberFormatException: For input string: "4ab25da0-3602-4f4a-bc5e-28bfefa5ca4c"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
	at java.lang.Long.parseLong(Long.java:589)
	at java.lang.Long.parseLong(Long.java:631)
	at com.orientechnologies.orient.core.metadata.schema.OType.convert(OType.java:344)
	at org.apache.tinkerpop.gremlin.orientdb.OrientGraph.convertValue(OrientGraph.java:256)
	at org.apache.tinkerpop.gremlin.orientdb.OrientGraph.lambda$null$6(OrientGraph.java:282)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.Iterator.forEachRemaining(Iterator.java:116)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
	at org.apache.tinkerpop.gremlin.orientdb.traversal.step.sideEffect.OrientGraphStep.lambda$elements$4(OrientGraphStep.java:98)
	at java.lang.Iterable.forEach(Iterable.java:75)
	at org.apache.tinkerpop.gremlin.orientdb.traversal.step.sideEffect.OrientGraphStep.elements(OrientGraphStep.java:93)
	at org.apache.tinkerpop.gremlin.orientdb.traversal.step.sideEffect.OrientGraphStep.vertices(OrientGraphStep.java:54)
	at org.apache.tinkerpop.gremlin.orientdb.traversal.step.sideEffect.OrientGraphStep.lambda$new$0(OrientGraphStep.java:46)
	at org.apache.tinkerpop.gremlin.process.traversal.step.map.GraphStep.processNextStart(GraphStep.java:139)
	at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
	at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50)
	at org.apache.tinkerpop.gremlin.process.traversal.step.map.FlatMapStep.processNextStart(FlatMapStep.java:48)
	at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:128)
	at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:38)
	at org.apache.tinkerpop.gremlin.process.traversal.Traversal.fill(Traversal.java:177)
	at org.apache.tinkerpop.gremlin.process.traversal.Traversal.toList(Traversal.java:115)
	at gremlin.scala.GremlinScala.toList(GremlinScala.scala:30)

If I change the index to:

fooClass.createIndex("V_Foo", OClass.INDEX_TYPE.UNIQUE_HASH_INDEX, "prop2", "prop1")

The error changes to:

Exception in thread "main" com.orientechnologies.orient.core.index.OIndexException: Impossible add as key of a CompositeIndex a value of type class java.lang.String
	DB name="test-1501851693982"
	at com.orientechnologies.orient.core.index.OCompositeCollate.transform(OCompositeCollate.java:62)
	at com.orientechnologies.orient.core.index.OIndexAbstract.getCollatingValue(OIndexAbstract.java:894)
	at com.orientechnologies.orient.core.index.OIndexOneValue.get(OIndexOneValue.java:47)
	at com.orientechnologies.orient.core.index.OIndexOneValue.get(OIndexOneValue.java:40)
	at com.orientechnologies.orient.core.index.OIndexAbstractDelegate.get(OIndexAbstractDelegate.java:58)
	at com.orientechnologies.orient.core.index.OIndexTxAwareOneValue.get(OIndexTxAwareOneValue.java:262)
	at com.orientechnologies.orient.core.index.OIndexTxAwareOneValue.get(OIndexTxAwareOneValue.java:40)
	at org.apache.tinkerpop.gremlin.orientdb.OrientGraph.lookupInIndex(OrientGraph.java:292)
	at org.apache.tinkerpop.gremlin.orientdb.OrientGraph.lambda$null$7(OrientGraph.java:283)
	at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.Iterator.forEachRemaining(Iterator.java:116)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
	at org.apache.tinkerpop.gremlin.orientdb.traversal.step.sideEffect.OrientGraphStep.lambda$elements$4(OrientGraphStep.java:98)
	at java.lang.Iterable.forEach(Iterable.java:75)
	at org.apache.tinkerpop.gremlin.orientdb.traversal.step.sideEffect.OrientGraphStep.elements(OrientGraphStep.java:93)
	at org.apache.tinkerpop.gremlin.orientdb.traversal.step.sideEffect.OrientGraphStep.vertices(OrientGraphStep.java:54)
	at org.apache.tinkerpop.gremlin.orientdb.traversal.step.sideEffect.OrientGraphStep.lambda$new$0(OrientGraphStep.java:46)
	at org.apache.tinkerpop.gremlin.process.traversal.step.map.GraphStep.processNextStart(GraphStep.java:139)
	at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
	at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50)
	at org.apache.tinkerpop.gremlin.process.traversal.step.map.FlatMapStep.processNextStart(FlatMapStep.java:48)
	at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:128)
	at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:38)
	at org.apache.tinkerpop.gremlin.process.traversal.Traversal.fill(Traversal.java:177)
	at org.apache.tinkerpop.gremlin.process.traversal.Traversal.toList(Traversal.java:115)
	at gremlin.scala.GremlinScala.toList(GremlinScala.scala:30)

I noticed this two (#126 #110) similar issues but not sure if it can be the same problem.
Any ideas what can be wrong?

[wolf4ood]

Hi @rtfpessoa

should be related in composite indexing usage.

Let me check

[wolf4ood]

Hi @rtfpessoa

how did you create the class?

[rtfpessoa]

Should be this:

val schema = graph.getRawDatabase.getMetadata.getSchema
val vClass = schema.getClass("V")
val fooClass = schema.createClass(s"V_Foo", vClass)
createClassProperty(fooClass, "field1") // int
createClassProperty(fooClass, "field2") // string

  private def createClassProperty[T](iClass: OClass, key: Key[T], mandatory: Boolean = true, notNull: Boolean = true)
                                    (implicit classTag: ClassTag[T]): Unit = {
    iClass.createProperty(key.name, OType.getTypeByClass(classTag.runtimeClass))
      .setMandatory(mandatory)
      .setNotNull(notNull)
  }

[rtfpessoa]

btw a colleague just found that this problem only happens with index type UNIQUE_HASH_INDEX, NOTUNIQUE_HASH_INDEX if you use for example UNIQUE, NOTUNIQUE it works fine.

[rtfpessoa]

Apparently the query only works if I also use both the filters of the index. If I do a query on only one of the index params it will fail.

Do you have any news?

[wolf4ood]

Hi @rtfpessoa

that's because the hash index cannot be used in order to do lookup with partial key.
If you want to use partial key you should use UNIQUE or NOTUNIQUE

[rtfpessoa]

@maggiolo00 but even with UNIQUE seems like it does not work.

Just to make this clear. If I have some queries that do not want to use the index how can I do it?

[wolf4ood]

hi @rtfpessoa

do you get the same exception with UNIQUE?

I don't think you can remove the Optimization Step per query but let me check

[rtfpessoa]

We tested with unique and I also get that problem.

[rtfpessoa]

Also when we use UNIQUE_HASH_INDEX or NOTUNIQUE_HASH_INDEX index even with the two keys we get:

play.api.http.HttpErrorHandlerExceptions$$anon$1: Execution exception[[OCommandExecutionException: Error on execution of command: sql.select EXPAND( rid ) from index:V_Commit_projectId_UUID_index where key = ?
        DB name="test"
        DB name="test"]]
        at play.api.http.HttpErrorHandlerExceptions$.throwableToUsefulException(HttpErrorHandler.scala:255)
        at play.api.http.DefaultHttpErrorHandler.onServerError(HttpErrorHandler.scala:180)
        at play.core.server.AkkaHttpServer$$anonfun$13$$anonfun$apply$1.applyOrElse(AkkaHttpServer.scala:252)
        at play.core.server.AkkaHttpServer$$anonfun$13$$anonfun$apply$1.applyOrElse(AkkaHttpServer.scala:251)
        at scala.concurrent.Future$$anonfun$recoverWith$1.apply(Future.scala:346)
        at scala.concurrent.Future$$anonfun$recoverWith$1.apply(Future.scala:345)
        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
        at play.api.libs.streams.Execution$trampoline$.execute(Execution.scala:70)
        at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
        at scala.concurrent.impl.Promise$KeptPromise.onComplete(Promise.scala:337)
Caused by: com.orientechnologies.orient.core.exception.OCommandExecutionException: Error on execution of command: sql.select EXPAND( rid ) from index:V_Commit_projectId_UUID_index where key = ?
        DB name="test"
        DB name="test"
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.orientechnologies.orient.client.binary.OChannelBinaryAsynchClient.throwSerializedException(OChannelBinaryAsynchClient.java:437)
        at com.orientechnologies.orient.client.binary.OChannelBinaryAsynchClient.handleStatus(OChannelBinaryAsynchClient.java:388)
        at com.orientechnologies.orient.client.binary.OChannelBinaryAsynchClient.beginResponse(OChannelBinaryAsynchClient.java:270)
        at com.orientechnologies.orient.client.binary.OChannelBinaryAsynchClient.beginResponse(OChannelBinaryAsynchClient.java:162)
        at com.orientechnologies.orient.client.remote.OStorageRemote.beginResponse(OStorageRemote.java:2138)
        at com.orientechnologies.orient.client.remote.OStorageRemote$27.execute(OStorageRemote.java:1160)
Caused by: java.lang.UnsupportedOperationException: iterateEntriesBetween
        at com.orientechnologies.orient.core.index.engine.OHashTableIndexEngine.iterateEntriesBetween(OHashTableIndexEngine.java:203)
        at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doIterateIndexEntriesBetween(OAbstractPaginatedStorage.java:2450)
        at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.iterateIndexEntriesBetween(OAbstractPaginatedStorage.java:2431)
        at com.orientechnologies.orient.core.index.OIndexMultiValues.iterateEntriesBetween(OIndexMultiValues.java:275)
        at com.orientechnologies.orient.core.index.OIndexAbstractDelegate.iterateEntriesBetween(OIndexAbstractDelegate.java:104)
        at com.orientechnologies.orient.core.index.OIndexTxAwareMultiValue.iterateEntriesBetween(OIndexTxAwareMultiValue.java:339)
        at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.searchInIndex(OCommandExecutorSQLSelect.java:2747)
        at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.assignTarget(OCommandExecutorSQLSelect.java:532)
        at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.executeSearch(OCommandExecutorSQLSelect.java:512)
        at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.execute(OCommandExecutorSQLSelect.java:488)