Uploaded image for project: 'SW'
  1. SW-647

PySparkling shell is failing in Spark 2.2.0

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.5
    • Fix Version/s: 2.2.6
    • Component/s: None
    • Labels:
      None
    • CustomerVisible:
      No
    • Sprint:

      Description

      It is not possible to run pysparkling at this point in Spark 2.2.0 (works in Spark 2.2.1):

      The command :
      ```hc = H2OContext.getOrCreate(spark)```
      produces

      ```Py4JJavaError: An error occurred while calling z:org.apache.spark.h2o.JavaH2OContext.getOrCreate.
      : scala.ScalaReflectionException: <none> is not a method
      at scala.reflect.api.Symbols$SymbolApi$class.asMethod(Symbols.scala:228)
      at scala.reflect.internal.Symbols$SymbolContextApiImpl.asMethod(Symbols.scala:84)
      at org.apache.spark.repl.h2o.H2OInterpreter$.getUserJars(H2OInterpreter.scala:95)
      at org.apache.spark.repl.h2o.H2OInterpreter.createSettings(H2OInterpreter.scala:67)
      at org.apache.spark.repl.h2o.BaseH2OInterpreter.initializeInterpreter(BaseH2OInterpreter.scala:102)
      at org.apache.spark.repl.h2o.BaseH2OInterpreter.<init>(BaseH2OInterpreter.scala:292)
      at org.apache.spark.repl.h2o.H2OInterpreter.<init>(H2OInterpreter.scala:42)
      at water.api.scalaInt.ScalaCodeHandler.createInterpreterInPool(ScalaCodeHandler.scala:100)
      at water.api.scalaInt.ScalaCodeHandler$$anonfun$initializeInterpreterPool$1.apply(ScalaCodeHandler.scala:94)
      at water.api.scalaInt.ScalaCodeHandler$$anonfun$initializeInterpreterPool$1.apply(ScalaCodeHandler.scala:93)
      at scala.collection.immutable.Range.foreach(Range.scala:160)
      at water.api.scalaInt.ScalaCodeHandler.initializeInterpreterPool(ScalaCodeHandler.scala:93)
      at water.api.scalaInt.ScalaCodeHandler.<init>(ScalaCodeHandler.scala:37)
      at water.api.scalaInt.ScalaCodeHandler$.registerEndpoints(ScalaCodeHandler.scala:132)```

        Attachments

          Activity

            People

            • Assignee:
              michal Michal Malohlava
              Reporter:
              michal Michal Malohlava
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: