We're updating the issue view to help you get more done. 

H2OContext can't be initalized on Databricks cloud

Description

In databricks environment, create cluster, add library - sparkling-water-assembly jar and create notebook with

1 val hc = H2OContext.getOrCreate(spark).

This will fail on class not found exception: org.spark_project.jetty.servlet...

The reason for this is that Spark normally uses shaded jetty in package org.spark_project.jetty and we use it as well https://github.com/h2oai/sparkling-water/blob/81d36ce63794ae5aa60912f6a1533da235f1becf/core/src/main/scala/org/apache/spark/h2o/ui/SparklingWaterInfoPage.scala#L29

However Databricks does not use this shaded jetty an uses regular jetty, that is why we see this exception.

The first idea won't work - change shaded jetty to regular jetty in the class above as we are relocating H2O jetty ot a new package and this class would be relocated as well. We might try to forbid relocation for this particular class as here, we need to use just spark jetty.

The current workaround is to disable SParkling Water UI Tab as

1 2 3 val conf = new H2OConf(spark).set("spark.ui.enabled", "false") val hc = H2OContext.getOrCreate(spark, conf)

Environment

None

Status

Assignee

Jakub Hava

Reporter

Jakub Hava

Labels

None

Release Priority

None

CustomerVisible

No

testcase 1

None

testcase 2

None

testcase 3

None

h2ostream link

None

Affected Spark version

None

AffectedContact

None

AffectedCustomers

None

AffectedPilots

None

AffectedOpenSource

None

Support Assessment

None

Customer Request Type

None

Support ticket URL

None

End date

None

Baseline start date

None

Baseline end date

None

Task progress

None

Task mode

None

Fix versions

Priority

Major