Add test for preemption during as_h2o_frame on high concurrency Databricks clusters

Description

We need to advise users whether high concurrency is safe on Azure Databricks.

High concurrency can cause a job to be killed when multiple users are connected to the cluster: https://docs.microsoft.com/en-us/azure/databricks/spark/latest/spark-sql/preemption#preemption

It appears that sparkling water is robust against it, however we need tests to validate this.

Status

Assignee

Unassigned

Reporter

Joseph Granados

Labels

None

CustomerVisible

No

testcase 1

None

testcase 2

None

testcase 3

None

h2ostream link

None

Affected Spark version

None

AffectedContact

None

AffectedCustomers

None

AffectedPilots

None

AffectedOpenSource

None

Support Assessment

None

Customer Request Type

None

Support ticket URL

End date

None

Baseline start date

None

Baseline end date

None

Task progress

None

Task mode

None

Priority

Major
Configure