Add support for Spark Dynamic Allocation

Description

For us not having Spark dynamic allocation is a major obstacle for Sparkling Water
deployment in a multi-tenant environment.

Would be awesome to have this fixed.

We looked at Sparkling Water last year, and we hoped we could start using SW once this is available.

Sparkling Water could just register for LiveListenerBus events as it has events
https://jaceklaskowski.gitbooks.io/mastering-apache-spark/spark-SparkListener.html

onExecutorAdded / onExecutorRemoved to scale SW's memory structures?

https://jaceklaskowski.gitbooks.io/mastering-apache-spark/spark-LiveListenerBus.html

Hope to see this fixed soon - we're excited to become SW users/customers but can't have it without Dynamic Allocation working.

Assignee

Unassigned

Reporter

Ruslan Dautkhanov

Labels

None

CustomerVisible

No

testcase 1

None

testcase 2

None

testcase 3

None

h2ostream link

None

Affected Spark version

None

AffectedContact

None

AffectedCustomers

None

AffectedPilots

None

AffectedOpenSource

None

Support Assessment

None

Customer Request Type

None

Support ticket URL

None

End date

None

Baseline start date

None

Baseline end date

None

Task progress

None

Task mode

None

Components

Affects versions

Priority

Critical
Configure