Investigate xgboost exhausts all the memory on a node (-xmx+(1.2*-xmx)) on wide datasets

Description

H2O instance launched with 5x dataset size and 120% extramempercent. Get from log -

`Current usage: 99.4 GB of 99 GB physical memory used; 151.4 GB of 1.2 TB virtual memory used. Killing container.`
`99 = 45 + 1.2 * 45`
(and the number `45` is coming from `-Xmx45g`)

Assignee

Jan Sterba

Fix versions

Reporter

Nidhi Mehta

Support ticket URL

Labels

None

Affected Spark version

None

Customer Request Type

None

Task progress

None

CustomerVisible

No

Priority

Blocker
Configure