Clouding process times out constantly leading to Cloud 1 under 4 error

Description

Following tests are regularly failing builds due to `Cloud 1 under 4` error:

Java 8 XGBoost Multinode JUnit / hex.tree.xgboost.BoosterParmsTest.testGetLocalized
Java 8 JUnit / ai.h2o.targetencoding.TargetEncoderModelTest.testTargetEncoderModel

On the screenshot it could be seen that test starts after about 50 seconds after clouding was started and cloud reaches 4 nodes only after tests have already been started.

Suggestion:
1) increase default clouding time from 30 secs to 60 secs
2) Improve logging so that it can be seen when we give up waiting for proper cloud. Add suggestion to increase timeout

Status

Assignee

Andrey Spiridonov

Fix versions

None

Reporter

Andrey Spiridonov

Support ticket URL

None

Labels

None

Affected Spark version

None

Customer Request Type

None

Task progress

None

CustomerVisible

No

Priority

Major
Configure