Uploaded image for project: 'Public H2O 3'
  1. PUBDEV-3971

Add metalearner_nfolds argument to Stacked Ensemble to enable cross-validation

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.16.0.1
    • Component/s: StackedEnsemble
    • Labels:
      None

      Description

      Just like all the other H2O algos, we should be able to perform k-fold cross-validation of the ensemble. To do a full cross-validation of the base model training + ensemble process, its a very computationally heavy operation. Also, the way that our SE code is set up – it assumes models have already been trained. So the only way to do this is to cross-validate the metalearner only.

      If we were add support for k-fold cross-validation of the full ensemble (not just the metalearner) in the future, then we'd probably set it up as a separate API where the user specifies just the parameters for the base models (instead of the pre-trained models themselves), similar to the `SuperLearner::CV.SuperLearner()` function in R.

        Attachments

          Issue links

            Activity

              People

              • Assignee:
                erin Erin LeDell
                Reporter:
                erin Erin LeDell
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: