Uploaded image for project: 'SW'
  1. SW-887

Make sure that we can use schemes unsupported by H2O in H2O Confoguration

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.1.34, 2.2.20, 2.3.9
    • Component/s: None
    • Labels:
      None
    • CustomerVisible:
      No
    • Sprint:

      Description

      For example, if we store LDAP login conf on `wasb` data lake as `H2OConf.set_login_conf("wasb://///"), this address is passed to H2O which then tries to read that file but successful as this data source is not supported.

      On the other hand, Spark can read from these paths, so what we can do is to add file using sc.addFile and then read using SparkFiles.get. This will add the file to the spark driver and we can than access it on the local filesystem.

      This can happen behind the `set_login_conf` call so the user does not need to worry about those

      CC: Jorge Hernandez Villapol

        Attachments

          Activity

            People

            • Assignee:
              Kuba Jakub Hava
              Reporter:
              Kuba Jakub Hava
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: