Uploaded image for project: 'SW'
  1. SW-696

Intermittent script test issue on external backend

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.26, 2.1.25, 2.2.11, 2.3.0
    • Component/s: None
    • Labels:
      None
    • CustomerVisible:
      No

      Description

      Script Tests / water.sparkling.scripts.ScriptPipelineHamOrSpamDL.hamOrSpamDeepLearning.script.scala

      Error Message
      java.lang.AssertionError: assertion failed: Value of term "answer2" should be true
      Stacktrace
      java.lang.AssertionError: assertion failed: Value of term "answer2" should be true
      at scala.Predef$.assert(Predef.scala:170)
      at water.sparkling.scripts.HamOrSpamTester$.test(ScriptsTestSuite.scala:246)
      at water.sparkling.scripts.ScriptPipelineHamOrSpamDL$$anonfun$15.apply$mcV$sp(ScriptsTestSuite.scala:230)
      at water.sparkling.scripts.ScriptPipelineHamOrSpamDL$$anonfun$15.apply(ScriptsTestSuite.scala:230)
      at water.sparkling.scripts.ScriptPipelineHamOrSpamDL$$anonfun$15.apply(ScriptsTestSuite.scala:230)
      at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
      at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
      at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
      at org.scalatest.Transformer.apply(Transformer.scala:22)
      at org.scalatest.Transformer.apply(Transformer.scala:20)
      at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:166)
      at org.scalatest.Suite$class.withFixture(Suite.scala:1122)
      at org.scalatest.FunSuite.withFixture(FunSuite.scala:1555)
      at org.scalatest.FunSuiteLike$class.invokeWithFixture$1(FunSuiteLike.scala:163)
      at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:175)
      at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:175)
      at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
      at org.scalatest.FunSuiteLike$class.runTest(FunSuiteLike.scala:175)
      at org.scalatest.FunSuite.runTest(FunSuite.scala:1555)
      at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:208)
      at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:208)
      at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:413)
      at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)
      at scala.collection.immutable.List.foreach(List.scala:381)
      at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
      at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:396)
      at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:483)
      at org.scalatest.FunSuiteLike$class.runTests(FunSuiteLike.scala:208)
      at org.scalatest.FunSuite.runTests(FunSuite.scala:1555)
      at org.scalatest.Suite$class.run(Suite.scala:1424)
      at org.scalatest.FunSuite.org$scalatest$FunSuiteLike$$super$run(FunSuite.scala:1555)
      at org.scalatest.FunSuiteLike$$anonfun$run$1.apply(FunSuiteLike.scala:212)
      at org.scalatest.FunSuiteLike$$anonfun$run$1.apply(FunSuiteLike.scala:212)
      at org.scalatest.SuperEngine.runImpl(Engine.scala:545)
      at org.scalatest.FunSuiteLike$class.run(FunSuiteLike.scala:212)
      at water.sparkling.scripts.ScriptPipelineHamOrSpamDL.org$scalatest$BeforeAndAfterAll$$super$run(ScriptsTestSuite.scala:221)
      at org.scalatest.BeforeAndAfterAll$class.liftedTree1$1(BeforeAndAfterAll.scala:257)
      at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:256)
      at water.sparkling.scripts.ScriptPipelineHamOrSpamDL.run(ScriptsTestSuite.scala:221)
      at org.scalatest.junit.JUnitRunner.run(JUnitRunner.scala:99)
      at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:114)
      at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:57)
      at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:66)
      at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
      at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
      at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
      at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
      at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
      at com.sun.proxy.$Proxy1.processTestClass(Unknown Source)
      at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:108)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
      at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
      at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
      at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:146)
      at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:128)
      at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:404)
      at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
      at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
      at java.lang.Thread.run(Thread.java:745)
      Standard Output

      Interpreter Response:
      import org.apache.spark.SparkFiles
      import org.apache.spark.ml.PipelineModel
      import org.apache.spark.ml.feature._
      import org.apache.spark.ml.h2o.features.ColumnPruner
      import org.apache.spark.ml.h2o.algos.H2OGBM
      import org.apache.spark.sql.types.

      {StringType, StructField, StructType}

      import org.apache.spark.sql.

      {DataFrame, Row, SQLContext}

      import water.support.SparkContextSupport
      import water.fvec.H2OFrame
      import org.apache.spark.ml.Pipeline
      import org.apache.spark.ml.h2o.algos.H2ODeepLearning
      smsDataFileName: String = smsData.txt
      smsDataFilePath: String = examples/smalldata/smsData.txt
      load: (dataFile: String)(implicit sqlContext: org.apache.spark.sql.SQLContext)org.apache.spark.sql.DataFrame
      import org.apache.spark.h2o._
      h2oContext: org.apache.spark.h2o.H2OContext =

      Sparkling Water Context:

      • H2O name: sparkling-water-scripts-tests-1706185573
      • cluster size: 2
      • list of used nodes:
        (executorId, host, port)
        ------------------------
        (0,172.16.2.182,54323)
        (1,172.16.2.186,54323)
        ------------------------

      Open H2O Flow in browser: http://172.16.2.184:54323 (CMD + click in Mac OSX)

      • Yarn App ID of external H2O cluster: application_1513873501160_9479

      sqlContext: org.apache.spark.sql.SQLContext = org.apache.spark.sql.SQLContext@7446115c
      tokenizer: org.apache.spark.ml.feature.RegexTokenizer = regexTok_e0ec7cfbaff9
      stopWordsRemover: org.apache.spark.ml.feature.StopWordsRemover = stopWords_397988ef9205
      hashingTF: org.apache.spark.ml.feature.HashingTF = hashingTF_f6015df94ca7
      idf: org.apache.spark.ml.feature.IDF = idf_a13318886981
      dl: org.apache.spark.ml.h2o.algos.H2ODeepLearning = dl_fc9eadeaa70d
      colPruner: org.apache.spark.ml.h2o.features.ColumnPruner = h2oColRemover_f7f9255362ec
      pipeline: org.apache.spark.ml.Pipeline = pipeline_a8ab901f4cb6
      data: org.apache.spark.sql.DataFrame = label: string, text: string
      model: org.apache.spark.ml.PipelineModel = pipeline_a8ab901f4cb6
      isSpam: (smsText: String, model: org.apache.spark.ml.PipelineModel, hamThreshold: Double)Boolean

      Printed output:
      Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0
      Determining driver host interface for mapper->driver callback...
      Possible callback IP address: 172.16.2.184
      Possible callback IP address: 127.0.0.1
      Using mapper->driver callback IP address and port: 172.16.2.184:56191
      (You can override these with -driverif and -driverport/-driverportrange.)
      Memory Settings:
      mapreduce.map.java.opts: -Xms6g -Xmx6g -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Dlog4j.defaultInitOverride=true
      Extra memory percent: 10
      mapreduce.map.memory.mb: 6758
      18/01/29 08:52:50 INFO impl.TimelineClientImpl: Timeline service address: http://mr-0xd10.0xdata.loc:8188/ws/v1/timeline/
      18/01/29 08:52:50 INFO client.RMProxy: Connecting to ResourceManager at mr-0xd7.0xdata.loc/172.16.2.187:8050
      18/01/29 08:52:51 INFO mapreduce.JobSubmitter: number of splits:2
      18/01/29 08:52:52 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1513873501160_9479
      18/01/29 08:52:52 INFO mapred.H2OYARNRunner: Setting application type to H2O
      18/01/29 08:52:52 INFO impl.YarnClientImpl: Submitted application application_1513873501160_9479
      18/01/29 08:52:52 INFO mapreduce.Job: The url to track the job: http://mr-0xd7.0xdata.loc:8088/proxy/application_1513873501160_9479/
      Job name 'sparkling-water-scripts-tests-1706185573' submitted
      JobTracker job ID is 'job_1513873501160_9479'
      For YARN users, logs command is 'yarn logs -applicationId application_1513873501160_9479'
      Waiting for H2O cluster to come up...
      H2O node 172.16.2.182:54323 requested flatfile
      H2O node 172.16.2.186:54323 requested flatfile
      Sending flatfiles to nodes...
      Sending flatfile to node 172.16.2.182:54323
      Sending flatfile to node 172.16.2.186:54323
      H2O node 172.16.2.186:54323 reports H2O cluster size 1 leader is 172.16.2.186:54323
      H2O node 172.16.2.182:54323 reports H2O cluster size 1 leader is 172.16.2.182:54323
      H2O node 172.16.2.186:54323 reports H2O cluster size 2 leader is 172.16.2.182:54323
      H2O node 172.16.2.182:54323 reports H2O cluster size 2 leader is 172.16.2.182:54323
      Cluster notification file (notify_sparkling-water-scripts-tests-1706185573) created.
      H2O cluster (2 nodes) is up
      Open H2O Flow in your web browser: http://172.16.2.182:54323
      Disowning cluster and exiting.

      For YARN users, logs command is 'yarn logs -applicationId application_1513873501160_9479'

      false
      false

      Standard Error
      Setting default log level to "WARN".
      To adjust logging level use sc.setLogLevel(newLevel).

        Attachments

          Activity

            People

            • Assignee:
              Kuba Jakub Hava
              Reporter:
              Kuba Jakub Hava
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: