When Stubbing function call it throws cannot be cast to org.apache.spark.sql.Dataset java.lang.ClassCastException:

[shikhil-gupta]

Hi Team, I have written this test but I am facing issue.

test("applyNewDeviceEnrolledChanges") {

val dao: DataAccessLayerDAO = mock(classOf[DataAccessLayerDAO])

val metricStartTimeStamp : Timestamp = new Timestamp(LocalDate.parse("2021-10-03").atStartOfDay(ZoneId.of("UTC")).toInstant.toEpochMilli)

val metricEndTimeStamp : Timestamp = new Timestamp(LocalDate.parse("2021-10-04").atStartOfDay(ZoneId.of("UTC")).toInstant.toEpochMilli)

val rollingPeriod : Int = 1

val metaDataProcessor : DeviceMetadataProcessor = spy(new DeviceMetadataProcessor(spark, dao, metricStartTimeStamp, metricEndTimeStamp, rollingPeriod))

doAnswer(new Answer[DataFrame] { override def answer(invocation: InvocationOnMock): DataFrame = { spark.emptyDataFrame } }).when(metaDataProcessor).loadMasterTableOfNthDay()

metaDataProcessor.process() }

**class DeviceMetadataProcessor(sparkSession: SparkSession, dataAccessDao: IDataAccessLayer, metricStartDate: Timestamp, metricEndDate: Timestamp, rollingPeriod: Int) extends IProcessor { var deviceMasterTableDataNthDay: DataFrame = null; override def process(): Unit = {

	deviceMasterTableDataNthDay = loadMasterTableOfNthDay()
    }
def loadMasterTableOfNthDay(): DataFrame = {
	var masterTable : DataFrame = dataAccessDao.getDeviceProcessedMetaData(deviceMasterTableTemplatePath, metricStartDate, DateUtilities.getStartOfDayTimeStamp(metricEndDate, -1 * rollingPeriod), 1)
}

}**

While stubbing loadMasterTableOfNthDay function of DeviceMetadataProcessor and I am trying to return empty dataframe but it throws me following errors.

Exception:-

com.microsoft.teams.deviceanalytics.core.test.DeviceMetaDataProcessorTest$$anonfun$1$$anonfun$apply$mcV$sp$1$$anon$1 cannot be cast to org.apache.spark.sql.Dataset java.lang.ClassCastException: com.microsoft.teams.deviceanalytics.core.test.DeviceMetaDataProcessorTest$$anonfun$1$$anonfun$apply$mcV$sp$1$$anon$1 cannot be cast to org.apache.spark.sql.Dataset at com.microsoft.teams.deviceanalytics.core.processor.DeviceMetadataProcessor.process(DeviceMetadataProcessor.scala:50) at com.microsoft.teams.deviceanalytics.core.test.DeviceMetaDataProcessorTest$$anonfun$1.apply$mcV$sp(DeviceMetaDataProcessorTest.scala:35) at com.microsoft.teams.deviceanalytics.core.test.DeviceMetaDataProcessorTest$$anonfun$1.apply(DeviceMetaDataProcessorTest.scala:20) at com.microsoft.teams.deviceanalytics.core.test.DeviceMetaDataProcessorTest$$anonfun$1.apply(DeviceMetaDataProcessorTest.scala:20) at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85) at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104) at org.scalatest.Transformer.apply(Transformer.scala:22) at org.scalatest.Transformer.apply(Transformer.scala:20) at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:186) at org.scalatest.TestSuite$class.withFixture(TestSuite.scala:196) at org.scalatest.FunSuite.withFixture(FunSuite.scala:1560) at org.scalatest.FunSuiteLike$class.invokeWithFixture$1(FunSuiteLike.scala:183) at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:196) at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:196) at org.scalatest.SuperEngine.runTestImpl(Engine.scala:289) at org.scalatest.FunSuiteLike$class.runTest(FunSuiteLike.scala:196) at org.scalatest.FunSuite.runTest(FunSuite.scala:1560) at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:229) at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:229) at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:396) at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:384) at scala.collection.immutable.List.foreach(List.scala:392) at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:384) at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:379) at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:461) at org.scalatest.FunSuiteLike$class.runTests(FunSuiteLike.scala:229) at org.scalatest.FunSuite.runTests(FunSuite.scala:1560) at org.scalatest.Suite$class.run(Suite.scala:1147) at org.scalatest.FunSuite.org$scalatest$FunSuiteLike$$super$run(FunSuite.scala:1560) at org.scalatest.FunSuiteLike$$anonfun$run$1.apply(FunSuiteLike.scala:233) at org.scalatest.FunSuiteLike$$anonfun$run$1.apply(FunSuiteLike.scala:233)

[shikhil-gupta]

@bbonanno Can you please look into this issue?

[ultrasecreth]

Given the syntax you're using, you don't seem to be using mockito-scala, would you mind refactoring that and try again? also pls format the code in the issues, is really hard to follow otherwise

[ultrasecreth]

To build on top of my prev comment, for this kind of weird errors, I'd need a repo with some code that can be run and reproduce the issue