automl-toolkit icon indicating copy to clipboard operation
automl-toolkit copied to clipboard

feature interaction: evaluation scoring on original input fields was too slow.

Open HCMY opened this issue 2 years ago • 0 comments

hey guys, i'm reading the source code and i would like to sincerely thank all those works you've done there, and public all of code too. But i noticed that some of code in "FeatureInteraction" is running too slow, for example:

`val nominalScores = nominalFields.map { x => x -> ColumnScoreData( scoreColumn( df, modelType, x, getFieldType("nominal"), totalRecordCount ), "nominal" )

}.toMap

val continuousScores = continuousFields.map { x =>
  x -> ColumnScoreData(
    scoreColumn(
      df,
      modelType,
      x,
      getFieldType("continuous"),
      totalRecordCount
    ),
    "continuous"
  )
}.toMap`

is there any suggestions for paralisim? looking forward your reply!

HCMY avatar Apr 19 '22 11:04 HCMY