Anish

Failure seems related to the error class related changes ?

@bogao007 - this PR is doing a lot. Could we please atleast a high level description of what files are being added, how they are being used and what features...

@bogao007 - test failure seems related ? ``` [error] /home/runner/work/spark/spark/sql/core/src/main/scala/org/apache/spark/sql/execution/python/TransformWithStateInPandasExec.scala:51:12: class TransformWithStateInPandasExec needs to be abstract. [error] Missing implementation for member of trait StatefulOperator: [error] def validateAndMaybeEvolveStateSchema(hadoopConf: org.apache.hadoop.conf.Configuration, batchId: Long,...

@bogao007 - license error seems related ? ``` Could not find Apache license headers in the following files: !????? /__w/spark/spark/python/pyspark/sql/streaming/StateMessage_pb2.py !????? /__w/spark/spark/python/pyspark/sql/streaming/StateMessage_pb2.pyi !????? /__w/spark/spark/sql/core/src/main/java/org/apache/spark/sql/execution/streaming/StateMessage.proto !????? /__w/spark/spark/sql/core/src/main/java/org/apache/spark/sql/execution/streaming/state/StateMessage.java ```

@bogao007 - pyspark-core errors are also relevant ? could you PTAL ? thx

@jingz-db @HeartSaVioR - PTAL, thx !

> General comment: JIRA ticket seems to be mentioning list / map state and PR title is mentioning value state. What's correct? Please make a correction. Apologies - JIRA ticket...

@HeartSaVioR @jingz-db - could you ptal ? Thx

> CI failure seems to be related. Yes thanks - fixed the issue. Should be resolved now. Will wait for CI to finish

@ericm-db - in the PR description, please update it to say what the new usage paradigm will look like as well