Kai Huang

Fix intel-analytics/BigDL#4468 This patch is a draft for dlrm workload. Need to refactor this before merge.

Support class_weight and sample_weight in tfpark KerasModel.fit

5

https://groups.google.com/g/bigdl-user-group/c/08SUl5pjDA0/m/R4sFE5NuBAAJ cc @yangw1234

Question: Have you tried feature-based approach without fine tuning BERT?

3

As described in BERT paper Section 5.4, it gives some experiment results of not fine tuning BERT parameters but use BERT as a feature extractor followed by 2 Bi-LSTMs. Wondering...

Fix duplicate repartition in tf2 estimator spark backend

8

## Description In fit, if input is dataframe, we will do repartition if partition < num workers when converting to SparkXShards. After converting to SparkXShards, the data should have num_partitions...

More flexibility in PyTorch train input and outputs

3

From studying vz-recommenders code: 1. Currently we only support features to be ndarrays, a single feature of shape (batch, ) and several features together of shape (batch, n). But if...

**[General]** - [ ] init_orca_context default cores=2, and users may not be aware of this setting and may just call it with default parameters. And in this case if user...

PyTorch Ray Estimator for customized data and train loop

2

- [ ] Check data loader return and model/loss forward return and raise error message if it isn't expected. - [ ] Provide an example for overriding TrainingOperator

https://github.com/intel-analytics/BigDL/blob/main/python/orca/src/bigdl/orca/learn/tf2/spark_runner.py#L25 uses BarrierTaskContext, doesn't work for spark 2.3

Example code styles are not included

3

https://github.com/intel-analytics/BigDL/blob/main/python/dllib/dev/lint-python#L23 https://github.com/intel-analytics/BigDL/blob/main/python/chronos/dev/test/lint-python#L23 https://github.com/intel-analytics/BigDL/blob/main/python/orca/dev/test/lint-python#L23 Currently only the files under `src` are tested, plenty of style errors in examples...

Per Fred's request: `export PYTHONUNBUFFERED=1`