mars

mars copied to clipboard

Mars lacks pandas API `combine` and `combine_first` to combine two source frames / series into one single result.

wjsi

**Describe the bug** A clear and concise description of what the bug is. **To Reproduce** To help us reproducing this bug, please provide information below: 1. Your Python version 2....

chaokunyang

[core] speeding up mars tiling

1

For some tileable graph, tile took about 40 minutes: ``` df = o.to_mars_dataframe(input_table_name, columns=["id","a","b"],chunk_size=10000) df = df.astype({'id': 'string','a':'float','b':'float'}) df.groupby(["id"])["a","b","id"].apply(predictor()).reset_index().execute() ``` `input_table_name` has about 4 billion rows.  **Describe the solution...

chaokunyang

[Discussion] Enhance evaluation module to facilitate JIT

3

Mars implements `DataFrame.eval` and collects operands (at https://github.com/mars-project/mars/blob/master/mars/optimization/logical/tileable/arithmetic_query.py) that can fit into a string expression applicable for pandas eval. While improving efficiency, this implementation has drawbacks. 1. When meeting non-string...

wjsi

**Describe the bug** `mars.tensor.array_equal` raises error when input tensor's dtype is string. **To Reproduce** To help us reproducing this bug, please provide information below: 1. Your Python version 2. The...

qinxuye

**Describe the bug** Mars integrates some deep learning frameworks(PyTorch, TensorFlow), these frameworks usually need to set some environments for distributed training, `TF_CONFIG` for TensorFlow, `MASTER_ADDR` for PyTorch. We use `ctx.get_worker_addresses()`...

hekaisheng

## What do these changes do? ## Related issue number Fixes #xxxx ## Check code requirements - [ ] tests added / passed (if needed) - [ ] Ensure all...

chaokunyang

Mars roadmaps and enhancement proposals

1

All proposals and roadmaps of Mars are listed here. Suggestions and discussions are sincerely welcomed. Roadmaps ---------- DataFrame (Phase 1: #495, done) Distributed engine (#493) Enhancement Proposals ------------------------ ### Project...

qinxuye

How does this compare to Dask?

9

kootenpv

Integrate paddlepaddle or Mindspore

1

**Is your feature request related to a problem? Please describe.** A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] I want to integrate...

tianlinzx