materialize icon indicating copy to clipboard operation
materialize copied to clipboard

[Epic] Support region-allocated data in dataflows end-to-end

Open antiguru opened this issue 1 year ago • 0 comments
trafficstars

Related to #25286.

Change timely, differential, Materialize to support containers on dataflow edges. This enables us to use specific containers, such as flatcontainer that limit the amount of memory allocations shared between threads, and improve locality.

Product outcome

Materialize uses less CPU cycles to compute a result.

Work items

- [ ] https://github.com/TimelyDataflow/timely-dataflow/pull/562
- [ ] https://github.com/TimelyDataflow/timely-dataflow/pull/556
- [ ] https://github.com/TimelyDataflow/timely-dataflow/pull/541
- [ ] https://github.com/TimelyDataflow/differential-dataflow/pull/478
- [ ] https://github.com/TimelyDataflow/differential-dataflow/pull/474
- [ ] https://github.com/TimelyDataflow/differential-dataflow/pull/476
- [ ] https://github.com/TimelyDataflow/differential-dataflow/pull/477
- [ ] https://github.com/TimelyDataflow/differential-dataflow/pull/471
- [ ] https://github.com/TimelyDataflow/differential-dataflow/pull/472
- [ ] https://github.com/TimelyDataflow/differential-dataflow/pull/473
- [ ] https://github.com/TimelyDataflow/differential-dataflow/pull/480
- [ ] Container-invariant sorter trait.
- [ ] Let builders in differential accept chains of data.
- [ ] Differential's half-join operator needs data in vectors.
- [ ] https://github.com/TimelyDataflow/differential-dataflow/pull/487
- [ ] Introduce `MzCollection` that wraps `Collection` in Materialize.
- [ ] https://github.com/TimelyDataflow/differential-dataflow/pull/487

Decision log

  • 1 January 1970. Issue created.

antiguru avatar May 03 '24 20:05 antiguru

📄 Content Checks

Updated: Fri, 17 May 2024 20:47:58 GMT

Found 2 error(s)

content/docs/auth/saml/adfs.mdx

Position Description Rule
738:3-738:89 Unexpected fully-qualified link to developer.hashicorp.com: https://developer.hashicorp.com/vault/docs/auth/saml. Replace with a relative path internal to Developer. Possibly: /vault/docs/auth/saml. ensure-valid-link-format
739:3-739:85 Unexpected fully-qualified link to developer.hashicorp.com: https://developer.hashicorp.com/vault/api-docs/auth/saml. Replace with a relative path internal to Developer. Possibly: /vault/api-docs/auth/saml. ensure-valid-link-format

CI Results: All Go tests succeeded! :white_check_mark:

github-actions[bot] avatar May 17 '24 20:05 github-actions[bot]

Build Results: All builds succeeded! :white_check_mark:

github-actions[bot] avatar Oct 24 '24 18:10 github-actions[bot]