ClickHouse icon indicating copy to clipboard operation
ClickHouse copied to clipboard

Enable projections and zero copy replication

Open alesapin opened this issue 2 years ago • 4 comments

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Now projections can be used with zero copy replication.

alesapin avatar Sep 09 '22 16:09 alesapin

I don't understand:

  • why do we need some separate methods to send projections, why don't send the data part as a whole?
  • what are the .proj files and why do we skip them? Maybe duplicate a comment about them?

alexey-milovidov avatar Sep 09 '22 18:09 alexey-milovidov

I still don't understand, what is special about projection files. Why don't treat everything as a directory tree while transferring?

alexey-milovidov avatar Sep 15 '22 02:09 alexey-milovidov

I still don't understand, what is special about projection files. Why don't treat everything as a directory tree while transferring?

Honestly I don't understand your question. We send projection name, we send whole projection checksums (name.proj) inside "parent" checksums. And what is special here? It's quite convenient to use recursive calls of the same functions to send and download projections. If we wound send part with projections just as directory tree we will need more complex logic to understand which file related to which projection and build projection parts inside loop. From my point of view it would be more or less the same complexity.

alesapin avatar Sep 15 '22 11:09 alesapin

Failures unrelated, let's merge

alesapin avatar Sep 21 '22 17:09 alesapin