prestodb
Remove stale Presto modules from the main repository
Presto has about 100 submodules, some of which rarely get updated from one release to another. Even there are some updates, most of the updates are about bug fixes and dependency updates. This happens especially on connector plugins.
- Too many submodules in the repository greatly slow down the development velocity, from building and testing perspective.
- The final presto server package and docker image are huge in size, affect the deployment process
Expected Behavior or Use Case
We will identify these submodules and create separate repositories in GitHub prestodb org. For each of these repositories, we will also provide release pipelines, and other supports, to ensure the compatibility with the updated Presto.
Presto Component, Service, or Connector
We will start with connectors.
Updates
- presto-cassandra
- presto-druid
I am going to work on presto-druid and presto-cassandra first. @tdcmeehan
Hi @tdcmeehan, the suggested step is to remove individual submodules from the repo along with the corresponding testing. A removed submodule will be in its own github repo, without updates to make it buildable. Whoever wants to update it later on is responsible for updating it, and I will take care of its release process.
@wanglinsong
A removed submodule will be in its own github repo, without updates to make it buildable.
Can you clarify what you mean by this?
@wanglinsong
A removed submodule will be in its own github repo, without updates to make it buildable.
Can you clarify what you mean by this?
This is just a code move, there are no updates at the beginning, so it is not buildable due to the inheritance nature of its pom.
We can't move the code and leave it in a broken state. When we move a module, it needs to build, and there needs to be CI that tests future changes. We can't do this later because who knows if we'll have time to do it later. It needs to be streamlined so it is easy to add new CI for each module that is moved out.
Then it will never be done. We (@elharo and me) think whoever needs those modules will work on it.
It's unacceptable to move the code and leave it broken. Let's examine the goals of this issue:
Too many submodules in the repository greatly slow down the development velocity, from building and testing perspective.
The final presto server package and docker image are huge in size, affect the deployment process
We don't need to break perfectly working modules to achieve these goals. Let's take a step back and think of another way, or, let's make it easy to set up CI for new modules (I don't even think it's hard to fix the build post move).