create-rust-app Use Cargo Workspaces in generated project to reduce compilation times

Currently, running the build scripts takes an unnecessary amount of time because all the scripts are treated like separate binaries but have the same dependencies as the backend, meaning that all the dependencies of the backend are compiled twice if you, for example, run cargo fullstack.

https://doc.rust-lang.org/book/ch14-03-cargo-workspaces.html

Mar 30 '23 20:03 AnthonyMichaelTDM

For example, running cargo backend will compile every dependency (upwards of 400-500 crates) once, execute the backend.rs binary, which will then compile every dependency again to run the server when it runs cargo watch.

Despite the fact that backend.rs has no dependencies.

Mar 31 '23 08:03 AnthonyMichaelTDM

Do you mind timing this on a fresh project with and without this workspace change? Also, could you share the output you get in your terminal? I'm not sure I understand the problem

Apr 02 '23 17:04 Wulf

oh sure, I'll get on that

Apr 02 '23 19:04 AnthonyMichaelTDM

Methodology notes

Command used to generate project: cargo run --bin create-rust-app -- create -c -d postgres -b actix-web test-project
ran cargo clean between each test, not doing so does speed up some compilations because build-artifacts are reused by cargo (but not cargo-watch)
used time cargo build --bin <binary> to measure runtime
for the "with workspaces" tests: edited/added manifests (included at end of this comment), moved files around, cleaned up dependencies, split dsync.rs and tsync.rs into binary and libary crates, etc.

Key Point:

all the binaries that run the backend (backend.rs, fullstack.rs) also have to compile it, and cargo-watch doesn't re-use the compilation artifacts from compiling backend/fullstack (whichever was used), this greatly increases the total time spent compiling things

Results

without workspaces

backend
- Crates compiled: 447
- time information: cargo build --bin backend 1087.23s user 106.52s system 645% cpu 3:05.07 total
dsync
- Crates compiled: 447
- time information: cargo build --bin dsync 1090.79s user 109.00s system 571% cpu 3:30.06 total
frontend
- Crates compiled: 447
- time information: cargo build --bin frontend 1052.26s user 106.69s system 607% cpu 3:10.74 total
fullstack
- Crates compiled: 447
- time information: these are all compiling the same dependencies, so they're all taking around 3 minutes
tsync
- Crates compiled: 447
- time information: these are all compiling the same dependencies, so they're all taking around 3 minutes
the backend server (test-project)
- Crates compiled: 447
- time information: these are all compiling the same dependencies, so they're all taking around 3 minutes

with workspaces

backend
- Crates compiled: 1
- time information: cargo build --bin backend 1.16s user 0.41s system 105% cpu 1.479 total
dsync
- Crates compiled: 47 (needs the dsync library)
- time information: cargo build --bin dsync 105.46s user 11.05s system 465% cpu 25.034 total
frontend
- Crates compiled: 404 (needs the create-rust-app libary)
- time information: cargo build --bin frontend 993.41s user 99.09s system 598% cpu 3:02.55 total
fullstack
- Crates compiled: 422 (needs create-rust-app and some other stuff)
- time information: cargo build --bin fullstack 992.44s user 101.83s system 579% cpu 3:08.87 total
tsync
- Crates compiled: 40
- time information: cargo build --bin tsync 65.34s user 8.26s system 325% cpu 22.612 total
the backend server (test-project)
- Crates compiled: 417
- time information: cargo build --bin test-project 958.04s user 94.59s system 534% cpu 3:16.95 total

Manifests

Click to expand

`./Cargo.toml` (root manifest)

[workspace]
members = [
  ".cargo/bin/dsync",
  ".cargo/bin/tsync",
  ".cargo/bin/fullstack",
  ".cargo/bin/frontend",
  ".cargo/bin/backend",
  "backend",
]

[workspace.dependencies]
dsync = "0"
tsync = "1"
create-rust-app = { version = "9", default-features = false, features = [
  "plugin_dev",
  "database_postgres",
  "backend_actix-web",
] }

[profile.dev]
debug-assertions = true

`./backend/Cargo.toml`

[[bin]]
name = "test-project"
path = "src/main.rs"

[dependencies]
actix-files = "0.6.0"
actix-http = "3.0.0"
actix-web = "4.0.1"
futures-util = "0.3.21"
create-rust-app = { workspace = true }
serde_json = "1.0.79"
tsync = { workspace = true }

[dependencies.chrono]
features = ["serde"]
version = "0.4.19"

[dependencies.diesel]
default-features = false
features = ["postgres", "r2d2", "chrono"]
version = "2.0.0-rc.1"

[dependencies.serde]
features = ["derive"]
version = "1.0.133"

[dependencies.tokio]
features = ["full"]
version = "1"

[package]
default-run = "test-project"
edition = "2021"
name = "test-project"
publish = false
version = "0.1.0"

`./.cargo/bin/backend/Cargo.toml`

[package]
name = "backend"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[[bin]]
name = "backend"
path = "src/backend.rs"
bench = false
test = false

[dependencies]

`./.cargo/bin/dsync/Cargo.toml`

[package]
name = "dsync"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[[bin]]
name = "dsync"
path = "src/dsync.rs"
bench = false
test = false

[lib]
name = "dsync_lib"
path = "src/lib.rs"
bench = false
test = false

[dependencies]
dsync = {workspace = true}

`./.cargo/bin/frontend/Cargo.toml`

[package]
name = "frontend"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[[bin]]
name = "frontend"
path = "src/frontend.rs"
bench = false
test = false

[dependencies]
create-rust-app = { workspace = true }

`./.cargo/bin/fullstack/Cargo.toml`

[package]
name = "fullstack"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[[bin]]
name = "fullstack"
path = "src/fullstack.rs"
bench = false
test = false

[dependencies]
create-rust-app = { workspace = true }
dsync = { path = "../dsync" }
tsync = { path = "../tsync" }

`./.cargo/bin/tsync/Cargo.toml`

[package]
name = "tsync"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[[bin]]
name = "tsync"
path = "src/tsync.rs"
bench = false
test = false

[lib]
name = "tsync_lib"
path = "src/lib.rs"
bench = false
test = false

[dependencies]
tsync = {workspace = true}

File tree of "using workspaces" test

Click to expand

command used: tree --dirsfirst ./ .cargo

./
├── backend
│   ├── src
│   │   ├── mail
│   │   │   ├── example.rs
│   │   │   └── mod.rs
│   │   ├── models
│   │   │   └── mod.rs
│   │   ├── services
│   │   │   ├── mod.rs
│   │   │   └── todo.rs
│   │   ├── main.rs
│   │   └── schema.rs
│   ├── views
│   │   └── index.html
│   └── Cargo.toml
├── frontend
│   ├── bundles
│   │   └── index.tsx
│   ├── public
│   │   ├── images
│   │   │   ├── favicon.ico
│   │   │   ├── logo192.png
│   │   │   └── logo512.png
│   │   ├── pwa.json
│   │   └── robots.txt
│   ├── src
│   │   ├── containers
│   │   │   ├── Home.tsx
│   │   │   └── Todo.tsx
│   │   ├── hooks
│   │   │   └── useQueryParam.ts
│   │   ├── images
│   │   │   ├── logo2.svg
│   │   │   ├── logo.svg
│   │   │   └── plus.svg
│   │   ├── types
│   │   │   ├── global.d.ts
│   │   │   ├── plugin-dev.d.ts
│   │   │   ├── rust.d.ts
│   │   │   └── untyped-modules.d.ts
│   │   ├── App.css
│   │   ├── App.tsx
│   │   ├── dev.tsx
│   │   ├── reportWebVitals.js
│   │   └── setupDevelopment.tsx
│   ├── tests
│   │   └── example.spec.ts
│   ├── package.json
│   ├── package-lock.json
│   ├── playwright.config.ts
│   ├── tsconfig.json
│   └── vite.config.ts
├── migrations
│   ├── 00000000000000_diesel_initial_setup
│   │   ├── down.sql
│   │   └── up.sql
│   ├── 00000000000001_utc
│   │   ├── down.sql
│   │   └── up.sql
│   └── 00000000000002_todos
│       ├── down.sql
│       └── up.sql
├── build.rs
├── Cargo.lock
├── Cargo.toml
├── diesel.toml
├── README.md
└── tree
.cargo
├── admin
│   └── dist
│       ├── admin.144d5956.css
│       ├── admin.144d5956.css.map
│       ├── admin.67d0b27a.js
│       ├── admin.67d0b27a.js.map
│       └── admin.html
├── bin
│   ├── backend
│   │   ├── src
│   │   │   └── backend.rs
│   │   └── Cargo.toml
│   ├── dsync
│   │   ├── src
│   │   │   ├── dsync.rs
│   │   │   └── lib.rs
│   │   └── Cargo.toml
│   ├── frontend
│   │   ├── src
│   │   │   └── frontend.rs
│   │   └── Cargo.toml
│   ├── fullstack
│   │   ├── src
│   │   │   └── fullstack.rs
│   │   └── Cargo.toml
│   └── tsync
│       ├── src
│       │   ├── lib.rs
│       │   └── tsync.rs
│       └── Cargo.toml
└── config

Apr 02 '23 20:04 AnthonyMichaelTDM

@AnthonyMichaelTDM thanks a bundle for the write-up and investigation

Apr 03 '23 21:04 Wulf

Use Cargo Workspaces in generated project to reduce compilation times

Methodology notes

Results

without workspaces

with workspaces

Manifests

./Cargo.toml (root manifest)

./backend/Cargo.toml

./.cargo/bin/backend/Cargo.toml

./.cargo/bin/dsync/Cargo.toml

./.cargo/bin/frontend/Cargo.toml

./.cargo/bin/fullstack/Cargo.toml

./.cargo/bin/tsync/Cargo.toml

File tree of "using workspaces" test

`./Cargo.toml` (root manifest)

`./backend/Cargo.toml`

`./.cargo/bin/backend/Cargo.toml`

`./.cargo/bin/dsync/Cargo.toml`

`./.cargo/bin/frontend/Cargo.toml`

`./.cargo/bin/fullstack/Cargo.toml`

`./.cargo/bin/tsync/Cargo.toml`