feat(database): add cloudflare d1 support

[ChadRosseau]

User description

SUMMARY

This PR adds support for the D1 database in Superset via a new external engine package:

• superset-engine-d1 – Enables Superset to connect to D1 databases.
• sqlalchemy-d1 and dbapi-d1 – Underlying packages required by the engine, recursively installed as dependencies of the engine.

The source code for these packages can be found in this organization

After installation, users can connect to D1 databases in Superset using the d1 engine spec. Example connection string:

d1://{cloudflare_account_id}:{cloudflare_api_token}@{cloudflare_d1_database_id}

This PR also updates docs/docs/configuration/databases.mdx with the appropriate documentation for Cloudflare D1.

TESTING INSTRUCTIONS

All packages have unit tests written. They can be installed by cloning the appropriate repositories and running poetry run pytest

Testing the functionality itself can be done by creating a Cloudflare account and following the connection process.

ADDITIONAL INFORMATION

• [x] Has associated discussion
• [ ] Has associated issue:
• [ ] Required feature flags:
• [ ] Changes UI
• [ ] Includes DB Migration (follow approval process in SIP-59)
  • [ ] Migration is atomic, supports rollback & is backwards-compatible
  • [ ] Confirm DB migration upgrade and downgrade tested
  • [ ] Runtime estimates and downtime expectations provided
• [x] Introduces new feature or API
• [ ] Removes existing feature or API

---

CodeAnt-AI Description

Add Cloudflare D1 database support and connection instructions

What Changed

• Superset documentation now lists Cloudflare D1 in the database reference with an install command and the expected connection string: d1://{cloudflare_account_id}:{cloudflare_api_token}@{cloudflare_d1_database_id}
• The project exposes a "d1" optional dependency group that installs the required D1 client packages to enable D1 connections
• A dedicated Cloudflare D1 section was added to the configuration docs with step-by-step install and connection details

Impact

✅ Can connect Superset to Cloudflare D1 ✅ Easier installation of D1 dependencies via the "d1" extra ✅ Clearer setup and connection instructions for Cloudflare D1

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

[rusackas]

Thanks for this! Pinging @betodealmeida for best practices advice here... having the db engine spec on pypi seems OK, but I wonder if it makes more sense (or not) to just add it to the Superset repo directly? ¯\_(ツ)_/¯

[ChadRosseau]

Thanks for this! Pinging @betodealmeida for best practices advice here... having the db engine spec on pypi seems OK, but I wonder if it makes more sense (or not) to just add it to the Superset repo directly? ¯_(ツ)_/¯

Thanks for this feedback, we actually had a pretty extensive conversation with @betodealmeida regarding external vs internal engine specs. We agreed that for a project like this where we may need to do more releases in the short term (for example if users run into needed features or bugs), then it's better to have the independent release cycle that comes with an external spec, as opposed to having to align our releases with those of Superset.

The main drawback of an external spec is that if the Superset internal API changes then we will need to adjust our implementation too. However, this is a more hypothetical issue that we think is worth dealing with in return for our own release cycles. Our various libraries are also open-source, so the community is also able to help with such adjustments if needed.

In the future after the implementation has been battle-tested by the community, there's also always the option to officially integrate it as an internal engine spec later on.

That being said, we are always open to other thoughts on the matter. Feel free to let us know if there's anything we haven't considered.

[betodealmeida]

Let's also add the package as an optional dependency here: https://github.com/apache/superset/blob/a18b62cf6b82c0ba73902941d5c4f807baa25016/pyproject.toml#L114

Something like:

d1 = [
    "superset-engine-d1>=0.26.3",
    "sqlalchemy-d1>=0.1.0",
    "dbapi-d1>=0.1.0",
]

This way to use d1 people can do:

pip install "superset[d1]"

And have all dependencies installed.

[codeant-ai-for-open-source[bot]]

CodeAnt AI is reviewing your PR.

[codeant-ai-for-open-source[bot]]

Nitpicks 🔍

🔒 No security issues identified

⚡ Recommended areas for review [ ] Sensitive Information Exposure The new D1 docs show an example SQLAlchemy URI containing an API token. URIs (and the Extras field) can be logged, stored in the metadata DB, or exposed in UI screenshots — this could leak credentials. The docs should explicitly warn about these risks and recommend secure alternatives (Secure Extra, environment variables, or SQLALCHEMY_CUSTOM_PASSWORD_STORE). [ ] URL Encoding / Special Characters Connection tokens and account IDs may contain characters that break URL parsing (e.g., '@', ':', '/'). The D1 section doesn't mention URL-encoding special characters (there are other places in the doc that cover encoding); add a cross-reference or inline guidance to avoid confusing failures.

[codeant-ai-for-open-source[bot]]

CodeAnt AI finished reviewing your PR.

[murphylee10]

Let's also add the package as an optional dependency here:

https://github.com/apache/superset/blob/a18b62cf6b82c0ba73902941d5c4f807baa25016/pyproject.toml#L114

Something like:

d1 = [
    "superset-engine-d1>=0.26.3",
    "sqlalchemy-d1>=0.1.0",
    "dbapi-d1>=0.1.0",
]

This way to use d1 people can do:

pip install "superset[d1]"

And have all dependencies installed.

Thanks for the suggestion @betodealmeida! We've added the d1 optional dependency group to pyproject.toml with the three packages included, and also added the alternate pip command to the database documentation.