Tinybird forward compatible
Description of changes
Changes to make the project compatible with Tinybird Forward:
- Add TYPE endpoint to endpoint pipes. This is the default in Tinybird Classic, and has to be explicit in Tinybird Forward.
- Add an explicit partition key, that was implicit in Classic and reflects the current schema of the data source.
- Remove VERSION from resources, rename the files to include the suffix of the last version, and always refer to resources with the suffix.
The first and second changes are completely inoffensive.
The third change deserves some comments: pipes will be updated atomically for Tinybird Classic with the usual tb push -f, but data sources will refuse to update. This is not an issue because the changes in files in this PR don't actually change the resources in Tinybird. However, removing VERSION and explicitly pinning the resources to the latest one will make iterating them with Tinybird Classic more cumbersome, if anyone does this.
Happy to discuss these changes in this PR, or you can email me at eclbg at tinybird.co if you prefer.
How to check full compatibility with Tinybird Forward
These steps rely on uv (installation)for working with the Classic and Forward CLIs without issues.
- Create a new Tinybird Classic workspace.
- In
mainbranch of the repo - Authenticate with Classic CLI:
uvx --from tinybird-cli tb auth --interactive. Select region and use admin email token. - Push files to workspace:
uvx --from tinybird-cli tb push lib/tinybird/ - Use the Forward CLI to check that the first Forward deployment detects no changes:
uvx --from tinybird tb --cloud deploy --check. Output should be:
Running against Tinybird Cloud: Workspace <your workspace>
» Validating deployment...
* No changes to be deployed
* No changes in tokens to be deployed
@eclbg is attempting to deploy a commit to the mftsio Team on Vercel.
A member of the Team first needs to authorize it.
@eclbg Thanks for the PR 🙌
Add an explicit partition key, that was implicit in Classic and reflects the current schema of the data source.
So the explicit partition key on the video_views__v1 is already applied by default, so pushing that update won't break anything on Tinybird, right?
However, removing VERSION and explicitly pinning the resources to the latest one will make iterating them with Tinybird Classic more cumbersome, if anyone does this.
based on your changes, is there anything I should do in the production side of things on Tinybird?
So in the future when I add a new pipe, I will add the version in the pipe's filename and then push to Tinybird.
A question to Tinybird Forward:
Is Tinybird Forward more than the local container env? The tb --cloud deploy will still deploy to the same Tinybird cloud I'm used to right?