open code tag up for extension

[srenatus]

trafficstars

Hello 👋

Thanks for Framework, it's been a pleasure to adopt ✨ and I'm planning to keep working with it.

I've been toying with adding Squint as an accepted language (ClojureScript transpiled to JS). My experiment's branch is here, the change is rather light; but it's also roughed in at this point, with some cut corners.

It would be pretty neat if one was able to mix-and-match JS and CLJS in Framework, like

```cljs echo
(defn squint [x]
    (> (:frequency x) 0.06))
```

```js echo
Plot.rectY(alphabet, {
  x: "letter",
  y: "frequency",
  fill: squint,
}).plot()
```

Now, I don't expect Framework to support Squint (although it would be pretty cool). What would be nice, however, would be to open the tags up to customization of some sort. Squint would just be a first example, I could imagine other cool projects getting unlocked by this 💭

I'd be happy to contribute, but I'd need guidance on how to cleanly allow such extensibility in JS.

What do you think?

[Fil]

Interesting idea! (Note: I took the liberty to edit your message to make the fence code apparent.)

[mbostock]

We could allow the project config to register additional code transforms for getLiveSource?

https://github.com/observablehq/framework/blob/97aa3cedfbc006490e0d0df630a22f35003aeb5f/src/markdown.ts#L52-L66

[srenatus]

We could allow the project config to register additional code transforms for getLiveSource?

I think that would be sufficient. Do you want me to just give it a shot? (Is there any precedent in the framework sources, where something additional can be registered?)

[srenatus]

Given SQL tags have landed (🎉), PRQL tags could also be nice and done with this.

[mbostock]

@srenatus Tracking PRQL separately with #1075; some progress in #1078.

[PawelStadnicki]

Upvote from me!

I’ve already experimented with F# as a data loader, and it works great. Now, I’d like to use F# in framework cells alongside JavaScript.

F# is primarily a backend language, but it has Fable, which transpiles F# to JavaScript, TypeScript, Python, Rust, and more. Using F#/Fable within the Framework would look identical to what @srenatus presents for CLJS.

It would be fantastic to have tags readily available at my disposal

[PawelStadnicki]

I have a follow-up question, and I would greatly appreciate an answer since I am not a professional Node.js developer, it may be a silly question for something that works by design.

I read the docs, but I'm not sure if/how the backend transpilation fits there. How does the Framework ensure integrity in the case of backend requests for live coding?

In particular, let’s assume I have code in any language that compiles to JavaScript on the backend:

let evaluate value = 1.67 // any value or computation

This code is sent to the backend for transpilation, and JavaScript code is returned that can be invoked in the browser. How do we ensure that my evaluate function is not mixed up with the requests of other simultaneous users of the dashboard?

Similarly, if I have a function like:

let continuation x = evaluate x // evaluate definition may vary between users

Are the definitions of my functions always isolated from other app users? Is this something that Node.js ensures by design, or does the Framework handle this isolation explicitly?

Many thanks in advance!

[Fil]

@PawelStadnicki for questions, please open a discussion.

[PawelStadnicki]

Thanks @Fil for pointing this out, done https://github.com/observablehq/framework/discussions/1875

[zachcp]

I love this idea as well!

[danielbodart]

I'd love this to be supported as well. I am currently building a prototype that allows BigQuery SQL to be replaced with calls to a server component that hides the original SQL.

So:

``` bigquery
select * from `dataset`
```

Becomes as

``` sql

  const sql = `select * from read_parquet('${url-to-server-component}/000000000000.parquet');`;

```

And generates a HTTP handler that executes the original SQL query and stores the results in a bucket on first call. Basically this is for semi dynamic data but allowing it to be cached and secured at the server (or edge).

Something like this:

export default async function handle(request: Request): Promise<Response> {
    const result = getFromBucket(request.url)

   if (result) return result; 

     const originalSql = sqlFor(request.url);

     const bqQuery = `EXPORT DATA OPTIONS (
         uri = 'gs://${this.bucket}/*.parquet',
         format = 'PARQUET', 
         compression = 'SNAPPY',
         overwrite = true)
     AS ( ${originalSql} )
);`;

    const [job] = await bigquery.createQueryJob({query});
    await job.getQueryResults();

   return getFromBucket(request.url)

}

So any news on this? Or suggestions for how you would like this to look? I.e would you expect this to be a config option, command line or file system convention or some other mechanism?

[danielbodart]

I have created a PR #2032 that adds support, please let me know what you think. As I saw in the other ticket reference adding to the config was the expected direction of travel.