Add support for `with-query`

[holyjak]

See the commits for details. Main:

---

DIY static query

We cannot use StaticQuery because the only way to get one int Gatsy is to use JSX (due to the way Gatsby parses/modifies source code via AST), which we can neither use nor produce from Cljs.

Our workaround is inspired by what Gatsby does:

• the with-query macro writes the query into a file and produces a property set to string including a special tag
• the createPages Gatsby hook looks for these files, executes the queries, and outputs corresponding .json files
• the custom Webpack loader query-replacement-loader.js searches the code for the aforementioned tags and replaces them with data from the .json files

Limitations:

• only raw GraphQL supported, without any Relay / Gatsby extensions
• if somehow the query is resolved after webpack has run (is that possible?!) then the page would not show the data unless we find out how to force Gatsby to re-run Webpack after createPages

---

The solution has 3 parts and is thus much more complex then I would like - but I don't see a simpler way. And being limited to only page queries could detract somebody from using Cljs.

[holyjak]

Feedback & improvement suggestion welcome!

[thheller]

Clever but I really don't like writing files during macro expansion. Let alone writing to a global directory where other projects may also end up writing pulling queries from other projects into the current.

There has to be a way to do this without hacking into webpack?

[holyjak]

I understand that. An alternative is to do it in the same time and way that you add the query to the pages.

Not sure what to do else then writing the query into a file:

• Embedding it into the .js file - how? How would we read it in createPages?
• Writing it into a directory inside the project - perhaps better than a global /tmp but then we need to clean its content, perhaps whenever cljs build starts? I thought about using site/.cache but gatsby can clean it after cljs has compiled so our .query files would be gone and not re-created until a cljs code change, so not optimal; perhaps something like site/.cljs-queries/?

Not sure how to do this without webpack. Perhaps I could try to inject the data during createPage, right after I get them, but I think I tried something similar and it triggered a compile loop in gatsby. Can try again.

What do you think?

[thheller]

I did not look into gatsby that much and don't know what createPages is capable of. If you have access to the pages that actually get generated you could inject a <script>var shadow$graphql = {<query-id>:<query-json>,...};</script> and reference that in the code generated by the macro (instead of replacing it in code which breaks source maps).

[holyjak]

Another idea: use NormalModuleReplacementPlugin to inject StaticQueries

The idea is to produce a JSX component that switches among StaticQueries based on the query sha and use that in runtime instead of the mock Switch we use at Cljs compile time.

How does it work:

1. In Cljs, we :require ["/dummy-static-query-switch.js"] that just contains something like module.exports.StaticQuerySwitch = (props) => null; the with-query macro produces something like [:> StaticQuerySwitch {:id "<query-sha>" :render (fn [data] (r/as-element ~body) }] + some metadata
2. In the shadow-cljs compile hook, we read metadata to create the real site/src/components/StaticQuerySwitch.js from all the queries in the namespace (see below)
3. In Gatsby webpack config, we use https://webpack.js.org/plugins/normal-module-replacement-plugin/ to replace dummy-static-query-switch.js with site/src/components/StaticQuerySwitch.jsx

I'm not sure this can actually be done but it is simpler (2 instead of 3 steps), does not create any unnecessary files, and uses Gatsby itself to handle the static queries

// the real StaticQuerySwitch.jsx:
export default const StaticQuerySwitch = (props) => {
  const query = {
    "<query sha1>": <StaticQuery query="graphql query1" render={props.render} />,
    "<query sha2>": <StaticQuery query="graphql query2" render={props.render} /> 
    ...
  }[props.id];
  return {query};
}

[thheller]

But that puts all queries into one file which probably is not ideal with regards to code splitting, eg. all pages will have all data.

[holyjak]

Good point. Now I am out of ideas :-(

I do not see any way how a page would know what static queries are needed to render it, so that we could ensure that only those are imported.

On the other hand, if we limit ourselves to using a simple static to render Layout used everywhere, this would not be a problem.

Wouldn't the proposal to inject <script>var shadow$graphql = {<query-id>:<query-json>,...};</script> (if possible) also include all data in all pages?

What we could do:

1. Similarly to the compile hook, if there is a pre-compile hook, we could generate dummy-static-query-switch.js with many distinct components instead of one: export const StaticQuery_sha1 = () => null; export const StaticQuery_sha2...
2. The macro would produce [:> StaticQuerySwitch_sha1 ...]
3. The post-compile hook would, like before, create site/src/components/StaticQueries.js with the real StaticQuery_<sha> components. We would still have 1 file but now with distinct components that hopefully Webpack can manage to split.

[thheller]

Not if you can do it per page since it would be pretty easy to extract which pages used which queries.

I'm a bit preoccupied with the shadow-cljs UI right now. I'll look into this when I'm done with with that. There should be a way to deal with this. Even its its just via the generates src/pages js files doing the mapping.

[holyjak]

Awesome, good luck with the UI!

it would be pretty easy to extract which pages used which queries

a query can be used somewhere deep in the component tree that makes up a page so I guess you would need an AST that can give you the tree of all functions reachable from the page function? Is that possible? Or is there a simpler way? Anyway, I will wait for what you come up with.