Use go templ instead of go template

[nithin-bose]

Go Templ is an alternate templating engine that can compile templates down to go code. I find the added compiler support to be really helpful for medium to large projects.

Is it possible to integrate it?

[mikestefanello]

Hi - great question. I am familiar with the project but I haven't yet given it a full try, especially within the scope of this. The last time I tried it out, which was a while ago, the Goland integration and tooling was lacking too much for me to feel comfortable considering it as a default for this project. I do need to try that again. Which IDE are you using and how has your experience been?

When I first came across templ, I was definitely impressed and completely onboard with the concept. Adding type-safety to that layer is a huge gain; not to mention other short-comings with standard templates. However, I was a bit hesitant to include it as a default here because it's quite a heavy dependency, requires you to know the language, patterns, add an IDE plugin, build step(s), etc. I'm not sure which is easier: moving to that or moving away from it.

There's nothing stopping you from using templ within pagoda. I can't say how much work that would be, but it will get pretty tricky, given that there's a lot of functionality baked in to the template renderer, everything else tied to the templates, and how they are built, parsed, and rendered, etc. If you do go down this path, please share your experience along the way. I will also be trying this out somewhat soon, if time permits, so I can evaluate it again and reconsider it being part of this.

[nithin-bose]

Hi @mikestefanello,

I am also just getting used to Templ. I used it in a couple small projects before and am planning to use it in a new project which is when I stumbled upon Pagoda. Pagoda has most of everything I need except for templ. Thanks for the time and effort you have put into Pagoda.

I am on VS Code with the templ plugin installed. It has autocomplete and validations. Yes, it leaves a lot to be desired, but its serving my needs so far. I am a huge fan of devcontainers so having the dependencies and IDE plugins installed is sorted. I agree with the rest of your points. There is a significant barrier to entry for go-templ. That said IMHO, learning go-templates or go-templ seems to have the same effort, especially considering that go-templ lets me use go in the template itself and most of the "magic" in the template renderer in pagoda might just go away.

I did start integrating templ into pagoda. I haven't got it working yet. So far, the following is my progress:

• Made template renderer an interface
• Created templ renderer
• Moved page cache out of template renderer so that both the template renderer and templ renderer can use it

As of now I am converting all gohtml files to templ files trying to keep the current structure as much as possible. Seems like I might have to change all the handlers as well and make some changes to the page module.

I am trying to keep the current template renderer and the templ renderer together. But as I am progressing I feel that just replacing the current template renderer with the templ renderer might be easier. I would love to know your thoughts about this.

I was planning to give a PR once I get this working, but if you are interested in the changes, I can keep in on a branch in my fork. Let me know.

[mikestefanello]

You're very welcome - I'm glad to hear that you're finding it useful.

I've been a bit busy lately, and I still need to wrap up the final cleanup of backlite so that's in a sufficient state (especially now that pagoda uses it). After that, I do intend to devote a good amount of time to re-investigating templ and seeing if it makes sense to incorporate it. I will certainly keep an eye on your branch but please also update this thread as you make progress and let me know what's working and what's not.

Thanks

[nithin-bose]

I am working on the go-templ branch in my fork.

As for the progress:

• I removed the go html renderer and replaced it with the templ renderer thereby making things a lot let complicated. I also removed the template renderer interface as it was no longer needed.
• Page cache is still outside the template renderer
• Since layout, pages and data in the page behave like function and function parameters in templ, the fields related to go html layout and Data in the page struct was removed in favor of a new templComponent field. I would like to remove this field and Title to somewhere else but haven't figured out where yet
• Converted all gohtml files to templ files
• The forms and data structs which were defined in the handlers packge was causing a circular dependency error when used from go-templ files. So moved them to a helpers package for the time being. I am looking for a better solution, suggestions are welcome
• Made changes to the Makefile to generate templ files on run

I am yet to re-implement the functions in funcMap so things like urls and images are broken or rather nothing other than rendering works as of now.

If you get time, do take a look at my branch and let me know if you have any suggestions. This rabbit hole is turning out to be a lot deeper than I thought.

[nithin-bose]

Updates:

• re-implemented the funcMap module, renamed it to funcs as there is no map anymore
• removed sprig as a direct dependency
• Updated link/url/file functions in templates

Everything seem to work fine now. Would be great if you can give it a try, in case I missed something

[mikestefanello]

Looks like great progress so far. I've only done a very quick review, but I will dig in more once I get some time to.

Circular dependencies was the first potential problem I thought of when I initially considered templ. I haven't put any though towards a solution yet, but as you mentioned, we'd definitely need a good pattern for it (something other than helpers, etc).

Some quick thoughts/observations:

• A noticed a few bugs clicking around the UI (paging is broken, the link on the error page is broken, the About link is broken).
• It would be nice to use consts for the route names in the components (they currently exist in handlers but can be moved).
• Maybe the Page concept doesn't quite fit any more, and something else should be considered? Not sure, just noting that we're not locked in to using it.
• Can the Funcs just be moved to templates so they can be called directly?
• Thinking (way) further ahead, I wonder if the form templates can be improved (ie, reusable element components, easier way to add the inline validation stuff, etc)
• Also thinking ahead, it feels like there should be a way to bake in the HTMX integration in a more seamless manner.

Big problem, for me, is it seems like the Goland plugin does not work at all.

[mikestefanello]

I've never tried gomponents and had a very negative reaction when I initially saw it, but I was curious if you've tried it out, since it seems to try to accomplish similar things as templ.

[nithin-bose]

Fixed the following:

• paging
• the link on the error page
• About link

Circular dependencies was the first potential problem I thought of when I initially considered templ. I haven't put any though towards a solution yet, but as you mentioned, we'd definitely need a good pattern for it (something other than helpers, etc).

I totally agree. helpers is too generic and has a tendency for eventually becoming a "dump all" package. A good pattern will need some thought and I would rather do it properly, if we are doing it at all. It'll be a major structure change anyways.

It would be nice to use consts for the route names in the components (they currently exist in handlers but can be moved). Yes, I was thinking of making the routeNames an explicit type. In fact I had given it a try but had the following problems:

• Calling it from handlers in the templ files causes a circular dependency so it has to be moved out of handlers, if so where to move it?
• If we move to a different package, add a route will require changes in two packages handlers and the new package where we are moving the route names to. IMHO changing multiple is a hassle.

Maybe the Page concept doesn't quite fit any more, and something else should be considered? Not sure, just noting that we're not locked in to using it.

Maybe moving the layout, component, title and pagination to a different rendering struct might be a good idea since rest of the fields are project level or request level and can be populated from something like a middleware and added to the context, which will avoid passing around page everywhere. What do you think?

Can the Funcs just be moved to templates so they can be called directly?

Yes it can be, but since the module requires web *echo.Echo which is not available in templates, it will have to be initialized in template_renderer anyways, so I didn't think it made much difference. But If think putting it in templates is better, it can be done.

Thinking (way) further ahead, I wonder if the form templates can be improved (ie, reusable element components, easier way to add the inline validation stuff, etc) Also thinking ahead, it feels like there should be a way to bake in the HTMX integration in a more seamless manner.

Yes agreed

I've never tried gomponents and had a very negative reaction when I initially saw it, but I was curious if you've tried it out, since it seems to try to accomplish similar things as templ.

I had the same negative reaction as you did. IMHO I am not sold on it

• html support is far more mature in IDEs etc
• It looks tedious (using Raw) if you have a lot of custom html attributes like if you use htmx, alpine or anything these days. Kind of beats the purpose IMHO
• Not sure how well and how many custom attributes can be supported if all they ever will be
• Personally, even though I try to avoid JS as much as possible unless the business needs an SPA or something, my background is HTML/JS and am more comfortable with them. This could change in the future though.

I haven't paid for an IDE yet, maybe I should :)

[mikestefanello]

Without IDE support, I really have a hard time taking templ seriously for this project. That's just such a blocker in my mind. Not to mention, all of the other various issues we've discussed. I do like the concept and objective of templ, and I certainly understand that standard Go templates leave a ton to be desired, but perhaps it's just not ready yet.

Are there any other libraries or approaches to consider and try out?

[nithin-bose]

Not at the moment.

I am planning to use pagoda with templ in my future projects. If I make any further changes, I'll push it to the branch and update here.

[leomorpho]

For anyone interested, I have a heavily modified fork that uses Templ instead of gotemplates. I personally haven't looked back. I trust my code so much more than back when I had to pass maps to components. I love the static typing, and it's got some quite fantastic template composition syntax. It's OSS: https://github.com/leomorpho/GoShip I'd be happy to merge some things back into Pagoda, but it's just gotten quite off-road from my initial pagoda fork...

[mikestefanello]

Looks great, thanks for sharing. I'll be sure to dig in and try it out once I have some time available. I'm sure I'll have some templ questions when I start exploring it more.

[PraveenGandhi]

My fork: PraveenGandhi/pagoda on top of @nithin-bose 's fork.

I am trying to replace Ent with SQLc. Hope it is a reference for anyone who wants.

Thank you @mikestefanello. As pagoda is well thought through and well organized, it is not much complicated to replace Ent with SQLc.

Disclimer:

1. I am not proficient in Go, hence there is room for improvement.
2. Not concentrated on Unit testing.

Thank you.

[PraveenGandhi]

I've never tried gomponents and had a very negative reaction when I initially saw it, but I was curious if you've tried it out, since it seems to try to accomplish similar things as templ.

I understand your thoughts, but I have seen similar thing in Python: FastHtml and python community loves it.

As far as I have understood it, it is general server rendered web framework. At the same time it has opiniated way with Pico CSS and HTMX

Other referrences: FastHtml Foundation FastHtml Tech Stack FastHtml Components Example

https://github.com/guettli/frow--fragments-over-the-wire HOTWIRE (HTML over the wire)

[mikestefanello]

@PraveenGandhi You're very welcome. Thanks for the kind words. Out of curiosity, why did you prefer SQLc over Ent? Are you planning to use Gomponents? Have you worked with it enough to be able to share your experience/feedback, etc? I have never really given it a try but I intend to at some point just to see.

[mikestefanello]

htmgo is another interesting (early) project that's similar to gomponents but looks like it has htmx baked in.

edit: I see that gomponents has some htmx support as well.

[nithin-bose]

My problem with libraries like htmgo is the whole write html in go thing.

In my experience when working on small or indie projects, I usually integrate prebuilt html themes (open source or even purchase from someone) which are mostly in html/css/js.

As for medium to large projects, I have generally had front end developer(s) create the themes also usually in html/css/js. Or go full out with a client side rendered framework like react, in which case go would only expose APIs.

In both of the cases above, converting html to go for the additional compiler support seems unnecessary, tedious and slow. Also something like templ seems like a good midway in both the cases, where I get compiler support in places where I interfere with front end code. I am really having trouble understanding who libraries like htmgo or gomponents are catering to. IMHO its more of a niche unless I am missing something.

Edit: I have used Streamlit in python for quick and dirty visualisations in internal dashboards but I dont think I will ever use it for anything customer facing (IMHO not too flexible) .

Just my 2 cents, please take it with a sack of salt :smile:

[mikestefanello]

I do agree, just like I originally did. I was just trying to keep an open mind and compile a running list of all options/alternatives to consider. It's interesting to see how each project approached the problem and perhaps we can see what each of them do well. I did give gomponents a serious try recently and I just don't see how it's practical or appealing. Maybe for very simple use-cases it can be, but I'm not sure. I imagine I would feel very much the same if I tried htmgo.

[PraveenGandhi]

why did you prefer SQLc over Ent?

I am new to GO ecosystem, I found SQLc simple to use, I have no opinoin about Ent, but in general I do not like full blown ORMs and prefer to use SQL and some kind of DSL like jOOQ, ExposedSQL

Are you planning to use Gomponents?

At the moment, I don't have any immediate plans to use Gomponents, but I'm open to exploring it in the future.

htmgo is another interesting (early) project that's similar to gomponents but looks like it has htmx baked in.

Thanks for the suggestion! I did notice that the development feedback loop with htmgo seems a bit slower compared to Pagoda, in terms of seeing changes reflected in the browser. However, I plan to give it another try and explore it further down the line.

[mikestefanello]

I'm renaming this issue and re-opening my exploration in to gomponents; and also requesting any feedback about it as well as potentially adopting it for this project. I started on an experimental branch (not yet pushed) to see how things work out switching from go templates (and my custom template renderer) to using gomponents. I judged it too quickly (and harshly) just from looking at the examples without really thinking things though (like focusing more on the pain points of using standard templates).

It will require a significant rework but I, so far, like the direction that it's heading in. It required a lot to make the standard templates somewhat easy and flexible to use, and even after all of that, it leaves a lot to be desired. Type safety is the obvious one but creating re-usable components in templates are really difficult. Yes, I have some examples, but they are either very simple or make use of sprig's dict function, which is a real headache to use. Or, they are calling out to Form methods directly.. I considered expanding the funcmap drastically to try to help with this, but even that doesn't work well. I think this problem would be amplified even more if you were using a utility-based CSS framework like Tailwind; which I know a lot of people prefer.

I'll push the branch once it's usable, but it'll take a good amount of time to design things in a way that works well, especially for developer experience.

[mikestefanello]

One concern I had about gomponents was performance, as compared to standard templates. There's a ton of nested function calls, and many slices, maps, objects initialized/allocated. Now, for a project like pagoda, it really shouldn't matter much as the idea is rapid development and it's very unlikely that you're building an app used by millions and millions of people where performance is absolutely critical. Developer experience and velocity is more important. But, I was curious.. so I simplified the current home page and recreated that with gomponents to benchmark both approaches. The results are interesting.

Using gomponents:

28650	     40779 ns/op	   32028 B/op	     857 allocs/op

Using the current templates:

16263	     62595 ns/op	   39070 B/op	     529 allocs/op

Using the current templates (first caching the compiled templates):

19015	     61210 ns/op	   37277 B/op	     529 allocs/op

According to this test/benchmark, gomponents is actually quite faster, allocates less total bytes per op, but does quite a lot more allocations.

Then, I realized that not every component you render is going to be dynamic, so there's no reason those cannot be cached in memory. I made a tweak to only cache the navbar (the top bar that contains the app name, search bar, and search modal). Just that one change provided a nice improvement:

29535	     39004 ns/op	   28042 B/op	     723 allocs/op

[mikestefanello]

Here's an initial, rough-draft, proof-of-concept using gomponents: https://github.com/mikestefanello/pagoda/tree/gomponents

Check the contact form and home page (the rest are haven't been changed yet and I only added a few elements on the home page). The contact form is the best example. I'll continue working on this branch so expect constant changes.

The amount of heavily-nested gomponents code is overwhelming at first glance, but working with it is actually very nice and easy: https://github.com/mikestefanello/pagoda/tree/gomponents/pkg/ui. Being able to create dynamic, re-useable, type-safe components, layouts, pages. forms, form elements, etc, is really nice and once you have the bases setup, creating more handlers becomes very easy. As you can see in this form example, not only is building a form very easy, but with the re-usable form elements, you don't have to manually code in the inline-validation on each form now.

This will ultimately eliminate the concept of the Page; as well as the template renderer and funcmap. It will also clean up form handling and introduce type-safety across the board.

Let me know if you have any feedback.

[Jimmy99]

What is the reason for moving the appname into ui ?

[mikestefanello]

What is the reason for moving the appname into ui ?

Good question. If the app name remains in the configuration, then we'd need to pass that along to the ui package somehow. That's tricky because we don't have a "renderer" any more, like we previously had with the template renderer, which was on the container, and made it very easy to inject the configuration dependency in to. I didn't want to build out a formal renderer simply for this reason; and I definitely don't want to have to require that every call from the handler pass it in. If I kept full-page caching, which I decided to remove (more on that later), then a renderer would probably make more sense since I'd also have to inject the cache as another dependency. Left with just the app name, I questioned if this really needs to be configuration. Will it ever change for an app you're building? Will you ever want to override that per-environment. I strongly doubt that so I felt that a simple constant was more than sufficient for it; especially since you really only will use it within the UI.

The full-page caching feature was easy to add when we had the concept of the Page; which handlers build and pass to the template renderer to be rendered. The Page contained everything about the response (status code, headers, data, template names, etc) which made it easy to capture an entire response and cache it. All of this changes now with the switch to gomponents. There's no template renderer and only explicitly what you want to render (models, forms) are passed to the ui layer (no more singular Page type for everything). This makes capturing the entire response quite difficult. It also comes back to the previous point about not wanting to have to build a formal renderer dependency to handle rendering. I certainly could have built out all of this, but I also had to stop and ask, is full-page caching really that desirable as a feature? I'm not sure.. but probably not, if I had to guess. It's very easy to cache data by injecting the cache as a handler dependency; and doing that in the handler gives you way greater control than the caching middleware I added. What do you think?

[Jimmy99]

As you say appname is only ever used in the ui so avoid the complexity and keep it there.

[mikestefanello]

@Jimmy99 I'm going back on what I said about config and I've moved the app name back to it. In practice, I do think there are many cases in which you'd want configuration available the ui, and especially for values that would be dynamic based on your environment. One example that I missed was being able to generate absolute URLs for links in emails; now that you can create an email body using a gomponent. I added a Host field to the app config to store the host so these URLs can be generated. To be able to really test that on a dev or staging environment, you'd want those links to point to the correct environment. Another simple case is something like including analytics JS. I can easily see you wanting to have a specific JS file or account ID per environment.

I did want to keep this, and rendering in general, as simple as possible. What I landed on was simple middleware that will store the configuration in the Echo context, and ui.Request will automatically extract it. I don't love it, but I found it better than the alternatives I was considering. This code has been pushed to the gomponents branch.

[mikestefanello]

The switch to gomponents is now merged. Please let me know if you have any feedback or come across any bugs, issues, etc. Thanks.

[neilyoung]

@PraveenGandhi Nice attempt to use templ. I would definitely favour this instead of gomponents (since it at least has a remaining touch of html).

I cloned your fork, but found a problem with a missing module (which you most likely added but not commited?)

go get github.com/mikestefanello/pagoda/pkg/db/sqlc

Would you mind to share this?