Add Django Forms support

[nerdoc]

I think I'd rather use a bug report here, as it should be tracked. in TetraJSONEncoder, you could add a few lines to encode a django model into JSON, e.g.

class TetraJSONEncoder(json.JSONEncoder):
    ...
    def default(self, obj):
        ...
        elif isinstance(obj, models.Model):
            # serialize model into json
            model = serializers.serialize("json", [obj])[1:-1]
            json_data = json.loads(model)
            # reformat .fields at root level, preserving .pk
            pk = json_data.get("pk")
            json_data = json_data.get("fields")
            json_data["pk"] = pk
            json_data["__type"]="model"
            return json_data

This just a basic implementation that works to encode a Django model into a Alpine.js usable JSON object. It doens't cover related fields, nor m2m, but works. The model is serialized into JSON, the fields dict is made the root item, and the pk field is added.

This is basically what Django-Unicorn does in its serializer. This is where Unicorn really does a great job - I think Tetra could learn from it here, or use bits of its code - it's also MIT licensed.

With this few lines above, I could easily create a demo component like this:

@default.register
class Person(Component):
    person = public(Person.objects.none())

    def load(self, pk) -> None:
        self.person = Person.objects.get(pk=pk)

    @public
    def save(self):
        # this does not work yet, as there is no decoder yet
        self.person.save()

    template = """
    <div>
        <input type='text' x-model='person.first_name'>
        <button @click="save()">Save</button>
    </div>
    """

and use it...

{% @ person pk=object.pk %} 

However, the encoding back into the save() method does not work, as it receives the blank dict (with correctly changed value from the input field, changed by Alpine model) instead of a model. I could not find a deserialization code in Tetra at the first glance, but with this here it should be easy to implement.

@samwillis Maybe I can help a bit or try to make a PR with simple working code + tests, if you give me a few hints how to implement the deserialization and how you want to handle models - maybe you have a completely different approach.

I would definitely reuse as much Unicorn code as possible, as it seems rather stable and covers edge cases.

One thing: you added the __type addition to your TetraJSONEncoder for datetime, set etc. I have picked up this approach and did it the same way. For models, the type "model" is not enough: you need to know the model to create an instance from it using e.g. apps.get_model(), so __type could be e.g. "model.." to parse the model out of it again. Maybe you know a better approach here too, or Unicorn does it better - I couldn't find out yet.

Django Querysets could be supported in one go the same way, of course.

[samwillis]

This is really nice, thanks!

My one reservation is making full Models public, there is a high risk of inadvertently exposing internal data - not all fields on a model should be exposed to JS. Or even worse accidentally making it possible to alter a field from the font end.

How about something like this, we have a separate decorator specifically for Django models that has parameters listing the public read/write fields?

# Readable only
person = public.model(Person.objects.none(), read=['first_name'])

# Or for read write (read implied)
person = public.model(Person.objects.none(), write=['first_name'])

# Or both
person = public.model(Person.objects.none(), read=['username'], write=['first_name'])

When the component is re-loaded we then need a special case for models instances, merging in any (and only!) updated public writable fields into the unpicked model instance ready for saving.

I'm sure you have seen that Tetra already has some custom handling for pickling models, it only saves a reference to them, and re-fetches them from the DB on re-load.

I'm more than happy for you to have a bash at implementing this!

[nerdoc]

Yes, the publishing of all model's data is a problem that has to be addressed. Unicorn does this by adding a list of field names to the Component that should not be exposed. But adding that using the public.model decorator is a very clean approach. I would even go one thing further than read/write: there are fields that should not even show up in the frontend, think of password fields (even if salted/hashed), or birthdays of persons etc. so I would say:

• at least fields=["first_name", "last_name", "dob"] should be a mandatory argument (as e.g. ModelForm does, allowing "__all__" for all fields)
• optionally readonly=["dob"], doing what you said above.

I wouldn't do read/write, as it would not allow for excluded fields. Alternatively, fields would default to "__all__" - But it would allow an "insecure" way to be the default. I think one should explicitely tell the system thatt all fields must be exposed.

[nerdoc]

Is there any time window for the implementation of model/querysets? In my project, I rely on this feature, as I (and I suppose most of tetra users) have many models to integrate... and I don't want to go back to where I came from ATM (sockpuppet, Unicorn, Turbo-Django, HTMX)... Tetra seems really cool - just lacking features, this one I'm missing the most.

So, as I can't really go on in my project, I can use my spare time as well helping you out with this one - even if I am not that experienced than you are - maybe I can do some of the work, just tell me.

[nerdoc]

I've implemented a do_model method with a fields parameter, and it works locally for getting the model into the frontend. But Could you give me a hint where to filter that fields out?

• TetraJSONEncoder? Has no easy access to the Public class/decorator's attributes like fields.
• BasicComponent._add_to_context() could check the fields and match them to the allowed. Feels a bit weird.
• BasicComponent.render()seems too late for me
• Component._call_public_method() seems too late for me too

[nerdoc]

Additionally you could add Form support, like FormView/ModelView does. This way you could add a form to a component (which is what you need all the time when you deal with objects) and use django's Form validation, which is maybe a better approach. For "micro" components, like a todo item, a form is not needed. For bigger ones, it could be easier to use Django's Forms framework. What do you think about that? I'll change the title to include Form support too.

[nerdoc]

When adding Model/QuerySet support, you also have to consider that base models are not serialized. Just tapped into that in Unicorn too.

[samwillis]

Hi @nerdoc, just a super quick message. You're probably thinking I've vanished, I've just been super busy with contract work. I'm going to try and catch up on Tetra over the next week or so.

[nerdoc]

Oh, I thought so (or struck by lightning, or abducted by aliens maybe...) No worries, I am tied up with work in my day job as well +kids & co. I've just been so over-active because I am so enthusiastic about tetra. If you like the idea of changing the name tetraframework -> tetra then do that task first, as the other guy is waiting for your message. Thanks for your great work here.

[burnhamd]

Also interested in this. Forms with validation are a large part of why I haven’t gone beyond the example projects. I’m new to the project but could possibly lend a hand also.

[nerdoc]

Hi @nerdoc, just a super quick message. You're probably thinking I've vanished, I've just been super busy with contract work. I'm going to try and catch up on Tetra over the next week or so.

@samwillis Is there any chance you continue working on tetra? There hasn't been *any action in 2 years, it seems that you abandoned this project, right? Just want to know.

[nerdoc]

Maybe the best bet would be to just "let" Tetra accept models as parameters, but only pass the pk instead of the object and let the load method fetch the object from the db. I think Sam did this with saving the component state this way too.

[nerdoc]

Hi @samwillis - I think I really need some help here. I am stuck since weeks, and can't get further. I've realized a FormComponent which kind of works. But I don't get the guts of Tetra's state encoding/decoding right.

Let's assume the following code:

class TestForm(forms.Form):
    text = forms.CharField()
    select = forms.Select(choices=((1, "foo"), (2, ("bar"))))

@default.register
class TestFormComponent(FormComponent):
    form_class = TestForm
    # language=html
    template: django_html = """
    <div>
    {{ form }}
    <button @click='reset()'>reset</button>
    </div>
    """

    @public
    def reset(self):
        self.text = ""
        # you can make other fields dependent on this one:
        # if self.text = "hello":
        #    self.form.fields["other_field"].queryset = Person.objects.filter(first_name=...)

This is what the FormComponent does. It takes a form class and creates all the component attributes from it's own attrs on the fly. This works pretty well at a first glance.

A form must not be kept in the state I suppose, so I create it in a _pre_load() method just before the overridable load() method from the data. And I automatically set x-model="foo" for each form attribute to the component (_connect_form_fields()), so the frontend/Js variables keep in sync with the HTML form field values and get to the server automatically. FormComponents have a submit() and validate() method, and like their Django counterparts, a form_valid() and form_invalid() method. So far, so good.

2 problems remain:

• I don't get the life cycle of the form right. Created shortly before load(), it can be modified in load() or public methods, like changing widget queryset attrs of fields, depending on other fields. Works great. But I really don't know where to really put the form creation in, as it is needed in resumed states too. _pre_load() is called there too, hence form is created, but I think I'm wrong here.
• There is a big problem when using ForeignKeys

I need to elaborate further:

When e.g. a ModelForm (which contains a FK) is used, the first time the component is rendered, everything is normal, html form field selected is None.

• Now, after the first option in the select is chosen by the user (options' values are PKs=ints), the Js field has the value "1"
• Then the validate() method is called, by clicking a test button.
• The state is resumed from the saved state before in from_state(), the form is created, and then component attrs set from data - here no custom decoding is done, as 1 is taken as int, no TetraJSONDecoder involved.
• But then, it somehow happens that the encoded state gets to the client, and the next time the user clicks the test button (to call validate()) the component attr doesn't get the 1 value, but gets the model instance. Which seems to be good, but it's not done the first time, and hence inconsistently.

@samwillis Maybe you could have a small glance into the form_component branch,especially at teh FormComponent and tell me what I am doing wrong with the form life cycle - or how would you have implemented a Form component (overview).

I'm running in circles already.

[nerdoc]

I've created a call graph for the component/state life cycle in Figma, if anyone is interested. As good as I could.

[nerdoc]

I changed the title to forms support, as models are basically implemented and working, see Todo example.

[nerdoc]

As form support does work, and just 2 things are not working, I'll open separate issues for those, to keep track easier. Basic form support is already implemented in the form_support branch and will be merged soon. So I'll close this issue for now, as it has become a bit overwhelming.