API interface description

[mark-lester]

Sequelise-restful had a method of requesting, via the HEAD of /api/<entity> essentially a JSON.stringify of the relevant sequelize.model. It didnt really work, once you added the associations there was a circular reference and it bombed. But also, I dont really want to have to first call /api/ which gave back an array of the entities it knows about, and then call HEAD on /api/entity for each of them. So I have this ugly thing below, which prints out all the models and skips any object keys starting with _ and a messy list of suspects. It's not something that needs to be within Epilogue, and certainly as it stands it's highly bespoke, I am just touching base that this isn't something Epilogue does, or wants to do. But having just said all that, it would be nice to actually operate on these resources I now make with Epilogue, and print them out, along with their sequelize.model and thus get the end points to assign to url and urlRoot in my Backbone Model and Collection extensions which I generate from the above. So that would make a nice and tidy epilogue-dictionary package, as long as we can work out a neat way of dumping sequelize models. FYI, my quest is a thing called WikiTimetable, which is an open access editor for GTFS https://developers.google.com/transit/gtfs/reference?hl=en . I picked this up again in August having abandoned my initial system (which worked, but not really). I've spent the last 10 weeks discovering node and cobbling this together https://github.com/mark-lester/Gemini, which is basically my application code but I am trying to factor out as much as possible.

restDictionary:function(){
    var self=this;
    return function(req, res){
        res.setHeader('Content-Type', 'application/json');
        var models=_.filter(self.models, function(model){
                return model.Gemini.excludeFromApi !== true;
            });

        res.send(JSON.stringify({
                                 models
        },function(k,v){
            if (k.charAt(0) === '_')
                return undefined;
            if (typeof v === 'object')
                switch(k){
                    case 'classMethods':
                    case 'instanceMethods':
                    case 'modelManager':
                    case 'Model':
                    case 'daoFactoryManager': 
                    case 'daos': 
                    case 'source': 
                    case 'target': 
                    case 'sequelize':
                    case 'GeminiHidden':
                    case 'hooks':
                    case 'lookup':
                        return undefined;
                }
            return v;
        }));
    };
},

[mbroadst]

I feel a bit uncomfortable adding that kind of feature to epilogue because I think it could be considered quite dangerous (you're basically providing a way for someone to remotely iterate your schema, and therefore vastly increase the attack vector). Having said that we could include something similar as long as it's protected behind a feature flag.

So to answer your question:

• does epilogue currently do this? no
• does epilogue want to do this? I haven't found need for it (and you're the first user to request such functionality), but I'm not strictly against it

[mark-lester]

I dont think it should go in either. I was just airing it. sequelize-restful was actually doing it, albeit with problems, so I just wanted to make sure it wasnt being done. Iterating over epilogue resources rather than sequlize models is what my next version will do. We can agree to disagree on whether this makes a system inherently more vulnerable than it presumably already was ;). All they have to do is open firebug and they'll get a pretty good idea as to what's gong on.

[mbroadst]

@mark-lester ah, I thought you meant providing a top level describe method (iterating all available models and therefore giving a map of what's available). I'm not actually against providing this on a model, given that a user already knows the endpoint - are you aware of any specs for JSON schema publishing? that'd probably be a good place to start

[mark-lester]

well yes i am, but not within epilogue. the application needs to know. my code is on github, mr nasty just needs to download and read. I'm reading this, thanks. http://json-schema.org/

[asaf050]

:+1: Could be very useful for creating forms rapidly with angular-formly or similar libraries and for client side validation using validate.js on the browser.

[mark-lester]

There's a load of information, including all the relations but also arbitrary modifiers like "this variable is a colour, so use the colour picker", which is in sequelize and you need to get to the MVC platform. For form building this is obviously critical. I have already been bounced by folks claiming it is a sin worthy of eternal damnation to put this stuff, e.g its a colour or even things like "must be bigger than this other field", in the model definition in the database. They think it should be carved into the UI code. They are wrong and I am right. I dont agree with the attack issues, "mr nasty" who wants to hack or crash your system will easily be able to work out the end points, obscurity is no defence especially if yours is a public project. The ability to load everything that's in sequelize that hasnt been contracted out of the API, and then the ability to dump all of that at the root of the API so the other side can construct a model hierarchy, is seriously useful imo. That said, I am not proposing this as some kind of default functionality within epilogue itself, but as both sides of this need to know how to you work with the wonderful REST builder, here is perhaps where we should discuss some things unless it's too much overhead/off topic. I have done this on backbone, not angular. If you are interested @asaf050 I would happily work with you to get the principle working on Angular. We just need to get Angluar to read a collection of model definitions (not the data models themselves, but the definitions for them) and then potentially instantiate them.

[atulhm]

I'm currently doing this. I send the model definition along with results in a response header. It's increased my productivity significantly and can confirm the value.

Not sure if it makes sense to send it with every list call but so far the cost/benefit is working out for us without the need for a second http call. I also include rest api links for related objects.

Hope this input is helpful.

Sent from my iPhone

On Jun 7, 2016, at 12:09 AM, Mark Lester [email protected] wrote:

There's a load of information, including all the relations but also arbitrary modifiers like "this variable is a colour, so use the colour picker", which is in sequelize and you need to get to the MVC platform. For form building this is obviously critical. I have already been bounced by folks claiming it is a sin worthy of eternal damnation to put this stuff, e.g its a colour or even things like "must be bigger than this other field", in the model definition in the database. They think it should be carved into the UI code. They are wrong and I am right. I dont agree with the attack issues, "mr nasty" who wants to hack or crash your system will easily be able to work out the end points, obscurity is no defence especially if yours is a public project. The ability to load everything that's in sequelize that hasnt been contracted out of the API, and then the ability to dump all of that at the root of the API so the other side can construct a model hierarchy, is seriously useful imo. That said, I am not proposing this as some kind of default functionality within epilogue itself, but as both sides of this need to know how to you work with the wonderful REST builder, here is perhaps where we should discuss some things unless it's too much overhead/off topic. I have done this on backbone, not angular. If you are interested @asaf050 I would happily work with you to get the principle working on Angular. We just need to get Angluar to read a collection of model definitions (not the data models themselves, but the definitions for them) and then potentially instantiate them.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[mark-lester]

@atulhm , are you dumping your own model def, or JSONifying the sequelize one. It's got more info in it, like the relational stuff. You can build a hierarchy in the client with this information which leads to some very generic solutions.

[atulhm]

it’s a mixture of both. since I use composite keys and sequelize doesn’t support that I had to create my own way of dealing with those relations.

I’ve created a data explorer tool that let’s me point to an api rest point and then proceed to explorer over 100 different tables in an editable grid. I’m thinking about blogging about it and turning it into a chrome extension or downloadable app — it mostly generic since there was no way I was going to hand code for all those tables.

On Jun 7, 2016, at 1:26 AM, Mark Lester [email protected] wrote:

@atulhm https://github.com/atulhm , are you dumping your own model def, or JSONifying the sequelize one. It's got more info in it, like the relational stuff. You can build a hierarchy in the client with this information which leads to some very generic solutions.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dchester/epilogue/issues/105#issuecomment-224213408, or mute the thread https://github.com/notifications/unsubscribe/AB8WlvBbpywYZJf5HVWipJDTh-7k7Fwaks5qJSszgaJpZM4GR-WN.

[mark-lester]

you can have a composite index in sequelize, and a virtual field that can be the concatenation of those fields. If it works directly off what is in sequelize then it doesnt matter how someone instantiates their sequelize stuff. I have a model loader, which does all the relationship adding using an obvious array of relation defs. I need to perform agricultural tasks on all these models, like adding relations to some super model, or manipulating the hooks. So I have got this concept now of being able to bless my sequelize schema with functionality like history (for a wiki style system) or an underlying replication system, or just enable concepts like a "sequence", a field which is a guranteed sequential list across some subset (e.g. you have a polypoints table of points in polylines. the need to be sequenced on the same PolyLineId) . It's very useful but of course others will have their own ways of establishing their sequelize models, and obviously, whatever we say in our definition, what matters is what sequelize thinks we've said. And of course, no we dont really want to do this on every call, but you know that. I think all this should go at the root of the api.