Revise to make use of the MarkLogic Node.js Client API

[dmcassel]

[grtjn]

I suspect this is not going to be a minor change..

[dmcassel]

I agree. But if the slush generator is going to get adopted as the common starting point for apps, I think it's important that it uses the official client API. Thoughts?

[joemfb]

Most of the front-end code in slush and related projects is built for a two-tier architecture (ie, direct use of the REST API). Our use of node is 1) as a proxy to the REST API, and 2) as a server for front-end assets. I understood the node client API to be built for a three-tier architecture. Wouldn't we have to substantially change the front-end components to use it?

[dmcassel]

Slush generated apps are three-tier, the middle tier is just pretty thin. Note that this doesn't have to be an all-at-once change -- right now, node knows where to look for user profiles. That interaction could go through the client API. Likewise, other functions that aren't simple pass-throughs could do the same.

For apps heading to production, you want the middle tier to control what endpoints the client has access to; the proxy is a convenience for rapidly building POCs (nothing wrong with that).

[grtjn]

I agree with Joe that the stack that is currently generated with this slush template is very much focussed on a mostly 2-tier design. A middle tier that is pretty much only doing proxying is hardly worth mentioning. Most important here is that you get a lot of angular directives and services that expect to communicate with MarkLogic REST-api, and also REST extensions that are expected to be used from UI directly. That doesn't mix well with an architecture that focusses on a proper 3-tier design.

Having said that, nothing stops us from allowing this template to generate another style of UI stack. One that uses the nodejs MarkLogic client to full extend. But that would mean creating entire new angular directives and services. Also not sure how to keep those generic, if the middle-tier can be custom-made for the app that you are building..

Personally I would try to avoid mixing both into one. E.g. using both nodejs client, and direct proxying of REST-api..

[dmcassel]

The slush generator is set up to pass through most requests, which is great for POCs (get things running quickly) but probably not for production apps (security important). In a production app, it would be reasonable to set up customized endpoints, which then use the Node Client API to talk to MarkLogic. I can see how that would be a pain for POCs, though.

[grtjn]

A thin or no middle tier doesn't automatically mean less secure.. ;-)

I still think it should be possible to support multiple stacks or at least variants..

[ryan321]

AFAIK, there isn't a way to specify a different username/password for each request with the ML node.js client. So for a multi-user app, there's no way to call rest API endpoints and specify the user at runtime. Creating a separate connection per user wouldn't be feasible either, so until there is a way to specify username/password per request, I don't know if we can adopt the ML node.js client in the slush project.

[dmcassel]

You specify the credentials when you make the connection to MarkLogic. I'd imagine the credentials would either be the defaults for guest users or user-provided, as you're currently doing. server.js currently has a function getAuth(); I could picture something similar that returns a database connection. Something like:

function getConnection(options, session) {
  var conn = require('../config.js').admin;
  if (session.user !== undefined && session.user.name !== undefined) {
    conn.user = session.user.name;
    conn.password = session.user.password;
  return ml.createDatabaseClient(conn);
}

[grtjn]

I'd suggest storing the connection on the session, if that is possible. That would save creating a new one for each request..

[joemfb]

I've done some work in ml-common-ng, trying to replace the query-builder with the API from the official node client (see joemfb/ml-common-ng#1 and the marklogic-integration branch). Such an integration would break backwards compatibility, but would massively increase the query-builder capabilities. And of course, the convergence with officially supported work is desirable.

I'd really appreciate feedback on that issue (the easiest way to test out the integrated query-builder would be to clone the project and modify/write tests).

[grtjn]

We reviewed internally, wanted to report back on that. Most votes were not to actively switch to node-client-api as basis for the (vanilla) app. We are definitely looking into making good use of it otherwise, like in ml-common-ng (see prev comment). We would also like to include some (commented out) examples of how the node-client-api could be used in the middle-tier.

Anyone got some examples? Maybe also worth a Cookbook entry, see #167?

[grtjn]

Pushing away