orango

orango copied to clipboard

This is a living document and may change based on comments provided by the community. This is an RFC for upcoming changes to Orango. Since the API is quite different from the current version or Orango, this will most likely be a Orango 2.0.

Background

Prior to using ArangoDb, I used MongoDb and Mongoose when developing in Node.JS. I wanted something similar to what I was used to with Mongoose. However, though both Mongo and Arango are NoSQL databases, they vary quite differently in syntax and purpose. Orango provides many of the common features that Mongoose provides. However, it is limited in its ability to support GraphQL and other more complex queries. I think its great to get started but falls flat as soon as you have to do something beyond single queries to a collection.

Goals

1. Simplify Orango to handle future changes in ArangoDb
2. Support complex queries that Orango's current Query Builder cannot / will not support
3. Allow developers to design / implement the methods they need without Orango getting in the way.
4. Better error handling
5. True support for Foxx and Web clients (currently not available)

Proposal

This is what I planning to implement in terms of Orango (2.0) in order to handle some limitations.

1. Written in TypeScript - This will compile to JS for web clients and Foxx but provide better support for autocompletion, typo reduction, documentation and better code coverage in unit tests.
2. Orango will be written more as a set of helpers. For example -- Orango's validator could be replaced by altogether because it is something you may or may not use. It is not an all or nothing deal. This eliminates Orango getting in the way of structure. This will also allow for a smaller set of code and the ability for others to contribute more easily.
3. There will no longer be "model" registration. Currently, this is limiting in that you have to register modes to a database. This also created a bunch of confusion and unexpected results if you didn't know what was going on. YOU WILL be able to still register collections and their indexes on each database but the "models" themselves will not be tied to any particular database.
4. Optional Query Builder. Queries will be performed using AQL. Orango will provide a query builder to generate queries but the Query Builder will is no longer be required to take advantage of validation or model conversion. The query builder will provide simple queries for common tasks but may not contain all query options provided by AQL (it just gets way too complicated and feels clunky)
5. JSON to Model can be done independently of other features. Even if you choose to use ArangoJS directly, you can still use a set of helper functions to do model conversion.
6. An ORM will still be provided but is optional. Orango will still provide an ORM but is no longer required to use all the features of Orango.

Some examples for New API:

let config = {}

class User {
  name: String = ''
  comments: Comment[] = []
}

class Comment {
  text: String = ''
  created: Date | undefined
}

async function connection() {
  await orango.db('examples').connect(config)
  console.log(orango.db('examples').connected) // true
  await orango.db('examples').disconnect()

  // alternative
  let examplesDb = await orango.db('examples').connect(config)
  console.log(examplesDb.connected) // true
  await examplesDb.disconnect()
}

async function count() {
  let aql = orango
    .qb()
    .collection('users')
    .count()
    .where({ active: true })
    .toAQL()

  let count = await orango.db('examples')
    .query(aql)
    .one()
}

async function findById() {
  let aql = orango
    .qb()
    .collection('users')
    .find('user')
    .where(`user._key == %id`)
    .toAQL()

  let result = await orango
    .db('examples')
    .query(aql, { id: 'eddie' })
    .one()

  let user = new User()
  json.unmarshal(result, user)
}

async function findOne() {
  let aql = orango
    .qb()
    .collection('users')
    .find('user')
    .one()
    .where(`user._key == %id`)
    .toAQL()

  let result = await orango
    .db('examples')
    .query(aql, { id: 'eddie' })
    .one()

  let user = new User()
  json.unmarshal(result, user)
}

async function findMany() {
  let aql = orango
    .qb()
    .collection('users')
    .find()
    .toAQL()

  let results = await orango
    .db('examples')
    .query(aql)
    .many()

  let users: User[] = []
  results.map(result => {
    let user = new User()
    json.unmarshal(result, user)
    users.push(user)
  })

  // alternative
  let users = await orango
    .db('examples')
    .query(aql)
    .many(User) // pass class in and unmarshal will happen automtically
}

async function findWithSort() {
  let aql = orango
    .qb()
    .collection('users')
    .find('user')
    .sort('lastName firstName')
    .toAQL()

  let results = await orango
    .db('examples')
    .query(aql)
    .many()
}

async function insert() {
  let userValidator = validator({
    name: { type: String, required: 'insert' }
  })

  let body = {
    firstName: 'John',
    lastName: 'Smith',
    name: 'John Smith'
  }

  let err = userValidator.validate(body)
  if (err) {
    // do something
  } else {
    let aql = orango
      .qb()
      .collection('users')
      .insert(body)
      .return()
      .one()
      .toAQL()

    let result = await orango
      .db('examples')
      .query(aql)
      .one()
  }
}

async function remove() {
  let aql = orango
    .qb()
    .collection('users')
    .remove('user')
    .where('user.active == false')
    .toAQL()

  let result = await orango
    .db('examples')
    .query(aql)
    .many()
}

async function subquery() {
  let userValidator = validator({
    name: { type: String, required: 'insert' }
  })

  let body = {
    firstName: 'John',
    lastName: 'Smith',
    name: 'John Smith'
  }

  let err = userValidator.validate(body)
  if (err) {
    // do something
  }

  // FOR comment IN comments FILTER comment.user == user._key RETURN comment
  let aqlComment = orango
    .qb()
    .collection('comments')
    .find('comment')
    .where('comment.user == user._key')
    .skip(0)
    .limit(3)
    .sort('created')
    .toAQL()

  // FOR user IN users FILTER user._key == %id
  //    user.comments = (FOR comment IN comments FILTER comment.user == user._key RETURN comment)
  //    user.message = ("Hello, world!")
  // RETURN user
  let aqlUser = qb
    .collection('users')
    .find('user')
    .where('user._key == %id')
    .set('comments', aqlComment)
    .set('message', 'Hello, world!')
    .toAQL()

  let result = await orango
    .db('examples')
    .query(aqlUser, { id: 'eddie' })
    .one()

  let user = new User()
  json.unmarshal(result, user)
}

This is an example on how one might use these helpers...

class User {
  name: String = ''
  comments: Comment[] = []

  _validator = new Validator({
    name: { type: String, required: 'insert' }
  })

  static _aqlFindById: String = orango
    .qb()
    .collection('users')
    .find('user')
    .where(`user._key == %id`)
    .toAQL()

  static async findById(id: String, dbName = 'examples') {
    return await orango
      .db(dbName)
      .query(this._aqlFindById, { id })
      .one(User)
  }

  validate(data: any) {
    return this._validator.validate(data)
  }
}

class Comment {
  text: String = ''
  created: Date | undefined
}

async function main() {
  let user = await User.findById("eddie")
  // User {name: "Eddie VanHalen"}
}

I scaffolded out a simple working example of some of the functionality and a couple of unit test...

// tslint:disable:no-expression-statement
import { QueryBuilder } from './lib/query';
import { Database, aql } from 'arangojs';
import test from 'ava';
import { Orango } from './index';
import { unmarshalCursor } from './lib/encoding/cursor';
import { unmarshal } from './lib/encoding/json';

let db = Orango.db('examples', {
  url: 'http://localhost:15100'
});

Orango.registerCollection(db, User)

class User {
  email = '';
  firstName = '';
  lastName = '';

  static get collectionName() {
    return 'users';
  }

  static get indexes():any[] {
    return [
      { type: driver.HashIndex, index: {"email"}, unique: true, sparse: false },
    ]
  }

  static async findWithMinimalHelpers(db: Database): Promise<User[]> {
    let collection = db.collection(this.collectionName)
    let query = aql`FOR doc in ${collection} RETURN doc`;
    const cursor = await db.query(query);
    let list = [];
    while (cursor.hasNext()) {
      const result = await cursor.next();
      let classInst = new User();
      let err = unmarshal(result, classInst);
      if (err) {
        throw err;
      }
      list.push(classInst);
    }
    return list;
  }

  static async findWithHelpers(db: Database): Promise<User[]> {
    let query = new QueryBuilder(this.collectionName).find('u').many();
    const cursor = await db.query(query);
    return (await unmarshalCursor(cursor, User)) as User[];
  }
}

test('query without helpers', async t => {
  let users = await User.findWithMinimalHelpers(db);
  console.log("#1", users)
  t.truthy(users.length === 5);
});

test('query with helpers', async t => {
  let users = await User.findWithHelpers(db);
  console.log("#2", users)
  t.truthy(users.length === 5);
});

Experimenting with a new AQL query builder. This will allow for guided auto-completion when building a query. Here is an early example of a graph query and AQL output...

let graph = new GraphQuery()
    .FOR('user', 'follow')
    .IN()
    .OUTBOUND('users/eddie', 'follows')
    .LIMIT(0, 10)
    .SORT('user.firstName', 'user.lastName DESC', 'user.email')
    .RETURN(f.MERGE('user', `{notify:follow.notify}`))
    .toAQL();

// AQL results
FOR user,follow OUTBOUND "users/eddie" follows 
LIMIT 0,10 SORT user.firstName, user.lastName DESC, user.email 
RETURN MERGE(user,{notify:follow.notify})

@roboncode I like the AQL query builder approach...

@maku Thanks, I am going to break out certain parts into their own repos to allow for:

1. Use in other libraries or as a standalone in order to use in Web clients, Foxx, etc
2. Better maintenance and history of changes for each part. The library for AQL Query Builder can be found here as I am working on it... https://github.com/orangojs/aqbjs

@roboncode Sounds great. I did a project with ArangoDB a while ago where I coded an abstraction to use ArangoDb myself. When I do a project in the future I will give the new stuff (Orango 2.0) a try (when it is already available)

Orango 1.0 is fully capable of handling most needs. So don't wait for Orango 2.0 before getting into it. :)

@roboncode would be interested to see Seeding and perhaps GraphQL compatibility, if thats under any consideration?

@roboncode Do you work on Orango 2.0? - if yes, I could imagine to help out in certain areas

@roboncode Would love to help on Orango 2.0 Really hope to see this coming to live!

@nirgn975 Seems not much activity in this project...