MongoDB support

[marshallmassengill]

trafficstars

Would love to see Mongoid support added. How can I help?

[antillas21]

Hi @marshallmassengill

I would love to finally have Mongoid support for this gem :)

If you want to help drive out this feature, please take a look at this gem's v-0-4-0 branch here: https://github.com/antillas21/ajax-datatables-rails/tree/v-0-4-0

This branch represents a rewrite on some methods and code organization for the gem. The idea behind this effort is to provide a better API to developers, and become the 1.0.0 version of the gem.

You will notice here in the AjaxDatatablesRails::Base that most methods are hook methods that have to be implemented by an ORM specific module, like the default AjaxDatatablesRails::ORM::ActiveRecord module.

So, if you'd like to help, you can:

• take a look at the code
• add an AjaxDatatablesRails::ORM::Mongoid module inside the lib/ajax-datatables-rails/orm/ directory
• implement the corresponding hook methods inside the module
• add the corresponding tests (this is very important)
• open a PR
• get credited by the feature in the CHANGELOG when the merge is completed

[marshallmassengill]

I'll start tinkering and let you know how far I get with it. I've got a project driving this so I have some motivation.

Thanks!

---Marshall

On Thursday, July 9, 2015, Antonio Antillon [email protected] wrote:

Hi @marshallmassengill https://github.com/marshallmassengill

I would love to finally have Mongoid support for this gem :)

If you want to help drive out this feature, please take a look at this gem's v-0-4-0 branch here: https://github.com/antillas21/ajax-datatables-rails/tree/v-0-4-0

This branch represents a rewrite on some methods and code organization for the gem. The idea behind this effort is to provide a better API to developers, and become the 1.0.0 version of the gem.

You will notice here in the AjaxDatatablesRails::Base https://github.com/antillas21/ajax-datatables-rails/blob/v-0-4-0/lib/ajax-datatables-rails/base.rb#L64-L78 that most methods are hook methods that have to be implemented by an ORM specific module, like the default AjaxDatatablesRails::ORM::ActiveRecord https://github.com/antillas21/ajax-datatables-rails/blob/v-0-4-0/lib/ajax-datatables-rails/orm/active_record.rb module.

So, if you'd like to help, you can:

• take a look at the code
• add an AjaxDatatablesRails::ORM::Mongoid module inside the lib/ajax-datatables-rails/orm/ directory
• implement the corresponding hook methods inside the module
• add the corresponding tests (this is very important)
• open a PR
• get credited by the feature in the CHANGELOG when the merge is completed

— Reply to this email directly or view it on GitHub https://github.com/antillas21/ajax-datatables-rails/issues/110#issuecomment-120090671 .

---Marshall

[antillas21]

Excellent! Let me know if you get stuck anywhere, perhaps I may help.

Really looking forward to what you build :)

[marshallmassengill]

I need some help with understanding what is going on with the calls to Arel. I'm not familiar with Arel. I've been reading about it and the nearest I can figure is that you are creating a new table from the search results. Is that correct?

What's going on with the join in the sort_column method?

---Marshall

On Thu, Jul 9, 2015 at 2:38 PM, Antonio Antillon [email protected] wrote:

Excellent! Let me know if you get stuck anywhere, perhaps I may help.

Really looking forward to what you build :)

— Reply to this email directly or view it on GitHub https://github.com/antillas21/ajax-datatables-rails/issues/110#issuecomment-120099459 .

[antillas21]

Hey @marshallmassengill ,

Inside the AjaxDatatablesRails::ORM::ActiveRecord module, we use Arel to build a search query targeting all and each of the columns defined inside the view_columns method (regular jQuery.dataTables approach) or a search query targeting a specific column (through the use of an additional jQuery plugin or hand-made approach following dataTables documentation).

You can see an example of the SQL query produced in these specs, you will notice we cast columns to TEXT, VARCHAR, or CHAR depending on the db engine used.

Now, as you will be implementing support for Mongoid, I'm not sure you will interact with Arel at all, I'd like to think you won't need to.

Perhaps the following is oversimplistic, but somehow I think this would be your starting point for implementing Mongoid support.

Hope all this helps to wrap your mind around the library. Let me know if I can help you with anything else.

[marshallmassengill]

That's actually pretty much what I have right now, albeit a bit more filled in. I removed a bunch of methods since it doesn't seem like I need them with Mongo handling the search the way it does.

I ran into an error related to javascript and trying to load the AjaxDatatablesRails::ORM::Mongoid module though. How does that get loaded? Is it just the config file that I need to worry about?

Thanks!

---Marshall

On Fri, Jul 10, 2015 at 12:20 PM, Antonio Antillon <[email protected]

wrote:

Hey @marshallmassengill https://github.com/marshallmassengill ,

Inside the AjaxDatatablesRails::ORM::ActiveRecord module, we use Arel to build a search query targeting all and each of the columns defined inside the view_columns method (regular jQuery.dataTables approach) or a search query targeting a specific column (through the use of an additional jQuery plugin or hand-made approach following dataTables documentation).

You can see an example of the SQL query produced in these specs https://github.com/antillas21/ajax-datatables-rails/blob/v-0-4-0/spec/ajax-datatables-rails/orm/active_record_filter_records_spec.rb#L144, you will notice we cast columns to TEXT, VARCHAR, or CHAR https://github.com/antillas21/ajax-datatables-rails/blob/v-0-4-0/spec/ajax-datatables-rails/orm/active_record_filter_records_spec.rb#L189-L217 depending on the db engine used.

Now, as you will be implementing support for Mongoid, I'm not sure you will interact with Arel at all, I'd like to think you won't need to.

Perhaps the following is oversimplistic, but somehow I think this would be your starting point https://gist.github.com/antillas21/1e155f2b3f3ccd3d50a6 for implementing Mongoid support.

Hope all this helps to wrap your mind around the library. Let me know if I can help you with anything else.

— Reply to this email directly or view it on GitHub https://github.com/antillas21/ajax-datatables-rails/issues/110#issuecomment-120449970 .

[antillas21]

Ok, make sure you:

• add the new mongoid module to the list of module requires here: https://github.com/antillas21/ajax-datatables-rails/blob/v-0-4-0/lib/ajax-datatables-rails.rb
• add the corresponding extend clause here: https://github.com/antillas21/ajax-datatables-rails/blob/v-0-4-0/lib/ajax-datatables-rails/base.rb#L99-L100

And that should be all that is needed to wire the new ORM module into the gem.

[marshallmassengill]

So I was making progress and I've hit a wall with something that seems to be more related to ajax or javascript than anything else. I'm not sure how to troubleshoot it to be honest.

When loading my site, I'm getting this error: SyntaxError: [stdin]:8:9: unexpected :

<%= stylesheet_link_tag    "application" %>
<%= javascript_include_tag "vendor/modernizr" %>
<%= javascript_include_tag "application", 'data-turbolinks-track' => true %>
<%= csrf_meta_tags %>

It seems to be coming from the application.js file but I have no idea what would be causing the issue. Any ideas or suggestions?

Thanks!

---Marshall

On Fri, Jul 10, 2015 at 1:00 PM, Antonio Antillon [email protected] wrote:

Ok, make sure you:

• add the new mongoid module to the list of module requires here: https://github.com/antillas21/ajax-datatables-rails/blob/v-0-4-0/lib/ajax-datatables-rails.rb
• add the corresponding extend clause here: https://github.com/antillas21/ajax-datatables-rails/blob/v-0-4-0/lib/ajax-datatables-rails/base.rb#L99-L100

And that should be all that is needed to wire the new ORM module into the gem.

— Reply to this email directly or view it on GitHub https://github.com/antillas21/ajax-datatables-rails/issues/110#issuecomment-120460344 .

[antillas21]

@marshallmassengill well... to be honest, as a rule of thumb, I always suggest to turn off turbolinks and only leave it on if you know what you're doing.

My recommendation would be:

<head>
  <%= stylesheet_link_tag    "application" %>
  <%= javascript_include_tag "vendor/modernizr" %>
  <%= javascript_include_tag "application" %>
  <%= csrf_meta_tags %>
</head>

[marshallmassengill]

Yeah, I actually went down that road initially thinking it was turbolinks but it doesn't seem to be. It's kind of a red herring since it is just a parameter for loading the application.js file, which is where the issue is originating.

Even without the turbolinks reference, I still get the same error. Some other interesting things are happening in the .coffee file:

$ -> $('#virtualMachinesTable').dataTable processing: true serverSide: true ajax: $('#virtualMachinesTable').data('source') pagingType: 'full_numbers'

That doesn't work, kicking out the same 500 error but this does at least load the page (albeit with no entries):

$ -> $('#virtualMachinesTable').dataTable processing: true serverSide: true #ajax: $('#virtualMachinesTable').data('source') pagingType: 'full_numbers'

So the issue seems to be coming from that line but I'm honestly not sure why and tracing it back further has been a struggle so far.

---Marshall

On Mon, Jul 13, 2015 at 1:34 PM, Antonio Antillon [email protected] wrote:

@marshallmassengill https://github.com/marshallmassengill well... to be honest, as a rule of thumb, I always suggest to turn off turbolinks and only leave it on if you know what you're doing.

My recommendation would be:

— Reply to this email directly or view it on GitHub https://github.com/antillas21/ajax-datatables-rails/issues/110#issuecomment-121000735 .

[antillas21]

If that's a coffee file, remember that indentation matters... I would also check that #virtualMachinesTable table would have a data-source attribute and that is pointing to the JSON version of the route that will instantiate the datatables class, otherwise this can be causing the trouble.

I may be available to help you debug this if you can/want to set a screenshare session (Google Hangout, preferably).

[marshallmassengill]

Ok... this is going to take me a little longer than I thought. I need a new keyboard since I just took mine out back and shot it for its treasonous tabbing behavior.

Thanks for the help.

I typed this via very quick pulsing of the USB lines to make the characters. #codesohard

---Marshall

On Mon, Jul 13, 2015 at 1:53 PM, Antonio Antillon [email protected] wrote:

If that's a coffee file, remember that indentation matters... I would also check that #virtualMachinesTable table would have a data-source attribute and that is pointing to the JSON version of the route that will instantiate the datatables class, otherwise this can be causing the trouble.

I may be available to help you debug this if you can/want to set a screenshare session (Google Hangout, preferably).

— Reply to this email directly or view it on GitHub https://github.com/antillas21/ajax-datatables-rails/issues/110#issuecomment-121005316 .

[antillas21]

LOL! No worries, as I said before, I am interested in finally adding support for mongoid (several people have already asked about it), so if I can help in any way to ease the process, don't hesitate in scheduling a hangout/screenshare to debug.

[marshallmassengill]

Alright,

I got past that and now I'm stuck on this one:

AjaxDatatablesRails::NotImplemented in VirtualmachinesController#index

    You should implement this method in your class and return an array
    of database columns based on the columns displayed in the HTML view.
    These columns should be represented in the ModelName.column_name,
    or aliased_join_table.column_name notation.

It seems like this should be pretty straightforward. Do I need to add an include to the controller?

---Marshall

On Mon, Jul 13, 2015 at 2:05 PM, Antonio Antillon [email protected] wrote:

LOL! No worries, as I said before, I am interested in finally adding support for mongoid (several people have already asked about it), so if I can help in any way to ease the process, don't hesitate in scheduling a hangout/screenshare to debug.

— Reply to this email directly or view it on GitHub https://github.com/antillas21/ajax-datatables-rails/issues/110#issuecomment-121008725 .

[antillas21]

No, the error should be fixed once you add a view_columns method in your datatable class as it's coming from here: https://github.com/antillas21/ajax-datatables-rails/blob/v-0-4-0/lib/ajax-datatables-rails/base.rb#L123-L131 and called here: https://github.com/antillas21/ajax-datatables-rails/blob/v-0-4-0/lib/ajax-datatables-rails/base.rb#L20-L22

So, your datatable class should define a view_columns method, say:

class MyMongoidDatatable < AjaxDatatablesRails::Base
  def view_columns
    @view_columns ||= [
      # this is the array of columns the error mentions
    ]
  end
end

[marshallmassengill]

That works. It shows up. It ain't pretty but the names are showing up finally. Looks like sort and search aren't working as expected yet.

---Marshall

On Mon, Jul 13, 2015 at 2:52 PM, Antonio Antillon [email protected] wrote:

No, the error should be fixed once you add a view_columns method in your datatable class as it's coming from here: https://github.com/antillas21/ajax-datatables-rails/blob/v-0-4-0/lib/ajax-datatables-rails/base.rb#L123-L131 and called here: https://github.com/antillas21/ajax-datatables-rails/blob/v-0-4-0/lib/ajax-datatables-rails/base.rb#L20-L22

So, your datatable class should define a view_columns method, say:

class MyMongoidDatatable < AjaxDatatablesRails::Base def view_columns @view_columns ||= [ # this is the array of columns the error mentions ] endend

— Reply to this email directly or view it on GitHub https://github.com/antillas21/ajax-datatables-rails/issues/110#issuecomment-121021568 .

[antillas21]

:+1:

[marshallmassengill]

Ok, so it is finally displaying content but I'm having trouble getting the sort and search to function. At first I thought it might be turbolinks just being a pain but it's been removed enough to satisfy me that that isn't the problem.

The table loads all of the records (provides the correct total) and creates a table with the first 10 entries but beyond that nothing seems to respond to mouse clicks.

Also, is there any difference between using ajax: and ajaxSource: inside of the coffeescript file? ajaxSource seems to work for me but just using ajax does not.

Thanks!

---Marshall

On Mon, Jul 13, 2015 at 2:59 PM, Antonio Antillon [email protected] wrote:

[image: :+1:]

— Reply to this email directly or view it on GitHub https://github.com/antillas21/ajax-datatables-rails/issues/110#issuecomment-121023193 .

[antillas21]

The actual JS code needed to write is outside the scope of this gem, and you should reference the jQuery dataTables plugin documentation for this.

I took a look at the options you can pass to the $().DataTable() constructor and can't find a reference to ajaxSource here: http://datatables.net/reference/option/ , though ajax is present.

Sorry to not being of much help on this.

[marshallmassengill]

Well... I got Sort and Paginate working I think... Not going to lie... it's exciting.

What's up with the view_columns method? How does it replace/augment the old searchable/sortable methods? Do search/sort apply to all of the view_columns?

I'm also not sure about the sort method I have. It seems to work for a single column but I'm not sure how to get that to work for multiple columns simultaneously. Is that supported?

Thanks!

---Marshall

On Tue, Jul 14, 2015 at 3:54 PM, Antonio Antillon [email protected] wrote:

The actual JS code needed to write is outside the scope of this gem, and you should reference the jQuery dataTables plugin documentation for this.

I took a look at the options you can pass to the $().DataTable() constructor and can't find a reference to ajaxSource here: http://datatables.net/reference/option/ , though ajax is present.

Sorry to not being of much help on this.

— Reply to this email directly or view it on GitHub https://github.com/antillas21/ajax-datatables-rails/issues/110#issuecomment-121362148 .

[antillas21]

:smile: humbled you find extending this gem exciting!

The idea behind replacing the old searchable_columns, sortable_columns methods with view_columns is this:

• if you inspect the params hash passed into the controller by the dataTables plugin, you'll notice that each column has sortable, searchable properties defined (see an example in our spec files)
• we realized it was redundant to have two separate column methods, when the view and the params hash provides us with all the info we need
• you can always declare in your JS code if you want to define a column as sortable, searchable. The default is, all columns are searchable and sortable, unless you fine-tune it via JS.

About multiple columns sorting, yes, in the ActiveRecord module, this is a supported feature, you can check the logs when clicking a column name first to sort and then shift+click another column to apply it. Here is the corresponding test to validate this.

[marshallmassengill]

Definitely helped.

I'm still stuck on the multiple column sorting but I'll figure that out easily enough... I suspect it's just not passing it back correctly.

I've run into an issue with search that I was a little worried about. My model includes references to other models and the search for mongoid doesn't seem to play nicely with those references. I'm going to have to figure out another way to deal with searching.

---Marshall

On Thu, Jul 16, 2015 at 1:59 PM, Antonio Antillon [email protected] wrote:

[image: :smile:] humbled you find extending this gem exciting!

The idea behind replacing the old searchable_columns, sortable_columns methods with view_columns is this:

• if you inspect the params hash passed into the controller by the dataTables plugin, you'll notice that each column has sortable, searchable properties defined (see an example in our spec files https://github.com/antillas21/ajax-datatables-rails/blob/v-0-4-0/spec/test_helpers.rb#L3-L39 )
• we realized it was redundant to have two separate column methods, when the view and the params hash provides us with all the info we need
• you can always declare in your JS code http://datatables.net/reference/option/columns if you want to define a column as sortable, searchable. The default is, all columns are searchable and sortable, unless you fine-tune it via JS.

About multiple columns sorting, yes, in the ActiveRecord module, this is a supported feature, you can check the logs when clicking a column name first to sort and then shift+click another column to apply it. Here is the corresponding test https://github.com/antillas21/ajax-datatables-rails/blob/v-0-4-0/spec/ajax-datatables-rails/orm/active_record_sort_records_spec.rb#L32-L41 to validate this.

— Reply to this email directly or view it on GitHub https://github.com/antillas21/ajax-datatables-rails/issues/110#issuecomment-122035940 .

[antillas21]

If it's of any help, here's how sorting is handled in the ActiveRecord module.

When a user selects several columns to sort by, this is what happens:

• user clicks a column in the view, results in the plugin adding this entry to the params hash
• user shift+clicks another column in the view, results in the plugin appending an entry like this to the params[:order] key in the params hash

Now, about searching on referenced models, I don't know how that works in mongoid, but for active_record, it's just a matter of adding the corresponding joins, example:

# inside your datatable class
def view_columns
  @view_columns ||= ['Client.name', 'Profile.address']
end

def get_raw_records
  Client.joins(:profile)
end

# based on a :client has_one :profile relationship between the models

This allows to create the correct query

WHERE clients.name ILIKE %value% OR profiles.address ILIKE %value%

You need to write the correct join syntax that may apply for mongoid. I've not used this before in my limited experience with mongoid and mongodb, so I am of no help here, sorry.

[marshallmassengill]

Yeah... mongo doesn't really do joins:

http://stackoverflow.com/questions/4067197/mongodb-and-joins

I'm going to think about it some more. I think there is a way to do this but it might not be as elegant. For now I've just disabled my search box. ;)

---Marshall

On Thu, Jul 16, 2015 at 3:27 PM, Antonio Antillon [email protected] wrote:

If it's of any help, here's how sorting is handled https://github.com/antillas21/ajax-datatables-rails/blob/v-0-4-0/lib/ajax-datatables-rails/orm/active_record.rb#L14-L20 in the ActiveRecord module.

When a user selects several columns to sort by, this is what happens:

• user clicks a column in the view, results in the plugin adding this entry https://github.com/antillas21/ajax-datatables-rails/blob/v-0-4-0/spec/test_helpers.rb#L31-L33 to the params hash
• user shift+clicks another column in the view, results in the plugin appending an entry like this https://github.com/antillas21/ajax-datatables-rails/blob/v-0-4-0/spec/test_helpers.rb#L32 to the params[:order] key in the params hash

Now, about searching on referenced models, I don't know how that works in mongoid, but for active_record, it's just a matter of adding the corresponding joins, example:

inside your datatable classdef view_columns

@view_columns ||= ['Client.name', 'Profile.address']end def get_raw_records Client.joins(:profile)end

based on a :client has_one :profile relationship between the models

This allows to create the correct query

WHERE clients.name ILIKE %value% OR profiles.address ILIKE %value%

You need to write the correct join syntax that may apply for mongoid. I've not used this before in my limited experience with mongoid and mongodb, so I am of no help here, sorry.

— Reply to this email directly or view it on GitHub https://github.com/antillas21/ajax-datatables-rails/issues/110#issuecomment-122059788 .

[antillas21]

Found that you may use includes? See Eager Loading section in this page: http://mongoid.org/en/mongoid/docs/querying.html#queries

Not really sure if any of this helps.

[washu]

when this brnch goes up to master if this isnt done, ill pop a fork and add my implementation for mongoid as a pr. i have it as a few patches to my rails core i can extract out. my mongoid orm is configured a little differently as columns = {"column" => { "fields": [], "type": :string|array }, ... }

[washu]

ive got a working model, once 4-0-0-0 is up ill make a pr with the data. it supports aggregate queries and normal document lookups. if your using mongodb 3.0 or greater aggregate call lets you left join now .. justa fyi ...

for example:

class StatementDatatable 
performs a left join on 2 collections to fill in the ids. add_stage setups up the starting stages, the aggregate datatable class handles building the sort, pagination and filtering stages

[kot-begemot]

Any news on Mongo Support?