Towards a CompHound Mongo Database Table View

We are already more than halfway through the Cloud Accelerator in Prague.

Happily, Kean Walmsley already talked about that, so no need for me to add anything more beyond pointing to my Cloud Accelerator Prague photo album:

Instead, I'll discuss what I've been up to during the last 36 hours – almost non-stop, by the way:

Steps towards the CompHound Mongo database table view
Express, ES6, React and more crash courses and best practices
A real scalable server side mongo database table view
CompHound road map

Steps Towards the CompHound Mongo Database Table View

I finally started moving forward with my CompHound component tracker project for my upcoming conference presentations at RTC Europe in Budapest end of October and Autodesk University in Las Vegas in December.

This project lives in its own CompHound organisation GitHub repository collection.

I laboured away really hard for the last 36 hours at creating a mongo database table view for it.

My initial attempts turned out be totally misdirected.

Miraculously enough, running into this dead end was also totally illuminating.

I worked all through the night until I finally understood, went to bed a six in the morning, got up again and had a nice walk in the sun at 8:30, coffee in the sun and now back to the office to write this...

The task I struggled with so hard is simply implementing a table view for the comphound database.

I first thought I would use some kind of intelligent component, e.g., based on React.

I also selected and explored several different react data grid and table libraries, e.g.:

I finally gave up on react and implemented my own much simpler initial custom table view solution using handlebars.

I then spent the rest of the night exploring ways to add sorting and filtering functionality to the basic HTML table element it generates using either pure old JavaScript or jQuery, e.g.:

Cut and paste HTML table filter script by Max Guglielmi
Table sorting, filtering, etc. by Matt Kruse
pbTable by Piero Blunda

I created the following series of releases of the corresponding CompHoundWeb implementations during the night:

0.0.3 – implemented instance view in HTML using handlebars
0.0.4 – added separate public css file to handlebars rendered view
0.0.5 – implemented static table sort and filter using old-fashioned js
0.0.6 – switched from dumb old javascript to jquery table sort and filter using pb-table.

Express, ES6, React and More Crash Courses and Best Practices

During my struggles with this, I happened to run into Sahat Yalkabov's very nice tutorial explaining how to create a character voting app using React, Node.js, MongoDB and Socket.IO.

Besides being well written and beautifully formatted, it also presents a large number of best practices setting up the various components:

Before that, I also had a look though Christopher Buecheler's dead-simple step-by-step guide for front-end developers to getting up and running with node.js, express, jade, and mongodb and Stéphane Derosiaux' journey through the creation of an app with Node, MongoDB, React, Gulp. While they are not as in-depth, experienced or professional as Sahat's tutorial, they still both provide helpful quick reads.

By the way, Stéphane is also a great fan of the Atom editor, cf. tired of Notepad++? Use Atom!, so I had a quick look at that as well and was quite favourably impressed.

Finally, I also found the overview of 21 react DataTable samples very helpful, which led me to take a deeper look at reactable.

As said, I ended up ditching it all for a simple handlebars template, only to realise later still that what I really need is a server-side database selection tool to avoid dumping the entire database contents to the client before even starting to browse.

A Real Scalable Server Side Mongo Database Table View

Working during the night through the slow hotel WiFi Internet connection, every attempt to retrieve the ca. thousand records from the mongolab-hosted remote test database took a significant amount of time, up to a minute or two.

The database is intended to manage an unlimited number of component instances from an unlimited number of projects of unlimited size, so the solution needs to be able to manage millions of entries.

Obviously, all the approaches described above are completely unfeasible.

I cannot believe that I did not realise that until I actually finished a rough initial solution.

Apparently, I was unable to take a step back and think until after successfully completing the hack.

The only solution for this is a dynamic server-side approach.

A realistic solution absolutely must strongly limit the number of entries to retrieve from the database and only extract the number required to populate the current view.

Once that was clear, I found a couple of new directions and samples to explore:

mongoose-datatable library and test project – server side data table request support for mongoose
Using mongoose-datatables with jQuery, NodeJS and MongoDb – article on using mongoose-datatable
mongoose-datatable demo – working demo of npm module mongoose-datatable

Summary:

Sorry for describing my convoluted path here in such detail...
Thinking about tables, filters, sorting, all on the client side...
Looked at React and other powerful table widgets...
Simplified to JavaScript modification of a table populated by Handlebars...
Implemented more sophisticated and modern table pimping using jquery...
Realised that I really cannot do this client-side at all, since the database might have millions of entries...
All these releases in between...
Now moving to mongoose-datatable server-side.

CompHound Road Map

The next steps on the CompHound project will include:

Implementing the mongoose-datatable data table view
Adding the View and Data API viewing component
Posting the server live to Heroku
Rerouting the nice new comphound.net domain to it

The current versions discussed above are the CompHoundWeb 0.0.6 node.js mongodb web server, CompHoundRvt 2016.0.0.1 C# REST API client populating it, and the (placeholder) landing page CompHound.github.io 0.0.1.

I was originally hoping to finish it all off this week, which is starting to seem a bit illusory...

The 3D Web Coder

Creative use of models and metadata on the web

September 17, 2015