We are already more than halfway through the Cloud Accelerator in Prague.
Instead, I'll discuss what I've been up to during the last 36 hours – almost non-stop, by the way:
- Steps towards the CompHound Mongo database table view
- Express, ES6, React and more crash courses and best practices
- A real scalable server side mongo database table view
- CompHound road map
I finally started moving forward with my CompHound component tracker project for my upcoming conference presentations at RTC Europe in Budapest end of October and Autodesk University in Las Vegas in December.
This project lives in its own CompHound organisation GitHub repository collection.
I laboured away really hard for the last 36 hours at creating a mongo database table view for it.
My initial attempts turned out be totally misdirected.
Miraculously enough, running into this dead end was also totally illuminating.
I worked all through the night until I finally understood, went to bed a six in the morning, got up again and had a nice walk in the sun at 8:30, coffee in the sun and now back to the office to write this...
The task I struggled with so hard is simply implementing a table view for the comphound database.
I first thought I would use some kind of intelligent component, e.g., based on React.
I also selected and explored several different react data grid and table libraries, e.g.:
I finally gave up on react and implemented my own much simpler initial custom table view solution using handlebars.
- Cut and paste HTML table filter script by Max Guglielmi
- Table sorting, filtering, etc. by Matt Kruse
- pbTable by Piero Blunda
I created the following series of releases of the corresponding CompHoundWeb implementations during the night:
- 0.0.3 – implemented instance view in HTML using handlebars
- 0.0.4 – added separate public css file to handlebars rendered view
- 0.0.5 – implemented static table sort and filter using old-fashioned js
During my struggles with this, I happened to run into Sahat Yalkabov's very nice tutorial explaining how to create a character voting app using React, Node.js, MongoDB and Socket.IO.
Besides being well written and beautifully formatted, it also presents a large number of best practices setting up the various components:
- New Express Project
- Build System
- Project Structure
- ES6 Crash Course
- React Crash Course
- Flux Architecture Crash Course
- React Routes (Client-Side)
- React Routes (Server-Side)
- Footer and Navbar Components
- Socket.IO - Real-time User Count
- Add Character Component
- Database Schema
- Express API Routes (1 of 2)
- Home Component
- Express API Routes (2 of 2)
- Character (Profile) Component
- Top 100 Component
- Stats Component
- Additional Resources
Before that, I also had a look though Christopher Buecheler's dead-simple step-by-step guide for front-end developers to getting up and running with node.js, express, jade, and mongodb and Stéphane Derosiaux' journey through the creation of an app with Node, MongoDB, React, Gulp. While they are not as in-depth, experienced or professional as Sahat's tutorial, they still both provide helpful quick reads.
As said, I ended up ditching it all for a simple handlebars template, only to realise later still that what I really need is a server-side database selection tool to avoid dumping the entire database contents to the client before even starting to browse.
Working during the night through the slow hotel WiFi Internet connection, every attempt to retrieve the ca. thousand records from the mongolab-hosted remote test database took a significant amount of time, up to a minute or two.
The database is intended to manage an unlimited number of component instances from an unlimited number of projects of unlimited size, so the solution needs to be able to manage millions of entries.
Obviously, all the approaches described above are completely unfeasible.
I cannot believe that I did not realise that until I actually finished a rough initial solution.
Apparently, I was unable to take a step back and think until after successfully completing the hack.
The only solution for this is a dynamic server-side approach.
A realistic solution absolutely must strongly limit the number of entries to retrieve from the database and only extract the number required to populate the current view.
Once that was clear, I found a couple of new directions and samples to explore:
- mongoose-datatable library and test project – server side data table request support for mongoose
- Using mongoose-datatables with jQuery, NodeJS and MongoDb – article on using mongoose-datatable
- mongoose-datatable demo – working demo of npm module mongoose-datatable
- Sorry for describing my convoluted path here in such detail...
- Thinking about tables, filters, sorting, all on the client side...
- Looked at React and other powerful table widgets...
- Implemented more sophisticated and modern table pimping using jquery...
- Realised that I really cannot do this client-side at all, since the database might have millions of entries...
- All these releases in between...
- Now moving to mongoose-datatable server-side.
The next steps on the CompHound project will include:
- Implementing the mongoose-datatable data table view
- Adding the View and Data API viewing component
- Posting the server live to Heroku
- Rerouting the nice new comphound.net domain to it
The current versions discussed above are the CompHoundWeb 0.0.6 node.js mongodb web server, CompHoundRvt 2016.0.0.1 C# REST API client populating it, and the (placeholder) landing page CompHound.github.io 0.0.1.
I was originally hoping to finish it all off this week, which is starting to seem a bit illusory...