FireRatingCloud Doc Modification Timestamp

The last task I discussed here was the FireRatingCloud batch upload functionality.

FireRatingCloud is a C# .NET Revit API add-in REST API MongoDB client of the fireratingdb node.js MongoDB web server.

The details of the research and development for this project is extensively documented in the FireRatingCloud GitHub repository.

The batch upload vastly improves the upload speed, since an upload of N records is reduced from N separate REST API calls to a constant 2 – one to delete existing records, and the second to batch upload all the new ones.

Another piece of functionality that I wish to implement for the FireRatingCloud sample is a real-time update, where the Revit add-in can poll the database for any changes that affect the current project, to read them back and update the BIM automatically in real-time.

I implemented something similar in my previous cloud database sample, the RoomEditorApp cloud-based, real-time, round-trip, 2D Revit model editor.

It communicates with the roomeditdb CouchDB database, and the latter returns a sequence number on each database modification.

I used the sequence number to determine which database record have changes since the last check, and hence need to be downloaded to update the BIM.

MongoDB does not provide such a built-in sequence number, so I have to implement something to replace it myself.

Here are the steps that I took on this quest:

Updating the database schema
Using the $currentDate operator
Setting schema timestamps to true
Implementing my own timestamp in C#
Download

As far as I can tell from a couple of quick internet searches, the easiest and most common approach seems to be to attach a 'last modification' timestamp to each document.

Let's go ahead and try that out.

Here are some StackOverflow search results that I encountered, initially on adding a timestamp:

That led me on to look further at automatic timestamp update on modification:

Updating the Database Schema

The mongo data model for a Revit door instance is defined by the door schema in model/door.js.

I could add a modified field to it like this:

var doorSchema = new Schema(
  { _id          : RvtUniqueId // suppress automatic generation
    , project_id : String
    , level      : String
    , tag        : String
    , firerating : Number
    , modified   : Date },
  { _id: false } // suppress automatic generation
);

However, on reading a bit further, I suspect that it might be possible to avoid explicitly adding the field myself after all.

Using the $currentDate Operator

Reading through how to auto update a timestamp on upsert in mongodb and the mongo documentation on the $currentDate operator, it seems that I can simply add a call to the operator to all the database controller calls.

The lengthy discussion on pre and post middleware not being executed on findByIdAndUpdate sounds rather scary, though.

However, apparently, an even simpler solution exists:

Setting Schema Timestamps to True

The discussions above led on to add created_at and updated_at fields to mongoose schemas, which mentions that

You can now use the #timestamps option with mongoose version >= 4.0.

I removed the modified field and set the timestamps option instead:

var doorSchema = new Schema(
  { _id          : RvtUniqueId // suppress automatic generation
    , project_id : String
    , level      : String
    , tag        : String
    , firerating : Number },
  { _id          : false // suppress automatic generation
    , timestamps : true }
);

After adding this, the batch update call added neither a createdAt nor an updatedAt field, but the upsert operation did indeed create the latter, at least:

{
  "_id": "194b64e6-8132-4497-ae66-74904f7a7710-0004b28a",
  "project_id": "qaSh_VLHTABQgzTeWedTLrOoriamVoTLY_BpjGwddhw=",
  "level": "Level 1",
  "tag": "jeremy",
  "firerating": 14,
  "updatedAt": {
    "$date": "2016-04-18T17:59:56.737Z"
  }
}

This seems to be one of those typical web development scenarios where everything you need has already been implemented somewhere, but it is next to impossible – or seems rather hard to me, at least – to determine exactly where and how to use it fully.

In this case, some operations perform as expected, others do not, and the documentation is meagre and confusing.

I therefore opted to define my own timestamp in the C# code, to ensure complete control – mainly due to my incomplete understanding of MongoDB and Mongoose, of course.

Implementing My Own Timestamp in C#

In order to avoid any time and date confusion issues, I am opting to store my own Unix timestamp, i.e., the number of seconds since 1970-01-01.

I am storing it as an integer value, and not letting anyone know that it is in fact a timestamp, to hopefully ensure that all parties involved leave it alone and don't try to meddle with it in any way.

I added it to the base DoorData class like this:

  public class DoorData
  {
    public string _id { get; set; }
    public string project_id { get; set; }
    public string level { get; set; }
    public string tag { get; set; }
    public double firerating { get; set; }
    public int modified { get; set; }
 
 
    /// <summary>
    /// Constructor to populate instance by 
    /// deserialising the REST GET response.
    /// </summary>
    public DoorData()
    {
    }
  }

I determine the Unix timestamp using this helper method:

  /// <summary>
  /// Gets a Unix timestamp representing 
  /// the current moment, i.e., the number 
  /// of seconds since 1970-01-01.
  /// </summary>
  public static int UnixTimestamp()
  {
    return (int) Math.Truncate(
      DateTime.UtcNow.Subtract( _1970_01_01 )
        .TotalSeconds );
  }

I can set the modified value in the DoorData constructor:

  modified = Util.UnixTimestamp();

The mongoose schema definition to hold it now looks like this:

var doorSchema = new Schema(
  { _id          : RvtUniqueId // suppress automatic generation
    , project_id : String
    , level      : String
    , tag        : String
    , firerating : Number
    , modified   : Number },
  { _id          : false } // suppress automatic generation
);

The document stored in the mongo database according to this schema looks exactly as expected:

{
  "_id": "194b64e6-8132-4497-ae66-74904f7a7710-0004b28a",
  "project_id": "qaSh_VLHTABQgzTeWedTLrOoriamVoTLY_BpjGwddhw=",
  "level": "Level 1",
  "tag": "jeremy",
  "firerating": 12345,
  "modified": 1461099506
}

Obviously, anyone who updates the database entries is now responsible for explicitly setting the correct modified value as well.

Using the built-in mongo functionality to do it automatically would be more comfortable, but how to reliably achieve that is unclear to me at the moment.

Now the path is open to start implementing the real-time BIM update by polling the database for all records belonging to the current project modified since they were last checked.

Download

The current versions of FireRatingCloud and fireratingdb discussed above are release 2016.0.0.24 and release 0.0.20, respectively.

The 3D Web Coder

Creative use of models and metadata on the web

April 19, 2016