Implementing Mongo Database Relationships

Let's look at how to implement relationships in a mongo database.

Yesterday, I made my very first steps exploring mongoDB to implement a mongoose based node.js database web server.

The example I am using is a hopefully simple project to implement a cloud-based multi-project version of the Revit SDK FireRating sample.

The relationships we are dealing with here are as trivial as they get: we are looking at doors in a building, represented by door family instances in a Revit RVT project file.

We need to be able to retrieve all doors for any given project. A project can contain any number of doors, and any number of projects can be added.

Mongo recommends two main approaches to handle relationships:

• Embedding documents
• Document references

In this case, the reference approach seems best suited.

In fact, our door-project relationship is quite similar to the book-publisher one described in the mongo documentation.

Identifying a Project

Yesterday, I described that I decided to use the UniqueId defined by Revit to identify the door instances in mongo.

This will work fine, as long as no two doors in Revit ever have the same UniqueId.

That can only happen if a project containing a door is copied and used as a basis for several different offspring projects.

For a Revit project, identification is significantly harder.

Theoretically, one could use the ProjectInfo singleton's UniqueId.

In practice, that is not reliable, because people apparently do indeed copy projects and create several branches from them.

Let's just hope they never do that after more elements have been added.

Next, I thought of identifying the project by the machine name and full path, e.g.:

  System.Environment.MachineName
    + separator
    + Document.PathName;

This can end up being a pretty long string. Since mongo becomes inefficient if the identifier is too long, I thought of compressing it and encoding to base64, or using a message digest or hash algorithm such as MD5 or the more secure and collision-proof SHA-2. The latter is available in .NET as System.Security.Cryptography.SHA256Managed.

However, all of the above only works for non-workshared projects. Workshared ones might potentially be able to use the Document.WorksharingCentralGUID property.

Once I got to this level of complexity, I decided to skip it and use the automatic mongo object id generation functionality for the project instead of defining my own. Avoid complexity, kiss, and handle both locally stored and central model projects the same way.

Adding a Project and Two Doors to the Database

As an initial proof of concept, here is a node server adding a project containing two doors:

// given: a Revit door element UniqeId;
// it must obviously be unique in the database.

var door_unique_id
  = '60f91daf-3dd7-4283-a86d-24137b73f3da-0001fd0b';

var mongoose = require( 'mongoose' );

mongoose.connect( 'mongodb://localhost/firerating' );

var Schema = mongoose.Schema,
    ObjectId = Schema.ObjectId;

var RvtUniqueId = String;

// use automatic Mongo ObjectId for project.

var projectSchema = new Schema(
  { computername        : String // .NET System.Environment.MachineName
    , path              : String // Document.PathName
    , centralserverpath : String // Document.GetWorksharingCentralModelPath().CentralServerPath
    , title             : String // Document.Title
    , numberofsaves     : Number // DocumentVersion.NumberOfSaves
    , versionguid       : RvtUniqueId // DocumentVersion.VersionGUID
    , projectinfo_uid   : RvtUniqueId // ProjectInfo.UniqueId
  }
);

var ProjectModel = mongoose.model( 'Project', projectSchema );

// use Revit UniqueId for door instances.

var doorSchema = new Schema(
  { _id          : RvtUniqueId // suppress automatic generation
    , project_id : ObjectId
    , level      : String
    , tag        : String
    , firerating : Number },
  { _id: false } // suppress automatic generation
);

var DoorModel = mongoose.model( 'Door', doorSchema );

var projectInstance = new ProjectModel();

projectInstance.computername = 'JEREMYTAMMIB1D2';
projectInstance.path = 'C:/Program Files/Autodesk/Revit 2016/Samples/rac_basic_sample_project.rvt';
projectInstance.centralserverpath = '';
projectInstance.title = 'rac_basic_sample_project.rvt';
projectInstance.numberofsaves = 271;
projectInstance.versionguid = 'f498e8b1-7311-4409-a669-2fd290356bb4';
projectInstance.projectinfo_uid = '8764c510-57b7-44c3-bddf-266d86c26380-0000c160';

projectInstance.save(function (err) {
  console.log( 'save project returned err = ' + err );
  if(!err) {
    var pid = projectInstance._id;
    console.log( 'project_id = ' + pid );

    var instance = new DoorModel();
    instance._id = door_unique_id;
    instance.project_id = pid;
    instance.level = 'Level 1';
    instance.tag = 'Tag 1';
    instance.firerating = 123.45;
    instance.save(function (err) {
      console.log( 'save instance returned err = ' + err );
      if(!err) {
        var instance2 = new DoorModel();
        instance2._id = door_unique_id + '2';
        instance2.project_id = pid;
        instance2.level = 'Level 2';
        instance2.tag = 'Tag 2';
        instance2.firerating = 678.9;
        instance2.save(function (err) {
          console.log( 'save instance2 returned err = ' + err );
        });
      }
    });
  }
});

This version of the code is captured as release 0.0.2 in the firerating GitHub repository, in case you would like to try it out yourself.

We can look at the result of running it in the mongo console.

First, let's drop any potentially pre-existing data:

> db.projects.drop()
true
> db.doors.drop()
true

Next, run this trivial little server:

$ node server.js
save project returned err = null
project_id = 5593a8733a003b852142e4eb
save instance returned err = null
save instance2 returned err = null

The results displayed in the mongo console look like this:

> db.projects.find()
{ "_id" : ObjectId("5593a8733a003b852142e4eb"),
  "projectinfo_uid" : "8764c510-57b7-44c3-bddf-266d86c26380-0000c160",
  "versionguid" : "f498e8b1-7311-4409-a669-2fd290356bb4",
  "numberofsaves" : 271,
  "title" : "rac_basic_sample_project.rvt",
  "centralserverpath" : "",
  "path" : "C:/Program Files/Autodesk/Revit 2016/Samples/rac_basic_sample_project.rvt",
  "computername" : "JEREMYTAMMIB1D2",
  "__v" : 0 }
> db.doors.find()
{ "_id" : "60f91daf-3dd7-4283-a86d-24137b73f3da-0001fd0b",
  "firerating" : 123.45,
  "tag" : "Tag 1",
  "level" : "Level 1",
  "project_id" : ObjectId("5593a8733a003b852142e4eb"),
  "__v" : 0 }
{ "_id" : "60f91daf-3dd7-4283-a86d-24137b73f3da-0001fd0b2",
  "firerating" : 678.9,
  "tag" : "Tag 2",
  "level" : "Level 2",
  "project_id" : ObjectId("5593a8733a003b852142e4eb"),
  "__v" : 0 }

We can easily select the doors belonging to a specific project by adding an appropriate query argument:

  db.doors.find( { "project_id"
    : ObjectId("5593a8733a003b852142e4eb") } )

Pondering the Callback Hell

Note how in the node.js code implementation, each addition of a new database entry is executed after the previous one completes.

Therefore, each subsequent save operation is wrapped in the callback function provided to the preceding call.

This causes the beginnings of an indentation avalanche.

Bear in mind that any project can potentially contain thousands of door instances.

These are the first beginnings of callback hell.

How can we avoid that?

Besides the solution suggested by callbackhell.com, the Stack Overflow thread on callback hell in nodejs provides some good, sensible suggestions.

Reading on, I find a yet better approach, because callback hell is a myth.

Adding a Project and Many Doors to the Database

Thus inspired, I notice that my issue is actually very manageable: although all my individual door instances do depend on the project and require its object id, they do not depend on each other, and can therefore be created in parallel.

Here is the code demonstrating addition of any number of doors to the project in a loop:

projectInstance.save(function (err) {
  console.log( 'save project returned err = ' + err );
  if(!err) {
    var pid = projectInstance._id;
    console.log( 'project_id = ' + pid );

    for( var i = 0; i < 10; ++i ) {
      var inst = new DoorModel();
      var s = i.toString();
      inst._id = door_unique_id + s;
      inst.project_id = pid;
      inst.level = 'Level ' + s;
      inst.tag = 'Tag ' + s;
      inst.firerating = 123.45 * (i + 0.1);
      inst.save(function (err) {
        console.log( 'save instance returned err = ' + err );
      });
    }
  }
});

It works fine, and the indentation never needs to exceed these three levels.

This updated version of the node.js code is stored as release 0.0.3 in the firerating GitHub repository.

Next Steps

This discussion addresses the first two items in yesterday's list of next steps:

• Define more complete schema that includes information about the containers, i.e., the Revit RVT BIM or building information model project files.
• Manage the relationships between the door family instances to their container projects.
• Add a REST API to populate and query the database programmatically.
• Implement a Revit add-in exercising the REST API from C# .NET.
• Re-implement the complete cloud-based Revit FireRating SDK sample functionality.

So let's continue with the rest anon.

Multi-Colour 3D Printing

One last little note before wrapping up for today: a new Autodesk patent reveals several advanced multi-color FDM 3D printing methods. The detailed description of these ideas is really cool! Check it out for yourself.