FireRatingCloud Round Trip and on Mongolab

I completed the initial FireRating conversion to the cloud.

Here are the steps it took to reach that point:

Revit project identification
Complete Revit Add-in Implementation
Demo Run Log
Demo Video
Mongolab – really in the cloud
Postman does more than cURL
Wrap-up, Download and Outlook

It now runs full circle, storing Revit building model door instance fire rating values for multiple projects in a cloud-based mongo database.

It implements the same three commands as the original FireRating SDK sample to create and bind the shared fire rating parameter, export the values, and import modified data back into the BIM.

I worked hard yesterday afternoon – and night, to tell the truth – to clean up my C# .NET REST API calls and the database structure.

Revit project identification

The main problem was the identification of projects, defining the possibility to search for and retrieve them and their doors again.

My initial attempts involved use of the Revit project information singleton object and its UniqueId property. That failed rather miserably, since existing projects can be copied to new ones, thus duplicating this identifier.

I ended up implementing a radical simplification.

The initial data structure exported a separate document for each project and referred to that from the door instances.

I now created a method to generate a single short string to identify which Revit project each door instance belongs to, and simply store that single string in each door instance document instead of the project document reference.

The method I use to identify the projects and generate a unique string for each currently looks like this:

  /// <summary>
  /// Convert a string to a byte array.
  /// </summary>
  static byte[] GetBytes( string str )
  {
    byte[] bytes = new byte[str.Length
      * sizeof( char )];
 
    System.Buffer.BlockCopy( str.ToCharArray(),
      0, bytes, 0, bytes.Length );
 
    return bytes;
  }
 
  /// <summary>
  /// Define a project identifier for the 
  /// given Revit document.
  /// </summary>
  public static string GetProjectIdentifier(
    Document doc )
  {
    SHA256 hasher = SHA256Managed.Create();
 
    string key = System.Environment.MachineName
      + ":" + doc.PathName;
 
    byte[] hashValue = hasher.ComputeHash( GetBytes(
      key ) );
 
    string hashb64 = Convert.ToBase64String(
      hashValue );
 
    return hashb64.Replace( '/', '_' );
  }

It builds a string of all the relevant data needed to identify the project, which might include information on the Revit server, central model, computer name, full project file path, etc., depending on your situation.

It creates a hash code of the concatenated string, encodes that in base64 format, and finally converts that to base64url, to it can be included in the URIs used to communicate with the mongo database.

In my case, for the URL encoding, all I worry about is the slash character '/', since including that in the URI messes up the routing.

According to the base64url convention, I simply replace it by underscore '_'.

Now I can use this string in the door instance data to identify the project it belongs to.

Its one and only purpose really is to enable me to retrieve all the door data for a given Revit project when importing modified data back into the building model.

Complete Revit Add-in Implementation

Let's look at the entire implementation of the three Revit external commands.

Note that the first one includes a lot of commented code discussing various ways to determine other categories that one might wish to equip with shared parameters.

#region Namespaces
using System;
using System.Collections;
using System.Diagnostics;
using System.Linq;
using Autodesk.Revit.ApplicationServices;
using Autodesk.Revit.Attributes;
using Autodesk.Revit.DB;
using Autodesk.Revit.UI;
#endregion
 
namespace FireRatingCloud
{
  #region Cmd_1_CreateAndBindSharedParameter
  /// <summary>
  /// Create and bind shared parameter.
  /// </summary>
  [Transaction( TransactionMode.Manual )]
  public class Cmd_1_CreateAndBindSharedParameter
    : IExternalCommand
  {
    // What element type are we interested in? The standard 
    // SDK FireRating sample uses BuiltInCategory.OST_Doors. 
    // We also test using BuiltInCategory.OST_Walls to 
    // demonstrate that the same technique works with system 
    // families just as well as with standard ones.
    //
    // To test attaching shared parameters to inserted 
    // DWG files, which generate their own category on 
    // the fly, we also identify the category by 
    // category name.
    //
    // The last test is for attaching shared parameters 
    // to model groups.
 
    static public BuiltInCategory Target = BuiltInCategory.OST_Doors;
 
    //static public BuiltInCategory Target = BuiltInCategory.OST_Walls;
    //static public string Target = "Drawing1.dwg";
    //static public BuiltInCategory Target = BuiltInCategory.OST_IOSModelGroups; // doc.Settings.Categories.get_Item returns null
    //static public string Target = "Model Groups"; // doc.Settings.Categories.get_Item throws an exception SystemInvalidOperationException "Operation is not valid due to the current state of the object."
    //static public BuiltInCategory Target = BuiltInCategory.OST_Lines; // model lines
    //static public BuiltInCategory Target = BuiltInCategory.OST_SWallRectOpening; // Rectangular Straight Wall Openings, case 1260656 [Add Parameters Wall Opening]
 
    public Result Execute(
      ExternalCommandData commandData,
      ref string message,
      ElementSet elements )
    {
      UIApplication uiapp = commandData.Application;
      Application app = uiapp.Application;
      Document doc = uiapp.ActiveUIDocument.Document;
      Category cat = null;
 
      // The category to define the parameter for.
 
      #region Determine model group category
#if DETERMINE_MODEL_GROUP_CATEGORY
      List<Element> modelGroups = new List<Element>();
      //Filter fType = app.Create.Filter.NewTypeFilter( typeof( Group ) ); // "Binding the parameter to the category Model Groups is not allowed"
      Filter fType = app.Create.Filter.NewTypeFilter( typeof( GroupType ) ); // same result "Binding the parameter to the category Model Groups is not allowed"
      Filter fCategory = app.Create.Filter.NewCategoryFilter( BuiltInCategory.OST_IOSModelGroups );
      Filter f = app.Create.Filter.NewLogicAndFilter( fType, fCategory );
      if ( 0 < doc.get_Elements( f, modelGroups ) )
      {
        cat = modelGroups[0].Category;
      }
#endif // DETERMINE_MODEL_GROUP_CATEGORY
      #endregion // Determine model group category
 
      if( null == cat )
      {
        cat = doc.Settings.Categories.get_Item( Target );
      }
 
      // Retrieve shared parameter definition file.
 
      DefinitionFile sharedParamsFile = Util
        .GetSharedParamsFile( app );
 
      if( null == sharedParamsFile )
      {
        message = "Error getting the shared params file.";
        return Result.Failed;
      }
 
      // Get or create the shared parameter group.
 
      DefinitionGroup sharedParamsGroup
        = sharedParamsFile.Groups.get_Item(
          Util.SharedParameterGroupName );
 
      if( null == sharedParamsGroup )
      {
        sharedParamsGroup = sharedParamsFile.Groups
          .Create( Util.SharedParameterGroupName );
      }
 
      // Visibility of the new parameter: the
      // Category.AllowsBoundParameters property
      // determines whether a category is allowed to
      // have user-visible shared or project parameters.
      // If it is false, it may not be bound to visible
      // shared parameters using the BindingMap. Note
      // that non-user-visible parameters can still be
      // bound to these categories. In our case, we
      // make the shared parameter user visibly, if
      // the category allows it.
 
      bool visible = cat.AllowsBoundParameters;
 
      // Get or create the shared parameter definition.
 
      Definition def = sharedParamsGroup.Definitions
        .get_Item( Util.SharedParameterName );
 
      if( null == def )
      {
        ExternalDefinitionCreationOptions opt
          = new ExternalDefinitionCreationOptions(
            Util.SharedParameterName,
            ParameterType.Number );
 
        opt.Visible = visible;
 
        def = sharedParamsGroup.Definitions.Create(
          opt );
      }
 
      if( null == def )
      {
        message = "Error creating shared parameter.";
        return Result.Failed;
      }
 
      // Create the category set for binding and
      // add the category of interest to it.
 
      CategorySet catSet = app.Create.NewCategorySet();
 
      catSet.Insert( cat );
 
      // Bind the parameter.
 
      using( Transaction t = new Transaction( doc ) )
      {
        t.Start( "Bind FireRating Shared Parameter" );
 
        Binding binding = app.Create.NewInstanceBinding(
          catSet );
 
        // We could check if it is already bound; if so,
        // Insert will apparently just ignore it.
 
        doc.ParameterBindings.Insert( def, binding );
 
        // You can also specify the parameter group here:
 
        //doc.ParameterBindings.Insert( def, binding, 
        //  BuiltInParameterGroup.PG_GEOMETRY );
 
        t.Commit();
      }
      return Result.Succeeded;
    }
  }
  #endregion // Cmd_1_CreateAndBindSharedParameter
 
  #region Cmd_2_ExportSharedParameterValues
  /// <summary>
  /// Export all target element ids and their
  /// FireRating parameter values to external database.
  /// </summary>
  [Transaction( TransactionMode.ReadOnly )]
  public class Cmd_2_ExportSharedParameterValues
    : IExternalCommand
  {
    /// <summary>
    /// Retrieve the door instance data to store in 
    /// the external database and return it as a
    /// dictionary in a JSON formatted string.
    /// </summary>
    string GetDoorDataJson(
      Element door,
      string project_id,
      Guid paramGuid )
    {
      Document doc = door.Document;
 
      string s = string.Format(
        "\"_id\": \"{0}\","
        + "\"project_id\": \"{1}\","
        + "\"level\": \"{2}\","
        + "\"tag\": \"{3}\","
        + "\"firerating\": {4}",
        door.UniqueId,
        project_id,
        doc.GetElement( door.LevelId ).Name,
        door.get_Parameter( BuiltInParameter.ALL_MODEL_MARK ).AsString(),
        door.get_Parameter( paramGuid ).AsDouble() );
 
      return "{" + s + "}";
    }
 
    public Result Execute(
      ExternalCommandData commandData,
      ref string message,
      ElementSet elements )
    {
      UIApplication uiapp = commandData.Application;
      Application app = uiapp.Application;
      Document doc = uiapp.ActiveUIDocument.Document;
 
      // Get shared parameter GUID.
 
      Guid paramGuid;
      if( !Util.GetSharedParamGuid( app, out paramGuid ) )
      {
        message = "Shared parameter GUID not found.";
        return Result.Failed;
      }
 
      // Determine custom project identifier.
 
      string project_id = Util.GetProjectIdentifier( doc );
 
      // Loop through all elements of the given target
      // category and export the shared parameter value 
      // specified by paramGuid for each.
 
      FilteredElementCollector collector
        = Util.GetTargetInstances( doc,
          Cmd_1_CreateAndBindSharedParameter.Target );
 
      int n = collector.Count<Element>();
 
      string json;
      string jsonResponse;
 
      foreach( Element e in collector )
      {
        json = GetDoorDataJson( e, project_id,
          paramGuid );
 
        Debug.Print( json );
 
        jsonResponse = Util.QueryOrUpsert(
          "doors/" + e.UniqueId, string.Empty, "GET" );
 
        if( 0 == jsonResponse.Length )
        {
          jsonResponse = Util.QueryOrUpsert(
            "doors", json, "POST" );
        }
        else
        {
          jsonResponse = Util.QueryOrUpsert(
            "doors/" + e.UniqueId, json, "PUT" );
        }
        Debug.Print( jsonResponse );
      }
      return Result.Succeeded;
    }
  }
  #endregion // Cmd_2_ExportSharedParameterValues
 
  #region Cmd_3_ImportSharedParameterValues
  /// <summary>
  /// Import updated FireRating parameter values 
  /// from external database.
  /// </summary>
  [Transaction( TransactionMode.Manual )]
  public class Cmd_3_ImportSharedParameterValues
    : IExternalCommand
  {
    public Result Execute(
      ExternalCommandData commandData,
      ref string message,
      ElementSet elements )
    {
      UIApplication uiapp = commandData.Application;
      Application app = uiapp.Application;
      Document doc = uiapp.ActiveUIDocument.Document;
 
      Guid paramGuid;
      if( !Util.GetSharedParamGuid( app, out paramGuid ) )
      {
        message = "Shared parameter GUID not found.";
        return Result.Failed;
      }
 
      // Determine custom project identifier.
 
      string project_id = Util.GetProjectIdentifier( doc );
 
      // Get all doors referencing this project.
 
      string query = "doors/project/" + project_id;
 
      string jsonResponse = Util.QueryOrUpsert( query,
        string.Empty, "GET" );
 
      object obj = JsonParser.JsonDecode( jsonResponse );
 
      if( null != obj )
      {
        ArrayList doors = obj as ArrayList;
 
        if( null != doors && 0 < doors.Count )
        {
          using( Transaction t = new Transaction( doc ) )
          {
            t.Start( "Import Fire Rating Values" );
 
            // Retrieve element unique id and 
            // FireRating parameter values.
 
            foreach( object door in doors )
            {
              Hashtable d = door as Hashtable;
              string uid = d["_id"] as string;
              Element e = doc.GetElement( uid );
 
              if( null == e )
              {
                message = string.Format(
                  "Error retrieving element for unique id {0}.",
                  uid );
 
                return Result.Failed;
              }
 
              Parameter p = e.get_Parameter( paramGuid );
 
              if( null == p )
              {
                message = string.Format(
                  "Error retrieving shared parameter on element with unique id {0}.",
                  uid );
 
                return Result.Failed;
              }
              object fire_rating = d["firerating"];
 
              p.Set( (double) fire_rating );
            }
            t.Commit();
          }
        }
      }
      return Result.Succeeded;
    }
  }
  #endregion // Cmd_3_ImportSharedParameterValues
}

All very straight forward, really.

Demo Run Log

Here is a complete demo run of the three steps – well, four, actually, counting the shared parameter binding:

Create and bind shared parameter
Export fire rating data from Revit

Modify externally:

> db.doors.find().forEach(function (u) {
...    u.firerating = 33; db.doors.save(u); });
> db.doors.find()

Import back into building model

Before doing anything else, clean up the mongo database from the console interface by removing the entire previous firerating data set:

> show dbs
firerating  0.078GB
local       0.078GB
> use firerating
switched to db firerating
> db.dropDatabase()
{ "dropped" : "firerating", "ok" : 1 }
> show dbs
local  0.078GB

We can examine the effect of this as logged by the mongo database daemon:

2015-07-09T08:27:39.955+0200 I COMMAND  [conn1] dropDatabase firerating starting
2015-07-09T08:27:39.958+0200 I JOURNAL  [conn1] journalCleanup...
2015-07-09T08:27:39.965+0200 I JOURNAL  [conn1] removeJournalFiles
2015-07-09T08:27:39.969+0200 I JOURNAL  [conn1] journalCleanup...
2015-07-09T08:27:39.969+0200 I JOURNAL  [conn1] removeJournalFiles
2015-07-09T08:27:39.972+0200 I COMMAND  [conn1] dropDatabase firerating finished

Let's also restart the fireratingdb node.js web server to make sure the latest updates are active:

Y:\a\src\web\mongo\firerating > node server.js
Firerating server listening at port 3001

I start up the FireRatingCloud add-in in the Visual Studio debugger, which launches Revit and loads a minimalistic test building model with just one door.

Initially, the door properties do not include the fire rating shared parameter, since it does not yet exist.

It appears – with an undefined value – after running the add-in external command Cmd_1_CreateAndBindSharedParameter to create it:

I export the shared parameters for this project to fireratingdb using Cmd_2_ExportSharedParameterValues and look at the result in the mongo console:

> show dbs
firerating  0.078GB
local       0.078GB
> use firerating
switched to db firerating
> show collections
doors
system.indexes
> db.doors.find()
{ "_id" : "194b64e6-8132-4497-ae66-74904f7a7710-0004b28a", "project_id" : "qaSh_VLHTABQgzTeWedTLrOoriamVoTLY_BpjGwddhw=", "level" : "Level 1", "tag" : "1", "firerating" : 0, "__v" : 0 }

I modify the fire rating value in the external database, e.g., setting it to 33:

> db.doors.find().forEach(function (u) { u.firerating = 33; db.doors.save(u); });
> db.doors.find()
{ "_id" : "194b64e6-8132-4497-ae66-74904f7a7710-0004b28a", "project_id" : "qaSh_VLHTABQgzTeWedTLrOoriamVoTLY_BpjGwddhw=", "level" : "Level 1", "tag" : "1", "firerating" : 33, "__v" : 0 }

Finally, I launch the third command Cmd_3_ImportSharedParameterValues to import the modified shared parameter value back into the building model.

The change is immediately reflected in the door properties:

Demo Video

Here is an 82-second video showing the addition of a few more doors and the full round trip data flow live:

Mongolab – Really in the Cloud

Cyrille suggests never installing mongo locally at all:

Instead you can use the free hosting service provided by mongolab.com.

This has several advantages. First, I do not have to install another service on my machine. Secondly, I can run and debug my code from all of my computers using the same dataset without having to worry about static IP addresses or location, e.g., while travelling. Another advantage is the fact that you automatically test and debug your solution with something real in the outside world, not just in your local development setup.

I followed Cyrille's advice and was surprised how quick, easy and useful it was.

It took me about ten minutes to sign up for a free mongolab account and create a database named firerating.

It took just about one minute more to switch my node.js server to use the mongolab hosted database instead of a local one.

Connecting obviously requires a user name and password.

For this demo, I used revit for both, so the database connection string becomes mongodb://revit:[email protected]:47742/firerating.

In the server implementation, all I did was change one single line of code:

// local database
//var mongo_uri = 'mongodb://localhost/firerating';

// mongolab hosted
var mongo_uri = 'mongodb://revit:revit'
  + '@ds047742.mongolab.com:47742/firerating';

I ran through a minimal test, and all looks good. Here is a snapshot of the firerating data of the first door instance hosted on mongolab:

I can browse and edit it freely right there and then.

Very handy indeed.

Thanks to Cyrille for the good advice!

Postman does more than cURL

Cyrille also points out another powerful and useful set of tools:

I found your cURL testing description very cool. However, what about people on Windows? They need to install it :-( or people who do not like command line stuff? I was thinking an article on Postman would be a nice addition (Chrome add-in).

Wrap-up, Download and Outlook

The fireratingdb node.js server obviously evolved since yesterday, in the following steps:

The Revit add-in C# .NET source code, Visual Studio and add-n manifest is provided in the FireRatingCloud GitHub repository, and the version discussed above is release 2016.0.0.6.

Where do we go from here?

Well, now that the mongo database is cloud hosted, I will obviously also move the node server to a globabally accessible place, for example Heroku.

For the Revit add-in, I would like to implement an external application to provide more convenient user interface access to the three commands.

I can also imagine implementing a batching functionality, to traverse all Revit projects one by one and run the three commands programmatically in each project.

What would you like to see?

The 3D Web Coder

Creative use of models and metadata on the web

July 09, 2015