CompHound, RestSharp, Mongo PUT and POST

I spent a few exciting and satisfying hours last night cleaning up my C# REST API client to use RestSharp instead of HttpWebRequest and the node.js mongo web server to enable PUT to create as well as update a record.

Here are all my topics for today:

Use percent for CSS font-size font sizing
The CompHound component tracker
GET, PUT and POST stupidity creating versus updating a record
Enabling PUT to create as well as update
Using RestSharp instead of HttpWebRequest
No need to format data as JSON

Use Percent for CSS font-size Font Sizing

Being rather perfectionistic, I do a lot of pure HTML editing and really care about the results.

I was therefore quite interested to discover a quite complete and trustworthy analysis and discussion of the topic by Kyle Schaffer, CSS font-size: em vs px vs pt vs %.

Oops. That site seems to have disappeared. Still, I'll leave the link in, in case it reappears one of these days.

Anyway, the executive summary of that discussion is simply, "use percent".

Back to my work at hand:

The CompHound Component Tracker

I started work on the CompHound component tracker project for my upcoming conference presentations at RTC Europe in Budapest end of October and Autodesk University in Las Vegas in December.

CompHound is a cloud-based universal component and asset usage analysis, visualisation and reporting.

So far, it consists of two modules:

The main JavaScript node.js web server driving a mongo database, now awaiting a user interface and 3D visualisation components.
A C# Revit add-in populating the database with components to analyse.

To keep these two projects nicely packaged together, I created the CompHound GitHub organisation and added them as subprojects to that:

CompHoundWeb – Cloud-based universal component and asset usage analysis, report and visualisation, a REST API node and mongo web server (server-side JavaScript).
CompHoundRvt – Revit add-in to populate the CompHound server (desktop C# REST API client).

I grabbed the code from my existing fireratingdb web server and FireRatingCloud Revit add-in to quickly achieve the following functionality for CompHound as well:

Set up the node.js web server
Implement a REST API for it
Drive the mongo database from it
Host the mongo database in the cloud on mongolab
Host the web server in the cloud on heroku
Retrieve and format the Revit model data
Send it to the server via REST API

Note that stepping through all of these tasks for the new CompHound project only took an hour or so in all, from scratch.

It took me several weeks to discover how to achieve all this first time around, for the FireRating project :-)

While doing so, I was once again irritated by the GET, PUT and POST issue I encountered in the firerating project for creating versus updating existing records.

I fixed them in the two new projects and published CompHoundWeb 0.0.1 and CompHoundRvt 2016.0.0.0, respectively.

Then I returned to the original FireRating samples to fix these issues there as well.

I'll discuss those improvements in more detail below.

GET, PUT and POST Stupidity Creating versus Updating a Record

The previous code to create a new record in the firerating C# add-in looked like this:

  jsonResponse = Util.QueryOrUpsert(
    "doors/" + e.UniqueId, string.Empty, "GET" );
 
  if( 0 == jsonResponse.Length )
  {
    jsonResponse = Util.QueryOrUpsert(
      "doors", json, "POST" );
  }
  else
  {
    jsonResponse = Util.QueryOrUpsert(
      "doors/" + e.UniqueId, json, "PUT" );
  }

POST is used to creata a new record, and PUT to update an existing one.

I need to use GET first to determine which one of these to call.

I cannot use PUT right away, because it does not create a new record, just silently fails to do anything at all.

I would rather just use one single PUT call to either create or update a record, regardless of whether it already exists or not.

That would save me from making two separate3 REST API calls, with the result of GET deciding whether to use PUT or POST.

Since PUT is idempotent, there is no reason – at least from the REST point of view – why this should not work.

Please also refer to the succinct and illuminating StackOverflow thread on PUT vs POST in REST.

I tracked down the reason for this in the underlying REST API implementation issuing the mongo database calls via mongoose.

PUT was driving the Update function, implemented like this:

exports.update = function(req, res) {
  var id = req.params.id;
  console.log('Updating ' + id);
  Door.update({"_id":id}, req.body,
    function (err, numberAffected) {
      if (err) return console.log(err);
      console.log('Updated %s doors', numberAffected.toString());
      return res.sendStatus(202);
  });
};

As said, when creating a new record, the target id is not found, the result in null, nothing is added and no error returned.

Easy to fix, once we tracked down the culprit:

Enabling PUT to Create as well as Update

I modified the update functionality by implementing update2 in controller/doors_v1.js as follows:

exports.update2 = function(req, res) {
  var id = req.params.id;
  console.log('Updating ' + id);
  Door.findOne({'_id':id},function(err, result) {
    if(result) {
      Door.update({"_id":id}, req.body,
        function (err, numberAffected) {
          if (err) return console.log(err);
          console.log('Updated %s doors', numberAffected.toString());
          return res.sendStatus(202);
      });
    }
    else {
      Door.create(req.body, function (err, door) {
        if (err) return console.log(err);
        return res.send(door);
      });
    }
  });
};

That creates a new record if none is found.

I wonder whether mongoose and mongoDB offer that functionality built-in?

The REST API PUT call is routed to the new method in routes.js:

module.exports = function(app) {
  var doors = require('./controller/doors_v1');
  app.get('/api/v1/doors', doors.findAll);
  app.get('/api/v1/doors/:id', doors.findById);
  app.post('/api/v1/doors', doors.add);
  //app.put('/api/v1/doors/:id', doors.update); // this one does not allow me to PUT a new instance only update existing
  app.put('/api/v1/doors/:id', doors.update2); // works more like POST + PUT, cf. http://stackoverflow.com/questions/630453/put-vs-post-in-rest
  app.delete('/api/v1/doors/:id', doors.delete);
  app.get('/api/v1/doors/project/:pid', doors.findAllForProject);
}

Using RestSharp Instead of HttpWebRequest

With that fixed, I also cleaned up my C# REST client a bit.

I switched from the convoluted use of HttpWebRequest to RestSharp, also available through NuGet.

Much cleaner.

The previous rather horrible Upsert method looks like this:

/// <summary>
/// GET, PUT or POST JSON document data from or to 
/// the specified mongoDB collection.
/// </summary>
public static string QueryOrUpsert(
  string collection_name_id_query,
  string json,
  string method )
{
  string uri = Util.RestApiUri + "/"
    + collection_name_id_query;
 
  HttpWebRequest request = HttpWebRequest.Create(
    uri ) as HttpWebRequest;
 
  request.ContentType = "application/json; charset=utf-8";
  request.Accept = "application/json, text/javascript, */*";
  request.Timeout = Util.Timeout;
  request.Method = method;
 
  if( 0 < json.Length )
  {
    Debug.Assert( !method.Equals( "GET" ),
      "content is not allowed with GET" );
 
    using( StreamWriter writer = new StreamWriter(
      request.GetRequestStream() ) )
    {
      writer.Write( json );
    }
  }
  WebResponse response = request.GetResponse();
  Stream stream = response.GetResponseStream();
  string jsonResponse = string.Empty;
 
  using( StreamReader reader = new StreamReader(
    stream ) )
  {
    while( !reader.EndOfStream )
    {
      jsonResponse += reader.ReadLine();
    }
  }
  return jsonResponse;
}

That is now replaced by a much simpler Put method using RestSharp:

/// <summary>
/// PUT JSON document data into 
/// the specified mongoDB collection.
/// </summary>
public static string Put(
  string collection_name_and_id,
  object data )
{
  var client = new RestClient( RestApiBaseUrl );
 
  var request = new RestRequest( _api_version + "/"
    + collection_name_and_id, Method.PUT );
 
  request.RequestFormat = DataFormat.Json;
 
  request.AddBody( data ); // uses JsonSerializer
 
  IRestResponse response = client.Execute( request );
 
  var content = response.Content; // raw content as string
 
  return content;
}

Note that the old method took a string specifying the JSON data to send.

The new one takes a native .NET object instead.

This is also much easier to implement:

No Need to Format Data as JSON

The JSON string was laboriously assembled by hand:

/// <summary>
/// Retrieve the door instance data to store in 
/// the external database and return it as a
/// dictionary in a JSON formatted string.
/// </summary>
string GetDoorDataJson(
  Element door,
  string project_id,
  Guid paramGuid )
{
  Document doc = door.Document;
 
  string s = string.Format(
    "\"_id\": \"{0}\","
    + "\"project_id\": \"{1}\","
    + "\"level\": \"{2}\","
    + "\"tag\": \"{3}\","
    + "\"firerating\": {4}",
    door.UniqueId,
    project_id,
    doc.GetElement( door.LevelId ).Name,
    door.get_Parameter( BuiltInParameter.ALL_MODEL_MARK ).AsString(),
    door.get_Parameter( paramGuid ).AsDouble() );
 
  return "{" + s + "}";
}

The .NET object is simpler and more efficient to assemble:

/// <summary>
/// Retrieve the door instance data to store in 
/// the external database and return it as a
/// dictionary-like object.
/// </summary>
object GetDoorData(
  Element door,
  string project_id,
  Guid paramGuid )
{
  Document doc = door.Document;
 
  string levelName = doc.GetElement(
    door.LevelId ).Name;
 
  string tagValue = door.get_Parameter(
    BuiltInParameter.ALL_MODEL_MARK ).AsString();
 
  double fireratingValue = door.get_Parameter(
    paramGuid ).AsDouble();
 
  object data = new {
    _id = door.UniqueId,
    project_id = project_id,
    level = levelName,
    tag = tagValue,
    firerating = fireratingValue
  };
 
  return data;
}

Download

As said, I updated both the existing FireRating and the new CompHound apps to use the new techniques, in their respective repositories:

The 3D Web Coder

Creative use of models and metadata on the web

September 10, 2015