Tuesday, July 10, 2012

salesforce_bulk v1.0

Several months ago I decided to write a ruby gem, salesforce_bulk, that integrates with the Salesforce Bulk API. There are several gems out that there can do the same for the non-bulk REST API, such as databasedotcom, but none of these use the bulk API, leading to frustration as you watch your daily API calls go through the roof.The salesforce_bulk gem keeps your governor limits in check by allowing you to do multiple operations in a single call, versus a single operation per call which is how the other gems operate.

When I first wrote this gem, I did so rather quickly thinking that no one would really use it so I didn't dedicate much time to improving it. It wasn't until recently that I noticed it had a several downloads, a couple of pull requests, and several mentions on stackoverflow, some blogs, and even on an official Salesforce guide.

I must admit that I was rather embarrassed to see my hastily written code being talked about by so many people. Several developers pointed out the fact that it lacked sandbox support and had near non-existent error reporting. This embarrassment has motivated me to improve gem and upgrade it from version 0.0.5 to version 1.0. In version 1.0 you'll find sandbox and asynchronous support (thanks to tonyjiang), better error catching, and better result reporting. I'll briefly go over these improvements here:

Sandbox Support


Asynchronous Job Processing Support

 upserted_account = Hash["name" => "Test Account -- Upserted", "External_Field_Name" => "123456"] # Fields to be updated. External field must be included  
 records_to_upsert = Array.new  
 salesforce.upsert("Account", records_to_upsert, "External_Field_Name", true) # last parameter indicates whether to wait until the batch finishes

Better Error Catching

Initializing the salesforce client with incorrect credentials will now result in an error similar to this:
 INVALID_LOGIN: Invalid username, password, security token; or user locked out. (RuntimeError)  
versus the dreaded "NoMethodError: undefined method `[]' for nil:NilClass" error.

Better Result Reporting

The results of a job are now returned in a format that is easy to read and manipulate. Take this job with 2 records, the first record being invalid (check the id) and the second being valid.
 updated_account = Hash["name" => "Test Account -- Updated #{Time.now}", "id" => "CLEARLY_AN_INVALID_ID"]  
 updated_account2 = Hash["name" => "Test Account -- Updated #{Time.now}", "id" => "001A000000sibbu"]  
 records_to_update = Array.new  
 job = salesforce.update("Account", records_to_update, true)  
 puts "result is: #{job.result.inspect}"  
The result of the puts statement will look like this:
 result is: {"errors"=>[{"0"=>"MALFORMED_ID:Account ID: id value of incorrect type: CLEARLY_AN_INVALID_ID:Id --"}], "success"=>false, "records"=>[#<CSV::Row "Id":"" "Success":false "Created":false "Error":"MALFORMED_ID:Account ID: id value of incorrect type: CLEARLY_AN_INVALID_ID:Id --">, #<CSV::Row "Id":"001A000000sibbuIAA" "Success":true "Created":false "Error":"">], "raw"=>"\"\",\"false\",\"false\",\"MALFORMED_ID:Account ID: id value of incorrect type: CLEARLY_AN_INVALID_ID:Id --\"\n\"001A000000sibbuIAA\",\"true\",\"false\",\"\"\n", "message"=>"The job has been closed."}  

The status and errors of each processed record is available through result.records, and the errors are available through a hash that is indexed in the same order that the records were added to the array. It is now easy to check the status of a job via result.success, which returns true of false. Note that result reporting is only available if the job is set to wait for processing. If waiting is turned off, then you will simply receive a generic message saying that the job has been queued.

I plan on maintaining this gem more than I have in the past. Hopefully many people have gotten use from it in its current state, and hopefully many more will get use from version 1.0.


  1. Hi, I need to do a query from Salesforce with relationships. Eg. Select a.object1__r.Name, a.object2__r from a where Id = id.
    Can you please confirm whether is it possible?

  2. agen bola Respect yourself first so you will get the respect of others, The more we are grateful, the more happiness we get poker qiu qiu

  3. First of all i appreciate the author for taking the effort to share an article about a very good topic. This article throws light on the new film : In Between Black and White!!!. What surprises me is that my favorite star Zara Phillips. who is a singer, song writer, actor and author, is also acting in this.Stump Removal Bronx

  4. Excellent Publish. I have been looking for this actual factor all 7 days. Awesome discuss for me, maybe nice for all audience of your site.schwimmbad dach