Avoiding side effects when traversing an IVsHierarchy in Visual Studio

There was a bug filed against Chutzpah reporting that Chutzpah’s Visual Studio extension was causing database connection error popup. I was surprised by this since all Chutzpah really does is traverse the project hierarchy and I wasn’t sure how this could result in that error. After digging in and searching online a bit I found the issue. The act of traversing a node in the IVsHierarchy can cause arbitrary side affects (like establishing a database connection!).  In order to determine if this can happen you must check the __VSHPROPID.VSHPROPID_HasEnumerationSideEffects property on an IVsHierarchy item. Once I found this the fix was simple, just skip any node that has that property set.


Chutzpah 2.0 Released

Chutzpah 2.0 is now available. This release contains many new and hopefully exciting changes. You can enjoy the new bits from CodePlexNuGet or go to the Visual Studio Gallery to get the updated Visual Studio Context Menu Extension  and Unit Test Explorer adapter for Visual Studio 2012.

Notable Changes

Streaming of test results

The previous version of Chutzpah would only report test results after the whole test file was finished. Version 2.0 re-architects how Chutzpah communicates with PhantomJS to stream the results in real-time. This change leads to a much more responsive test running experience. The streaming change enabled the addition of the TestStart, FileLog and FileError methods to the execution callback interfaces.

public interface ITestMethodRunnerCallback
  void TestSuiteStarted();
  void TestSuiteFinished(TestCaseSummary testResultsSummary);
  void FileStarted(string fileName);
  void FileFinished(string fileName, TestCaseSummary testResultsSummary);
  void TestStarted(TestCase testCase);
  void TestFinished(TestCase testCase);
  void ExceptionThrown(Exception exception, string fileName);
  void FileError(TestError error);
  void FileLog(TestLog log);


HTML Test Files are run in place

When running Chutzpah on an existing HTML test harness it will no longer parse this file and generate its own test harness in the temp directory. Chutzpah now trusts the HTML test harness you give it and will just run the file. This will help support many of the fringe scenarios that were not supported well before. For JavaScript files Chutzpah will still generate its own test harness.


Parallel Test File Execution

You can now specify through both the command line and through a Visual Studio options page a level of parallelism for Chutzpah. By default this is 1 but if you have a large number of test files putting a larger number results in a significant performance increase.

From the command line the new argument is

 /parallelism n           : Max degree of parallelism for Chutzpah. (Defaults to 1)

In Visual Studio you can set this in the existing options page for the context menu extension and in the new options page for the Unit Test Explorer adapter:


Testing Mode Drop Down

In the screenshot above you may have noticed the Testing Mode option. This is a new configuration option for the Chutzpah test adapter. This lets you choose whether Chutzpah will scan and run HTML files, JS files or both.


Improved RequireJS support

I will write a full post to go into more detail about this but Chutzpah is now able to run tests that are using RequireJS. This support isn’t perfect and you may need to tweak your RequireJS testing setup to make it work. There are a couple examples checked in which demonstrate this functionality. For an HTML test file see here and for running a JS file directly see here.

Upgraded versions of Jasmine and QUnit

When generating its own test harness Chutzpah will now use QUnit 1.9 and Jasmine 1.2


And as always if you have any ideas/bugs/suggestions or you want to contribute code please check out the Chutzpah CodePlex page.


Chutzpah test adapter for Visual Studio 11 Beta

Visual Studio 11 Beta was released today and one of the many cool new features is the Unit Test Explorer.


The Unit Test Explorer functions using a plugin model where it scans for test adapters and uses them to find and run tests. You can download adapters from the Visual Studio Gallery which currently hosts the xUnit.net, NUnit and Chutzpah adapters.

With the addition of the Chutzpah adapter Visual Studio 11 Beta can discover and execute QUnit and Jasmine unit tests. VS now provides a nice graphical interface to see and run the available tests.



The Chutzpah adapter is still very young and has much room for improvement but I wanted to get something out there early to get feedback. Feel free to provide feedback on the Chutzpah CodePlex page. Soon I will also be upgrading the standard Chutzpah Visual Studio plugin to VS 11 so you still have the option of a TestDriven.net type experience for JS tests.


The problem with the EnvDTE ItemOperations.NewFile method

During my work on the latest version of the Snippet Designer I was struggling with solving an issue with the EnvDTE method ItemOperations.NewFile. The purpose of the NewFile method is to create a new file in the same manner as if you created it from the File->New->File  dialog in Visual Studio. I used this method as part of the Snippet Designer’s Export as Snippet feature. The NewFile method was working fine (I thought) until I started receiving some bug reports from French and German users. After looking into it deeper I learned that the way I was using the NewFile method would not work in non-English installations of Visual Studio. To understand why you need to know a little bit more about ItemOperations.NewFile.

The NewFile method is define as:

Window NewFile(
	string Item,
	string Name,
	string ViewKind

When I would launch a new snippet file I would call the method like:

 Dte.ItemOperations.NewFile( @"General\Text File", "SnippetFile1.snippet");

The MSDN documentation for the first argument of this method is

The virtual path to the item based on the tree nodes from the left pane of the dialog box and the item name from the right pane.

This is where the problem was, the virtual path is language dependent! So while in English the correct path is  “General\Text File” in Spanish it is something like “General\Archivo”.  At this point I looked into this further to see if anyone else has this issue with no luck.

This left me with two options:

  1. Figure out each language specific virtual path
  2. Find a different way to have Visual Studio launch the file

Option 1 is an annoying and fragile solution so I started looking for alternate methods. Eventually I got the idea to try to send the command directly to the Visual Studio shell’s IOleCommandTarget::Exec method. The IOleCommandTarget interface helps define the command design pattern which VS uses. Many components in VS implement this interface (explicitly or implicitly) so that they can handle commands from the user.

Getting the IOleCommandTarget for the Visual Studio shell is pretty simple once you know that there is a service for this called SUIHostCommandDispatcher

private IOleCommandTarget GetShellCommandDispatcher()
    return GetService(typeof (SUIHostCommandDispatcher)) as IOleCommandTarget;

Then looking through the VSConstants.VSStd97CmdID list I found the FileNew command id

FileNew     Command id:”File New” on File menu.

With the both the command target and the command id I was able to execute the command:

private bool LaunchNewFile(string fileName)
    IntPtr inArgPtr = Marshal.AllocCoTaskMem(200);
    Marshal.GetNativeVariantForObject(fileName, inArgPtr);

    Guid cmdGroup = VSConstants.GUID_VSStandardCommandSet97;
    IOleCommandTarget commandTarget = GetShellCommandDispatcher();
    int hr = commandTarget.Exec(ref cmdGroup,
                                (uint) VSConstants.VSStd97CmdID.FileNew,
                                (uint) OLECMDEXECOPT.OLECMDEXECOPT_DODEFAULT,
    return ErrorHandler.Succeeded(hr);

One thing to note above is that in order to pass the file name for FileNew command into the Exec method you need to marshal the string into a native object since IOleCommandTarget is just a managed interface on top of native COM interfaces.

After all of that I was able to correctly launch a new file through Visual Studio that works regardless of locale.


OpenWithTest 0.4 Released

OpenWithTest 0.4 has been release on the Visual Studio Gallery.

If you already have it installed you should get an update in your extension manager.

Release Notes

  1. Added an OpenWithTest toolbar button. This button allows you to quickly enable and disable the plugin.  I find this useful since sometimes I am just exploring code and don’t want to have all the test files open. However, once I am ready to code I can easily turn it back on.


    You can add this button by right clicking on the toolbar and selecting OpenWithTest.

  2. Added the ability to delete your index file. It is possible the the index file the plugin creates for your project may eventually get out of date.  In this case in may be helpful the purge the index and start fresh.  I added a new option menu that allows you to do this.  After you delete the index for a solution you need to close and reopen that solution for a new index to be built.

  3. Improved error handling.