Chutzpah 4.1 – Visual Studio Debugger Support

A week ago Chutzpah 4.1 which finally adds the ability to debug your unit tests directly from Visual Studio.




Debugging from Visual Studio

Thanks to the awesome PR of Martin Connell you can now debug your JS tests from inside of the IDE. You can set breakpoints in your JS files and launch using the Visual Studio Debugger by clicking:

Debug Tests when using the Chutzpah Test Adapter Plugin


Run Chutzpah With -> Debugger when using the Chutzpah Context Menu Extension

Chutzpah will then launch Internet Explorer and attach the Visual Studio Debugger to it.


NOTE: Before this will work you must ensure script debugging is enabled in IE by going Tools->Internet Options->Advanced and unchecking both Disable script debugging settings.


Chutzpah 4.0 – Batching, Inheritance and more

Chutzpah 4.0 cleans up some of the legacy baggage while adding the much requested feature of test file batching.


Test File Batching

Chutzpah has always created an HTML harness for each of your test files. This has the advantage of isolated one test from another but it has the big draw back of limiting throughput of running all your tests. Especially for people with large suite of tests parallel execution of individual test harnesses will not be as fast as batching them all together in one harness. With this release Chutzpah now adds support for both modes. When you set the following in your chutzpah.json file:

"EnableTestFileBatching": true

It tells Chutzpah to put all the test files under this chutzpah.json file into one HTML harness (see full sample). Because the settings file determines the grouping of files that get combined into one harness it gives you the ability to determine what tests you want to get loaded into one harness. Chutzpah can then run those harness in parallel which can give you increased throughput. In the future (based on user feedback) batching may become the default.


Settings File Inheritance

The addition of the test file batching feature above encourages the use of multiple chutzpah.json files to help control how files are batched and parallelized. This presents a problem though since your project will often have a set of common files and you would rather not list them multiple times in different settings files. To address this you can now inherit settings from a parent chutzpah.json file. In your chutzpah.json file (see full sample) you write:

"InheritFromParent": true

Which instructs Chutzpah to traverse recursively up the folder hierarchy and look for another chutzpah.json file. If found it will merge the two files and combine their settings. The inheritance between a parent and child settings works as follows when both the parent and child files contain the same setting:

Setting Name What happens?
References References from child template are prepended with the references from the parent template
Templates Templates from the child template are prepended with the templates from the parent template
Tests Not inherited
*All Others* Child template overwrites the parent template setting


Multiple Includes/Excludes for Tests and References Settings

The Tests and References settings now have Excludes and Includes properties (in addition to the old singular Exclude and Include setting). The new plural settings let you specify an array of patterns. This provides more flexibility when describing what files you do and do not want to include for a folder. For example:

    "Tests": [
        { "Path": "someFolder/someOtherFolder", "Includes": [ "*Dir1/*.js", "*Dir2/*.js"], "Excludes": [ "*Dir1/test4.js", "*Dir2/test2.js" ] }


Legacy Compilation Support Removed

The 4.0 release removes the legacy compilation support. Chutzpah no longer ships with a built in TypeScript and CoffeeScript compiler. Please use the compile setting now to configure how Chutzpah handles TypeScript or CoffeeScript files.


Chutzpah 3.3.0

Chutzpah 3.3.0 is now available at your usual locations.


  • Update to QUnit 1.16
  • Update to Jasmine 2.1.3
  • Improve parsing of Chutzpah.json files (finally allow comments)
  • Skip paths that are too long while discovering tests
  • Integrate test result transformers into Chutzpah.json file
  • Add LCOV test result transformer
  • Add TRX test result transformer
  • Add source map support for code coverage
  • Fix issue with requirejs urls containing querystrings
  • Move HTML template injection from HEAD tag to BODY tag


Test Result Transforms

Chutzpah has long had support to transform your test results into the JUnit XML format but this was only exposed from the command line. With recent additions this feature has been greatly improved: Chutzpah now supports JUnit, LCOV (for code coverage) and TRX file formats. In addition, you can now specify what transforms you want and where the output files should live using the chutzpah.json file.

For example the settings below will output the results as lcov, trx and junit to the specified paths whenever Chutzpah runs.

    "Transforms": [
        { "Name": "lcov", "Path": "C:\temp\lcov.dat" },
        { "Name": "junit", "Path": "C:\temp\junit.xml" },
        { "Name": "trx", "Path": "C:\temp\trx.trx" }


Code Coverage Source Map Support

One annoyance with Chutzpah’s code coverage was that is always showed coverage in terms of the .js files even if your source files were TypeScript or CoffeeScript. To address that problem this release contains a new option as part of the compile settings called UseSourceMaps. This will instruct Chutzpah to look for source map files in your configured compile output directory. If it finds them it will map your coverage results to the lines of the source files and generate the coverage HTML accordingly.

Here is an example config using this setting:

    "Compile": {
        "UseSourceMaps": true,
        "Mode": "External",
        "Extensions": [ ".ts" ],
        "ExtensionsWithNoOutput": [ ".d.ts" ]
    "Tests": [
        { "Path": "Tests.ts" }
    "References": [ 
        { "Path": "SourceMappedLibrary.ts" }
    "CodeCoverageExcludes": ["*Tests.ts"]

Chutzpah is now on GitHub

Today, I moved Chutzpah from CodePlex to GitHub. I made this move for a few reasons

  1. The feature set GitHub offers has grown increasingly compelling.
  2. There is a more active community on GitHub and I am hoping this will lead to more user engagement.
  3. I have received feedback from multiple Chutzpah users that they would like to have it moved to GitHub.


Source Code

Migration of the source code was simple since it’s all git.



The migration of the documentation was simple but time consuming. CodePlex uses a different syntax than MarkDown so I manually ported all the pages. If you notice any issues likes broken links please let me know.



The issue migration was interesting. I have a couple hundred closed/active issues on the CodePlex site. I considered just leaving the old ones on Codeplex but I really like having them all in one place. When I searched to find if anyone wrote a tool to do this I found the CodePlex Issues Importer, perfect! Well, there was a catch. That project hasn’t been updated in three years which meant it was using an old version of the GitHub API and was screen scraping looking for non-existent elements on the CodePlex site. I still started playing with this project and with some minor changes I was able to get it to work and import all of my issues to GitHub! I sent a PR to the CodePlex Issues Importer project with my fixes to help anyone else who wants to migrate.


What’s Left

There are still a few links that point to the CodePlex site in the VSIX packages. I will update these during the next release.


Avoiding side effects when traversing an IVsHierarchy in Visual Studio

There was a bug filed against Chutzpah reporting that Chutzpah’s Visual Studio extension was causing database connection error popup. I was surprised by this since all Chutzpah really does is traverse the project hierarchy and I wasn’t sure how this could result in that error. After digging in and searching online a bit I found the issue. The act of traversing a node in the IVsHierarchy can cause arbitrary side affects (like establishing a database connection!).  In order to determine if this can happen you must check the __VSHPROPID.VSHPROPID_HasEnumerationSideEffects property on an IVsHierarchy item. Once I found this the fix was simple, just skip any node that has that property set.