Chutzpah 2.0 is now available. This release contains many new and hopefully exciting changes. You can enjoy the new bits from CodePlex, NuGet or go to the Visual Studio Gallery to get the updated Visual Studio Context Menu Extension and Unit Test Explorer adapter for Visual Studio 2012.
Notable Changes
Streaming of test results
The previous version of Chutzpah would only report test results after the whole test file was finished. Version 2.0 re-architects how Chutzpah communicates with PhantomJS to stream the results in real-time. This change leads to a much more responsive test running experience. The streaming change enabled the addition of the TestStart, FileLog and FileError methods to the execution callback interfaces.
public interface ITestMethodRunnerCallback
{
void TestSuiteStarted();
void TestSuiteFinished(TestCaseSummary testResultsSummary);
void FileStarted(string fileName);
void FileFinished(string fileName, TestCaseSummary testResultsSummary);
void TestStarted(TestCase testCase);
void TestFinished(TestCase testCase);
void ExceptionThrown(Exception exception, string fileName);
void FileError(TestError error);
void FileLog(TestLog log);
}
HTML Test Files are run in place
When running Chutzpah on an existing HTML test harness it will no longer parse this file and generate its own test harness in the temp directory. Chutzpah now trusts the HTML test harness you give it and will just run the file. This will help support many of the fringe scenarios that were not supported well before. For JavaScript files Chutzpah will still generate its own test harness.
Parallel Test File Execution
You can now specify through both the command line and through a Visual Studio options page a level of parallelism for Chutzpah. By default this is 1 but if you have a large number of test files putting a larger number results in a significant performance increase.
From the command line the new argument is
/parallelism n : Max degree of parallelism for Chutzpah. (Defaults to 1)
In Visual Studio you can set this in the existing options page for the context menu extension and in the new options page for the Unit Test Explorer adapter:
Testing Mode Drop Down
In the screenshot above you may have noticed the Testing Mode option. This is a new configuration option for the Chutzpah test adapter. This lets you choose whether Chutzpah will scan and run HTML files, JS files or both.
Improved RequireJS support
I will write a full post to go into more detail about this but Chutzpah is now able to run tests that are using RequireJS. This support isn’t perfect and you may need to tweak your RequireJS testing setup to make it work. There are a couple examples checked in which demonstrate this functionality. For an HTML test file see here and for running a JS file directly see here.
Upgraded versions of Jasmine and QUnit
When generating its own test harness Chutzpah will now use QUnit 1.9 and Jasmine 1.2
And as always if you have any ideas/bugs/suggestions or you want to contribute code please check out the Chutzpah CodePlex page.
Hi Matthew,
I’m just starting out with backbone, requirejs & jasmine and chutzpah looks like it’ll be a great help, thanks!
Any news on the blog post regarding the improved requirejs support? I’m trying to marry up the example to my solution but I’m struggling with what’s needed to run the js tests directly rather than through html.
Yea I have been meaning to get to writing more about how to get Chutzpah to work well with Require.js but I haven’t had a chance. In the meanwhile you can take a look at the integration test I use to verify this functionality in Chutzpah:
Jasmine Require.js in Chutzpah
Take a look at that and let me know if you have any questions.
Hi, I managed to get the tests working but I have to save “all.tests.js” to pick up any amendments to my tests in Chutzpah. Is this expected?
It’s still a bit confused by the file structure and the “references.js” files but I’m getting there.
Thanks again!
That is unfortunately expected. In the setup I referenced Chutzpah will monitor the test file which is just than file that references other tests. It would be nice if Chutzpah could monitor all referenced files and re-fresh tests when any changed.
Hi, does this post still apply in version 2.3? I am not able to run the tests with requirejs? is the references.js required?
What version of require.js are you using? If you think there is an issue please file a work item on chutzpah.codeplex.com and attach a sample zip that shows your issue.
Thanks!
Hello Matthew, thanks for this great tool!
I’m having a hard time getting RequireJS to work with Chutzpah in Visual Studio 2012. The links to example files in the above “Improved RequireJS support” section are broken. Do you have updated links?
Thanks!
Dave
I am currently working on making require.js support better. Will blog about the release when done.
Wow, thanks for the lightning-fast response!!
That’s very cool that you’re working on making RequireJS support better. But, does that imply that there’s not a workable Chutzpah + RequireJS solution today? I have an immediate requirement to set up headless unit testing for a JavaScript library that depends on RequireJS. If I can’t make RequireJS work with Chutzpah, then I can’t use Chutzpah. It’s ok if that’s the answer — I recognize that you’re donating your time — but if there’s a reasonable way for me to get this working, then I’d rather do that, because Chutzpah is cool. :-)
Thanks!
Dave
There is a workable way to make it work today, take a look at this chutzpah integration test
Thank you! I was able to get this working after looking at your integration test. For future readers of this post, I recommend:
Get Chutzpah & Jasmine (or QUnit) working first.
Add Matthew’s RequireJS folder (including its contents) into your project.
This should cause Matthew’s tests to appear in VS’s Test Explorer pane. Run the tests to verify that they work.
Gradually transform the folder structure to get it to be the way you want. I had to do this in small moves, because it seemed like Chutzpah lost track of the tests easily. I also found that if I touched any of the files containing tests, I also had to touch all.tests.jasmine.js last in order for Chutzpah to pick up the tests again.
Thanks again!
Sorry about the formatting above. I meant to do a list, and must have missed a tag.