« Regular Expression Tester Updated | Main | This is the end »

AS3 Code Coverage

Note A code coverage tool is now available at http://code.google.com/p/flexcover/.

This maybe a little crazy but I'm going to throw it out there. I've been thinking about a way to get code coverage reports for Actionscript 3. We make heavy use of FlexUnit at work, but I'm always curious just how much of our code is really covered with the tests. I've not found anything about code coverage tools with the few Google searches that I've run hence my search for other solutions.

My current idea is to leverage the Flex debugger. Something along the lines of creating a break point on every line in ever file that you want to check coverage of. As the break points get hit, remove them. Once all of the test code is done running, see if there are any break points that haven't been hit.

This isn't elegant by any means but it should be possible to test the theory with a few hours work. If someone knows of a tool that already does this I'd love to hear about it. Otherwise I'll see how my hacking goes.

Tags: as3 fdb flex testing

Posted by Daniel Rinehart on June 29, 2006 9:23 PM |

Comments

We are looking for the same thing, and want to see what you come up with! We have two additional thoughts: 1) there's a company that specializes in code maintenance tools & they've given us a quote to adapt their code coverage, profiling and clone detection (finds duplicated code fragments by parsing code base into abstract tree & comparing to find duplicated fragements); they have the core for multiple languages & would only need to adapt to actionscript; their name is Semantic Designs; we'd seriously consider collaborating with other companies to go to these folks with money from a few customers to get the work done & propose the results be shared. 2) a poor mans version of #1 would be using some parsing knowledge to insert something like profiling data at each conditional and use the flex profiler to dump the data - it might be munged into some semblance of code coverage. Thoughts? Feel free to contact - email is san at saplingNOSPAMsystems dot com

Posted by: Steve Sanderson | July 7, 2006 1:13 AM

Has anyone settled on a workable solution yet? I have been looking for something like this for a while and haven't seen anything yet. Thanks

Posted by: Lance | August 3, 2006 5:27 PM

I was thinking about the need for such a tool... But how do you propose to drive the Flex debugger itself? I suppose you can do it, since it's somehow interacting with Eclipse Debugging Framework... Let me know; also, I'd like to offer help, I am somewhat familiar with EDF...

Posted by: Gregory Golberg | January 12, 2007 9:43 PM

I was thinking about code coverage in AS3 as well. Here's my idea for how to attack the problem: http://www.colettas.org/?p=60

Posted by: David Coletta | January 21, 2007 2:45 PM

Dan, We have a working prototype of (basic) code coverage that we plan to deploy on dpUInt soon. It uses the flex profiler classes and works fairly well for identifying methods that are never touched. It is not yet powerful enough to notice if conditional paths are all executed. Mike

Posted by: Michael Labriola | February 13, 2008 12:35 PM

Here's my take on how to do this the right way, so that 1) the code doesn't have to be touched in any way, 2) every line of source code is identified in coverage reports, and 3) the debugger and profiler are kept out of the picture (dealing with that stuff just makes everything more complicated and fragile at runtime and will seriously kill performance). Now that Flex 3 SDK is open sourced, we can modify the compiler itself to instrument every line of code. Each line of code like this: 
    // line 123 of SomeClass.as
    a = b.c();
turns into something like this: 
    SomeClass._trackCoverage(123); a = b.c();
The class also gets instrumented: 
    private static var _coverageCount:Array = [];
    private static var _coverageInit:* = Coverage.registerClass(SomeClass);

    private static function _trackCoverage(lineNumber:uint):void
    {
        if (lineNumber in _coverageCount)
            _coverageCount[lineNumber]++;
        else
            _coverageCount[lineNumber] = 1;
    }
Finally, there needs to be some reporting tool that walks all the classes at the end that registered themselves and writes a report somewhere. I'm thinking of doing it. Everyone needs this stuff!

Posted by: Joe Berkovitz | March 27, 2008 11:13 AM

Code coverage would be a nice. I've seen tons of articles on Test Driven Development for flex, but none of them mention source code metrics. Why don't we have metrics for flex? If we are going to use OOP and TDD with flex, then we could definitely benefit from having metrics available such as line coverage, cyclomatic complexity, LCOM, etc...

Posted by: Kevin Carr | April 6, 2008 11:52 PM

Just to close the loop on my earlier post -- I have in fact created a code coverage tool along the lines of what I described. It is posted on Google Code at http://code.google.com/p/flexcover/. Alex Uhlmann and I are going to continue moving it forward. It does class/function/line coverage now, and is being extended for branch coverage.

Posted by: Joe Berkovitz | April 14, 2008 8:40 PM