Open Testware Reviews

Sclc Metrics Tool

Contents

Overview

Maturity

Project activity

Platforms

Support

Documentation

Installation

Implementation

Sclc is implemented in a single Perl script. Run against itself, it counts 1008 non-comment source lines. There are 680 comment lines. Function headers vary from a single line to a long standardized template. There is a sprinkling of comments throughout the code that should help experienced Perl programmers understand the script.

I like the modular design of the definitions for each language that sclc parses. I tried once to add code to support the Limbo programming language, but after several minutes of studying the code and the comments I wasn't quite able to get the gist of it. I ended up using the -language option to tell it that Limbo programs were shell code, because they both use the same comment syntax, and I think the line count results ended up being accurate except for the overall summary for shell code. If I had wanted to spend more time teaching sclc to understand Limbo, I'm pretty sure I could have done it with some experimentation and/or help from the author.

There are no test cases that ship with the tool. The script does use the "use strict" mechanism to enforce good programming practices, but it does not use "use warnings" to catch errors at run-time.

The author states that sclc is "ancient Perl 4 code," with only minimal porting to use common Perl 5 coding standards. This should only be a concern to Perl purists who both want to work with a newer style implementation and aren't willing to help update the sclc code.

Performance

Parsing source code tends to be cpu-intensive. I tested sclc by running it recursively on all files in the DejaGnu 1.4.3 source distribution with 66,000 total lines of code. It took an average of 2 minutes on a 266 MHz Windows machine, 48 seconds on a 600 MHz Mac, and 27 seconds on an i686 Linux machine of unknown origin.

Similar tools

Clc is a predecessor to sclc, also written by Brad Appleton. It works for C, C++, and Perl, and unlike sclc, it also counts source statements as well as lines of code. When I use clc to measure sclc, it agreed with sclc's assessment of total lines of code, but disagrees significantly on the number of non-comment source statements, 1008 according to sclc, and 1656 according to clc. I don't trust the numbers from clc - a simple grep shows that there are only 1480 lines that don't start with a comment character. I think the two tools tend to agree more on C code. The moral is - it doesn't hurt to use two different tools as a sanity check.

Clc is dated February 14, 1995, a few months before sclc originally appeared. It's on Chris Lott's "Metrics collection tools for C and C++ Source Code" page, which includes several other tools that are either abandoned or are outdated copies. With some detective work you can find more recent copies of some of them.

One such tool listed on Lott's page is sloccount. You'll find the latest version at http://www.dwheeler.com/sloccount/. Sloccount can parse 27 languages, more than twice as many as sclc, it probably has better heuristics for automatically determining the language used within a file, and it seems to have a larger user community than sclc. I compared the NCSL numbers from both sloccount and sclc after analyzing a directory of Linux kernel source files, and the numbers were identical.

So why do I stick with sclc? I like sclc's user interface better. It's more difficult to specify the files you want to process with sloccount, especially if you just want to check one file. Sloccount's output is more cluttered. It probably wouldn't take much effort to make sloccount easier to use, but at first glance, I like sclc better.

Limitations

1. Sclc counts lines of code, not source statements. Counting source statements is probably more accurate when you're dealing with code developed with a variety of coding styles, because raw line counts are sensitive to coding style. Using the -delim-ignore option may make sclc's NCSL numbers roughly similar to a source statement count.
2. The tool often makes wrong guesses about language - for example, it thought a Cascading Style Sheet (.css) file was C code, it counted a makefile.in file as shell code rather than a makefile, it parsed an RTF file as Pascal, and it counted Expect and incr Tcl files as shell code. The workaround is to use the -language option to make it smarter about recognizing file extensions, and exclude documentation and other files you don't want it to count.
3. The numbers are likely to be different if you're using the "-diff" option and you use a context diff rather than the default diff format. The sum of the inserted and deleted lines tends to stay the same, but the individual counts vary.
4. With the "-diff" option, sclc doesn't try to count changed lines. All modifications are reported only in terms of additions and deletions.
   
5. The tool gives no output if you give it the "-diff" option and its input stream is empty. I'd prefer to get positive confirmation that sclc ran.
6. I had to go to the source code to find out that the regular expressions given as arguments to "-name" and "-except" are sandwiched between ^ and $ anchors automatically. The "filename must completely match" comment in the man page didn't get this point across to me.
   
7. Sclc doesn't run preprocessors on files. After macro expansion, conditional compilation, etc., the number of lines of code that actually get compiled can grow or shrink. That's probably okay, and probably the way most metrics tools work, but it's something to be aware of.
8. The AESL value changes when using the -delim-ignore option, because this option changes the NCSL count and the AESL is based on NCSL. It seems strange that the assembly equivalent for a file would change if you change the way you measure the source code.
9. The "Totals" and "LangTotals" arguments to the "-sections" option are ignored if you're only analyzing one source file. This might make it more difficult to parse the output with a script that doesn't know ahead of time how many files will be analyzed. There is also no mention in the documentation that these two sections are omitted by default if there's only one file.
10. It would help if the column headings were repeated before the summary, to help the user remember what each column represents. I once read the AESL column thinking it was NCSL, which resulted in an order-of-magnitude error in the numbers I reported, and I didn't catch the error for several days.
    
11. If you put the same filename on the command line twice, there is no warning and the file is counted twice in the totals.
    
12. If you ask sclc to process a file that doesn't exist, it gives a misleading error such as: "sclc: Can't determine programming language for nosuchfile."
    
13. The man page is garbled with escape characters on my Cygwin/Windows 2000 configuration using TERM=cygwin or vt100. Probably not an sclc bug. I also sometimes saw an error on Linux that resulted in the man page coming out as raw POD markup.
14. If there are 100,000 or more lines in a source file, the columns in the output don't line up.

• Fatal error using ActiveState Perl on Windows to process a file that doesn't have an extension - "Unmatched ) in regex" at line 1244.
• If I enter a bad command-line argument, a man page is printed that obscures the error message.
• Reports erroneously that an empty file with no extension is a binary file.
• Gives no output for an empty file with an extension (e.g. empty.c).
• Tries to process directories as source files when not using -recurse. Either reports them as binary files or says it can't determine the programming language.
• Silently ignores context diff's (diff -c) when using the -diff option.
• The synopsis in the man page is wider than the standard 80 character line on Linux and HP-UX.

Observations

Sclc is a tool that can do simple static analysis on 13 different programming languages (if you count makefiles as a language). It uses a command-line interface that looks like this when you ask it to analyze itself:

$ sclc sclc
Lines  Blank  Cmnts   NCSL     AESL
=====  =====  =====  =====  ==========  =======================================
 1824    167    680   1008     15120.0  sclc  (Perl)

It shows a total line count, the number of blank lines, the number of lines containing either full-line or inline comments, and the number of non-comment source lines (NCSL). Note that if your code has lines with inline comments, those lines will be counted in both the comment and NCSL total. The AESL metric refers to "assembly-equivalent source lines," discussed further in the Documentation section earlier. I don't use the AESL metric, and I could suppress it from the output using the "-counts" option.

These kinds of metrics, especially the raw line count (often called the LOC or KLOC metric, for "lines of code" or "thousands of lines of code") that you could generate with a simple tool like the Unix "wc" utility, have been the cause of much academic hand-wringing because there are so many ways to misuse the metrics. I'm going to presume that if you're considering using a tool like sclc, you've done some background reading so that you understand the limitations of these simplistic metrics. Here are a few references to get you started--Software Metrics: Successes, Failures and New Directions by Norman E. Fenton, and for an interesting but hard to implement alternative, Managing (the Size of) Your Projects: A Project Management Look at Function Points by Carol Dekkers.

Despite the shortcomings, LOC and NCSL metrics are very common software metrics. In fact, for both of my previous reviews, I have reported NCSL metrics for the tools I reviewed, using data generated by sclc. These metrics give us a very rough idea of how complex the programs you're dealing with are. I've also used sclc during a consulting engagement to demonstrate the magnitude of the hundreds of thousands of lines of code they were trying to wrangle.

If you're analyzing more than one file, you get a column total at the bottom of the output, plus a file count. And if you're dealing with more than one programming language, you'll get a breakdown by language. Here's an example of a further refinement of the DejaGnu analysis I mentioned earlier, to help sclc figure out the programming language and to tell it to ignore certain files.

$ sclc.pl -language .exp=tcl -language .itcl=tcl -except '.*\.(rtf|in|am)' \
-recurse -ignore -counts Lines+Blank+Cmnts+NCSL .
Lines  Blank  Cmnts   NCSL
=====  =====  =====  =====  ===================================================
   36      1      1     34  ./.clean  (shell)
  901    134    314    453  ./aclocal.m4  (shell)
   23      4     11      8  ./baseboards/a29k-udi.exp  (Tcl)
   37      8     15     14  ./baseboards/arc-sim.exp  (Tcl)
...
34277   3985   6952  23504  ----- Tcl -----  (256 files)
  409     63     38    310  ----- C++ -----  (2 files)
  345     44     16    289  ----- C -----  (6 files)
24652   2588   2882  19291  ----- shell -----  (51 files)
 2223    214    606   1416  ----- Lisp -----  (1 file)
61906   6894  10494  44810  ***** TOTAL *****  (316 files)

This still needs further tuning, because there are still documentation files like ".clean" above that shouldn't be counted, m4 files that might should be counted but not as shell scripts, and all the ".in" and ".am" files that I haven't figured out. So what counts as code that needs to be counted? Tools aren't going to help much with that conundrum.

Sclc is packaged with the cdiff tool, which is a wrapper on top of the cleardiff command in the ClearCase configuration management system. Sclc has a few command line options for interfacing with cdiff. It probably wouldn't take much effort to enhance sclc to talk to other configuration management systems as well. But sclc works just fine if you don't use ClearCase.

Note that the URL for sclc is a redirect to a different site. Be sure to take note of the bradapp.net URL, because the underlying page has changed recently, and it may change again.

There are many freeware metrics tools lurking around the Internet, especially for C code, though most of them are orphaned. Sclc is slightly rough around the edges, but it's my favorite among the tools I've tried.