Using Perl Scripts

Software QA Magazine, Vol. 2, No. 3, 1995 Tester's Toolbox column

by Danny Faught, faught@tejasconsulting.com
Copyright 1995, Danny Faught


This is the first in a series of articles that will focus on 
testing technology implementation.  The first topic is perl, a
valuable tool in a tester's toolbox.  I will touch on a few of the ways
you might be able to use perl in a test environment, and some of the
issues you should consider when deciding to use perl.  At the end
is a list of resources you can use to find out more.

Perl is a freely available scripting language that seems to be growing
in popularity.  It is more powerful and typically runs faster than other
choices such as batch files or Unix shells.  It runs faster because
more of the commands are available as system calls or are built into
perl, where many other types of scripts require you to call external
programs for most of the real work.

Perl code is often faster to develop and easier to maintain than C
code.  It does not have as many arbitrary limits as sed and awk.
Programmers who are familiar with any of these languages will easily
recognize many of perl's constructs.

Virtually every Unix and Unix-like platform runs perl.  It is
also available on MS-DOS, Windows/NT, Macintosh, OS/2, VMS, and
AmigaDOS.  Because perl grew up under Unix, some of its functionality may
not be present on non-Unix ports.  You can sometimes find pre-built
perl binaries for your system if you want to get started quickly.  See
the perl FAQ for more information, or look in a newsgroup dedicated to
your system.

Perl began as a language to support text processing.  Its regular
expressions are the richest I have ever seen.  With regular expressions
and other string manipulation functions, it is easy to do complex text
transformations with a surprisingly small amount of code.  Perl also
can be used to manipulate binary data, and it is a good tool for some
tricky (but small-scale) data juggling.  Associate arrays provide a
powerful hash table mechanism, without the programmer even having to
understand what a hash bucket is.  A package mechanism allows the perl
programmer to protect the name space, and local variables may use
either dynamic or static scoping.


What testing tasks can perl help with?
--------------------------------------

The OS test environment at Convex uses a variety of languages at all
levels.  You will find perl scripts from the top-level user interface
down to the functional tests.  Because the test environment is
supported on several different operating systems, a benefit of using
scripts is that the same executable script runs on all operating
systems.  Of course, we have to build and install the perl interpreter
on each system, and we sometimes have to use conditional code to make a
script portable.  But when we update a script we do not have to
recompile it for each system.  We simply make the change and make the
new version of the script available.  This shortens the
edit-compile-debug process by one step.

Perl is well-suited as a systems programming language, and that
makes it a good choice for developing test drivers and front ends.
Perl's powerful text processing capabilities make it easy to implement
elaborate features such as configuration files and complex command line
options that you might be reluctant to even try to implement in a
language like C.

You might also find that unit tests are a good use for perl.  You may
not use all of its power in a test case, but it is nice to be able to
use a built-in feature when you do need to do something fancy.  Using
other scripting languages, I often run into challenges posed by the
limited nature of the language.  These challenges do not occur often
with perl.

There are many other tasks that perl is useful for.  Any time you want
to summarize or transform data from one format to another, a perl
script may be able to help.  I remember when a manager wanted to
combine several test schedules and correlate them.  He figured that it
would take a week or more of struggling with a spreadsheet, so he decided
to abandon the idea.  A few hours later, I presented him with the
combined, correlated, and sanity-checked schedules, along with the perl
script prototype that produced them.


Should you do everything in perl?
---------------------------------

Even perl die-hards admit that perl is not the perfect choice in all
situations.  When a perl script is using external utilities to do all
its work, it consists of nothing but calls to system().  In that case,
it is more efficient and less cluttered to just use a shell script (for
simplicity, I will refer to Unix shell scripts, DOS batch files, and
the like as shell scripts).  Even so, I sometimes find myself calling
perl from shell scripts to do the more complex chores (I rarely use 
sed or awk, since perl is a superset of both).

Sometimes you have to choose a more standard language if perl is not
installed on the system.  You may have to use a scripting language that
is supported by the vendor, and it is hard to find system vendors who
ship and support perl.  Of the vendors who do ship perl, many do not
keep up with the recent revisions.  My group decided to support perl
ourselves since we were not willing to give it up, so we require that
perl be installed on systems that run our tests.

Ironically, perl scripts are much more likely to be portable than shell
scripts.  This is because shells often have to call external programs
which vary widely in location, name, and functionality.  Perl breaks
the toolbox paradigm of having small sets of features in several
different utilities by building in many of the features you will need.

If you need to automate interactive processes, especially those that
require a controlling terminal, expect is a better tool than
perl.  Look for more information in a future column.  There are
efforts to build expect-like libraries for perl, but so far none of
them match expect's capabilities.

It is not clear whether perl is a good language for large programming
projects.  There is some evidence that perl is good for "programming in the
medium", but it has not been proven that perl is suitable for
programming in the large.  My group has a few perl scripts, ranging
from 800 to 8000 lines, that might work better in a compiled language.

You *can* do practically everything in perl.  In fact, many standard 
Unix utilities have been ported to perl.  But you should not forget
to consider whether another choice would work better.


New and supported or old and stable?
------------------------------------

When deciding to use perl, you have a choice between version 4 and
version 5.  The production release of perl 5 has been available since
October, 1994.  Perl 5 was almost a complete rewrite from the perl 4
code; the short list of improvements in the FAQ is 32 items long.  The
most significant changes are the introduction of object-oriented
support, nested data structures, and a flexible extension mechanism.
Object-oriented scripts?  You bet.

The perl 5 extension mechanism, using dynamic loading where available,
allows you to add in any number of C/C++ or hybrid perl/C/C++
extensions with a simple perl command.  Using perl 4, such extensions
must be specially built and are difficult to combine.  You must call a
different perl binary, such as "sybperl" (Sybase extensions) or
"curseperl" (perl with curses).  One of the most promising perl 5
extensions is the POSIX module, which gives access to POSIX.1 functions
and makes scripts more portable.  The POSIX module can also make perl
more comfortable for seasoned C programmers.

There are a few reasons why some perl users stick with perl 4.  First,
they are concerned about the incompatibilities between perl 4 and perl 5.
There are a few, but perl 5 is generally very helpful about telling you
what needs to be changed; touching up existing scripts for perl 5
is usually not difficult.  Second, stability is a serious concern.  Perl 5
is still in flux and has a lot of new code to debug.  You have to make
sure you have the latest patch level.  Perl 5's performance also
has not caught up with perl 4 yet.  

Because perl 4 is no longer supported, many people will assume you have
perl 5 when answering questions (by "supported," I mean that there is a
chance that bugs will be fixed by some kind soul out there).  If you
stick with perl 4, be sure to get patch level 36, and get familiar
with perl 5 features anyway to avoid confusion.

Personally, I am playing with perl 5 with a fairly ambitious new front
end.  I have tried several of the new features, and I have found
several isolated bugs that I have had to work around.  I have not yet
recommended that my group make a wholesale upgrade from perl 4 to perl
5, but I am looking for ways to slowly ease people into it.  I think if
you stick mostly with the basic perl 4 features, you should not have
much trouble with perl 5.


Perl resources
--------------

In the tradition of Usenet, you should start with the FAQ (Frequently
Asked Questions) file when you have questions about perl.  The perl
FAQ contains a quarter of a megabyte of information, from a brief 
introduction to obscure technical details.  It also explains how
to get the perl source so you can build it on your machine.  You can
retrieve the FAQ via ftp from rtfm.mit.edu in /pub/usenet/comp.lang.perl/.

For books about perl, the definitive reference is Programming Perl (ISBN
0-937175-64-1), by Larry Wall (the author of perl) and Randal Schwartz.
For users just learning perl, Learning Perl (ISBN 1-56592-042-2) by
Randal Schwartz is highly recommended.  See the FAQ for a list of the
other books available.  There is not currently much coverage of
perl 5 features in any of the books, but an updated version of
Programming Perl is in the works.

Another source of information is the man pages.  When printed
out, the man pages make a book of their own, and they are more 
up-to-date than any of the published books.  Type "man perl" for the
man page if you are on a Unix system, and consult with your system 
administrator if that does not work.  Starting with perl 5, the
information is split into several man pages; "man perl" will give
you a list of these.  You can also access the man pages at
http://www.eps/mcgill.ca/perl/perl.html using a web browser.

If you have questions about perl that are not answered anywhere else,
the comp.lang.perl.misc Usenet newsgroup is a great place to look.
A large community of perl users hang out in the newsgroup.
Watch comp.lang.perl.announce for periodic announcements.  Expect
other groups to sprout up in the perl hierarchy.  The venerable 
comp.lang.perl is no longer available, so be sure to use
comp.lang.perl.misc instead.

Finally, for a motley collection of most of the above, use the 
World Wide Web.  http://mox.perl.com is a good jumping off point.
http://www.cis.ufl.edu/perl/ also seems to be a good place to browse.
Both of these point you to several other web sites.