Adding new functionality to Perl can be accomplished in a number of ways. Library routines can be written, modules can be created, etc. Sometimes, however, the added functionality is inappropriate or impossible to implement in Perl. For instance, the code may be compute-intensive or need to invoke a system facility which is not available from Perl itself.
At this point, we turn to Perl extensions, hybrid modules which allow C code to be used from your Perl program. Extensions are frequently programmed in XS, a special-purpose variant of C with facilities that allow transparent linkage to Perl scripts. If you are already comfortable in C and Perl, XS programming should not be a real challenge.
Once the extension code is written, however, you'll need to build it into a shared library, allowing your Perl scripts to use it. This involves:
Although I've personally coded in C, I've not yet done any extensions programming (and I don't even have a C compiler on my Macintosh!). So, this month I've turned the column over to someone who knows more about this topic than I do, and who is willing to share his knowledge with you. The specifics are for MacPerl, but even non-MacPerl programmers should be able to glean some useful information.
Introductions over, readers, please welcome Arved Sandström.
I'll see you next month.
- Vicki
p.s. If you're a MacPerl user with knowledge to share, I'm looking for other guest columnists for future issues. Just send me a note at macperl@perlmonth.com
Extensions aren't significantly harder to build for MacPerl than for Unix Perl, but they do require some groundwork. Most of this groundwork corresponds to what a Unix systems administrator would also have to do at some point in time - it's somewhat disingenuous to extol the simplicity of Unix extension builds without taking the big picture into account.
On both systems, in any case, the setup effort is limited. Perl is installed once, a development environment is installed once, and so forth. These procedures are simpler on Mac OS, but building the typical extension on Mac OS is more work (balanced, perhaps, by the oft-heard complaint of Unix users that they can't figure out where or how to install their personal modules.
I thought I'd throw that last part in to excite heated debate. :-) Currently I do more work on MkLinux than I do on Mac OS, so I don't actually have an axe to grind. Just let's be fair - Mac OS has always been a good programming environment, despite much opinion to the contrary.
In this column I propose to provide an overview of building extensions for MacPerl. Many of the technical aspects are covered in existing tutorials (references [1] and [2]), and of course also in existing documentation for Perl XS ([3] and [4]) and the SWIG wrapper generator ([5]).
Extensions involve compiling and linking C, so you'll need a Mac-based IDE (Integrated Development Environment) for C. The most popular choices are Metrowerks CodeWarrior and Apple Computer's MPW (Macintosh Programmer's Workshop).
The obvious advantages of the latter are that it doesn't cost anything and that a large body of Unix-inspired tools come with it. See http://developer.apple.com/tools/mpw-tools for more information on MPW. You can download MPW itself from ftp://dev.apple.com/developer/Tool_Chest/Core_Mac_OS_Tools/MPW_etc.. For information on CodeWarrior, see http://www.metrowerks.com.
For the specific purpose of building MacPerl extensions for PowerPC platforms, there appears to be no reason to favor one IDE over the other. CodeWarrior is, however, clearly better when it comes to building CFM68K (68K with Code Fragment Manager) extensions. Nevertheless, both development environments may be used to build either PowerPC or CFM68K extensions.
You must also have the MacPerl source distribution; see http://www.macperl.com/depts/getmp.html for details. It is also helpful to pick up some sample XS code as a starting point. Chris Nandor's MacPerl Module Porters page http://pudge.net/cgi-bin/mmp.plx is a useful compendium of already-ported modules.
Most of the work involved in building an extension revolves around writing the extension interface. There are two methods for preparing interfaces and transforming them to C code that is ready for compiling and linking.
The first method, as mentioned above, is XS. CPAN extensions use XS, because the standard build process is oriented to it. Once you've written an XS interface, the xsubpp XS compiler can be used to convert it to C, which is then compiled (along with any other C source files), and linked (along with any needed C libraries).
If you just want to build existing Perl extensions for MacPerl, you probably don't need to learn XS. Refer to the Building Extensions for MacPerl tutorial ([1]) for technical details.
The other method for interfacing C code to Perl is SWIG, the Simplified Wrapper Interface Generator. Although SWIG can be run directly on header files, or even on C source files, the recommended approach is to prepare an interface file, which serves the same purpose as an XS file. The SWIG distribution comes with excellent documentation. For more information on actually building a working extension for MacPerl, using MacSWIG, please refer to the MacPerl Extensions and MacSWIG tutorial ([2]). SWIG for MacOS is available from ftp://ftp.swig.org/pub/MacSWIG1.1p2.sea.hqx; a console app and MPW tool built from SWIG 1.1p5 are available as part of the MacPerl SWIG tutorial.
When to use XS, and when to use MacSWIG? Well, if the extension module came from the CPAN, the decision has been made for you - it uses XS and the XS file will already be present. Otherwise, there are no hard and fast rules for when to use SWIG and when to use XS. Certainly SWIG has less of a learning curve; it is also better suited than XS to the typical C library interfacing task.
Here are some loose guidelines that you may find useful. Bear in mind that these are "where do I start" guidelines, and hence more applicable to a person unfamiliar with both approaches. (When an API (Application Programming Interface) is referred to, below, I'm assuming that an interface needs to be written for a C library or existing C source files.)
Use SWIG if:
Use XS if:
These guidelines need some elaboration. For example, let's assume that we have a C library which contains the function
double func1(int a, char *b);
If the desired Perl equivalent is going to be called as
$fpnum = &perlfunc($intnum, $somestring);
then there is, essentially, a one-to-one correspondence between number and types of arguments and function return type. This is what I'm referring to as as-is, or direct translation. If you intend to translate all functions in just this fashion, SWIG is more convenient. You can basically copy and paste every function declaration from the header file into the interface file, add a few statements, and invoke SWIG.
This kind of bulk "as-is" translation is, in fact, how SWIG is intended to be used. Many C libraries are amenable to this kind of interfacing. However, in my opinion, XS offers more flexibility to the interface writer who needs fine control over exactly how the C function is called. And this is where the real art of extension building comes in - planning the interface, and deciding how to map C functions to corresponding Perl subroutines.
For a variety of examples of how to use SWIG, I recommend looking at the MacPerl SWIG tutorial. There are examples to be had elsewhere, but these are often quite intricate, and not really suitable for pedagogical purposes.
For an excellent example of using both the XS file and the Perl module itself to provide a great deal of value-added to an existing C library, refer to the XML-Parser distribution on the CPAN. This starts with James Clark's expat library, and adds considerable Perl functionality to it through the interface (and the interface is both the XS file, and the accompanying .pm file).
It should also be mentioned that XS is quite adequate to the task of coping with complex datatypes. In fact, once the datatypes get really gnarly then you'll likely want the hands-on that comes with using XS. The XS Cookbooks ([6]) and many of the MacPerl Toolbox modules offer numerous examples. A not insignificant initial investment of time and practice is required to write competent XS at this level, which is why I do not recommend it for novice extension builders.
But I leave the final judgment up to the informed reader who has used both SWIG and XS extensively. My over-all guideline is this: familiarize with both, and decide for yourself.
The original purpose for XS was to write interfaces for C libraries. The pendulum has swung, and it is probably more likely to be used to write C specifically for Perl. I'm not going to say that this is bad, but I do urge module developers to use XS as a last resort. That is, convince yourself that you simply can't satisfy a speed or functionality requirement without going to C.
Please don't write XS just because you can. Without naming names, there are CPAN modules which incorporate XS and really don't need to. If you absolutely must, consider delivering both an XS and a nonXS version of your module.
I hope that this report on the state of the union, as far as MacPerl extensions are concerned, will serve to whet your appetite. Basically, any portable C can be as easily interfaced to MacPerl as it can be to Unix Perl (depending on the Unix variant, it could well be easier to do the port to MacPerl :-). The benefits are enormous - Perl is already a powerful language; the ability to extend it with custom C further extends its capabilities.
I urge you to avail yourself of the references. Despite initial appearances, it really is quite easy to do basic XS and SWIG for MacPerl. I won't lie and say that advanced XS and SWIG is easy - it's still very much bleeding-edge stuff in many ways.
We are eagerly awaiting a new and improved MacPerl, so this is not about hitching your wagon to a dying horse. Bear in mind, too, that expertise with XS and SWIG is completely transferrable across operating systems.
Good luck with your MacPerl extension building! I hope to hear from you on the MacPerl mailing lists. (See http://macperl.com/depts/mlists.html/ to join)
Arved Sandström Arved_37@chebucto.ns.ca
This is the mantra that you'll see time and time again in CPAN module README's:
perl Makefile.PL
make
make test
make install
The Building Extensions for MacPerl tutorial ([1]) covers actual procedures for building with Mac OS. The CodeWarrior MPW procedure, which is also discussed in MacPerl: Power and Ease (by Chris Nandor and Vicki Brown [7]) is very similar to the standard process.