The most universal scripting language for Linux is?
We are writing scripts for Linux systems, there has been some debate over what would be the most universally Linux present scripting language to use. Bash, SH, Posix? What?
«sh»? What «sh»? The Thomson shell? the Bourne shell? bash, ksh, pdksh, ash, zsh as found as /bin/sh on system X or Y? The POSIX sh specification, The SUSv3, SUSv4 sh specification, the LSB sh specification? «sh» by itself doesn’t mean anything.
if this is for software builds and install scripts, you might want to check out Autotools, which attempts to solve cross-system compilation issues.
@sch The most portable common intersection, «obviously». In case people unfamiliar with Unix shells get misled by your comment, the POSIX spec is implemented by nearly all contemporary shells that dare to go by the name sh , and the non-conforming sh that don’t are ancient and rare nowadays (e.g. Bourne). One can rattle off an ever growing list of extensions and variations but if the goal is «universal» or portability, one should be going in the opposite direction. Gilles’ answer covers the details in more depth.
8 Answers 8
There are two programming environments that are available on every unix-like operating system, that are Turing-complete and that are able to call other programs: awk, and sh, the Bourne/POSIX shell family. AWK is oriented towards text processing (it complements more specialized utilities), while sh is oriented towards being a glue language to put programs together. Sh is the universal scripting language on Linux and across the unix world.
The POSIX standard defines mandatory features of sh itself and associated utilities. Most unix-like systems comply with POSIX 1003.1-2004 (a.k.a. Single Unix v3, a.k.a. the Open Group Base Specification issue 6); the latest version of that standard is POSIX 1003.1-2008 (a.k.a. Single Unix v4, a.k.a. the Open Group Base Specification Issue 7).
Every Linux and unix or Unix-like system has a Bourne-style shell at the path /bin/sh , and any non-antique system has a POSIX-compliant shell (barring the occasional bug). Every modern unix-like system (including Linux) supports shebangs, so it automatically runs scripts in /bin/sh if the first line is #!/bin/sh . There are POSIX systems where sh is located in another place (typically emulation layers on OSes you wouldn’t think of as being really Unix-like).
Embedded Linux systems may have a stripped-down BusyBox system that doesn’t implement all POSIX features. BusyBox has a large number of compile-time options to accommodate small-footprint systems, so it’s difficult to know what to expect in advance, you have to tailor your scripts to a particular device. BusyBox is the most common small-footprint implementation of sh and assorted utilities; another one you might encounter is the extremely reduced shell environment in Android (later versions are less anemic).
Non-embedded Linux systems almost always have either dash or bash as /bin/sh . Dash is a small and fast shell that implements little more than POSIX features. Bash is a larger shell with more features.
Non-embedded Linux systems almost always have Bash installed as /bin/bash . Hence, for portability on non-embedded Linux systems, you can assume that bash is available. Among the useful additional features of bash are arrays, the ability to cope with dot files conveniently, the pipestatus variable to obtain the return status of all the commands in a pipeline, additional comparison operators for file times, and (in recent versions) regular expression matching.
One of the characteristics of shell programming is that you aren’t just using the sh program, you’re also using a number of utilities. Most of the file manipulation and text processing utilities on Linux are the GNU coreutils (on embedded systems, they’re usually from BusyBox).
If you need portability beyond Linux, your best bet is to stick to POSIX. Other unix variants may not have bash installed (bash is part of the standard installation on OSX, but an optional package on *BSD and most commercial unices). Almost all unix variants other than Linux and OSX (i.e. *BSD and commercial unices) have some version of the Korn shell, at least pdksh. Many of bash’s convenient extensions are from ksh, so it can be useful to write scripts that can run under both, but detecting where bash or ksh is located on an unknown system can be a bit of a pain.
The shell can’t do everything. If you need a more sophisticated language, the two more common choices are Perl and Python (anything else is far behind as a unix scripting language). Perl is the traditional scripting language, and few non-embedded Linux systems lack it, but Python is gaining ground (boosted in part by being the recommended scripting language for Ubuntu). In the non-Linux world, Perl is part of the base installation on OSX and OpenBSD; it’s optional but very commonly installed on FreeBSD, and optional but often installed on NetBSD.
Mostly agree with all of this. Just some different emphases: * some distros are BusyBox-based but not necessarily embedded (I use one, Alpine). (Granted, the answer does say «almost always».) * BSDs are a large class of Unix-like systems where you can’t assume bash by default. * dash can do almost everything bash can, it just sometimes demands more care. * If you need a more sophisticated language, yes Perl and Python are most common choices, but awk is even more ubiquitous and is adequate for many purposes. It’s also commonly underestimated. But it’s lighter-weight than Perl and Python.
A fair warning, though — FreeBSD dropped Perl from their default install some time ago. Aside from that, I don’t know of any other distro who doesn’t have Perl in their base install.
@dubiousjim Only Linux (non-embedded, or a good enough approximation in practice) and OSX have bash in their default installation; *BSD have pdksh or mksh, and commercial unices have ATT ksh.
- sh, but stick to POSIX-specified facilities.
- bash, but don’t forget to explicitly specify it in the shebang or you might get dash instead.
- Python. Almost everyone uses it.
- Perl. But you get to write it.
After that, no one really cares since there isn’t much you can’t do with just those.
I agree with @ignacio perl is #4 and python is #3. The reason is obvious. I think Python is evolution of perl.
@ashwin: nope, python is not an evolution of, or even like, perl. perl is a language by sysadmins for sysadmins. python is a language by programmers for programmers. That difference is crucial. They both have their uses and while there can be a lot of overlap in use-cases, for some tasks perl is the better choice, and for others python is clearly superior.
Ruby and PHP were inspired by Perl. Python is the result of a physics experiment wherein they created anti-Perls in a supercollider. (I have nothing against Python. Plusses and minuses, plusses and minuses.)
Ordinarily, I’d say sh . but since you specified Linux, I’ll say bash — it’s guaranteed to be on every Linux system around (well, excluding the obscure pedantically-tiny ones that fetishize minimalism :).
If you don’t have to care about non-linux portability (and if you don’t need to run on tiny distros or in embedded Linux devices like plastic-box routers), then you may as well make use of the significant enhancements that it offers over plain sh . Otherwise use sh .
After bash (and sh ), the next most «universal» scripting language for Linux would be some dialect of awk — usually either mawk or gawk . If you stick with plain awk and avoid gawkisms your script should run fine on almost any Linux system (it may be missing on tiny distros or embedded devices). Most Linux systems will have both mawk and gawk available, but on some distros (debian, for example) mawk is installed by default and you have to install gawk yourself if you want it.
Next would be perl . AFAIK, the base perl language is installed by default on all common Linux distros, so that makes it a good choice. Even more fortunately, there is very little version incompatibility with perl5 releases (although perl 5.12 or it might have been 5.14 finally got around to removing some obscure features that had been deprecated for about 15 years. ample warning not to use them) so unless your coding style is truly weird and you like to ignore over a decade’s worth of «don’t do that» warnings, your perl scripts will run just fine nearly anywhere. The language is robust and powerful and can do everything that awk and sed can do and more. With a little effort it can do the things that sh is traditionally good for too (e.g. running external commands and using/piping the output). The standard perl libraries are pretty comprehensive, too — covering more than just the basics.
The only catch with perl is that there is also an enormous library of CPAN modules to do just about anything you can think of (and lots more that may never occur to you) — and not all of them will be available on every system with perl. They’re usually of very high quality so it’s easy to get into the habit of just using them — but if you do make use of them, you’ll have to make sure they’re installed. Many CPAN modules are pre-packaged for Linux, and the remainder are easily installed with the cpan tool (or dh-make-perl on debian/ubuntu/etc to turn a CPAN module into a .deb package)
I’d like to be able to say python next, but I really can’t. There’s a lot to like about python but it’s not included by default on many Linux systems and, frankly, it’s version compatibility (both for python itself and its allegedly «standard» libs) is a complete shambles. Some distros make an excellent effort at sorting out the mess, and some don’t. Neither are helped by the fact that python is basically a language written for programmers by programmers (as opposed to sysadmins) and they don’t seem to think that the system their code will be installed on is at all important. their code is really super special so they don’t need to concern themselves with boring stuff like integration into existing systems.
(Don’t get the wrong idea from my sarcasm here — I really like python as a language, I just hate the fact that version and dependency management is a PITA. It’s like going back 20+ years to the era of manually hunting for obscure bits of code and patches to get something compiled and running on your proprietary *nix)