FAQ Policies
FAQ : CGI scripts


Writing, installing and using CGI scripts



The CSE webservers can do more than simply serve static html pages. They can also process dynamic content of all kinds, including embedded PHP, server-parsed HTML (.shtml files ) and CGI applications (both interpreted languages such as perl, python or shell scripts, and compiled executables)

You may also want to check out the World Wide Web or Creating Website pages.

How do I write a CGI script?

For more details on CGI scripting than you ever wanted to know, check out Google Directory's CGI section.

For all things PHP-related, visit http://www.php.net



Okay, I have a script I want to run. Where do I put it, and how do I access it?

While PHP and SHTML files can be served from anywhere under your public_html directory, CGI scripts and executables themselves must be placed under your public_html/cgi-bin directory, or they will not run, without extra provisions being made.

For example, if you create a script called /home/YourUserName/public_html/cgi-bin/test, you can run this script using the URL: http://cgi.cse.unsw.edu.au/~YourUserName/cgi-bin/test

Make sure that your script is executable! (ie. chmod +x test.pl)

If you really need to run a CGI program outside of your cgi-bin directory, you can enable it with the following .htaccess directive:

<Files "mycgiprogram">
SetHandler application/x-setuid-cgi
</Files>





The actual processing of dynamic content is handled not by www.cse.unsw.edu.au itself, but rather by cgi.cse.unsw.edu.au - a DNS pointer to a pair of load-sharing CGI servers. Where possible, links to dynamic content should point directly to cgi.cse rather than www.cse, however some fairly smart URL handling should redirect most requests automatically.



Help! It doesn't work!

First up, the most common CGI/PHP errors boil down to permissions issues. Here are the minimum permissions needed in order for everything to work:

  • Static files (html/txt/jpg etc) are handled by www.cse, which runs as the w3serv user. These files therefore either need to be owned by the w3serv user/group, or to be set other-readable (mode o+r, or 644).
  • Directories do not need to be readable by w3serv, but they do need to be executable by that user (mode o+x, or 601). Note that the entire chain of directories leading to your content, whether static or dynamic, from public_html downwards, need to be set mode o+x. Take for example the file public_html/foo/bar/somefile.php. If public_html, foo or bar are not accessible by the w3serv user, the file will not be accessible.
  • Dynamic content (PHP scripts, SHTML pages and CGI programs) need to be owned by the user in whose home directory they reside. This means that /home/someuser/public_html/cgi-bin/test.pl must be owned by someuser. Files owned by anotheruser will fail to run. In the case of group-owned directories (such as class/project accounts), this is obviously a problem. To address this, use the priv takeover command to change ownership of the files to the group account. If you use this command, ensure that you enable group-write access, as you will lose owner access to the files when the ownership changes. The target group must be a member of the GroupCGI class - contact System Support to have this added.
  • All dynamic content is executed by the user that owns the home directory. If you wouldn't have permission to run the program yourself, (or any programs that it in turn executes) from a shell, then it won't work on the CGI server, either.
  • PHP and SHTML files need to be readable by the owner, but do not need to be executable (mode ug+r or 600).
  • CGI programs, in addition to residing under public_html/cgi-bin, must be executable by yourself (mode ug+rx, or 700)




How do I access files in my home directory?

Short answer: you can't.

Long answer: Because CGI scripts are by their very nature designed to be executed by the general public, we consider them to be inherently untrusted. Even the most innocuous-looking script could have exploitable security flaws that could result in loss or leakage of sensitive information, or compromise the security of your account, if given access to all of your files. To prevent this, the CGI servers are set up so that they can only access files under your public_html directory.

Your public_html directory is accessible via /web/YourUserName - this is the filesystem path you should use to access data files etc. in your scripts. This path is available on all CSG-linux machines, allowing you to use and test your scripts from the command line without having to reconfigure them.



It still doesn't work properly! How can I see debug information?

CGI programs on our servers are handled by a program called CGIwrap, which provides a secure, controlled setuid environment for CGI execution, along with a handy debugging facility.

To debug a script, set the handler for your script to be the one with debugging. This is best explained with an example. Say you need help debugging http://cgi.cse.unsw.edu.au/~YourUserName/cgi-bin/somedir/problem.cgi. You should create a .htaccess (note leading period) file in the same directory, with text like:

<Files "problem.cgi">
SetHandler application/x-setuid-cgid
</Files>



This tells the webserver to use the debugging-enabled cgiwrap instead of the usual one.

Note that the .htaccess file must be made readable by the webserver by using chmod 644 .htaccess.



I'm getting CGIWrap errors, how do I fix them?

The other common-but-cryptic CGIWrap error message is Script does not have same UID (Script does not have same UID), which basically means that the file is not owned by the owner of the home directory in which it resides - and thus needs to be chown'ed (usually using priv takeover) before it will run.



What about security?

For a very comprehensive introduction to this somewhat fraught subject, see the WWW Security FAQ.

For details on restricting web access to specific users / hosts / etc, see Restricting Web Access

Writing insecure server-side applications is all too easy, and an awful lot of the pre-built scripts freely available on the web have glaring security shortcomings. To ensure that security is not compromised by ill-considered (or even malicious!) programs, we have placed a number of security restrictions on CGI processes.

By restricting access to user home directories, running scripts as the file owner, and requiring that file ownership match the home directory, and preventing excess resource usage, we can effectively prevent badly-written, insecure, exploited, or even malicious programs from adversely affecting our systems to any great degree.

This doesn't mean that you shouldn't keep a defensive-coding attitude, though. Some useful security tips include:

  • Ensure that your scripts are not world-readable, especially where they contain sensitive information such as database passwords
  • Where appropriate, restrict access with .htaccess files - consider restricting access to specific users/classes/groups, restricting access to on-campus machines, etc.
  • Wherever sensitive information is involved, including any kind of password authentication, please use SSL for greater security.




Error Messages

You may also want to check out the World Wide Web or Creating Website pages.

Real UID could not be changed!

One of the more common (and also the more cryptic) CGIWrap errors is Real UID could not be changed!. This is usually caused by process limits on the CGI servers. We enforce fairly strict limits on the number of simultaneous processes that can be run on the CGI servers by any one user, and also on the amount of resources they consume. If running your script would exceed these limits (for instance, a badly-behaved script failing to exit, consuming vast quantities of CPU time or RAM or forking multiple background processes), then access will be denied, and this error will be returned.

To fix this, you can use the priv cleancgi command. To list your CGI processes, use priv cleancgi list, and to kill them, use priv cleancgi clean.

Of course, you should always code defensively in order to avoid this sort of situation. Set timeouts on potentially lengthy operations, and ensure that you clean up any background processes that you launch.



Script does not have same UID

This error most frequently occurs in the case of group accounts.

Our CGI setup requires that any CGI scripts run from your home directory are owned by you. This error message occurs when a CGI script is not owned by the right person.

Read the Help Me section for information on solving this problem, including using the priv takeover command.



Error 500: Malformed header from script

There are many reasons you could get a 500 error. It basically means "something else went wrong with the script". However, they usually include further details, and in this case we are talking about one that starts with "Malformed header from script".

This error message occurs when your CGI script does not return a Content-Type header (and possibly other headers) before a blank line. It is often the result of errors from your code being output before the reply headers.

Your best bet is to turn on Debugging Mode to see the full text of the error, and fix it.


WebSVN on CSE CGI servers

Configuring WebSVN is essentially a matter of downloading the sources and changing one line in a configuration file. But, the process can be time consuming because several software components are involved, some of which users have no control over, getting useful diagnostic information requires some effort, and the cgi environment is imperfectly replicated by a user's terminal session.

1. Download PhP source from: http://websvn.tigris.org/servlets/ProjectDocumentList;jsessionid=24F17B3F5279F7DE3BB39F064A2C4A03

2. Extract the file into a cgi-bin directory
tar xzf websvn-2.0.tar.gz -C $HOME/public_html/cgi-bin


3. Recommended: add a version-independent link so that URLs will not change if the websvn version does:
ln -s $HOME/public_html/cgi-bin/websvn-2.0 $HOME/public_html/cgi-bin/websvn


4. Create a config.php file:
cp $HOME/public_html/cgi-bin/websvn/include/distconfig.php $HOME/public_html/cgi-bin/websvn/include/config.php


5. Edit the config.php file, adding a line: $config->parentPath('/web//');

NOTE: A path beginning with /web is essential! /home//public_html will NOT work. The path is to a server-side svn repository, thus it should NOT begin with 'file://'.

6. Place a copy of the svn repository itself (NOT a checked out directory) directly in the $HOME/public_html directory.

Aside: Everything in the repository will be accessible over the internet. To grant access to specific subfolders it is necessary to migrate them into a new repository (with dump, filter, create, load, see http://svnbook.red-bean.com/en/1.1/ch05s03.html, and also http://svn.tartarus.org/svn-tools/svndumpfilter2?view=markup).

7. Access the repository: http://www.cse.unsw.edu.au/~/cgi-bin/websvn A list of the repositories in public_html should be displayed. If not, verify that step 5 was completed correctly and check the file permissions on the repository (for files: o+r, for directories: o+rx).

8. Clicking on one of the repositories should bring up a list of folders and files. Several configuration errors can cause a blank page to appear instead. To see a meaningful error message, one can edit
$HOME/public_html/cgi-bin/websvn/listing.php;
after the showTreeDir function, add a line:
echo " test: ";


An error like: XML error: no element found (3) at line 3 column 0 byte 45 indicates that websvn cannot access the repository. An svn command will also be displayed. Check that it works directly at a terminal session, but note that the subversion installation called by the cgi server is different to that available to most users through a terminal session. In particular it might be a different version.

For more debugging information one can edit $HOME/public_html/cgi-bin/websvn/include/svnlook.php; in the getLog function find the line beginning $cmd = quoteCommand(... Add another line afterward:
$cmd = "svn --version";
Then inside the while (!feof($handle)) loop, after $line = fgets($handle); add a line:
echo " line:$line: ";
Accessing the page should now reveal the version of the svn command.

An error like: svn: Expected format '3' of repository; found format '5' indicates that the cgi version of svn is too old. Until it is upgraded, repositories will have to be downgraded, see http://svn.collab.net/repos/svn/trunk/notes/repos_upgrade_HOWTO

SVN instructions by tbourke 20080808

Tags for this article: CGI htaccess php scripts web