Just a quick note on the progression I’m making to get the mailserver components working on Leopard. At first I thought it would be simple, just recompile the lot and be done with it. But as most of you might have noticed, a lot has changed under the hood for Leopard. most notably was the removal of all the netinfo stuff and replacing it with dscl.

I’ve got the basic stuff working with some workarounds but still need to test SSL, maildrop and DSpam. But to reveal some of the stuff that needs to be done in a different here are some of the changes to be made in the documentation:

Postfix has no changes to the install instructions.

Instead of niutil to create the vmail user we use dscl:

sudo dscl . -create /Users/_vmail
sudo dscl . -create /Users/_vmail UserShell /usr/bin/false
sudo dscl . -create /Users/_vmail UniqueID 102
sudo dscl . -create /Users/_vmail PrimaryGroupID 102
sudo dscl . -create /Users/_vmail NFSHomeDirectory /var/empty
sudo dscl . -passwd /Users/_vmail ''

To compile courier-auth you need this line added to the configure fase:

export MACOSX_DEPLOYMENT_TARGET=10.5 \
./configure --with-authmysql \
--without-authpam \
--with-mysql-libs=/usr/local/mysql/lib/mysql \
--with-mysql-includes=/usr/local/mysql/include/mysql

IMAP 4.3.0 compiles without a problem, but won’t run properly yet. I’ve got IMAP 4.2.1 working without any change in how you need to compile it. Update: tried it again and it all works, my mistake! Sorry…

On to the next item on the todo list, will keep you posted on my progress.

It’s not been that long ago but Sam Varshavchik has done it again and updated Courier IMAP. Some of the fixes include:

• Added support for GnuTLS as an alternative to OpenSSL.
• Fix certain courier-authlib misconfigurations from being reported as false clock skew errors.
• Remove the \Draft flag from messages that are automatically moved to Trash after expunge, to have clients treat them as ordinary messages, if pulled from Trash.

I’ve downloaded, compiled and tested it and found no issues on my test server.

In the last year I had some discussions with DIYMacServer users on how to upgrade their machine or even move their setup to some new hardware. The biggest issue or problem they encountered was how to safely move the emails stored on the server. In this post I will try to describe one way (there are more possibilities to do this) that is pretty safe and you won’t lose any mail.

Before you start please check if you have enough diskspace to make an archive of the mail directory, it might have become very big. You can check the size of the directory by using the command:

sudo du -hc /usr/local/virtual

The last line from the output will tell you how much space you need on your disk. If you haven’t got enough room you can ask your users to empty their Trash and Sent mail in their email application or remove some items you have stored on the server yourself. If that ain’t possible you’ll need to buy an extra drive (external or internal, depending on the machine you’ve got).

If you have users other then yourself using the mail-server please inform them in advance that the mail-server is going off-line for a short period at a time that is convenient for you to perform this task. This is only about moving the mail, not the websites and I’m assuming you’ve got the other machine ready and setup to take over.

When the time has comes, take the IMAP server off-line, to prevent users from accessing their IMAP folders and accidentally do something between backup and restore actions. If you’ve got POP3 configured as well please stop that as well.

Stop Postfix to prevent incoming mail to be lost when it arrives between backup and restore actions.

Now everything is stopped and you are ready to make a backup of the email directory.

You can do this by using the following command:

sudo tar -czf mailbackup.tar.gz /usr/local/virtual

This will create a big file called ‘mailbackup.tar.gz’, you can copy/move/ftp this file to any destination you would like.

To unpack the backup and restore it to it’s new destination please use the following command:

cd /
sudo tar -xzf /locationofarchive/mailbackup.tar.gz

Replace ‘locationofarchive’ with the path to where you stored your backup archive.

If you used the same setup on both machines the mail directory should have the proper security settings and ownership. You can test this by using the following command:

sudo ls -l /usr/local/virtual

The owner of the directories should be postfix or vmail, depending on your setup.

Now you can start the IMAP server first and check with your own mail client if you can access all your email again on the server. If there is a problem please check your logfile (/var/log/mail.log for a possible explanation). If it all works turn on the Postfix server and send yourself an email using another mailserver (use a webbased service like Gmail, Yahoo mail or Hotmail) and check if the email will arrive in your mailbox. Keep an eye on the logfile to see the email coming in.

If everything is working for you inform your users that it is all right to use the mail-server and if they have any issues that they please inform you about it.

Good luck!

Somehow the PHP.net guys forget they have an announcement mailinglist to tell everyone a new version is released. I had a kind and very thoughtfull reminder in my mail from a happy DIYMacServer user telling me that a new version has been released.

So gentleman, start your download program and warm up your compiler. The PHP installation documentation has been updated and also has a Leopard configuration for all you early adopters.

The new release boasts the following fixes:

• Fixed dl() to only accept filenames. Reported by Laurent Gaffie.
• Fixed dl() to limit argument size to MAXPATHLEN (CVE-2007-4887). Reported by Laurent Gaffie.
• Fixed htmlentities/htmlspecialchars not to accept partial multibyte sequences. Reported by Rasmus Lerdorf
• Fixed possible triggering of buffer overflows inside glibc implementations of the fnmatch(), setlocale() and glob() functions. Reported by Laurent Gaffie.
• Fixed “mail.force_extra_parameters” php.ini directive not to be modifiable in .htaccess due to the security implications. Reported by SecurityReason.
• Fixed bug #42869 (automatic session id insertion adds sessions id to non-local forms).
• Fixed bug #41561 (Values set with php_admin_* in httpd.conf can be overwritten with ini_set()).

and the following enhancements:

• Upgraded PCRE to version 7.3
• Updated timezone database to version 2007.9
• Added ability to control memory consumption between request using ZEND_MM_COMPACT environment variable.
• Improved speed of array_intersect_key(), array_intersect_assoc(), array_uintersect_assoc(), array_diff_key(), array_diff_assoc() and array_udiff_assoc() functions
• Fixed bug #43139 (PDO ignores ATTR_DEFAULT_FETCH_MODE in some cases with fetchAll())
• Fixed bug #42785 (json_encode() formats doubles according to locale rather then following standard syntax)
• Fixed bug #42549 (ext/mysql failed to compile with libmysql 3.23)
• Over 60 bug fixes.

Note: There are more recent instructions on compiling Apache on Leopard in the documentation set: Installing Apache

As people are compiling Apache on Leopard and because its not the usual configure, make, install procedure. I thought a separate blog post explaining it would help people to get it all working.

There are 2 ways to define the setup, go for the default Leopard install method and overwrite the current Apache installation or install a separate instance. The separate instance falls in line with all the other DIYMacserver documentation. The Leopard install will keep the functionality of the preference pane intact. You choose !

The difference in the configuration is made in the ‘config.layout’ file which determines where all the packages should be installed. For both methods copy the layout below and add it to the ‘config.layout’ file.

DIYMacserver installation configuration
<Layout DIYMacServer>
  prefix: /usr/local/apache2
  exec_prefix: ${prefix}
  bindir: ${exec_prefix}/bin
  sbindir: ${exec_prefix}/bin
  libdir: ${exec_prefix}/lib
  libexecdir: ${exec_prefix}/modules
  mandir: ${prefix}/man
  sysconfdir: /etc/httpd
  datadir: /Library/Webserver
  installbuilddir: ${datadir}/build
  errordir: ${datadir}/error
  iconsdir: ${datadir}/icons
  htdocsdir: ${datadir}/Documents
  manualdir: ${datadir}/manual
  cgidir: ${datadir}/CGI-Executables
  includedir: ${prefix}/include
  localstatedir: /var
  runtimedir: ${localstatedir}/run
  logfiledir: ${localstatedir}/log/httpd
  proxycachedir: ${runtimedir}/proxy
</Layout>

Leopard installation configuration

<Layout Leopard>
  prefix: /usr
  exec_prefix: ${prefix}
  bindir: ${exec_prefix}/bin
  sbindir: ${exec_prefix}/sbin
  libdir: ${exec_prefix}/lib
  libexecdir: ${exec_prefix}/libexec+
  mandir: ${prefix}/share/man
  sysconfdir: /etc+
  datadir: /Library/Webserver
  installbuilddir: ${prefix}/share/httpd/build
  errordir: ${prefix}/share/httpd/error
  iconsdir: ${prefix}/share/httpd/icons
  htdocsdir: ${datadir}/Documents
  manualdir: ${datadir}/share/httpd/manual
  cgidir: ${datadir}/CGI-Executables
  includedir: ${prefix}/include+
  localstatedir: /var
  runtimedir: ${localstatedir}/run
  logfiledir: ${localstatedir}/log+
  proxycachedir: ${runtimedir}/proxy
</Layout>

If you’ve edited the ‘config.layout’ file you are ready to run the configuration script. Change the ‘–enable-layout’ to reflect your choice in configuration. So fill in either DIYMacServer or Leopard to choose, the example uses DIYMacServer as the layout.

./configure --enable-layout=DIYMacServer \
--enable-mods-shared=all \
--with-ssl=/usr \
--with-mpm=prefork \
--disable-static \
--disable-unique-id \
--disable-ipv6 \
--enable-ssl \
--enable-dav \
--enable-cache \
--enable-proxy \
--enable-shared \
--enable-logio \
--enable-deflate \
--with-included-apr \
--enable-cgi \
--enable-cgid \
--enable-suexec

You can now run:

make

and if it all goes well:

sudo make install

If you run into errors or some strange thig happens which you can’t explain, please post you questions and the error into the forum on this site:

The configuration and all the other posts related to apache in the documentation set are still valid, you still can use those.

While trundling to my subscribed RSS feeds (still too many and I’m still pruning them regularly) I noticed John Gruber mentioning his friend Dan Benjamin writeup of his MySQL compilation instructions article. I’ve read it and it’s pretty good and really well written for an audience who have no experience at the Terminal.

I do however not gree with all his instructions which are pretty much geared to his need to get Ruby on Rails working for him. The most important mistake I see is that he doesn’t go for the shared libraries options. He is compiling static libraries and we know from experience (read the older blogpost son the subject) that that options causes issues with for instance our PHP compilation.

Another item that keeps me wondering is why he’s removing the default StartupItem which works fine even wth your own compiled installation and replace it with a new launchd script. But I would recommend to read the article yourself and draw you own conclusions.

By the way, the installation instructions for MySQL, Apache and PHP have been updated to reflect the impact that Leopard had on them. I’m still working on the mailserver instructions.

As I already blogged before, there where some changes in the user creation. I noticed the new notification for the mysql user which changed in Leopard into _mysql, the underscore appears on almost all the users that aren’t connected to people. The only users without an underscore are ‘daemon’, ‘nobody’, ‘root’ and me.

But more has changed I found out. As where I first used the Netinfo tools to create users for daemons to have them use it, like the vmail user for the virtual mail directory. But in Leopard these tools are gone! There is also no GUI tool to have a look at them.

There is a new tool in town, it’s called the ‘Directory Service command line utility’, ‘dscl’ for short and is command line only! See this article in the Apple knowledgebase and the dscl man page here.

Here is a example on how to use the tool which you have to use as root or use with the sudo command. To find out the UID of the postfix user you use the following command:

$ sudo dscl . -read /Users/_postfix

Which returns:

AppleMetaNodeLocation: /Local/Default
NFSHomeDirectory: /var/spool/postfix
Password: *
PrimaryGroupID: 27
RealName:
Postfix Mail Server
RecordName: _postfix postfix
RecordType: dsRecTypeStandard:Users
UniqueID: 27
UserShell: /usr/bin/false

This means another task for rewriting the documentation. Which makes me a bit sad.

Tiger shipped with the heavily outdated Apache 1.3.x and PHP 4.x. While both of these versions where working and helped people to get started most of us liked working on the newer releases of that software. Besides the Apache Software Foundation has moved into security update, and critical bug fix only mode for the 1.3.x series of the web server and as we all know the PHP project as announced end of life for PHP 4. Its a good time to move on as I showed you how to do on my blog.

To my surprise however I found when plating around with my Leopard install it has as default the latest version of Apache and PHP installed per default. You still need to activate the PHP module yourself by uncommenting it in the httpd.conf configuration file which is in a new location by the way.

The new location of the configuration and the modules is in ‘/etc/apache2/’ instead of the default ‘/etc/httpd/’ you would expect. I’m sticking to the last one by the way for my installation.

The modules are installed in ‘/usr/libexec/apache2/’ and there are some remarkable modules name in there. What would you think of a module called mod_bonjour, I wonder what that one does. Another one is mod_auth_svn which would indicate a default installation and wonder me it’s installed. Mod_perl and mod_fast-cgi are there as well. I will try to mimic the availabilty of these modules in my install scripts.

PHP is another question, with php-info() you can retrieve the configure command-line, it looks like:

./configure '--prefix=/usr' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--disable-dependency-tracking' '--with-apxs2=/usr/sbin/apxs' '--with-ldap=/usr' '--with-kerberos=/usr' '--enable-cli' '--with-zlib-dir=/usr' '--enable-trans-sid' '--with-xml' '--enable-exif' '--enable-ftp' '--enable-mbstring' '--enable-mbregex' '--enable-dbx' '--enable-sockets' '--with-iodbc=/usr' '--with-curl=/usr' '--with-config-file-path=/etc' '--sysconfdir=/private/etc' '--with-mysql-sock=/var/mysql' '--with-mysqli=/usr/bin/mysql_config' '--with-mysql=/usr' '--with-openssl' '--with-xmlrpc' '--with-xsl=/usr' '--without-pear'

Which looks a lot what I had with some extra’s, I’m looking into which ones I will add.