Migrating away from postfix/dovecot to Zimbra 8 on a Debian 6 server

May 24th, 2014 | By | Category: Internet, Linux / Freebsd

What a better way to spend a glorious sunny saturday afternoon than trying to get a very complicated open source exchange platform to run on a production server powered by an OS that isn’t supported?  As with all things me, there’s no easy way to go about doing this so I’m digging in my heels for a fight and figured I’d blog about it as the amount of resources on the topic do seem a bit slim and spread out

So I personally created a ZCS installation directory in/home/zimbra/install . In that dir I put the scripts below and then unarchived the .tgz file downloaded below here.

Let’s start with setting up some scripts to make the user migration process easier.

This script will help you to migrate your Postfix+MySQL (and optionally Postfixadmin) installation to Zimbra.

It reads the “mailbox” and “aliases” table used by Postfix with MySQL backend and generates a bash script that will re-create the same email accounts and aliases/forwards on your zimbra server.

Check php.ini in CLI version (not CGI) for: safe_mode = Off

Mysql/PostFix Migration:

http://wiki.zimbra.com/wiki/Migrating_from_Postfix_and_MySQL_with_bash

 

[spoiler intro=”Script code”]

#!/usr/bin/php

// Postfixadmin (http://postfixadmin.sourceforge.net/) to Zimbra
// (www.zimbra.com) migration script
//
// History:
// Based on work from: Jaros�[34m~Baw Czarniak
// Enhanced by NERvOus (www.nervous.it) on 1-12-2009

<?php
/////////////////////////////////////////////////////////

$user=”changeme”;
$pass=”changeme”;
$db=”postfix”;
$table_mbox=”mailbox”;
$table_alias=”alias”;
$file=”exported.sh”;

/////////////////////////////////////////////////////////
echo “This script generates a bash script called: $file
The script contains the commands to re-create the mboxes and
aliases on zimbra server.\n\n
“;

$mydb = mysql_connect(‘localhost’,$user, $pass) or die (‘Error connecting to server’);
mysql_select_db($db);
mysql_query(“SET CHARACTER SET utf8”);
mysql_query(“SET NAMES utf8”);

$query = “SELECT username,password,name,maildir,quota,domain
FROM $table_mbox”;
$dane = mysql_query($query) or die (‘Error during query for ‘.mysql_error());

echo “Writing to $file …\n”;
$fh = fopen($file, “w”);

fwrite($fh, “#!/bin/sh -x\n\n”);

while ($row = mysql_fetch_array($dane, MYSQL_NUM))
{
$data_mbox = “zmprov ca “.$row[0].” dsfs123hsdyfgbsdgfbsd displayName ‘”.$row[2].”‘\n”;
$data_mbox .= “zmprov ma “.$row[0].” userPassword ‘{crypt}”.$row[1].”‘”.”\n”;
fwrite($fh, $data_mbox);
}
// skip domain aliases, aliases that forward to themselves and aliases
// for which a mbox exists
$query = “SELECT address, trim(trailing ‘,’ from goto) AS dest
FROM “.$table_alias.”
WHERE address NOT LIKE ‘@%’
AND address NOT IN (SELECT username FROM $table_mbox)
HAVING address != dest”;
$dane = mysql_query($query) or die (‘Error during query for ‘.mysql_error());
while ($row = mysql_fetch_array($dane, MYSQL_NUM)) {
// multiple dests
unset($rawdest_r);
unset($dest_r);
$rawdest_r = preg_split(‘/,/’, $row[1]);
foreach ($rawdest_r as $dest) {
if ($dest != $row[0]) {
// don’t forward to itself
$dest_r[] = $dest;
}
}
if (count($dest_r) > 1) {
// distribution list
$data_list .= “zmprov cdl $row[0]\n”;
foreach ($dest_r as $dest) {
$data_list .= “zmprov adlm $row[0] $dest\n”;
}
}
if (count($dest_r) == 1) {
preg_match(‘/@(.*)$/’, $row[0], $matches);
$acct_domain = $matches[0];
preg_match(‘/@(.*)$/’, $dest_r[0], $matches);
$alias_domain = $matches[0];
if ($acct_domain == $alias_domain) {
// we are adding an alias, not a forward
// try to create alias both for a normal
// account and for a distribution list. One of the
// commands will fail, pity.
$data_alias .= “zmprov aaa $dest_r[0] $row[0]\n”;
$data_alias .= “zmprov adla $dest_r[0] $row[0]\n”;
} else {
// we are adding a forward
$data_alias .= “zmprov ca $row[0] ” . rand_str(11) . “\n”;
$data_alias .= “zmprov ma $row[0] zimbraprefmailforwardingaddress $dest_r[0]\n”;
}
}
}

// first create all distribution lists, last all aliases
// We cannot create aliases for distribution lists that do not
// exist yet
fwrite($fh, $data_list . $data_alias);
fclose($fh);

echo “Done.

Now copy exported.sh to zimbra server and run:
# su – zimbra
$ sh ./$file
“;

function rand_str($length = 32, $chars = ‘ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz1234567890’)
{
// Length of character list
$chars_length = (strlen($chars) – 1);

// Start our string
$string = $chars{rand(0, $chars_length)};

// Generate random string
for ($i = 1; $i < $length; $i = strlen($string))
{
// Grab a random character from our list
$r = $chars{rand(0, $chars_length)};

// Make sure the same two characters don’t appear next to each
// other
if ($r != $string{$i – 1}) $string .= $r;
}

// Return the string
return $string;
}
?>
[/spoiler]

 

Mailbox Migration.

http://wiki.zimbra.com/wiki/Maildir_to_zmmailbox_with_bash

The script expects to find the mailboxes in the current path! The structure of maildirs must be as follows:

domain/username/{cur|new|tmp}
domain/username/subfolder1/{cur|new|tmp}
domain/username/subfolder2/{cur|new|tmp}
...

After the script has run, every zimbra user will have a subfolder in his mailbox called ‘old-mbox’ (feel free to change the name at the beginning of the script). The hierarchy of subfolders is maintained under old-mbox.

[spoiler intro=”Script code”]

#!/bin/bash

# Postfix virtual transport -> Zimbra mailbox migration
# written by NERvOus (http://www.nervous.it) - 2009-12-25
# minor fixes by NERvOus (http://www.nervous.it) - 2013-07-23

# base folder where msgs will be imported
BFOLDER="old-mbox"
ZMMBOX="/opt/zimbra/bin/zmmailbox"
BDIR=$(pwd)

echo You must run $0 from inside /var/mail/virtual directory
echo $0 expects to find the mailboxes in the current path!
echo The structure of maildirs must be as follows:
echo 
echo "domain/username/{cur|new|tmp}"
echo "domain/username/subfolder1/{cur|new|tmp}"
echo "domain/username/subfolder2/{cur|new|tmp}"
echo ...
echo 
echo All folders will be stored in a subfolder called $BFOLDER
echo The hierarchy of subfolders will be maintained under $BFOLDER.
echo 
echo Press Enter to start, CTRL+C to abort.
read

# handle folders with a space inside, they are more common than you may
# think
IFS='
'

for p in $(find . -type d -name cur); do
        DOMAIN=`echo $p | cut -d'/' -f2`
        UNAME=`echo $p | cut -d'/' -f3`
        DIRNAME=`echo $p | cut -d'/' -f4`
        # this may fail, if folder already exists. Who cares. Ignore the
        # error.
        echo createFolder /$BFOLDER | $ZMMBOX -z -m $UNAME@$DOMAIN
        if [ "$DIRNAME" == "cur" ]; then
                echo Importing INBOX folder for $UNAME@$DOMAIN from $BDIR/$DOMAIN/$UNAME/ 1>&2
                # we are importing the top level folder
                echo addMessage --noValidation /$BFOLDER $BDIR/$DOMAIN/$UNAME/cur | $ZMMBOX -z -m $UNAME@$DOMAIN
                echo addMessage --noValidation /$BFOLDER $BDIR/$DOMAIN/$UNAME/new | $ZMMBOX -z -m $UNAME@$DOMAIN
        else
                DIRNAME=$(echo $DIRNAME | sed -e 's/\/cur$//')
                echo Importing folder $DIRNAME for $UNAME@$DOMAIN from $BDIR/$DOMAIN/$UNAME/$DIRNAME 1>&2
                echo createFolder \'/$BFOLDER/$DIRNAME\' | $ZMMBOX -z -m $UNAME@$DOMAIN
                echo addMessage --noValidation \'/$BFOLDER/$DIRNAME\' \'$BDIR/$DOMAIN/$UNAME/$DIRNAME/cur\' | $ZMMBOX -z -m $UNAME@$DOMAIN
                echo addMessage --noValidation \'/$BFOLDER/$DIRNAME\' \'$BDIR/$DOMAIN/$UNAME/$DIRNAME/new\' | $ZMMBOX -z -m $UNAME@$DOMAIN
        fi
done

[/spoiler]

Alternatively there is a method using perl instead of bash here: http://wiki.zimbra.com/wiki/Mbox_to_maildir_with_Perl

A lot of other great migration pages can be found here: http://wiki.zimbra.com/wiki/Category:Migration

 

Let’s get this party started

https://library.linode.com/email/zimbra/install-zimbra-debian-6-squeeze

Now that we have our migration prep taken care of ahead of time we can start the full installation.  I believe I’ve read something in the past saying that I would have to uninstall my current postfix/dovecot solution to make room for zimbra but I can’t find that so I’m going to go ahead with teh installation on the current server here and see when I run into my first brick wall.

 

Download Zimbra Server:

Because the latest version of Zimbra 8 doesn’t support Debian 6 at all you have to install the Deprecated Ubuntu 10.04 version, found here:

http://files2.zimbra.com/downloads/8.0.7_GA/zcs-8.0.7_GA_6021.UBUNTU10_64.20140408123937.tgz

 

Setting up the server:

There’s some pretty basic stuff to take care of here first if you haven’t already:

 

Setup a unique hostname for your server

#echo “yourservernamehere” > /etc/hostname #hostname -F /etc/hostname

Setup your hosts file

Replace the IP addresses below with those appropriate to your server

#vi /etc/hosts

127.0.0.1   localhost   localhost
12.34.56.78 yourdomain       yourservernamehere

 

 Install some basic packages needed by zimbra:

apt-get install sudo libpcre3 libgmp3c2 lib32gmp3-dev libgmp3-dev sysstat libexpat1 libidn11 perl-modules wget lzma

Download Zimbra

Download the installer into your installation dir
#wget http://files2.zimbra.com/downloads/8.0.7_GA/zcs-8.0.7_GA_6021.UBUNTU10_64.20140408123937.tgz
#tar -xf zcs-8.0.7_GA_6021.UBUNTU10_64.20140408123937.tgz
#vi zcs-8.0.7_GA_6021.UBUNTU10_64.20140408123937/util/utilfunc.sh

Replace this line:
PREREQ_PACKAGES=”sudo libidn11 libgmp3 libstdc++6″
with this:
PREREQ_PACKAGES=”sudo libidn11 libgmp3c2 libstdc++6″

 

Uninstall postfix and dovecot

This is why I’ve been putting this install off for months.  I am going to have to uninstall postfix completely as it conflicts with the zimbra MTA installation, meaning our mail server is going offline and when you do smoething like that on a production environment you want to make sure to set aside 6-8 hours just incase all hell breaks loose as you won’t be able to go to sleep until it’s fixed, so better to start early on a cleared schedule than after work.

So we want to run something like this:

sudo apt-get remove postfix postfix-mysql

sudo apt-get remove dovecot-common

 

 

Install libaio1

You are going to need this for Zimbra’s mysql server to initialize properly:

sudo apt-get install libaio1 libaio-dev

Downgrade dpkg:

Due to stricter behavior in the version supplied in Debian 6, you’ll need to temporarily downgrade your dpkg binary before proceeding. Issue one of the following commands

#wget http://security.ubuntu.com/ubuntu/pool/main/d/dpkg/dpkg_1.15.5.6ubuntu4.8_amd64.deb

#dpkg -i dpkg_1.15.5.6ubuntu4.8_amd64.deb

 

Install Zimbra

Time to install the beast and see what explodes!  So go into the directory where you ZCS is all unarchived and run the following:

./install.sh –platform-override

The installer will spew a bunch of agreements and whatnot on the screen, read em and accept them and then you’ll be promped with a bunch of options.

For a great rundown of where some of these packages are connected, check out this page: http://www.zimbra.com/docs/os/7.0.0/multi_server_install/wwhelp/wwhimpl/common/html/wwhelp.htm#context=OS_MultiServer_Install_7_0&file=OS_MultiServer_Install_7_0.Zimbra%20Mailbox%20Server%20Configuration%20Options.html

  1. Install zimbra-ldap [Y]
    • This installs Zimbra’s Lightweight Directory Access Protocol.  It’s what zimbra users to look up and write information to your server.  I’d install it but if you want more info on what LDAP is all about head here: http://www.gracion.com/server/whatldap.html
  2. Install zimbra-logger [Y]
    • I’m going to assume you want logs 🙂
  3. Install zimbra-mta [Y]
    • This is the Mail Transfer Agent and by the sounds of it it’s something you are going to want.  Read more about it here: http://wiki.zimbra.com/wiki/Zimbra_MTA
  4. Install zimbra-snmp [Y]
    • This is a monitoring package to track server usage, system logs and other runtime info : http://www.zimbra.com/docs/os/6.0.8/administration_guide/9_Monitoring.12.10.html
  5. Install zimbra-store [Y]
    • Info is hard to find on this specific module but I think it is the storage engine for everything and seems like something you need
    • If you install fails here check out this thread: http://www.zimbra.com/forums/installation/925-installation-fails-zimbra-store.html
  6. Install zimbra-apache [Y] N
    • This is the next brick wall we run up against in my situation because we are trying to install Zimbra on a production server with apache and nginx already running, something which is very clearly not advisable according to some of the discussion here: http://www.zimbra.com/forums/installation/64348-apache-zimbra-8-05-same-host-possible.html
    • From what’s being said in that thread above it would appear that we are now able to clamp Zimbra to a specific IP address to keep it out of the way of Apache.  Lucky for me there are 10 IP’s assigned to my server so we can dedicate one strictly Zimbra by going to Configure->Servers->->IP Adress Bindings
    • Reading what’s listed here: http://www.zimbra.com/docs/ne/6.0.8/single_server_install/quick_start.1.06.htmlZimbra Apache is installed automatically when Zimbra Spell or Zimbra Convertd is installed.” It would appear that those are the only 2 things absolutely needed by Zimbra
  7. Install zimbra-spell [Y] N
    • You going need to need zimbra-apache for this so this is a no go
  8. Install zimbra-memcached [N]
    • Running this on a single server causes issues it wuold appear (http://www.zimbra.com/forums/administrators/53568-completely-remove-memcached.html)
    • It’s mainly to speed up communications between the ldap and the proxy server if they are on separate machines so in our case not needed
  9. Install zimbra-proxy [N]
    • Same as the above, this is not something we need when running Zimbra on a single server so the answer is no

After all of this you are going to get a nice warning message that will look something like this

Checking required space for zimbra-core
Checking space for zimbra-store

Installing:
zimbra-core
zimbra-ldap
zimbra-logger
zimbra-mta
zimbra-snmp
zimbra-store

You appear to be installing packages on a platform different
than the platform for which they were built.

This platform is DEBIAN6_64
Packages found: UBUNTU10_64
This may or may not work.

Using packages for a platform in which they were not designed for
may result in an installation that is NOT usable. Your support
options may be limited if you choose to continue.
Install anyway? [N]

Being the badasses that we are, we’re going to answer yes when it wants us to say no! It’s going to give us one last warning saying the system will be modified and ask us to continue or not to which we reply yes.

After which you should see something like this:

Removing /opt/zimbra
Removing zimbra crontab entry…done.
Cleaning up zimbra init scripts…done.
Cleaning up /etc/ld.so.conf…done.
Cleaning up /etc/security/limits.conf…done.

Finished removing Zimbra Collaboration Server.

Installing packages

and then it’s going to install a bunch of packages but don’t get too excited yet there pilgram as we have a ton of port conflicts to work through.  Before we get to that though, with any luck Zimbra was able to detect your existing mail server domain and set it as the default. After that it should detect all your IP’s and start checking for port conflicts, which in my case were plenty:

Port conflict detected: 80 (zimbra-store)
Port conflict detected: 443 (zimbra-store)
Port conflict detected: 443 (zimbra-store)

So Zimbra really doesn’t like the fact that my existing apache installation is using the 2 ports it wants and we’re going to have to address these in the next screen.

You will be shown an admin menu and overall it looks like everything is pretty much good to go(port conflicts aside) except the admin password hasn’t been set.  So let’s start with the conflicting ports as those are going to need to work for this thing to get off the ground.  Go ahead and change them from 80 and 443 to 60080 and 60443 respectively.  After that you’ll need to set the admin password and you should be go to apply the changes and move onto the rest of the installation. It’s goign to warn you again that your system will be modified, but what did we expect? 🙂

After this it should go through the steps to generating certs and setting up the server and then initializing everything.  This takes more than a couple of minutes so if you’ve got some chores to do around the house, now’s the time to go take care of them.

Operations logged to /tmp/zmsetup.05242014-135208.log
Setting local config values…done.
Initializing core config…Setting up CA…done.
Deploying CA to /opt/zimbra/conf/ca …done.
Creating SSL zimbra-store certificate…done.
Creating new zimbra-ldap SSL certificate…done.
Creating new zimbra-mta SSL certificate…done.
Installing mailboxd SSL certificates…done.
Installing MTA SSL certificates…done.
Installing LDAP SSL certificate…done.
Initializing ldap…done.
Setting replication password…done.
Setting Postfix password…done.
Setting amavis password…done.
Setting nginx password…done.
Creating server entry for yourdomain.com…done.
Setting Zimbra IP Mode…done.
Saving CA in ldap …done.
Saving SSL Certificate in ldap …done.
Setting service ports on yourdomain.com…done.
Adding yourdomain.com to zimbraMailHostPool in default COS…done.
Setting zimbraFeatureTasksEnabled=TRUE…done.
Setting zimbraFeatureBriefcasesEnabled=FALSE…done.
Setting MTA auth host…done.
Setting TimeZone Preference…done.
Initializing mta config…done.
Setting services on yourdomain.com…done.
Creating domain yourdomain.com…done.
Setting default domain name…done.
Creating domain yourdomain.com…already exists.
Creating admin account admin@yourdomain.com…done.
Creating root alias…done.
Creating postmaster alias…done.
Creating user spam.xfxu0xp7v_@yourdomain.com…done.
Creating user ham.jji4oyqxy@yourdomain.com…done.
Creating user virus-quarantine.y3otijugm@yourdomain.com…done.
Setting spam training and Anti-virus quarantine accounts…done.
Initializing store sql database…done.
Setting zimbraSmtpHostname for yourdomain.com…done.
Configuring SNMP…done.
Setting up syslog.conf…done.
Starting servers…done.
Installing common zimlets…
com_zimbra_tooltip…done.
com_zimbra_url…done.
com_zimbra_srchhighlighter…done.
com_zimbra_date…done.
com_zimbra_clientuploader…done.
com_zimbra_attachmail…done.
com_zimbra_viewmail…done.
com_zimbra_attachcontacts…done.
com_zimbra_bulkprovision…done.
com_zimbra_proxy_config…done.
com_zimbra_email…done.
com_zimbra_ymemoticons…done.
com_zimbra_phone…done.
com_zimbra_adminversioncheck…done.
com_zimbra_cert_manager…done.
com_zimbra_webex…done.
Finished installing common zimlets.
Restarting mailboxd…done.
Creating galsync account for default domain…done.

You have the option of notifying Zimbra of your installation.
This helps us to track the uptake of the Zimbra Collaboration Server.
The only information that will be transmitted is:
The VERSION of zcs installed (8.0.7_GA_6021_DEBIAN6_64)
The ADMIN EMAIL ADDRESS created (admin@yourdomain.com)

Notify Zimbra of your installation? [Yes]

 

After this it will setup the zimbra crontab, move some logs around and Viola!

Restoring DPKG

We're all done with the zimbra installation so we need to restore the latest version of dpkg so that our system will work.  You can do that by typing
#apt-get -f install

Now the Fun Begins

Well the average person would get to this point and have a beautifully running Zimbra server all ready to go.  We unfortunately are not one of those people as we decided to install this on a production server with apache and postfix and dovecot already running.  Yay for us!  If you remember above we have quite a few conflicting ports that are preventing us from connecting to this lovely platform that is now happily purring along in the background.  So we have to find a work around to deal with the conflict on ports 80 and 443 I’m going to start with 80 and 443 as I am dying to get in and play with the web interface, this being my very first web mail setup it’s one of the features I really am excited about.

 

Getting the web interface up

http://wiki.zimbra.com/wiki/ZimbraApache – This seems like an excellent starting place, although as you can expect it’s not going to be all roses here.

the first bit of advice is to change the zimbra ports like this:

zmprov ms mail.yourdomain.com zimbraMailPort 60080
zmprov ms mail.yourdomain.com zimbraMailSSLPort 60443
zmprov ms mail.yourdomain.com zimbraMtaAuthHost zimbra.mydom.com /etc/init.d/zimbra restart

Now you should be able to go to https://mail.yourdomain.com:60443 and see a nice lovely login that takes you into your admin users mail box.

You should also be able to go to https://mail.yourdomain.com:7001 and access your Zimbra webmin

 

 

So that’s the first big hurdle up and running.. We have a semi-functional mail server but now we have to deal with the errors in the webmin area and that’s going to be a whole new post 🙂

 

Other fun tidbits:

If you encounter the following: scan_dir_push: open directory defer: Permission denied it’s probably because you changed some perms somewhere and need to run /opt/zimbra/libexec/zmfixperms.  Should fix you up

Tags: , , , , ,

Leave a Comment