"Upgrading" KMail from Debian Wheezy to Devuan Jessie

After upgrading Debian Wheezy to Devuan Jessie, the first time KMail starts, you get a message about having to complete a migration process because KMail 2 stores things differently from KMail 1, and you are clearly informed that the process must not be interrupted. I assume that it also cannot be reversed once complete…

I have what I suppose is quite a lot of email folders on my machine (203).

I use three different POP3 accounts to collect my email from, and I have a correspondingly large number of filtering rules to put the downloaded email into the correct folders when it arrives.

The migration process simply deleted all my filtering rules, and all new email is now arriving in my "Inbox".

There's no warning in advance that this is going to happen; that the migration tool can't bring the filtering rules across - it just says "migration process complete", you start KMail again, and all your new email simply arrives in the Inbox. When you go to check the list of filtering rules you've built up, it's empty. I can't find a way to import the old rules from a backup of my machine, either. This is going to be tedious to repair.

I've found postings such as https://lists.debian.org/debian-kde/2014/03/msg00015.html suggesting that people have found their filtering rules are not working after a migration from Wheezy to Jessie, but in my case the filter rules list is simply empty.

Fortunately, I am a cautious / suspicious / paranoid sort of person (ie: I don't trust that anything will work as well as someone else tries to tell me it will, so I take steps to make sure I don't lose anything when it turns out that whatever-it-is goes wrong), therefore I did not simply "update my desktop computer".

I made a complete copy of the machine (and of course I have backups of all my data), and I upgraded the copy. That way, if anything went wrong, turned out to be a show-stopper, or otherwise got in the way of me working with the new system, I could simply revert to the machine as it was previously, and either carry on using it and wait until the upgrade process works better, or try some other way (with a bit of hindsight) of doing the upgrade.

So, fortunately I still had a fully functioning version of KMail 1, containing all the folders, emails, and filtering rules. I started it up, went into Settings and Filtering Rules, and Exported the rules to a network share which I could access from the new machine.

Somewhat naïvely I then expected to be able to restore my home directory from a backup (containing my old KMail 1 settings, folders, etc) and repeat the KMail 1 → KMail 2 migration process.

Oh, no. Nothing as simple as that. Instead of the migration tool starting up, telling me it'll take a while and I shouldn't interrupt it, and then getting on with its job, it started telling there was some problem with the Akonadi server (The what? This is some background part of KDE, which apparently gets used by KMail even more in the new version than it used to be in the old one, and as far as I'm concerned, anything which runs in the background like that should not produce fatal errors when the user is simply repeating some action which "worked" perfectly well the first time. If it does run into a problem and something needs repairing, the error message should very clearly tell the user what needs doing to fix things, because it's a reasonable assumption that most people never even know it's on their system, let alone how to fix it when it breaks.)

So, anyway, I found quite a few forum entries from people having the same problem, and in my case it was solved by doing:

akonadictl stop
rm -rf ~/.local/share/akonadi/
rm -rf ~/.config/akonadi/
rm ~/.kde/share/config/akonadi*
akonadictl start

Re-running the KMail migration process then did what it had done the first time - migrated all my folders and emails (although note that it takes a remarkably long time to be able to see the list of emails in the folders - there's some additional migration process after KMail has finally started which gradually works through all your folders and makes the emails in them actually visible in the message list), with no filtering rules.

Remembering that the migration process doesn't carry the account passwords across, I deliberately refused to answer the questions about "what's your password for account X?" until I had Imported the filtering rules. That way, the rules would be in place before the machine started downloading new emails, and I wouldn't end up with an Inbox containing a few hundred, or thousand, emails (depending on how long this migration process had taken since I shut down the old system and took a copy of its disk).

So I'm now sitting, a couple of hours later, looking at a KMail 2 screen which looks pretty similar to the old KMail 1 screen, with all my familiar folders, seeing newly-arriving emails getting filtered into the correct places, and wondering just how long the folder migration process is going to take. A lot of the folders show that they contain no emails at all (which is completely wrong - many of them contain several thousand) and some simply show "4 emails, 4 new" without letting on that there are a few hundred, or thousand, old ones in there too. I can't work out what order the folders are being processed in, but one by one, they start showing the right number of emails, and I get the usual list of subjects, and can click on one and see the body of the email too. The progress bar at the bottom simply says "synchronising email folder xxx", where xxx is the one it's working on at the time. "top" shows that kmail is using 100% of a CPU core, and akonadi_maildis is using 100% of another one. mysqld is busier than I'm used to on this machine too - around 15% of a CPU.

I've now just seen the message (in the status bar at the bottom, not in a dialog box) "Moving old messages from folder Squid to folder Squid failed".

The Squid folder shows me 5092 emails, of which 57 are new (it's the folder which email from the Squid mailing list goes onto).

So, what am I supposed to do about that failure? What failed? Have I lost anything? Where do I get more information about the problem?

That strikes me as the sort of message a programmer puts into an application when they've just found out their manager has cancelled their holiday request. "Okay, I need to tell the user there's a problem, but I'm damned if I'm going to be helpful about it." Thanks, whoever did that.

Now I find out I get a status bar (KDE's main screen status bar, not KMail's) notification every time a new email comes in. Who the hell wants that in 2017? 25 years ago, yes, a new email was an interesting event, but these days? I get emails arriving every few seconds - I don't want to be told about each one individually! And besides, I hadn't selected that option in my old KMail settings, so why has someone else turned it on for me during the migration? (For reference, you can turn it off again from Settings, Configure Notifications, select "New email notify" from the easily-overlooked little drop-down menu at the top, then highlight "New email arrived" and remove the tick from "Show a message in a popup".) Grrrr.

Several more hours later, I still don't have a usable email client. It's stopped using 100% of a CPU core, but I'm not convinced it's "synchronised" all its folders, and almost no matter what I ask it to do, it just sits there with a progress bar moving from left to right and back again like a pig swimming through treacle.

I can ask it to "remove duplicate messages" from a folder with only 1000 or so emails in total. Chug, chug, chug… After half an hour or waiting for it to say that it had done anything worthwile, I cancelled that.

It's not like this is a feeble machine, either - 6-core 3.3 GHz CPU, 16 Gbytes RAM, system partition on SSD, home partition on Raid5 HDD.

I can click on a folder name and get a list of the 80 emails in the message list; I can click on one of the emails in that list and get… nothing. There's no preview of the email shown in the preview pane (that just shows "KMail - The KDE email client - Retrieving Folder Contents - Please wait…") For a folder with only 80 emails in it!? Has somebody programmed the equivalent of a 14.4kbps modem inside this thing?

It really rather bothers me that some programmer has realised how abominably slow this version of KMail is at doing something you wouldn't even notice the time for in the previous version (you click on an email; KMail 1 shows you the content - what could be more fundamental in an email client?), and instead of filing a bug report to rescue the poor pig who's trying to swim through the treacle, simply creates a pretty window content saying "Retrieving folder contents, please wait…" and leaves it at that, satisfied with a job well done.

If I try right-clicking on an email and ask for a Print Preview, I get a progress bar saying "Please wait while the message is transferred" (from where, to where, I have no idea - this is a downloaded email sitting on the hard disk of my computer), and that swimming pig tries to make its way through the treacle again. I gave that half-an-hour's chance before cancelling it, too - with no useful result.

Same thing for a right-click on a folder and then "Folder properties" - I get a progress bar saying "Retrieving folder properties" (From where? Who knows, but it's not an efficient process…). I'm pretty certain that one can outlive my patience as well.

All in all my experience of KMail2 so far is that it's useless.

I'm just hoping that it's doing something clever in the background with my 25 Gbytes of emails, and once it's finished it'll let me actually read one of them.

Oh, I've just had another system pop-up message on the main KDE status bar. This one says "Resource KMail folders is broken". I wonder what the hell that means and what I might be expected to do about it. Someone's had a right fun time with these bloody useless error / notification messages, I can tell.

So, it looks like KMail has finished "synchronising" all my folders (How can I tell? Well, every folder now has the total number of emails, and the number of unread emails, displayed against the name, and when I click on the name of a folder, I no longer see a "synchronising" message at the bottom, before the system shows me the list of emails in it.)

However, is it now responsive and usable? Ha! Like hell it is.

I started my standard morning email ritual today - look at every folder with new emails in; if it's a mailing list, scan quickly for anything interesting, and mark things as "read" so I can spot more as they come in during the day. If it's a customer-related folder, look for whether someone wants something, has replied to something of mine, or an alert has come in, etc. Finally, for the more generic folders, delete or move most of the new stuff into "spam" as appropriate.

I found one folder which needed 8 emails deleting. As I press "delete" for each one, a pig-in-treacle-slow progress bar starts up at the bottom right of KMail, saying "Moving messages". Those "progress" bars (sorry, I think that's a complete misnomer, but I have to call them something you'll recognise so you know what they look like) are still there an hour later.

The bit that makes the whole thing unusable, though, is that now those bars are there, I can't actually do anything with the rest of my email. I went through a few more mailing list folders, and scanned the cursor down each of the new emails, to mark them as "read". They're still showing as "new" an hour later.

More important, I can click on a folder and get the list of emails, but if I click on an email, I can't read it. The preview pane simply shows "Retrieving Folder Contents - Please wait…"

There's no way this is usable. Email is the most important thing on a computer for me (with a browser as a close second), because I use it for almost all of my business communications. If it's a bit slow, I'll get irritated by it. If it sits there doing nothing of any consequence for an hour and in the meantime actually preventing me from reading any emails, it's damaging my business.

KMail is now able to tell me "Moving messages to wastebin failed."

It's saying I cannot delete emails I no longer want.

Did the programmer who realised it was necessary to put that message in not even think about giving me a clue as to why I can't delete my emails (ie: move them to the wastebin), so I can at least have a go at fixing the problem?

Cynically I suspect that they know there is nothing I can do to fix the problem, so there's no point in giving me any more details. Until they change what KMail is trying to do in the background, this problem exists. "No user-serviceable parts inside" :(

Searching doesn't work, either.

I'm looking at an Inbox folder which contains only 46 emails. I can see that several of them are from "SIPprot" and have the subject "SIP Attack Notification". I'd like to move them to the folder they belong in, so I use Find Messages to search for emails where "From contains SIPprot". The result? "Search complete" and an empty results window. This is just bollocks. How could anybody release software in this state??

On day 3, I've just tried deleting some emails again. I'm looking at a folder containing 641 emails, and I've selected 6 of them (each between 7kbytes and 16kbytes in size) to delete. I press "delete" and immediately all 6 emails go grey in the message list. So far so good. A "progress" bar appears at the bottom saying "Moving messages". I did that at 09:30; it's now 14:30 and the emails are still in the message list, and the "progress" bar is still there. This is a complete joke (except it's not funny).

Oh, and of course throughout that process I never got to see the actual body of any of the emails - the preview pane simply shows me "Retrieving Folder Contents; Please wait…". It seems to be the default message for KMail 2.

I may have waited long enough (50 hours…!) for KMail 2 to be usable.

Then, again, I may not, because it still isn't.

Go up
Return to main index.