20 Jun 2008

Import and archive your old emails with GMail

Posted by kilbot | Filed under: News

GmailThere’s been a bit of buzz about Google Apps lately, particularly in web hosting circles with Dreamhost founder Josh Jones recommending his clients use GMail for email rather than their own servers. The Kilbot Factory will be offering Google App integration for our clients in the near future, but anyone can use GMail to archive old emails right now, for free. Here’s how…

The problem – 12,000 old emails
I have kept pretty much every email since 2003, that’s when I set up my first POP3 account to download mail to my desktop. Previous to that I was using webmail and had to carefully pick and choose which emails to keep so that I wouldn’t go over my 200Mb limit. Downloading all my email to the inbox worked fine until 2005 when the volume of emails I was receiving increased dramatically, space wasn’t a problem but the usability of my email client dropped significantly – Outlook just didn’t like searching through a few thousand emails at a time. I switched to Thunderbird which generally handles search better, but it too became swamped, so in 2006 I began moving my old emails into folders marked 2003 Archive, 2004 Archive etc and exported them to an external hard drive and that was that… until I need to quickly retrieve an email from 2005, in which case I import the archive then search then delete … ugh.

So what I needed was somewhere to stuff all my mail, it had to have quite a bit of room because my archive mail and current mail was already tipping 5Gig, and it had to be searchable, fast. And for this GMail fits the bill perfectly.

The solution – GMail
Obviously we need a way to get the mail into GMail, ideally Google should include an option to import your mbox or pst files and suck in all your email at once … but they don’t have that option yet, so we will have to use a slightly more painful, but nonetheless effective solution: IMAP.

You have probably heard of IMAP and POP3 as ways to get your email from a server, there are differences between the two protocols, but put simply POP3 is used when you want to download and remove email from a server and IMAP is used when you want to sync your local email with the server email. Using IMAP we can get GMail to ‘sync’ with our archives and suck up all that email.

Step 1: Make sure IMAP is enabled in GMail by going to your settings menu.

Step 2: Set up your favourite email client (Outlook, Thunderbird etc) to access your Gmail account. There are instructions on how to set up a range of email clients on the Google Support website. Make sure your email client is not going to store a copy locally which would defeat the purpose. Mine is set to not download messages over 1kb.
Save Disk Space

Step 3: Once GMail is set up on your email client you can add folders, for example, 2003 Archive, 2004 Archive etc. In Thunderbird my folders looked like this:
GMail Folders

Step 4: Drag and drop your email into the appropriate archive folder. You could try dragging whole folders at a time but I found this troublesome; every now and then GMail would throw up an error which means you have to start again. I found it safer to drag and drop a few hundred emails at a time, starting with the smallest emails and then working up to the large ones. Email over 10Mb might have to be done in smaller batches still, and there seems to be a maximum size limit of around 30Mb; I had one email with attachments that was 36Mb and it threw an error everytime … I guess at that size you should be archiving the attachments separately.

The syncing process is not terribly fast so it will require a little patience (it took me about 2 days to get all the email up, chunk at a time), but eventually you should have something like this:
Gmail Labels
Storage

2.69Gb, around 12,000 email messages, easily searchable and stored for free.

  • sarah p.

    Great tutorial, thanks! Just a few questions.

    When you copied your local mail to the IMAP Archive folders, did you create separate sub folders for sent & received mail or just dump it all in?

    In either case, will the archived mail retain the “replied” flag that links a received message with your reply or does it somehow treat it as a “conversation” when sent/received mails are mixed together?

    Thanks! Sarah

  • http://www.kilbot.com.au kilbot

    Hi Sarah,

    I treated my GMail folders like a bucket, sent and received in the same folder. It generally groups emails in a conversation, but I have noticed a little weirding from time to time, like conversations spilt in two and even two completely different conversations mixed together.

    So far this hasn’t caused a problem, I almost exclusively use GMail through the search filter to get information, then I’m back to my regular email client. I guess it may cause problems if you had to follow a long conversation.

    k.

  • http://minhaaj.blogspot.com minhaaj ur rehman

    fabulous tutorial