Pegasus Mail and Mercury Developer News Archives
October 2021 - Updates and reassurances A long time between updates — my apologies for that... There's been a lot going on. Community Relaunch It has taken far longer than it should have, but the Pegasus Mail and Mercury Community is now fully up and running again in its new home on my main server. The fact that it has taken so long is entirely my fault: I wouldn't want the excellent work of Rolf Lindby and Peter Strömblad to be tainted by my own personal inefficiency, so it's very important to make it clear that the delay stems entirely from my own indecision and unfamiliarity. I'm not really much of a social media user — I don't have Twitter, Instagram, Tik-Tok or other such accounts, and I even got rid of my Facebook account many years ago when it finally dawned on me how intrusive and privacy-indifferent it was. This unfamiliarity has meant that I've been a bit mentally-blocked on the Community, and at times have struggled to make quite simple calls on its appearance and function, which has delayed things far more than was ever necessary. Having said that, I believe the final result is good, and once we get the inevitable initial rough edges sorted out, I think it will serve my user community well. I can't offer enough thanks to Rolf and Peter for their help in getting the new software set up and porting the data from the old site, and I'd like to offer my special appreciation to Peter for supporting the original community site for so long. OAUTH2 Update I'm going to keep this as neutral as I can. Those of you who know me well know that I regard OAUTH2 as probably the worst, most incompetently-conceived 'standard' I have ever seen in my more than 30 years working with e-mail, but the reality is that the large corporations have spoken, so OAUTH2 is what we are going to have. I have been working as time permits on an OAUTH2 implementation for Pegasus Mail that will allow connections to GMail, and recent announcements from Microsoft have now forced me to add the various Microsoft Mail services to the mix. For those of you unfamiliar with the fact, OAUTH2 is not a standard - it's more of a concept: services who implement it are free to do it however they want, and client developers like myself are required to write custom modules specific to each service, and to register specifically with those services. It's an absolute dog's breakfast - oh, hold on, I said I was going to be neutral... Right. Short version: I am working on OAUTH2 implementations for GMail and MS and will have them out as soon as possible. 64-bit Windows Versions With Microsoft's recent announcement that Windows 10 will be end-of-life in 2025, and that from that time all applications will have to be 64-bit, I have had a number of people ask me what my intentions are. Well, you needn't worry. The only reason I have never produced 64-bit versions of Pegasus Mail or Mercury is because there is absolutely no advantage for either me or my users in doing so - literally no advantage at all... But for many years I have written my code in a manner that makes 64-bit versions a simple matter of changing a couple of lines of code in a makefile and everything follows from there - it's easy, and not at all complicated. There *may* be some issues with some of the third-party code I use, in particular the editor module, which is very expensive, but that's a financial difficulty, not a coding or support difficulty, and I'm sure we'll be able to find a way around it. In short, though, producing 64-bit versions of my programs is easy, and will not take long when it comes time to deal with it. New Versions New versions of both Pegasus Mail and Mercury are imminent - Mercury will probably be up by the time you read this, and Pegasus Mail isn't more than a week or two away. These are not v5, but they contain some v5 technologies: WinPMail in particular has a hugely improved editor module that should be significantly better at editing the bizarre range of HTML data we see in e-mail these days. I know you've heard me say this before, but these will *definitely* be the last members of their respective v4.x code trees - after this, we're into the new technology versions. Truly. Really. I'm sure you'll believe it when you see it. <ironic grin>. Goodbye, XP, it's been fun knowing you... This is just a heads-up that the imminent WinPMail and Mercury releases will be the last to have any kind of support for Windows XP. In order to work with newer versions of Windows, I need to migrate to a new Microsoft compiler (Visual Studio 2019), in which the runtime libraries no longer work on XP. Given that XP has been beyond end-of-life for a number of years now, I would hope that this won't be an issue, but I thought I'd mention it, just in case. Windows 7 will remain supported for the foreseeable future, even though it too is past end-of-life: this is mainly because it was the last version of Windows that wasn't either pig-ugly, or badly privacy-invasive, things that are important to me, and I know will be important to many of my users as well. The pmail.com website and HTTPS I've had a number of people ask me why the www.pmail.com website doesn't use HTTPS (secure access). The reason for this is because it is hosted on an Amazon.com service (S3) that doesn't support HTTPS. I've also been asked why you have to enter 'www.pmail.com' instead of just 'pmail.com'; once again, the problem is to do with Amazon: I can only process links to the top-level domain if I have Amazon doing all my domain name servicing, something which won't work for me. My medium-term plan for dealing with these issues is to migrate the website to my own server, leaving just the downloads in Amazon S3, but that's not an imminent change. Cheers! -- David --
November 2020 - Christmas updates 2020... What a year! I'm not going to go into any of it - like me, you've all been living it, and are no doubt pretty sick of it (although not literally, I hope). Well, I can't offer a cure for the ills of the world, but I can at least offer some news of updates to both Pegasus Mail and Mercury that will be out in time for Christmas. Both programs' updates will feature new OpenSSL libraries: we've now finally managed to get the OpenSSL 1.1.1 tree compiling and linking, so SSL support is now completely up-to-date and assured for a number of years (the 1.0.2 tree we had been using is now at end-of-life). This won't mean a lot to normal users, but should allow the programs to integrate seamlessly with large providers for the foreseeable future. There are also quite a few bug fixes in both programs, in particular some improvements in Mercury's IMAP support that should improve reliability for sites with a lot of iOS devices, and a new text editor module for Pegasus Mail that should do a better job of editing replies to HTML messages. I had hoped to have OAUTH2 support available for Pegasus Mail in particular, but this has ended up being a more difficult proposition than even I expected (and I was expecting it to be pretty awful). I've had to develop a comprehensive HTTPS client to make it work, and while I'm confident I'll have it operating soon, I don't want to hold up the updates for any longer because of it. As soon as the OAUTH2 module is complete, there will be another update - I am aiming for late January at this point. Pegasus Mail v5 and Mercury v5 development continues. I realize that many of you will think of these programs as vaporware by now, and it's hard to describe how frustrating I find it that I can't show you all the work that has gone in behind the scenes. Mercury v5 is much closer than Pegasus Mail v5 - essentially, only one component needs to be finished before V5 can enter beta testing (the mailing list editor, which has proven much harder to split into service and UI components than the rest of the program). Regarding Pegasus Mail, a lot of the redevelopment involves modules such as the new message store, which have been written to be completely standalone - a lot of what remains is coding support for these new modules into the program itself, which is unfortunately still a lot of work. I'm not in a position to offer any kind of date for Pegasus Mail v5 - I've done that in the past and created a lot of trouble for myself by being dramatically optimistic... But I offer you my sincere promise that it *will* happen, and that I believe the result will be good when it does. Finally, as noted elsewhere in the "What's new" section, redevelopment of the Community Site is progressing well: all users and postings have been imported from the old site into the new CodoForum version, and we're now at the point of adjusting the look and feel of the new site to make it a bit more familiar to our users. We hope to have the new site up and running very soon - watch this space for more information as it comes to hand. That's it for now - please keep safe and be kind, even if some days it's frustratingly difficult to do that. Cheers! -- David -- December 2019 - Several Issues Ahh, December - the silly season. Turkey or ham, pavlova or apple pie, family and friends... Work should be the last thing on anyone's mind, but this is also a useful time to bring closure to the events of 2019 and look forward to the next year, so here we go... In this instalment of the Developer News: After cataract surgery on November 26th I have made a steady recovery (hampered only by a minor but persistent infection which now appears to be under control) and now have good vision that will allow me to resume normal workloads in the New Year. Thank you to everyone who has sent me their kind wishes and support - I really appreciate it. Many of you who use Google's GMail service may recently have received an ominously worded message from them advising that as of 2021, you will no longer be able to access their service using so-called "less secure applications". Instead, you will be required to use an authentication method called OAUTH2 in order to access your GMail account. Now, this is a "hot button" issue for me - it will get me started, and if I'm not forcibly held down, it can be very hard to stop me from ranting almost indefinitely. I'll resist the urge to let loose, though, and simply say that (1) "Less-secure applications" is crude scare-mongering by Google that means little and is mainly intended to serve their political agenda, and that (2) OAUTH2 is among the most poorly-standardized and ill-considered hacks I've ever encountered in thirty years of serving the Internet. That said, GMail's size makes it an effective bully, and regardless of what I might think about OAUTH2, as they say in Star Trek, "resistance is futile". Accordingly, we have been working on implementing support for OAUTH2 in Pegasus Mail, and already have a basic working proof-of-concept on which we can build going forward. We expect to have working OAUTH2 support for GMail (and at this stage, *only* GMail) well before it will be necessary for continued use of the service. One of the main flaws of OAUTH2 is that it has to be implemented on a service-by-service basis, and at this time we do not have plans to support services other than GMail, although we will review this position over time as required. Our OAUTH2 support will also be available for Mercury users who use MercuryD to access GMail accounts, but it may take a little more effort to use. We will publish more about this as work proceeds. We will shortly be releasing an update for both Pegasus Mail and Mercury that implements a more modern version of the OpenSSL libraries used to provide secure access to remote servers and services. The update will include a version of OpenSSL called 1.0.2t, which is already right near the end of its support life, but which should be fine in the short term. The reason we are not issuing support for the current 1.1.1 build of OpenSSL is because we simply have not been able to produce a build that will pass its own internal self-tests. Once we've got this issue sorted out, there will be another update that includes the more up-to-date version of OpenSSL. The interim release will be suitable for use in Pegasus Mail v4.73 and later, and Mercury v4.7 and v4.8. After a hiatus because of my eye problems, work has recommenced on Mercury v5 and Pegasus Mail v5. Mercury will be the first to be released, mostly because it is smaller and has less user-interface than Pegasus Mail. I hope to have builds of Mercury v5 available to testers and interested users in the first three months of the New Year, while Pegasus Mail will probably be slightly further out. I appreciate everyone's ongoing patience with the seemingly neverending delays in these releases - I can only promise you that there is progress, and that I'm totally committed to getting these new versions released. ------------------ All my very best to everyone for the Christmas season, and my hopes for a better New Year for all. July 2019 - The Eyes Have It In January this year, while I was taking a brief holiday with some friends in Christchurch (a city about 350km north of Dunedin, where I live), I ended up being admitted to hospital for emergency eye surgery on a detached retina. It was probably just as well that my friends are both doctors, since it meant that the condition was caught early. The surgery was successful, but since that time I have had almost constant problems with my vision, partially as an after-effect of the operation, but mostly because of a rapidly-growing cataract that has formed on the lens of my left eye: for the last three months, I have had almost no useful vision in that eye. I am scheduled for priority cataract surgery to correct the problems, after which I should be back to normal (or even, they tell me, a little better than before the initial problem), but at this stage the best estimate the hospital can give me for when it will happen is "sometime in the next four months". The effect this series of issues has had on my life, and on Pegasus Mail and Mercury, is quite considerable. My right eye has always been weaker than my left eye, but it now has to handle the entire burden of my vision. What this means for my user community is that I can only do screen work for about half an hour at a time before getting eye strain: I'm still working, but at a reduced rate, and my ability to do things like answer mail is heavily reduced. Warning: raw emotional outpouring alert! Now that I've got that off my chest, let me give you a small progress update. Mercury v5 is very near to going into beta - I just have to finish the mailing list manager overhaul that is required to support the new split interface and it will be ready. I do not expect a long beta period for v5 - I have done huge amounts of testing as I've worked on it (I had to - just about every source module in the program has changed significantly). WinPMail has been lagging behind a bit as I put my efforts into Mercury, but I would rate progress as good, and believe I should be on target for rough public betas before the end of the year. WinPMail needs fewer structural changes than Mercury, but far more modernization, and there's much more of it. I will put out another developer update soon laying out some of the changes you can expect to see in the new versions of both programs, but for now, I'm afraid, my half hour is up, and I'm squinting so hard at the screen I can feel my eyelids seizing up. All the best! -- David -- June 2018 - Building Blocks The release of WinPMail v4.73 gives me an opportunity to describe a little of the process that goes into developing a program like this - in particular, I'd like to provide a little insight into the new help system in v4.73, which is the first Pegasus Mail Version 5 feature to make it to release. The new help system became a necessity when Microsoft retired the old WinHelp help format a few years ago, because they simply didn't replace it with anything suitable for developers like me who had an investment in it. The HTMLHelp program they provided was adequate for many developers, but suffered from a crucial flaw: because it depended on Internet Explorer for its HTML rendering, it could not load help files on shared volumes (IE won't do this, supposedly for security reasons). There's absolutely nothing worse than being stressed by a problem with a program, pulling up its help system, and seeing "Navigation Cancelled" instead of a help page, yet this is what happened if you tried to open an HTMLHelp help file that was installed on a server or NAS shared volume. There were mitigations for the problem, but they were poorly documented and technically demanding for anyone who wasn't a trained sysadmin. So, after looking around to see what else was available and finding almost nothing, I began rolling my own using components I already had for Pegasus Mail. It was a long job - I estimate that developing the help compiler, the help display module, and the user interface for the help system took me about 400 hours over a period of years, and I know for a fact (having done it recently) that converting the old RTF source I used for the Pegasus Mail WinHelp help file to HTML for use in the new system took me 150 hours (yes, I was counting — it was a HORRIBLE job). "But it's just a help system!" I can hear you saying; "it's just a little text and a window to display it in!". Well, the WinPMail help pages consist of more than 250 separate help files, all cross-indexed and linked, each formatted as consistently as possible: that's roughly 20,000 lines, or (for those of you who can recall my last developer update) about a third of a War-And-Peace, and that's just the basic text, without considering the markup, indexing and table of contents generation. The source code for the help compiler and the display module make up about another 15,000 lines of code - that's just the reality of working with a format as rich and complicated as HTML. All of this just to replace something that Microsoft decided to remove because they couldn't be bothered keeping it up to date... That said, though, the result is pretty good, and I now have a help system that doesn't depend on anything outside my control, and that I can use in both Pegasus Mail and Mercury — I have only one piece of code to maintain, completely self-contained, unaffected by any changes to the programs that host it. And it is this that is really the goal of this update - to describe a process I am now applying to all my new development: code that stands on its own, can be used in either system, is available to extension developers, and requires minimal support in the long term. This approach to developing things as building blocks gives flow-on benefits that can be guite interesting — so, as an example, the module I'm just finishing now that implements full S/MIME capability will allow Pegasus Mail to send and receive a full range of modern encrypted data, but will also allow Mercury to send out its list mail with digital signatures as proof of origin; in a similar fashion, the new Pegasus Mail Contact Manager (the replacement for the aging addressbook system) could easily serve as the basis for a Mercury vCard server — and there are countless other similar examples where having the one self-contained module will allow both programs to benefit: I'm expecting that there will even be crossovers I haven't anticipated yet. Interesting times indeed. All my very best to you all, -- David --
April 2017 - "How many WaPs is that?" I'm sure many people must be wondering if I'm still alive... I have to apologize for being so uncommunicative for so long - as I've remarked in other posts, for someone who has put most of his adult life into trying to help people to communicate, I'm pretty poor at doing it myself. I'd love to tell you why I find it so hard to do regular updates, but I can't, because I'm not really sure. It just seems to be a task I find unusually difficult: a lot of the time, it feels like I only have dry, technical things to report, and I guess I just don't imagine my community being too interested in that stuff. Still, there's no doubt I need to work on my communication skills. My community of users have to be the most patient people in the world: each week I get a few messages asking me how Pegasus Mail v5 and Mercury v5 are going, but they're never angry or impatient, always encouraging - and I have to say I find that amazing... I've been saying that both programs were "near" for almost as long as I can remember, yet my community seems to be willing to accommodate delay after delay with a steady good grace I find quite humbling. I hope, in this posting, to give you an idea of why these releases are taking so very long. To start with, you'll need some background. The first version of Pegasus Mail for Windows to send a message appeared back in 1993 - that's twenty-four years ago. Since that time, I calculate there have been fifty-six releases in various forms, each release building on the ones before it in some way or other. That's a lot of releases, each one representing many, many changes to the source code I write to produce the program. Over the years, that source code has become ever larger and more complex; furthermore, because I'm human, my style has grown and evolved over the years, and the way I wrote things back in 1993 is almost invariably not the way I would do them now, so a lot of the code exists in a mixture of styles that makes maintenance very difficult. By 2010, it had become absolutely clear to me that I could no longer reliably maintain the Pegasus Mail source code as it stood - it had become too large and had too much history and compromise in it... Add that to the fact that Mercury had also grown large and unwieldy, and I realized that I was going to have to find some way to reorganize and unify the programs - come up with common internal components so that I was maintaining less code, and the code I was having to maintain was more modern, more consistent, more streamlined. At the time, I thought I could do this as a gradual process - replace a piece here, a piece there, and the programs would progressively converge on a new, maintainable standard... But I was wrong - or at best, naive. I would start working on modernizing one part of the program, only to find that if I were to do the work properly, I could salvage almost none of the old code - I would have to write it anew. Then I'd find that rewriting that part of the program couldn't proceed until I'd rewritten another part of the program on which it depended, so the process would start once more - and so on and on, over and over again. Although it certainly wasn't my original intention, what I actually ended up doing was rewriting almost the entire program from scratch: this wasn't a process of modernization, as I had initially planned, but a process of writing a completely new program using an older one as a model. It can be hard to get perspective on this process - on how large the work really is... But over time, I've come to look at what I do in terms of units I call "WaPs" - short for "War and Peace": Tolstoy's "War and Peace" (Oxford World Classics edition) has about 71,500 lines of text, so I defined 71,500 lines as 1 WaP, and use that to measure the size of the work I do. At the moment, the totals are roughly as follows:
That means that the code I am working on is currently roughly the size of nine copies of War and Peace... That's a lot of lines. I further estimate that in the last four years, I have updated, added or removed around 200,000 of those lines of code (2.797 WaP), and although that's an estimate, it's on the low side if anything. The cold reality of these numbers is that the combined bulk of Pegasus Mail and Mercury is now beyond the ability of a single programmer to maintain and improve unless ways can be found to streamline the process as much as possible, and it's mainly this that I have been focusing on for the last few years. I have no intention of giving up on these programs that have been part of my life for so long, but I simply have to find ways of making them manageable. So what have I actually done so far? Here's a potted summary, by no means complete:
The last two are so important that they deserve some elaboration: the object framework (which I call "OIF" internally) is the basis for all the new code, and will allow third-party add-ons to be first-class citizens, with the same access to the internals of Pegasus Mail and Mercury as the programs themselves: this desire to open up the "guts" of the programs in a consistent, effective manner that made it easy for people other than me to create powerful new features for them was a major goal of the redevelopment, and required enormous effort, although I believe the results are really good. The new Message Store is an example of how difficult some of this stuff can end up being; I have been working on it for several years and in that time have ended up scrapping what I had done and restarting twice. The Message Store is the low-level code that implements the folders and messages you work with on a day-to-day basis, and I wanted something that provided a clear way forward to future capabilities without sacrificing compatibility with existing data; getting the right balance between past and future makes this process unusually difficult - had I been able to scrap everything and simply start from scratch, it would have been a far simpler process, but I value data compatibility above almost everything else: it is a matter of pride to me that a folder created by the DOS version of Pegasus Mail in 1990 is still readable by the latest Windows version in 2017, and will remain readable by v5, even though newer and vastly better folder formats will also be available. Hindsight, as the old saying goes, has 20-20 vision, and in hindsight, I would have to say that I could have handled the process of updating and modernizing my programs considerably better than I have. Had I realized when I started what I was getting myself into, I would certainly have tried to find a different strategy, some way of making the process cleaner, more compartmentalized, and more inclusive of my user community... But in the end, the code would still have had to be written, and it was always going to take a long time no matter how I approached it. I hope this rather over-wordy update (roughly 0.002 WaP in size) offers at least some explanation why the process of bringing out the next generation versions of Pegasus Mail and Mercury is taking such a long time: in offering it, I would also like to offer my thanks to my community of users, testers and supporters for their considerable patience and encouragement, and my hope that they will stick with me for just a little longer. I really do feel like there's light at the end now. All my very best to you all, -- David -- September 2015 My, my! Time flies when you're having fun, and sometimes even when you're not, it seems. The last time I updated this page, I was struggling with a nasty deadlock issue in Mercury; since that time, I have managed to fix that issue and bring out a public beta of Mercury v4.80, but the struggles didn't end there. The public beta exposed a number of other unrelated issues that it has taken time to fix, but we're finally there now, and Mercury/32 v4.80 is an official release, with a far greater new feature set than I ever intended it to have. I also believe it will be one of the best and most stable releases I have ever done: I am quite proud of it, and of the effort my test team has put in to it. You might ask, why has this taken so long? Why should it be such a struggle simply to bring out a new version of a well-established, reliable program? The answer is mainly 'history'. Mercury was originally written in 1993 as a special kind of program called an 'NLM', for a now largely forgotten network operating system called 'Novell NetWare'. I produced the first Windows version of Mercury in 1995, using quite a bit of code from the original NLM program. Since 1995, Mercury has grown and changed continually, as living software does, and has had a profusion of capabilities and features added to it that are far in excess of anything ever contemplated in the original code. The problem is that this accretion of features has been done on an underlying code platform that is now more than twenty years old, and the result has been an almost chaotic blend of code styles and practices, with workarounds for various versions of Windows thrown in for good measure. This means that it gets more and more difficult to maintain the program with each new version of Windows and each new feature that is added. About five years ago, the difficulty of this process reached an apex, where the time taken to add any new feature, even quite a small one, had to be offset by a disproportionately huge investment in time and effort in testing and validation. Since that time, I have ploughed enormous ongoing effort in rewriting and modernizing the code to remove complications, improve robustness and reliability, and to make it much more maintainable, but this is a huge job - more than 100,000 lines of code already, and still counting. Then there's the problem of backwards compatibility: this is the idea that a data file you created back in 1993 should continue to be usable by versions of the program released in 2015. I have always tried to make backwards compatibility a hallmark of the work I do - for example, a folder created in the original DOS version of Pegasus Mail in 1990 can still be read by both Pegasus Mail for Windows and the Mercury IMAP server in 2015. But these levels of backwards compatibility come at a price, and the older the program gets, the greater that price becomes. There is a huge amount of code and effort in both programs devoted to supporting data files and formats that many people probably no longer use. And of course, there's the size of these programs: they are now very large - each on its own is a full-sized commercial-quality software package, yet there's only me writing, maintaining and documenting them. To give you a point of reference, I have a friend who runs a successful company making software not much larger than mine, and he employs more than fifty programmers, graphic artists, documentation specialists, web developers and support staff. Of necessity, this imbalance of size versus capacity means that I can simply no longer work as fast or produce as many updates as I could when the programs were smaller and more manageable; it also means that I have to choose carefully the things on which I focus my efforts - and I have to be pragmatic about that: I will always choose to focus on things like reliability and real-world features over pretty interface skins because I believe those are the things that most benefit my users, even though neglecting the eye-candy aspect of the programs comes at a cost, since for many people that's all they really see when they're evaluating them. To me, it's one of the sadder facts of the modern world that many people are much more interested in how something looks than in how well it will serve their needs, but I guess that's just the way it is, and grizzling about it, while possibly therapeutic, is largely pointless. <grin> There is some good news though... Mercury v4.80 is much closer to version 5 than it is to version 4.7 - a lot of stuff is in there but not currently exposed, and as a result, I can assure you that you won't be seeing an untoward delay in getting the first v5 builds out once the dust has settled on this release. It's a slow process, and it has been very arduous, but the results are near, and I believe they will be good. November 2013 Each year, the Queen of England makes a seasonal broadcast to her people around Christmastime, in which she gives the folk a good old pep-talk. A few years ago (I think just after Lady Diana had been killed), she used a latin phrase to describe the year past - she referred to it as an "Annus Horribilis", or "Horrible year". Well, this year has been one of those for me. In a case of life imitating Monty Python, every absurd thing that could go wrong seems to have gone wrong this year, and I will not deny that at times I have been skirting the abyss of desperation here in Dunedin. The most lingering and destructive issue has been a nasty problem that somehow got introduced into Mercury at some point during the significant code rewrite process that was needed leading up to version 5. This problem manifests itself as occasional, sporadic deadlocks - situations where Mercury simply stops responding. It's not a crash as such, so neither the Mercury Loader nor the Windows Service Manager will restart it, and it just sits there doing nothing until someone restarts it manually. Of all the problems you could have in a piece of mission-critical software, this is just about the worst, so it was obvious that no release could occur until the problem was fixed. Unfortunately, the problem is intermittent - it might take anywhere from twelve hours to a week to manifest itself, which made tracking it down extremely difficult. In the end, to simplify the task of debugging, I moved forward a process I had hoped to defer for one more release - I ported the code from the ancient Borland compiler I was using to a much more modern version of Microsoft Visual Studio. Although this took some time, it has given me much better tools to deal with the problem and will, I hope, allow me to bring out the v4.80 release I had hoped to bring out in May before the end of the year. On the positive side, the delay caused by the deadlock problem has given plenty of time to test the completely rewritten TCP/IP code based on OpenSSL, and it's now working very well. A much-requested change, this will make Mercury's SSL support much more interoperable with other systems, and will allow you to use real SSL certificates issued by CAs. During the port to Visual C, I've also done a lot of work tidying up and modernizing the Mercury code - you may not realize it, but Mercury has been around for over twenty years now, and just as was the case with Pegasus Mail a couple of years ago, there was a lot of accumulated cleanup that really had to be done during the port. For Pegasus Mail: The recent revelations about the unprecedented intrusion by security services into material that most normal people would rightly believe to be private has affected me enormously - I have found the whole "breach of faith" aspect of the saga incredibly upsetting. As a result, I have made a conscious decision to redirect my current work efforts in Pegasus Mail towards increased security and privacy. I am now adding strong encryption facilities to the program, with the particular aim of making them so easy to use that there will be no reason not to do so. The new Contact Manager, already a pivotal feature for version 5, becomes even more important when seen in this light: I have now found and implemented a fully-encrypted version of the SQL database I use for the Contact Manager, meaning that even if someone steals your contact file, it will be useless to them. I have also heavily beefed-up the provisions for storing encryption-related settings and certificates in the Contact Manager. In time, I will also support encrypted folder formats in the program, and we will finally have a truly secure mail client. Other work in Pegasus Mail has involved completely replacing the spelling checker with a full multi-lingual version based on the same spelling checker as used in OpenOffice and Libre Office, completely rewriting the program's TCP/IP module to use OpenSSL and fixing a host of small bugs. We hope to have an early edition of Pegasus Mail v5 available at least for public preview early in the New Year. Finally, on a personal note, I feel I have to apologize for communicating poorly. It's kind of ironic that I got into this field because I was passionate about making it possible and easy for people to communicate with each other, but that I'm not actually that good at doing it myself. For the most part, it's simply a case of burn-out - I've been handling too much mail for too many years (nearly twenty-five years, in the case of Pegasus Mail), and I've reached a point where I rapidly become overwhelmed by even quite modest volume. This doesn't mean that my commitment to my community is any the less, though: I remain totally and utterly devoted to providing tools that fit the needs of my people, and to ensuring that they remain as relevant and capable as it is in my power to make them. March 2013 Well, it's been a long time between drinks, as they say. 2012 was a frustrating year in which I did a lot of work, but ended up without having a lot to show by way of releasable products. Things are rolling on, though, and I expect 2013 to be a year where many of the fruits of those labours reach the public view. Pegasus Mail The main focus for Pegasus Mail during 2012 was to replace the aging addressbook subsystem with something much, much better. The existing Pegasus Mail addressbook code is now very old - some of it dates from the time of the DOS version of the program - and it has reached the point where not only is it practically useless in terms of real functionality, but it is very difficult to maintain. The replacement I have developed for it is a fully-fledged Contact Manager, with an incredibly rich and extensive range of data attributes and capabilities. It will tie deeply into Pegasus Mail and will provide new functionality only dimly imagined in past versions of the program. It also showcases a lot of development I have done on user interface, both in terms of my philosophy towards UI, and the tools I have written to create it. I believe it has the best user interface experience I have ever produced, and that it will significantly expand the capabilities of the program as a whole. As a teaser, here's a simple screenshot of the current state of the Contact Manager UI in action.
Other developments include replacing the entire spelling checker subsystem with a completely internationalized version using standard dictionaries (over 80 languages available at last count), and improving the performance of network-related operations (such as POP3 and SMTP), including migrating to secure networking code based on OpenSSL. Mercury/32 Two things in particular have occupied my attention in Mercury development over 2012: the first is a migration to using the OpenSSL libraries in the program to handle secure communication. Moving to OpenSSL promises much higher levels of interoperability with other systems and makes it much easier to use "real" certificates (i.e, those issued by Certification Authorities). As part of this process, I have put a lot of time into making my TCP/IP code in general more robust and improving performance where possible. The second focus in Mercury/32 is the complete decoupling of the Mercury user interface from the worker code that actually handles mail. This has become increasingly important with more modern versions of Windows that do not allow service processes to expose any kind of UI, but was always going to be something I would have to address in any case. While a very large and complicated job, this work is progressing well, and several of the Mercury protocol modules are now converted to the split UI/worker process model. A side-effect of this work will be that tools people have been requesting for a long time, such as easy ways of managing mailing lists remotely, will become practical to write. Interfaces Probably one of the areas where the most work has happened in the last year is one that no user will ever directly see - the internal interfaces Pegasus Mail and Mercury use to communicate and manage data. I've been working towards a very highly-integrated object-based interface for a number of years, and the last year has seen this work finally come to fruition. The Mercury inter-process communication abilities required to separate the user interface from the worker processes are directly supplied by this new object-based interface, which is now a fixture in both programs. While my user community won't normally be aware of all the work that has gone into this, it was a crucial step in making the programs maintainable for the future, so it was pretty important stuff. MailStore MailStore, the replacement foldering management layer I've been working on for a long time now, has taken somewhat of a hiatus in 2012. The problem, as it often has been in the past, is the IMAP code for accessing remote mailboxes: this code has required a major rewrite, but is unfortunately based on internet standards that are very difficult, time-consuming and (dare I say it) soul-destroying to implement properly. I decided that other, more visible additions to the programs, such as the Contact Manager and the Mercury UI separation were more important in the short-to-medium term, so I have put MailStore on the back-burner while I work on those. Mailstore does, however, remain a key development goal for both programs, and will eventually find its way into production. There's a lot on the horizon, and a lot to look forward to this year. I'd like to take a moment to thank those of you who have stayed with me and have generously donated in support of Pegasus Mail. I'd also like to break with the reserved, almost shy lack of self-promotional sense that makes it very difficult for me to ask for support and say that donations from my users are increasingly important in times as hard as these; I appreciate any financial help you feel able to provide and hope as many of you as possible will find a way to offer your assistance. 30 November 2011: I realize that it's been a long time since the last entry on this page, and I apologize for that: the delay has come about through a combination of work overload and system migration. The work overload centres around the vast, long-term, ongoing overhaul of now-ancient code in both Pegasus Mail and Mercury, a process that will secure the future maintainability of both programs. In the course of this year, I have rewritten large swathes of the underlying code and have produced more than 50,000 lines of new code, including completely new IMAP client code for the new back-end message store, a full object interface that will be the underpinning of all future development, a comprehensive dynamic string library, and new code to use the industry-standard OpenSSL libraries for secured communications. I have also begun the laborious process of moving the Mercury code from the old Borland compiler I have used for so long to Microsoft Visual C: fortunately, having already done this in the larger and more difficult Pegasus Mail codebase, I expect to find the Mercury port easier and quicker. Clearly, then, there's a lot of work going on "behind the scenes": much of the time, this process feels like walking through treacle - the progress is slow, and it's frustrating having so little to show for the hundreds of hours of work I've done. The only consolation here is that the real benefits will start to flow quickly and clearly once I move to full development of Pegasus Mail v5.0 and Mercury v5.0. In the non-development area, I have now moved all work on Pegasus Mail and Mercury to Windows 7, and have migrated the development and maintenance of this web site to Adobe DreamWeaver, which should make it considerably easier to produce updates to the site and to this page more frequently. Moving development to Windows 7 allows me to consider innovations such as 64-bit versions of the programs, and installation options that more closely follow the Microsoft-mandated directory layouts for software. I have not yet personally tried the Windows 8 betas that Microsoft has released, but members of the test team who have tried them have not currently noticed any problems with either program in the desktop interface of the new version, and should any issues arise during the beta period for Windows 8, I am now in a good position to be able to address them. On the release front, Pegasus Mail v4.63 and Mercury/32 v4.74 are now in final testing. Both programs are maintenance releases that fix numerous small problems (Pegasus Mail has over 60 such fixes, Mercury around 40). They should be released around the first week of December. 20 February 2011: Pegasus Mail v4.61 has now been released. This is really an interim release – it falls between v4.52 and v5.0, but we felt it was important to bring it out to deal with problems in the way other programs are now insisting on formatting their HTML mail (essentially, sending the entire HTML message on a single unwrapped line... <grrr>). That said, there are also some powerful new capabilities in there, and the new look-and-feel gives an idea of what we are trying to develop for the program in the future. On the developer front, things have slowed down a little in the last month as we got v4.61 ready to release (releasing a new version is an extraordinarily difficult and time-consuming process, which is one reason why there are fewer releases these days than there once were), but the new contact manager code is now in its late development stages and should be going into testing shortly: I expect an initial public release containing the new contact manager and further UI improvements sometime in the next couple of months. Work on integrating OpenSSL is proceeding well, hampered mainly by the relative lack of documentation available for OpenSSL, but we expect to have it integrated into both Mercury and Pegasus Mail in the very near future. Finally, the new back-end message store for both programs (see below under the August 2010 entry for more on this) has taken a back seat to releases and other development for the last month or so, but will be back into full development now that v4.61 has been released. 2 November 2010: Progress continues on the new contacts manager for Pegasus Mail version 5. The new contact manager uses an SQL-based back-end for storage, and offers an incredibly rich range of attributes and options: it will exist alongside the now aging Pegasus Mail addressbook format, and while existing addressbooks will continue to work, there will also be options to import their contents into the new contact manager database. The new code has been designed heavily with external synchronization in mind, but at least initially, we will not be including any built-in synchronization clients: there is an extensive API that other developers can use to perform synchronization (down to the attribute level) though, and as real-world usage solidifies after release, we will evaluate what synchronization clients might be most appropriate to provide as standard options. We hope to release an initial version of Pegasus Mail version 5 with the new contact manager in place either late in December this year, or early in 2011. A subsequent version with the entirely-new message store I have been working on for over a year will follow at a later stage. I'm personally very excited about the new contact manager - I believe it could have an enormous impact on the way many people work in a world of e-mail overload. On the Mercury front, we have begun the process of integrating support for OpenSSL into all the Mercury protocol modules. This should provide more robust SSL support, which is becoming increasingly important as the Internet grows. We are also investigating what will be necessary for Mercury to exist in the IPV6 world - we do not expect this to be a large issue, and believe we can have IPV6 support in place well before most sites have any real need for it. Once the OpenSSL transition is complete, there will be a new release of Mercury with that code in place, probably in December 2010 or January 2011. 31 August 2010: Welcome to the new developer news page! I'm conscious that there can often be quite long gaps between releases for both Pegasus Mail and Mercury, so this page is aimed at keeping you informed of what we're working on during those gaps. The intention is to update the page once a month or so, or whenever there's a significant development in the programs. I have spent much of the last year working on the single biggest change to the programs since 1993 - specifically, a completely new message store that will be common to both programs. The Message Store is the "back end" of the program, responsible for organizing and controlling access to mailboxes, folders, messages and other mail-related data. It constitutes about a third of all the programming in Pegasus Mail, and about a quarter of the programming in Mercury, so it's easily the single biggest discrete chunk in both pieces of software. Having an industrial-strength message store that is common to both programs will hugely improve the reliability and maintainability of the whole system, and is a crucial part of the process of taking the programs into the future. As well as the mail store, Pegasus Mail now has an entirely new suite of beautiful user interface graphics by Sven Henze, the development of which was funded entirely by the Pegasus Mail Thousand - thank you all! I'm also working on a new contact manager to replace the aging address book format - the intention is that the contact manager will be as good as or better than anything else in the industry. Finally, Michael in der Wiesche has developed a new HTML renderer for Pegasus Mail that uses the rendering power of Internet Explorer to produce top-class HTML displays. The new renderer will be shipped with the next release as a built-in option - it will still be possible to use the existing renderer, but for higher-grade rendering, the IERenderer can be enabled with a single click. We expect to release an initial Pegasus Mail v5 version with these features before the end of the year, and another version with the new message store early in 2011. Mercury is undergoing extensive internal redevelopment, with the primary initial aim of separating the user interface from the core transports. The aim is to make the server run as a true Windows service all the time, and have configuration and console monitoring done either over HTTP or using a dedicated remote app. We're also redeveloping the way Mercury handles domains, to allow true multi-domain and MX domain support, and the most important long-term goal is to have comprehensive, well-integrated webmail as a core part of the program (this, in fact was the primary driver behind the message store redevelopment). That's all for now - watch out for more updates over the next few months. Cheers! -- David Harris --
[ Page modified 25 December 2019 | Content © David Harris |
