On the Way to Morphia 2.0

Morphia 2.0 is finally here. The path here was a bit more arduous than I expected back when it all began. I’d like to take a few minutes to explain some of that history before we get to the details of what’s new. My hope is that you’ll have a better understanding what went in to the work so you’ll have a clearer picture of why some of the decisions have been made. You can, of course, skip ahead to that section if you’d prefer but for those who are interested in some of history, come with me.

Some History

I first joined 10Gen (The MongoDB company!) in June of 2013 in order to pick up the maintenance of Morphia among other duties. It had been sitting dormant for some time and was in need of some focused attention. Quite a few issues had filled up the backlog so this was no inconsequential task. I was not the original developer so there were certain decisions I had to live that I might have done differenty. Over time, I made my own decisions that I might make differently today. There was a lot of legacy code that had grown organically over time and it became increasingly difficult to update and test this code.

Once I worked through enough of the backlog trying to clear out years-old issues, I started thinking about to fix this maintenance burden. The 3.0 series of the MongoDB Java Driver introduced the concept of codecs which seemed an ideal solution for what I was trying to do. Toward the end of my tenure at MongoDB I focused on building out what would eventually become the PojoCodec system with an eye to building the next generation of Morphia on that infrastructure. I was never able to complete that work as I took another position outside MongoDB and so moved on.

After I left MongoDB, Morphia didn’t see much attention again for some time. Unsure of its future inside MongoDB, I forked the repository and began work on my Morphia 2.0 vision. I figured I could file one massive pull request and boom Morphia 2.0 would be unleashed! It didn’t quite work out that way. I made some decisions early on in this fork that didn’t quite work out as well as I’d hoped. In the interim, I talked to MongoDB about their plans for Morphia and my interest in taking it on as a community project. MongoDB surprisingly agreed and Morphia was moved out of the MongoDB github organization to a new community organization. Morphia was officially “mine” to work on once more.

Once again, I found myself behind a backlog of issues to work through. I began to fix these issues as quickly as I could and to release more releases on the 1.x branch. Significant work went in to the 1.5.x releases in terms of both bug fixes and internal refactorings. Morphia was again seeing updates, which was great, but it destroyed any hopes of merging in my 2.0 work. So I decided to scrap it all.

Using what I’d learned in the abandoned 2.0 fork, I restarted with the 1.5.x branch and began applying lessons learned. In the end, the final result is much nicer and much more maintainable than either 1.x or my original fork. One of the lessons learned, however, was the value of simplicity and opinionated APIs. And that brings me to what you’re likely really interested in.

What’s New?

Morphia has lost some weight

The biggest lesson learned in my failed fork was that there were too many code paths doing too many similar things. So the biggest thing you’ll notice in 2.0 is a much slimmer API. There are far fewer overloads and far fewer variations on different operations. With fewer methods to choose from in the API (most notably on Datastore), the learning curve for Morphia should be much more friendly. Morphia simply funnels more work through fewer methods. Rather than overloads for many of the different ways to execute various options, there are just a handful of methods and all the variations are handled primarily through the use of Option classes. In general, these Options mimic the APIs offered via the driver so there should be little context switching for those familiar with those APIs in the driver.

Fluent No More?

One of the big selling points in prior versions of Morphia has been its fluent API particularly with the Query interface. This API, as it turns out, led to some difficult internal implementations and some serious obstacles to extending and enhancing the API. In 2.0, the operators part of a Query isn’t necessarily “fluent” any longer. Instead, a series of Filter references are used to define the query filter. (This was largely driven by complications in the implementation of compound queries using the and/or operators.) Surprisingly, this code is actually much less verbose in practice than in earlier versions.

What is more “fluent,” however, are the non-query operations. In 1.x, e.g., to do an update you’d have to create a Query, then create an UpdateOperations, and then pass both to the Datastore. That API has always felt awkward to me and as of 2.0, it is no more. Instead, you can now do something as simple as datatore.find(SomeClass.class).delete(). Other operations are also available but rather than requiring that awkward Datastore step, they are now all invoked off the Query.

Reworked and completed operator library

The aggregation API in 1.x has always been incomplete. It was actually intended as a prototype but was never marked as such so folks started using it and I couldn’t justifiably break those apps to make corrections. Truth be told, a good chunk of that API was written over the Atlantic Ocean in the middle of the night on my way to Devoxx. It was really was meant to be a rough cut.

Morphia 2.0 completes that job. It introduces a completely new API (in a new package to not break existing uses). In addition, it has complete coverage of all the operators supported in the server. (That’s not quite true as there’s one or two operators that don’t make any sense to use from client code that are not implemented.) Similarly, 1.x is missing a number of query filters and update operators which 2.0 now provides. There are automated tests now that attempt to ensure that when new operators are added, Morphia will get them soon after.

Compatibility and future evolution

There are a lot of changes under the hood that we haven’t even touched on. A more focused migration guide on the website will cover those in more detail. However, I’d like to cover one last major piece: compatibility. There’s a new query implementation in 2.0 that users will automatically pick up simply by upgrading. Should you need the legacy implementation (and in some cases you just might), you can configure that implementation using the legacy() method on MapperOptions to configure something that matches what you’d find in 1.5. Otherwise, you’ll get all new mapping options. If you’re upgrading an existing project, you’ll probably want to use the legacy settings until you understand the difference.

The new query and aggregation interfaces are currently in an experimental package. These are, however, completely safe to use. I am currently using them in a number of my own projects and they work quite nicely. By the time a 2.1 lands, I hope to have enough feedback and have made any tweaks to the API to promote them all to a “final” API. Any such changes now should be incremental barring any massive oversights and so far I have seen no reason to believe there are any. I know that this approach is slightly unusual but I’m hoping to avoid as much API wart lock in as I can. If you’re uncomfortable with this approach, the legacy query APIs are a completely viable option in the meantime.

I know this is a wordy post and if you’ve made it this far you have my thanks and admiration. This release represents of 3, really 4, years of planning, dreaming, and working like mad hours in to the night to make it happen. I am terribly excited about this release not only for how it’s turned out but also for all the plans I have to build on it. I do hope you’ll be as pleased with this release as I am. But, certainly, if you find something off, don’t hesitate to file an issue and we’ll figure it out together. Thank you so much for reading and thank you for being a Morphia user.