The Deacon Project Droid+Beacon – Open-Source Push Notifications for Android & Java


Deacon Sighting: Make Me Droid

Posted by Dave

It's always fun when Deacon users and developers write in to let me know they've successfully integrated Deacon into the project or product. I received one such note this week... The developers of Make Me Droid have incorporated Deacon into their application builder, allowing their users to incorporate Android push notifications into apps they build with the system. Push notifications are delivered using Make Me Droid's server, so there's no need for users to worry about setting up a notification backend.

We wish the team at Make Me Droid much success with their product launch, and welcome them to the Deacon community!

Filed under: Information, News No Comments

Burn-in testing Meteor with Deacon

Posted by Dave

On a few occasions in blog comments and on the Deacon Project mailing list, interested folks have asked just how many Deacon clients a Meteor server can handle. The discussions have mostly revolved around theoretical limits imposed by Meteor's codebase, or the Linux OS under which it runs. Rather than perpetuating the discussion on hearsay and conjecture, I decided to write up a little Meteor test suite and commit it as part of Deacon to enable some objective experimentation.

My test, which is located in the source tree's org.deacon.test package, performs some simplistic load testing of the Meteor server using a whole bunch of DeaconService instances. It runs in pure Java, so it can be run from your PC rather than an Android device. Along with some delays to prevent race conditions, the first test - aptly named the Simultaneous Subscribers Test - successively generates tens of thousands of Deacon instances, connects them all to the same channel on a single Meteor server, and measures the worst-case push notification turnaround times by sending pushes containing the system time in milliseconds as the payload. When the pushes are received, the payload times are compared with the system's new current time to provide a latency measurement. The Simultaneous Subscribers Test runs until the push latency reaches a pre-defined maximum (default: 1 second), or a maximum number of subscribers are created (default: 1 million).

I'm working on a second test to prod Meteor for its maximum achievable channel count. As subscribers are successively added on the client side, each will join a new channel. Test push messages - again containing CurrenTimeMillis() payload data - will be sent to each channel, and latency measured until a maximum latency or channel count is reached.

As far as the architecture required to support widely-adopted mobile applications goes, these tests are very simplistic. In my case, they measure latency over a LAN (which is good, as it reduces the influence of intermediary bottlenecks on the results) but all pushes terminate at a single machine (which is bad, as they must be crammed through a single network interface and CPU). A distributed test, however, would require far more effort, infrastructure and coordination - so the tests in this little suite offer a good first-order look at Meteor's capabilities when used as a mobile push server. Another caveat to these test results is the way latency is measured: while using the System's CurrentTimeMillis() value as a payload is clever, it inherently lumps together the latency of push client delivery with that of connecting to the channel controller and delivering the push in the first place. I believe this is responsible for many of the outliers in the chart below.

So how'd it do on my runs? Squeaking along on my 1.8GHz Celeron-powered server (an aging but trusty Dell Poweredge 600SC with 512MB of RAM), I was able to connect 32,768 simultaneous Meteor subscriptions with a maximum latency of about 0.7 seconds, which is where I had configured the test to stop. The file descriptor limit on this machine (/proc/sys/fs/file-max) is 65,536 with only 800-or-so open descriptors at idle, so I imagine I had a lot more headroom remaining. Last night, I added a charting library and re-ran the test, this time setting the stop value around 25,000 (I wanted it to complete before I woke up this morning). The results: 25,000 instances saw a maximum latency under half-a-second, with a typical latency between 40 and 100ms:

Simultaneous Subscribers Test Result - Click for much more detailed version

This chart shows the worst-case latency observed at each subscriber count. As you can see, the results are distributed largely bimodally between around 40ms and 100ms values, with higher outliers. The linearity of this chart seems to indicate that extrapolation to far more client instances would be entirely feasible.

My next step is to complete the channel count test as well as revamp the stop conditions for this test so that it can be run over longer periods and accumulate a much larger quantity of clients. Further, I'll downsample the chart data and add error bars, so that is conveys more meaningful data with fewer raw datapoints. Stay tuned!

[Image credit: Wikimedia Commons]

Filed under: Information 4 Comments

9 Million Reasons

Posted by Dave

Apparently it's been a good year for Sony: their Xperia line of Android phones has certainly been a contributing factor to a 39-million-Euro profit-before-taxes for 2010. As part of their recently-released 2010 financial report, they prominently announce the sale of over nine million Xperia devices last year. Thus far, not a single one of those 9 million phones (rooted-and-rom'd devices excepted, of course) runs any version of Android newer than 2.1. And if you read my previous post, you know what that means: no C2DM push notifications.

Those Xperia devices represent 9 million reasons for developers to carefully consider how they implement push in their applications. That isn't to say "use Deacon, we're better!" (because admittedly, by several metrics, we're not!) but rather to suggest that a silver-bullet strategy for push notifications leaves a huge potential market unserved. While certain apps could provide "push-disabled" versions for pre-2.2 devices, developers then risk offering disparate user experiences that result in polarized opinions of their product. Rather than falling back to polling or disabling push altogether, if a developer plans to maintain a source tree for pre-2.2 devices anyway, why not use Deacon as a fall-back push provider? Given that the same application server that feeds Google's C2DM backend could be easily adapted to act as a Meteor event controller in the case that a client connects from a pre-2.2 handset.

[Xperia image: Wikimedia Commons / Espen Irwing Swang]


Push Apps for All? Or just Some?

Posted by Dave

This past May, when Google unveiled version 2.2 of Android - codenamed Froyo - the Deacon team wrung out some commentary on the addition of built-in push capability. The gist? We think it's great that there are a few different ways to do push on Android, and we think Deacon will continue to be a good choice for many developers.

One point we made in our Froyo discussion was that many Android devices won't ever be able to use Google's C2DM push solution. A quick look at the latest adoption numbers from Google is evidence of that - nearly half of the millions of Android phones in use worldwide are running pre-2.2 operating systems. Deacon is a great way to push-enable apps on older devices - but what about devices that ought to have 2.2?

I'm referring, of course, to Samsung and T-Mobile's shenanigans of late. Yesterday, Androidspy broke a rumor from "a reliable source" that accused the two companies of withholding an otherwise-ready Froyo update for the wildly-popular Vibrant handset, in hopes of bolstering sales of the forthcoming Vibrant 4G+ model. Today, AndroidGuys followed up with some commentary which opines that such tactics - if true - are "downright wrong." Speaking only for myself, I couldn't agree more. Because not only does holding back the Vibrant's Froyo update negatively impact users, it also artificially constrains the user base of developers who push-enable their apps with C2DM.

There are well over 9 million Galaxy-S-based phones in users' hands worldwide. A sizable portion of these are doubtless the Vibrant flavor - and run Android's push-incapable "Eclair" variant. While Samsung has (indirectly, and in my opinion unconvincingly) denied the rumors, they've certainly given us another facet of the mobile application ecosystem to consider: should app developers hedge their technology choices?

In this case, integrating Deacon as a push platform not only enables compatibility with older devices that can't run Android 2.2, but also insulates app developers against manufacturers or carriers who won't keep their otherwise-capable handsets up to date. The distinction is between technical and political limitations - and Deacon can help with both.

I'll be the first to forgive Samsung and T-Mobile if they come clean and make the Vibrant's Froyo situation right - but we should still learn from the free lesson that this rumor offers up. Likewise, I'm a big fan of C2DM. I enjoy C2DM apps on my own handset, and every push option on Android has its optimal use cases - that's the beauty of a platform that offers choice. To that end, I'm also readying a post comparing Deacon and C2DM, and providing a rundown of the situations where each excels. Stay tuned...


Testing Deacon on Devices

Posted by Dave

Around this time of the year, it's not uncommon (in many Western nations, at least) to hear songs about a certain saint of Christmas who's "making a list, and checking it twice". The Deacon Project has kicked off a list of its own, but rather than keeping track of kids' shenanigans, this one tracks how well various Android devices play with the Deacon push notifications library. Without further ado...

The Deacon Device Testing List

The goal of the Device Testing List is to catalog the results of developers testing Deacon on their devices. There are a lot of Android devices on the market, in different form factors, with different software versions and manufacturer/carrier customizations. Deacon is designed to avoid dependence on any one device's capabilities or OS version - a strong ally in the fight against fragmentation. Nonetheless, the Deacon team can't possibly test the library in all the different hardware/software environments where it might be used. Such testing can reveal bugs and help improve Deacon - so I'm hopeful that those who are running Deacon-powered apps on the spectrum of Android devices will log their results on the Device Testing List.

Keep an eye on the list as we develop the format and add devices. So far, only the devices that I have personally tested appear - but I hope that the Deacon community changes that quickly!

Note: As far as I can tell, you just need a GitHub account to edit the Deacon Wiki, which hosts the Device Testing List. If anyone has trouble making edits, please let me know by comments or e-mail!

Filed under: Information 2 Comments

Wormburner: C2DM comes to Deacon

Posted by Dave

About once a year, my wife and I go to a driving range and remind ourselves of all those golf muscles that we don't use the other 364 days of the year. Inevitably, my first few swings result in nothing but a "whiff", and perhaps the ball falls off the tee. Then I usually manage to bounce a ball along the ground for a few dozen yards, before I finally manage to connect with one.

That's about where C2DM compatibility is with Deacon right now. The last few evenings I've been reading about C2DM development and hacking away at a little code for it. A new branch in the repository, c2dm-compat, is the temporary home for the results of those efforts. If you're interested in this facet of Deacon, I hope you'll check out the branch - or even the commit - and offer any wisdom or suggestions you may have in the form of comments. I've never written a C2DM-powered app, nor do I have time to set up an AppEngine backend for testing, so I'm pretty much working off the documentation and whatever nuggets of wisdom can be gleaned from the Chrome-to-phone example.

If you're a GitHub user, please make any comments or suggestions right on the applicable commit page (for now, that's commit f334e13, but that will change). If you're not on GitHub, consider the comment form for this post your "suggestion box"! To make C2DM compatibility in Deacon work well, I'm going to need your help.

And in the mean time, maybe I'll spring for a golf lesson or two.

Filed under: Information No Comments

Push Opportunity: “Push as the new Premium”

Posted by Dave

It's no secret that the newspaper business is hurting these days - subscription rates and readership of "dead tree" newspapers have been on the decline since the mid-1990s, and some companies are rethinking the decision to provide free online versions of their content. As the population ages and society becomes more-connected, people continue to find and rely on new ways to keep informed, and new delivery channels. While my parents subscribed to our local newspaper for decades, my wife and I have never had a daily paper delivered to our home.

If there's one thing technology such as web delivery and smart phones should catalyze, it's the development of novel business models around content delivery, monetization and specialization. News can already be more customized than ever, whether along topical, geographic or other special-interest lines. It can be delivered via more channels, and the rise of smartphones, eBook readers and MIDs has only expanded the means by which we consume journalistic content. At the same time, all this choice and connectivity has come with a steep price - news outlets still need to pay to keep the lights on, and in recent years it's gotten much harder to charge for the value they generate.

There's a key differentiator that smartphones, and their near-ubiquitous connectivity, can offer: push notifications. (How'd you guess?!) Just as web browsers supplanted newsprint for those in search of digital delivery, push notifications can offer a more real-time alternative. In 1995, your local newspaper could bring you breaking news the morning after it happened. In 2000, they published it to their web site, and you found out if you happened to point your browser there. 2005 brought the rise of RSS feeds, letting software take care of all that repetitive polling for you - but feeds are still a little too geeky for mainstream. Today, however, mobile apps and push technology offer a disruptive opportunity to actually reverse the pattern of consumption.

So, newspapers of the world - do we have your attention? Turning off your online editions, choking back your feeds or continuing to rely on banner ad revenue are not the ways to stay relevant in today's digital marketplace. Offering real-time, pushed news that's tailored to users' unique interests and gets them informed before other news outlets can? That just might set you apart.

[Image: Sanja Gjenero, used under license]


Push Topics: Security

Posted by Dave

Note: This post is part of a series on the Deacon Blog that will address topics in implementing Push Notification systems. Naturally, this coverage is oriented toward the Deacon Project, but the topics and issues discussed are often widely applicable.

"Software security" - it's been a hot topic in the mobile phone world (or at least my feed reader) lately, what with multiple iPad and iPhone related security snafus at AT&T, and the release of the first major-label security software for Android phones by Norton/Symantec. With lots at stake in the mobile world - sensitive little things like GPS locations, contact lists and phone records - security should be a consideration in any mobile computing endeavor. Push notifications are certainly no exception.


Bass, Clements and Kazman define a system's security as it's "ability to resist unauthorized usage while still providing its services to legitimate users." [1] We can decompose this requirement into two equally-important areas: determining which users are "legitimate", and restricting access to only those users. In push notification systems (and those based on Meteor in particular), there are also two types of users: push subscribers (those that receive notifications) and channel controllers (those that send notifications). Last in the rundown of areas to consider comes data transport: push notifications can traverse untrusted networks as they are generated, sent and received.

But before we dive into the security mechanisms that can be used with Deacon and Meteor, there's something important to keep in mind. For efficiency's sake, remember that push notifications should be as small as possible to minimize the load on the server. This is Google's advice when using Android's C2DM system; a notification's payload should only contain a "tickler" for the receiving app (such as an ID number or URL), which should then handle subsequent retrievals or processing after it receives the push. To enforce this behavior, Google limits C2DM payloads to 1024 Bytes - no such limit exists in Deacon, but it's still a good idea to follow the "tickler" approach.

So what can be done to secure push notification systems? It boils down to three relatively-simple practices: don't accept subscriptions from unauthorized users, use secure connections for channel controllers, and don't send data payloads that require secure transport. We'll take a look at all three after the jump...


Quick Update: Beta Progress

Posted by Dave

For every time you feel like you've covered a lot of ground in software development, it seems like there are often other times when progress grinds down. We bit off a big chunk with the Alpha release of Deacon, and achieved a lot of functionality in just a few weeks' time. But without feedback, it's hard to maintain that cadence.

If you've tested the Deacon Push Notifications library, or are considering giving it a try, here's how you can help:

As a reminder, Deacon isn't just for Android apps! The core of Deacon, the DeaconService class, is pure Java and can be used in any Java application. We then use a wrapper around the DeaconService class to provide Android-specific functionality (such as network connectivity detection or context awareness).

Of course, our plea for assistance with automated testing is still open - feel free to comment here on the blog, or head over to Stack Overflow and add your two cents there...please?

Filed under: Information 2 Comments

Countdown to Beta … little help?

Posted by Dave

According to the Deacon project roadmap, tomorrow is supposed to be the day we release our Beta edition. The library, at least, is feature-complete for Beta at this point - mostly because we identified very few new features between Alpha and Beta releases, choosing instead to focus on increasing the robustness and eliminating (sometimes preemptively) bugs.

Source: Wikimedia commons (public domain)

What's missing is testing. We haven't got much expertise in the realm of testing Android applications - in particular, because there are a lot of different conditions that Deacon needs to be subjected to in order to test out its error-handling capacity, we'd really like to automate this testing. F8'ing the emulator over and over gets old very quickly! In particular, we'd like to be able to write tests that automatically toggle the emulator's (or a real device's) network connection off and on, and switch between 3G and WiFi connections. So, we turn to you, the dev community - can anyone recommend a clean way to do this without resorting to hacking around with airplane mode?

If you can help, please leave a comment on this post, check out the corresponding open question at Stack Overflow, or hit us up on the Deacon project mailing list! We'd offer you a free copy of Deacon, but since that's already open-source, the best we can offer is to buy you a beer if we should ever meet (say, at Google IO 2011?)...

As for that Beta release - well, there aren't any giant nagging to-dos holding it back, but by the same token little has changed since Alpha. It would certainly make a nice 30th birthday present for me, but I feel it should offer a bit more value. Hanging out the "Beta" flag might encourage more developers to test out Deacon, which would hopefully garner some feedback and even more opportunities for us to improve the library. For tonight, I think I'll sleep on it...