Burn-in testing Meteor with Deacon
Posted by Dave
On a few occasions in blog comments and on the Deacon Project mailing list, interested folks have asked just how many Deacon clients a Meteor server can handle. The discussions have mostly revolved around theoretical limits imposed by Meteor's codebase, or the Linux OS under which it runs. Rather than perpetuating the discussion on hearsay and conjecture, I decided to write up a little Meteor test suite and commit it as part of Deacon to enable some objective experimentation.
My test, which is located in the source tree's org.deacon.test package, performs some simplistic load testing of the Meteor server using a whole bunch of DeaconService instances. It runs in pure Java, so it can be run from your PC rather than an Android device. Along with some delays to prevent race conditions, the first test - aptly named the Simultaneous Subscribers Test - successively generates tens of thousands of Deacon instances, connects them all to the same channel on a single Meteor server, and measures the worst-case push notification turnaround times by sending pushes containing the system time in milliseconds as the payload. When the pushes are received, the payload times are compared with the system's new current time to provide a latency measurement. The Simultaneous Subscribers Test runs until the push latency reaches a pre-defined maximum (default: 1 second), or a maximum number of subscribers are created (default: 1 million).
I'm working on a second test to prod Meteor for its maximum achievable channel count. As subscribers are successively added on the client side, each will join a new channel. Test push messages - again containing CurrenTimeMillis() payload data - will be sent to each channel, and latency measured until a maximum latency or channel count is reached.
As far as the architecture required to support widely-adopted mobile applications goes, these tests are very simplistic. In my case, they measure latency over a LAN (which is good, as it reduces the influence of intermediary bottlenecks on the results) but all pushes terminate at a single machine (which is bad, as they must be crammed through a single network interface and CPU). A distributed test, however, would require far more effort, infrastructure and coordination - so the tests in this little suite offer a good first-order look at Meteor's capabilities when used as a mobile push server. Another caveat to these test results is the way latency is measured: while using the System's CurrentTimeMillis() value as a payload is clever, it inherently lumps together the latency of push client delivery with that of connecting to the channel controller and delivering the push in the first place. I believe this is responsible for many of the outliers in the chart below.
So how'd it do on my runs? Squeaking along on my 1.8GHz Celeron-powered server (an aging but trusty Dell Poweredge 600SC with 512MB of RAM), I was able to connect 32,768 simultaneous Meteor subscriptions with a maximum latency of about 0.7 seconds, which is where I had configured the test to stop. The file descriptor limit on this machine (/proc/sys/fs/file-max) is 65,536 with only 800-or-so open descriptors at idle, so I imagine I had a lot more headroom remaining. Last night, I added a charting library and re-ran the test, this time setting the stop value around 25,000 (I wanted it to complete before I woke up this morning). The results: 25,000 instances saw a maximum latency under half-a-second, with a typical latency between 40 and 100ms:
Simultaneous Subscribers Test Result - Click for much more detailed version
This chart shows the worst-case latency observed at each subscriber count. As you can see, the results are distributed largely bimodally between around 40ms and 100ms values, with higher outliers. The linearity of this chart seems to indicate that extrapolation to far more client instances would be entirely feasible.
My next step is to complete the channel count test as well as revamp the stop conditions for this test so that it can be run over longer periods and accumulate a much larger quantity of clients. Further, I'll downsample the chart data and add error bars, so that is conveys more meaningful data with fewer raw datapoints. Stay tuned!
[Image credit: Wikimedia Commons]
9 Million Reasons
Posted by Dave
Apparently it's been a good year for Sony: their Xperia line of Android phones has certainly been a contributing factor to a 39-million-Euro profit-before-taxes for 2010. As part of their recently-released 2010 financial report, they prominently announce the sale of over nine million Xperia devices last year. Thus far, not a single one of those 9 million phones (rooted-and-rom'd devices excepted, of course) runs any version of Android newer than 2.1. And if you read my previous post, you know what that means: no C2DM push notifications.
Those Xperia devices represent 9 million reasons for developers to carefully consider how they implement push in their applications. That isn't to say "use Deacon, we're better!" (because admittedly, by several metrics, we're not!) but rather to suggest that a silver-bullet strategy for push notifications leaves a huge potential market unserved. While certain apps could provide "push-disabled" versions for pre-2.2 devices, developers then risk offering disparate user experiences that result in polarized opinions of their product. Rather than falling back to polling or disabling push altogether, if a developer plans to maintain a source tree for pre-2.2 devices anyway, why not use Deacon as a fall-back push provider? Given that the same application server that feeds Google's C2DM backend could be easily adapted to act as a Meteor event controller in the case that a client connects from a pre-2.2 handset.
[Xperia image: Wikimedia Commons / Espen Irwing Swang]
Push Apps for All? Or just Some?
Posted by Dave
This past May, when Google unveiled version 2.2 of Android - codenamed Froyo - the Deacon team wrung out some commentary on the addition of built-in push capability. The gist? We think it's great that there are a few different ways to do push on Android, and we think Deacon will continue to be a good choice for many developers.
One point we made in our Froyo discussion was that many Android devices won't ever be able to use Google's C2DM push solution. A quick look at the latest adoption numbers from Google is evidence of that - nearly half of the millions of Android phones in use worldwide are running pre-2.2 operating systems. Deacon is a great way to push-enable apps on older devices - but what about devices that ought to have 2.2?
I'm referring, of course, to Samsung and T-Mobile's shenanigans of late. Yesterday, Androidspy broke a rumor from "a reliable source" that accused the two companies of withholding an otherwise-ready Froyo update for the wildly-popular Vibrant handset, in hopes of bolstering sales of the forthcoming Vibrant 4G+ model. Today, AndroidGuys followed up with some commentary which opines that such tactics - if true - are "downright wrong." Speaking only for myself, I couldn't agree more. Because not only does holding back the Vibrant's Froyo update negatively impact users, it also artificially constrains the user base of developers who push-enable their apps with C2DM.
There are well over 9 million Galaxy-S-based phones in users' hands worldwide. A sizable portion of these are doubtless the Vibrant flavor - and run Android's push-incapable "Eclair" variant. While Samsung has (indirectly, and in my opinion unconvincingly) denied the rumors, they've certainly given us another facet of the mobile application ecosystem to consider: should app developers hedge their technology choices?
In this case, integrating Deacon as a push platform not only enables compatibility with older devices that can't run Android 2.2, but also insulates app developers against manufacturers or carriers who won't keep their otherwise-capable handsets up to date. The distinction is between technical and political limitations - and Deacon can help with both.
I'll be the first to forgive Samsung and T-Mobile if they come clean and make the Vibrant's Froyo situation right - but we should still learn from the free lesson that this rumor offers up. Likewise, I'm a big fan of C2DM. I enjoy C2DM apps on my own handset, and every push option on Android has its optimal use cases - that's the beauty of a platform that offers choice. To that end, I'm also readying a post comparing Deacon and C2DM, and providing a rundown of the situations where each excels. Stay tuned...
Deacon Demo App: Prettied up and Ready to Rock
Posted by Dave
For quite a while (read: since October) I've been meaning to spend some time improving the Deacon demonstration application. What we had before was OK - it connected to the Deacon test server, showed some messages coming in, and generally indicated that something was happening behind the curtain. But I wanted to add more configurability, and more visibility into Deacon's state. Most importantly, I wanted to switch to running Deacon in a service, rather than within the app's user interface activity.
With the 1.0 release of the Deacon Demo app, you can choose the channel to subscribe to on the Deacon project's test server, as well as stop and re-start the Deacon instance. Icons and text indicate whether the Deacon instance itself is running, as well as whether the device has a network connection. If you navigate away from the Deacon Demo app's UI, the corresponding service will create a notification if any push messages are received that contain a prime number payload - clicking the notification in the pull-down list will bring you back to the Deacon Demo activity.
You can download this app directly from GitHub using the QR Code below. Make sure you have the "Unknown sources" checkbox ticked in your device's Application Settings!
Please log any problems you encounter, or feature requests, at the Deacon-Demo issue tracker. For a look at the source code or more project details, check out the Deacon Demo GitHub project page.