New Mailing List
Posted by Dave
Back in October, a multi-disk failure at our hosting provider caused the un-recoverable loss of the Deacon Project mailing list... Unfortunately, the list archives (which contained some useful technical info) and the member list fell victim to Mr. Murphy. There have been a few support requests since then, mostly handled in the comments here, or (more appropriately) on the Deacon Issue Tracker. This has been fine, but I wanted to move back to having a central place to collaborate, ask questions, and discuss Deacon.
Since GitHub, despite being awesome in almost every other conceivable way, doesn't seem interested in adding any manner of forum or Q&A venue, I decided to set up a Google Group for this purpose. You can find it (and join it!) here:
http://groups.google.com/group/deaconproject
Links will be updated momentarily...
New Deacon Test Server – Live!
Posted by Dave
Happy 2012, everyone! A few months ago, the generous folks at Attendit.se donated a virtual server for Deacon testing. As of today, that server is live, running the standard set of Meteor test channels for Deacon clients.
The server address is: test.deaconproject.org (83.140.103.26)
The available channels are:
- 2sec
- 10sec
- 1min
- 10min
Deacon Mailing List: Nuked
Posted by Dave
Bad news to report, folks: Apparently, the hosting provider where the Deacon project's blog and mailing list are served experienced a triple-disk failure on their RAID-6 array. Beginning about September 28th, the Deacon Mailing List archives and membership data were lost. As you can see, the blog is just fine...
I am working on finding an alternate provider for the Deacon mailing list; for now, the blog will stay on its current host, but I would like to explore options for porting it to GitHub Pages...
Update 2011-10-12: I've pulled the Deacon mailing list down from host's mailman server. The redirect for existing links will be broken for a while - I'll post an update here if/when I set up a new mailing list. Since it saw very little use, either for Q&A or planning discussions, I'm debating whether to set up something new at all. In the absence of any sort of discussion group feature from GitHub, I'm leaning toward Google Groups. If you have a strong desire to see the Deacon mailing list resurrected (or any opinion on the matter) feel free to sound off in the comments...
Updating: Deacon and how we build it
Posted by Dave
Thanks to the testing efforts of GitHub user Bonna and our contributors at Attendit, we've uncovered a somewhat nefarious bug between Deacon and Android that leaves a parent app hanging if Deacon's connection "goes dark" without an explicit disconnect event. While this isn't the sort of event you see with emulators and comfy lab environments, it does happen out in the real world of fluctuating coverage and unpredictable 3G/4G/WiFi transitions. Thanks to some debug output from Bonna's gingerbread device, we think we've got it resolved - a fix is currently being tested in the feature-pingcheck branch, and will be merged into master if it resolves the issue.
Speaking of branches, there's something else I've been working on. Ever since reading through Vincent Driessen's nothing-short-of-stellar Git branching-model workflow write-up, I've been looking forward to using it on Deacon. After spending some time with biz-partner Pat sand-boxing around with Vincent's equally-awesome git-flow extensions and Justin Hileman's git-flow autocompletion, we're hooked. I'm already using git-flow with Deacon on my local development box, and when we're ready to merge in the bug fix I described earlier, I'll push the new branch structure to the github repository. This also means that the default branch on github will change to develop, and that master will only hold release snapshots that are used to build the .jar files found in the downloads section.
If you're using Git for version control or SCM, I definitely recommend you check out git-flow. Git is extremely flexible, and imposes very little structure on branching and merging - which is part of what makes it so powerful. But with no tool-imposed discipline, a repository with several contributors can quickly become a rat's nest of branch clutter and inconsistent tags. git-flow imposes a minimal amount of discipline that can really clean up a project's repository, and fits nicely and unobtrusively into just about any software development process. Now...if only we could improve people's commit messages!
The Deacon Project: 1 Year Later
Posted by Dave
One year ago, I was working through my masters degree in software engineering. Google had not yet introduced Android 2.2 "Froyo" or cloud-to-device messaging (C2DM), and I had only purchased my first Android device (a Motorola Droid) a few months prior. Not only had Google not announced the "Gingerbread" or "Honeycomb" variants of Android - people had barely started guessing which confections would mascot these coming releases! It was amidst this zeitgeist that the Deacon Project sparked to life, late in the evening on April 16th, 2010, born of my desire to see equity in push capabilities between Android devices and their sundry fruit-themed competition.
Before the Deacon team reached our Alpha release, Google introduced C2DM with the arrival of Froyo. The team opted to press on, to offer developers an independent push alternative where they could control the entire pipeline and support older devices. At a proof of concept level, I believe the team achieved that goal. We got the attention of many app developers, peaking at nearly 2000 web page views per month. At one point, we were even offered a sponsorship; it would later fall through, but not without injecting a healthy dose of enthusiasm.
Unfortunately, we're learning that the cost of maintaining an open source project isn't zero. Domain names and hosting aren't free, coding and testing and responding to e-mails consume precious time, and motivating participation isn't easy. Thus far, our Pledgie button has gone unused, and adoption hasn't been strong enough to give the project much self-sustaining momentum. There have been many projects that flourish with the efforts of just a handful of curators (dcraw, jhead and dnsmasq come to mind), but strong uptake and personal usefulness are often requisite motivators for ongoing development. Frankly speaking, that simply hasn't happened here - and as a result, it's been difficult to attract contributions in the form of code or testing, and correspondingly difficult to achieve a self-sustaining critical mass.
I still believe that The Deacon Project could take off. It offers a value proposition - complete infrastructure control and support for all Android devices - that's beneficial for many mobile-app business models. For now, though, I will be scaling back my efforts to test and polish the Deacon library to free up time for some other projects. Of course, I'll be happy to continue answering questions via the mailing list, making minor improvements in the codebase, and watching the issue trackers. I'd also be thrilled (honored, even!) to receive code contributions in the form of pull requests and patches - and interested parties can certainly inquire about direct commit access. If you or your company really wants to see Deacon grow and mature, please contact me - with the right interest, conditions and support, there is always the potential to go further.
Quick Update: Capacity Testing
Posted by Dave
Another late-night coding session, another learning experience... It seems that the Linux per-process open file descriptor limit that had me a bit concerned has a bark that's worse than its bite:
dave@server:~$ ps aux | grep meteord
meteor 10613 76.2 6.6 11156 8356 pts/1 R Feb23 13:11 meteord daemon
ddr4179 11732 0.0 0.6 3004 768 pts/2 R+ 00:00 0:00 grep meteord
dave@server:~$ ulimit -n
1024
dave@server:~$ sudo lsof -p 10613 | wc -l
1884
As it turns out, some limits in the meteord.conf file were preventing persistent connections from many of the Deacon instances in my previous test, and so only a limited number were able to hold a connection at any given time. I've fixed that now, and the tests seem to be running a bit more smoothly. More results as I get them...
Testing: Epilogue to Chapter 1
Posted by Dave
Outside the delightful task of cleaning and organizing our basement, this weekend offered some prime coding time - and I spent it chewing through a new and improved Deacon+Meteor test suite.
Unfortunately, the results have been more cause for confusion than celebration. The new suite uses a pair of dedicated machines - one server, one client - on an isolated network with PTP-synchronized clocks and a socket-based side channel for control. It successfully addresses the error introduced by including push send time in the round-trip duration, but exposes many more weaknesses in my previous tests.
While I didn't seem to encounter this limit before, I'm now having great difficulty getting either the client or server machines past 1,024 connections - an upper-bound imposed by Linux's per-process file descriptor limit. This limit is configurable, but still seems to be leading me on a merry chase. I also discovered that the previous test didn't account for Deacon instances that weren't actually connected to the server - so I've added some extra accessors and state tracking to Deacon to prevent the test from advancing before all the clients are ready.
I'm still making progress - so stay tuned and hopefully I'll have some better news to share soon. In the mean time, suggestions are always welcome via the comments or mailing list...
Heads-up: Meteor test server downtime
Posted by Dave
Just a quick note for Deacon developers: If you're using the test server, there may be some brief downtime this coming Thursday, between 1-6PM Eastern Time. I am switching Internet providers, and this requires redirecting "home.daverea.com" to the new modem's IP address.
Thanks for your patience!
Help Wanted: Bayeux Hacker!
Posted by Dave
You wouldn't know it to watch theis blog and the Deacon Repository lately, but there has been a lot happening with the Deacon project behind the scenes.
For one thing, I've been working on a complete re-architecting of the Deacon library to better support extensible application-side interfaces, and the addition of new push protocols (and servers) on the upstream side. Which brings me to the subject of today's post... If you're familiar with the Bayeux Protocol (or want to get familiar with it!) and know some Java, the Deacon Project could use your help. In the coming weeks, we'll be looking to implement the Bayeux protocol in Deacon for compatibility with the Cometd, Glassfish and Jetty servers.
If you're interested in helping write the Deacon Bayeux client, please hit us up in the comments!
There's more to come, too - I've been collaborating with multiple app developers who are integrating Deacon, and we've been subjecting Deacon to some much-needed testing and tweaking. Watch for some fresh documentation (in the Wiki) on setting up a Meteor/Deacon system end-to-end, and changes in the codebase and API that will bring enhanced extensibility and better cohesion.
[Image: Kandyjaxx via Flickr, CC licensed]
Back up and running
Posted by Dave
Thinking back on the last month or so, I'm reminded of a scene from the film Batman Begins, after the destruction of Gotham is thwarted and the camera pans across the newly-rendered wreckage. My office has seemed a bit like this scene recently!
A week or so after school finished, the [admittedly-aftermarket] solid state disk in my trusty HP laptop went haywire, taking my development environment (which was about as comfortable as my favorite pair of jeans) with it. In order to get back up and running - and back to work at my new job - I replaced the hard drive and band-aided a new system together. But a long-term solution was needed, so I set about spec'ing a new machine.
A few weeks, a few out-of-town trips, and a few hours' worth of work later, my laptop is back to 100% and getting used to sharing my work table with a speedy new desktop. And as of tonight, I've got the Deacon project loaded back up and running in Eclipse. I committed a few documentation tweaks tonight, while the experience of ridding my JUnit console of "Java.Language.RuntimeException: Stub!" was fresh in my mind. Keep an eye out for more soon!