In a previous blog post, I discussed the idea of a virtual scripting layer that operated over the Sensorpedia data sources. Each data source, instead of just exposing links to data, will also expose a data-oriented programming interface. Users will be able to write and upload scripts that can transform raw data and communicate with external applications. By associating scripts with tags, users will be able to easily share and incorporate functionality into a data source by applying the appropriate tag to the data source. So, for example, if a user has associated an HTML parsing script with the tag HTML, other users can benefit by applying the same tag to their data source.

So what will these scripts actually look like? Although I still haven’t quite decided on the final syntax,  I’d like users to write these scripts in multiple languages. That way users comfortable with Java can program in Java while trendier users can write these scripts in Ruby. One way we can accomplish this is by exploiting the many languages that are available for the Java runtime (Java, JRuby, Jython, Clojure, etc.). The only thing we’ll impose is some minor syntax additions that are shared across all languages that

define the data-events the script can handle. For now, the scripts kind of look like this:

script ScriptName

dep “DependantData”

event “DependantData”
// Language-specific code

end

Each script defines a set of data-event methods that are invoked whenever data with the appropriate name is published (a publication may be associated with some data value). Data can be published via several sources, including from the network (so that scripts can react to external tools), timer mechanisms (for periodic sampling), or from

other methods. Once invoked, methods can, in turn, publish other data elements and thus trigger additional methods. Since each script works independently and are loosely coupled, the user will be able to load new scripts without affecting or even knowing about the internal operation of other scripts.

Although there’s still a long way to go with respect to both the specification and implementation, enough has been implemented that  I’ve been able to rewrite the Tables demo using this new framework. Instead of hard-wiring Tables to interact with the DASMet tower webpages, Tables interacts with a set of scripts I wrote associated with the Tables and DASMet tags. These scripts are then grouped together as a virtual sensor node. Tables, in turn, communicates with these virtual sensor nodes over the internet using a simple text-based protocol (see Figure). Unlike the old demo, Tables can now interact with any data source (besides DASMet) that apply the Tables tag and includes a script that exposes sensor data.

Obviously this short description is not enough for users to begin writing their own scripts, but hopefully it should give you an idea of where I’d like to go with this. There’s still a lot of work to do, including better session support and better integration with the Sensorpedia database.  Also, besides evolving the final syntax, users can only write scripts using Java. Once I integrate some other language support, I’ll post another blog entry explaining the more technical details of how these things are written, loaded, and executed.

As we move Sensorpedia from a limited beta (with a sneak peek) to an open beta, we have an important decision to make. We are currently evaluating multiple authentication options for Sensorpedia. Some of the options we are currently evaluating include:

• OpenID
• Google Friend Connect
• Facebook Connect
• Twitter oAuth

Each of these options has pros and cons. We need to consider both technical issues (ease of implementation, robustness, networking/firewall limitations) and functionality (features, user base, flexibility). Our original plan for Sensorpedia was to simply use OpenID to offload the authentication process by supporting a number of (all?) OpenID providers. That is valuable in and of itself, but we’d still have to develop and maintain all social networking functionality ourselves. That certainly offers a lot of flexibility, but it’s hard to ignore the benefits one gains from leveraging the existing social networking capabilities of APIs from companies like Google, Facebook, and Twitter. Each of these company’s offerings have some real strengths. Google Friend Connect is easy to use and has a wide variety of widgets available. It also doesn’t tie you to a specific social network. Facebook Connect is attractive because of its size (currently 300+ million users!) and you immediately get the benefit of users’ existing social graphs for sharing and managing groups for access control requirements. Twitter is also nice because of its popularity, asymmetrical connections, and because it’s used more for non-personal communication (which probably fits Sensorpedia’s user base better). The situation is further complicated now that Google and Facebook both support OpenID and Facebook even allows you to log in with your GMail credentials.

So which path should we take? We need a solution that we can implement quickly and also gives our users the greatest set of features for sharing information, managing their social graph, and supporting data mashups. We also want to keep in mind the desire to open source the Sensorpedia software and make it available to run within an enterprise and on secure networks. (Think Wikipedia / Intellipedia.) Would the social networking functionality provided by Google, Facebook, or Twitter have to be reimplemented anyway for such scenarios? These are all issues that we are discussing internally on the Sensorpedia team. We’d love to get your input as well. Please share your thoughts in the comments or get in touch with us on Twitter, Facebook, or LinkedIN.

As we move forward in this area, we’ll begin consolidating our discussion and documentation on the Sensorpedia Developers Wiki.

We value your input and look forward to hearing from you!

This summer I worked on creating an iPhone application for Sensorpedia. The goal of the project was to develop a software application that can be run on a mobile computing device and used to transmit distributed data readings from citizens and first responders in near-real time. This application will permit citizens to rapidly report event information, such as disaster occurrences, security anomalies, and accounts of emergencies. It will allow field users to contribute to Sensorpedia. It is currently in transition from the simulator to the hardware. Here is a presentation on the application (view full screen to see the presenter text):

Here is the poster I presented about it to the 2009 ORNL summer student poster session on August 5, 2009:

Thank you to David Resseguie for all of your help and ever-willing guidance, Bryan Gorman and the rest of CSED for being a great division to work with, ORNL and ORISE for allowing me to come back and live among the other summer students, and thank you to those funding Sensorpedia for making this sort of summer educational program possible. It has been a great summer; I am glad to have learned a host of new technical skills and am happy that I had the opportunity to put them to use for Sensorpedia.

Well, it’s around the beginning of August and it’s time to head out back to our universities and hometowns. First of all, I’ve got to say that I’ve had a blast working and learning with these guys at ORNL! I started this internship with knowledge in C++ alone and as a result ended with skills in HTML, XML, CSS, JavaScript, jQuery, and AJAX.

Throughout the summer, I have registered Tennessee Department of Transportation traffic cameras and United States Geographic Survey river level sensors to Sensorpedia. I’ve also completed the basic operations of my third party application entitled Sensorpedia H2O! I have learned a lot of programming techniques and also got hands-on  experience in the work industry which is priceless.

So, in all, I loved my experience at ORNL with the Sensorpedia team and I’m really going to miss all the guys! And it’s a wrap! –Brandon Rives
(more…)

It’s August, and the time of interns at Sensorpedia for the summer is coming to an end.  This internship has been a fantastic experience in every possible way, and has helped me develop professionally and technically, to say nothing of the immense and invaluable knowledge of presentation and communication techniques that David has imparted on us.  My sincerest thanks go out to all who have funded us and made this summer possible.

The Sensorpedia Python Library

But let’s talk about Sensorpedia, shall we?  This summer, I’ve focused mainly on the Python library, which while far from really being complete, is now stable and creating feeds.  The Python library provides a “Pythonic” frontend to Sensorpedia, such that registration feeds that are large or that need to be dynamically generated can be generated with a more intuitive and simple programming interface, without having to know anything at all about ATOM or XML.  The library is already being used in several projects and sensor feeds:

• Brandon Zachary’s air quality sensors from the EPA
• Chris Tomkins-Tinch’s iPhone app ("We too are sensors.")
• A feed generated by Chris from Harvard’s Citysense
• ~2000 ICAO weather sensors (updated with current data readings once every hour)
• Several hundred NOAA buoy sensors located offshore of various American coastlines

Feedback from some of these feeds and projects is already helping me decide what the next iteration of the library will look like.  We’re hoping to open source the library and other parts of Sensorpedia such that  people not necessarily associated with the lab can contribute as their interests dictate.  I’m hoping that in the future, regardless of my affiliation with the lab, I’ll be able to continue contributing to this library as a personal project in conjunction with some of my related projects.

As of this writing, the current version of the library is 0.2.3.  You’ll note that we’re nowhere near 1.0 yet, and this is to indicate that the library, as with the rest of Sensorpedia, is still very much in progress.  The interface can (and probably will) change quite a bit as the development cycle renews itself.  That being said, I consider the library to be very stable under normal operation, and would put it forward as a painless way to generate Sensorpedia registration feeds programmatically.

The current version of the library can be found here.

Today I’d like to introduce the first iteration of a python library written to ease the process of generating ATOM feeds for Sensorpedia.  With spylib, no knowledge of XML or ATOM is required to publish a sensor group, though an understanding of the various tags and what Sensorpedia wants to do with each of them is advised. (The definitive source of this information can be found in the Sensorpedia API.)

Creating a sensor registration document is easy as Py:

myfeed = sensor.SpGroup("efa6e019-eca4-4fd6-a37b-c57ad2b1f441")
myfeed.title = "Cool Sensors near Akron, OH"
myfeed.georss_point = [41.090737,-81.496582]
 
mysensor = sensor.SpSensor("efa6e019-eca4-4fd6-a37b-c57ad2bFALLS", title = "Cuyahoga Falls, OH")
mysensor.georss_point = [41.166441,-81.536858]
 
myfeed.addsensor(mysensor)
atomtext = myfeed.toprettyxml()

Some Quick Links:

• Project Home Page
• API Reference

In this blog post, we’ll discuss the rationale behind the library, and provide a tutorial.
(more…)

The Sensorpedia API Documentation is now viewable for non-BETA testers as well. If you’d like to edit pages or leave comments, you’ll still need to be an official “collaborator”. If you’d like to be added as a collaborator, please let us know and we’ll add you to the list and give you further instructions.

The Sensorpedia development team is happy to announce that a draft version of the Sensorpedia Application Programming Interface (API) documentation is now available for BETA testers! The document describes our use of the Atom Syndication Format as a basis for the Sensorpedia RESTful web services. The API is designed to provide an easy mechanism for interfacing autonomous and incompatible sensor data to existing and future enterprise applications. To utilize the Sensorpedia framework, sensors and sensor systems are registered with Sensorpedia using Sensorpedia’s Atom-based API. Registered sensors may communicate and publish data with either standards-based or proprietary protocols. Applications such as the Sensorpedia web application and other third party software discover and subscribe to sensor data using the Sensorpedia API. Some data providers may choose to use the Sensorpedia API directly to post sensor data updates. Others may use the framework to provide links to existing systems or services that provide users the ability to subscribe directly to the sensor system.

(more…)