Open Data the Fakoli Way
Everywhere you look these days you will find Open Data initiatives. From train timetables to super-computer generated climate models, a plethora of data sets are becoming available online as machine readable data sources. The US Federal Government has committed to Open Data initiatives as a way to improve government transparency and foster an environment for innovation in the community at large.
A lot of the work involved in these initiatives lies in providing APIs that software developers can use to access the data sources and make use of them in their own applications. The term "API" (short for Application Programmer Interface) is getting rather overloaded these days. In its most general form it means any collection of functions or classes that a programmer may use to interact with a system. In this context Fakoli is an API for PHP developers wishing to develop web applications, as are Drupal, Joomla, CakePHP, etc.In this context of Open Data, however, API generally refers to web-based services for querying or submitting data that return their results in easily manipulable machine-readable formats. The most common approach these days is to expose RESTful Web Services (which have gained rapid acceptance due to the simplicity of their approach), although SOAP is still used for more complex interfaces.
With this focus on Open Data APIs, one question you need to ask when choosing a framework on which to base your new web application is "how easy will it be for me to support Open Data APIs, both to share my own data and to take advantage of the resources provided by others?" The answer with Fakoli is "very easy, indeed".
Case Study: Use of Open Data in http://www.foodfrequencyonline.org/
First a little background. Back in September my company, Sonjara, was involved in USAID's Hacking for Hunger weekend hackathon. As our contribution to that effort we built a fully functional nutrition surveying toolkit using Fakoli in under 36 hours (hey, it's not bragging if it's true!). One of the rules of the hackathon was that any solution had to involve the use of Open Data in some form. Our solution used open data sets from the World Bank to provide climate information to the user. We also used a publicly accessible but not API-enabled Excel file containing the official ISO Country Codes list, so that we could correctly categorize our geographic data and access the correct country data from the World Bank site.
Example Code - Retrieving temperature data from World Bank
When creating a food frequency survey, we wanted users to be presented with climate data for their selected country, to help them determine the best time of year to survey the population. To do this, we pulled in data from the World Bank. The first step was to create a climate_data component (to allow us to potentially re-use the code in other projects), then within that component we implemented an action handler that we could call from Javascript to retrieve the data from the World Bank.
<?php
Fakoli::using("climate_data", "iso_country_codes");
$country_id = checkNumeric($_GET["country_id"]);
if (!$country_id) ajaxReturn("");
$country = new ISOCountryCode($country_id);
$data = json_decode(getRemote("http://climatedataapi.worldbank.org/climateweb/rest/v1/country/cru/tas/month/{$country->iso_3_alpha}"));
$out = array();
foreach($data as $datum)
{
$out[] = round($datum->data, 1);
}
echo json_encode($out);
?>
The action handler serves two purposes:
- It circumvents cross-domain issues with XMLHttpRequest, since we are accessing information from our serving domain, and
- It provides us with an intermediate step where we can massage the retrieved data to better suit our purposes
The key to the above code is the getRemote() function - one of Fakoli's standard utility functions. getRemote() handles the calls to the cURL library and makes the remote HTTP request to retrieve the data. Once we have retrieved it we decode the JSON format, massage the data to 1 decimal place of precision and re-encode it as JSON for the output. Our Javascript layer then uses this to populate a data set in a Histogram provided by Fakoli's standard svg_charts component, and the data is displayed to the user as shown in the screenshot below:
Another Example - Exposing Data as a Queryable API
The previous examples shows how we can consume Open Data and use it to enhance our functionality and user experience, but to be a full Open Data citizen, we really have to put our money where our mouth is, and make at least some of our data open and queryable by the general public.
Suppose we wanted to take the ISO Country Codes that we had imported from an Excel spreadsheet and make those available as a RESTful API. It turns out that with the latest release of Fakoli (3.1.8) this is even easier than consuming data. Fakoli's Rich Data Architecture means that all items in our application's data model are able to serialize themselves to and from a variety of formats, including XML and JSON. In Fakoli 3.1.8 we have added a new framework class called APIHelper. This class takes a datamodel class and implements a query API based upon that classes structure.
To create an API for our ISO Country Codes based on our imported ISOCountryCode class all we need to do is create an action handler script with the following code:
<?php
Fakoli::usingFeature("api_helper");
$api = new APIHelper(ISOCountryCode);
$api->query();
?>
Yup, that's all you need. APIHelper makes use of Fakoli's SearchParameters framework class that uses introspection on the target class to provide all the search logic that translates query parameters into SQL. The query parameters for the API are based on the data fields in the target class, with search behavior suffixes to control the search. For instance, to retrieve all the country codes, you would make a call to the API like so:
http://www.foodfrequencyonline.org/action/iso_country_codes/query
To retrieve the data for Angola, the call would be:
http://www.foodfrequencyonline.org/action/iso_country_codes/query?name:equal=Angola
To retrieve all the countries that start with the letter A:
http://www.foodfrequencyonline.org/action/iso_country_codes/query?name:startsWith=A
All the currently valid country codes:
http://www.foodfrequencyonline.org/action/iso_country_codes/query?end_valid_year:from=2012
.and so on. Not bad for two lines of code? But wait - there's more (as they say on TV). We can also specify the output format for the data, using the _format parameter:
We will be adding a few refinements to APIHelper over the next few weeks, such as automatic documentation of the API parameters, but hopefully this will have illustrated how building your application on a rich data model makes Open Data less of a nice-to-have and more of a no-brainer.
