GRDDL Primer

GRDDL is a mechanism for Gleaning Resource Descriptions from Dialects of Languages. It is a technique for obtaining RDF data from XML documents and in particular XHTML pages. Authors may explicitly associate documents with transformation algorithms, typically represented in XSLT, using a link element in the head of the document. Alternatively, the information needed to obtain the transformation may be held in an associated metadata profile document or namespace document. Clients reading the document can follow links across the Web using techniques described in the GRDDL specification to discover the appropriate transformations. This document uses a number of examples from the GRDDL Use Cases document to illustrate, in detail, the techniques GRDDL provides for associating documents with appropriate instructions for extracting any embedded data.

Status of this Document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

This document is a Working Group Note, developed by the GRDDL Working Group.

As of the publication of this Working Group Note the GRDDL Working Group has completed work on this document. Changes from the previous Working Draft are indicated in a log of changes. Comments on this document may be sent to [email protected] (with public archive). Further discussion on this material may be sent to the Semantic Web Interest Group mailing list, [email protected] (also with public archive).

Publication as a Working Group Note does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

Introduction

GRDDL provides an inexpensive set of mechanisms for bootstrapping RDF content from XML and XHTML. GRDDL does this by shifting the burden of formulating RDF away from the author to transformation algorithms written specifically for XML dialects such as XHTML. In this document the term HTML is used to refer to the XHTML dialect of HTML [XHTML].

GRDDL works through associating transformations with an individual document either through direct inclusion of references or indirectly through profile and namespace documents. For XML dialects the transformations are commonly expressed using XSLT 1.0, although other methods are permissible. Generally, if the transformation can be fully expressed in XSLT 1.0 then it is preferable to use that format since GRDDL processors should be capable of interpreting an XSLT 1.0 document.

While anyone can create a transformation, a standard transform library has been provided that can extract RDF that's embedded directly in XML or HTML using tags as well as extract any profile transformations. GRDDL transformations can be made for almost any dialect, including microformats.

This document may be read in conjunction with the GRDDL Use Cases [GRDDL-SCENARIOS] which describes a series of common scenarios for which GRDDL may be suitable. Readers desiring the technical details of the GRDDL mechanism or wishing to implement GRDDL themselves should refer to the GRDDL Specification [GRDDL].

GRDDL and XHTML: Scheduling a Meeting

One persistent and troublesome problem is discovering precisely when and where your friends are together so that you can schedule a meeting. In our example, a frequent traveller called Jane is trying see if at any point next year she can schedule a meeting with all three of her friends, despite the fact that all of her friends publish their calendar data in different ways. With GRDDL, she can discover if they can meet up without forcing her friends to all use the same centralized Web-based calendar system.

GRDDL provides a number of ways for GRDDL transformations to be associated with content, each of which is appropriate in different situations. The simplest method for authors of HTML content is to embed a reference to the transformations directly using a link element in the head of the document.

Microformats are simple conventions for embedding semantic markup for a specific domain in human-readable documents. One of Jane's friends has marked up their schedule using the hCalendar microformat. The hCalendar microformat uses HTML class attributes to associate event-related semantics with elements in the markup, as shown in Robin's calendar:

To explicitly relate the data in this document to the RDF data model the author needs to make two changes. First, she needs to add a profile attribute to the head element to denote that her document contains GRDDL metadata. In HTML, profiles are used to link documents to descriptions of the metadata schemes they employ (see HTML specification, Meta data profiles). The profile URI for GRDDL is http://www.w3.org/2003/g/data-view and by including this URI in her document Robin is declaring that her markup can be interpreted using GRDDL.

Then she needs to add a link element containing the reference to the specific GRDDL transformation for converting HTML containing hCalendar patterns into RDF. She can either write her own GRDDL transformation or re-use an existing transformation, and in this case there's one available for calendar data. The link element contains the token transformation in the rel attribute and the URI of the GRDDL transformation itself for extracting RDF is given by the value of the href attribute.

The profile URI in the Robin's new GRDDL-enabled calendar file signals that the receiver of the document may look for link elements with a rel attribute containing the token transformation and use any or all of those links to determine how to extract the data as RDF from Robin's calendar.

Referencing Through Profile Documents

Individual publishers of data using popular vocabularies can also give users of their data of being transformed into RDF without having to even add any new markup to individual documents. This is done by referencing GRDDL transformations in a profile document referenced in the head of the HTML. Other XML vocabularies may use their namespace documents for the same purpose. or namespace document. This method requires no work from the content author of individual documents but requires that the profile document contain a reference to a GRDDL transformation and be accessible to the GRDDL client, and so may require work from the creator and maintainer of the dialect. Yet this is a good use of time, since once the transformation has been linked to the profile document, all the users of the dialect get the added value of RDF.

Another of Jane's friends, David, has chosen to mark up his schedule using Embedded RDF. Embedded RDF has a link to a GRDDL transformation in its profile document.

Note that in this document the profile attribute does not contain a reference to the GRDDL profile. Instead it references the standard profile URI for Embedded RDF which does contain the GRDDL metadata. Anyone wishing to get the RDF data out of David's page can fetch the Embedded RDF profile URI to obtain the following profile document:

This document contains a reference to the GRDDL profile which again indicates that the profile may contain references to GRDDL transformations that can be applied to David's calendar, even if David does not explicitly link these transformations to his calendar. Jane's agent applies a standard transformation for profile documents to the Embedded RDF profile document in order to find a link to a transformation for all Embedded RDF documents, including David's HTML document. This transformation for all Embedded RDF documents, http://purl.org/NET/erdf/extract-rdf.xsl, is identified in the profile document using the rel attribute ofprofileTransformation. This process may be replicated with any vocabulary that has a profile URI.

Microformat-enabled web-pages on the Web may not be valid XHTML. For this purpose, one may wish to use a program like Tidy (or some other algorithm) to make the web-page equivalent to valid XHTML before applying GRDDL [GRDDL-SCENARIOS]. Also, many microformats may not have profiles with transformations. A user can always take matters into their own hands by applying a GRDDL transformation for a microformat directly to the web page in order to get RDF. This is risky since if the author of the document or microformat vocabulary does not explicitly license a GRDDL transformation, the responsibility for those RDF is now in the hands of the user.

Jane would like to meet with David and Robin, but does not want to manually check all their calendars, a process that is tiresome and prone to human error. To solve this problem, Jane decides to use a GRDDL implementation that converts both Robin and David's calendar to RDF. Jane stores her calendar directly in RDFa, a way of embedding RDF directly into HTML. She can use a GRDDL Transformation for RDFa to convert RDFa to RDF/XML, in order to get her entire schedule in RDF/XML.

One of the advantages of the RDF data model is that RDF data can be easily merged by adding it to a RDF store, so Jane can merge and query all the calendars together once they are transformed into RDF. Jane uses SPARQL [SPARQL] to query her data, which automatically merges the calendar data sources before running the query. SPARQL (The SPARQL Protocol and RDF Query Language) is a query language for RDF with a syntax similar to well-known data-base query languages. Online forms for submitting SPARQL queries can be found at on the this wiki.. Her scheduling SPARQL query looks like this:

The SELECT line determines which variable will appear in the results, here one of the start dates, one of the stop dates, a location and a summary. The FROM lines identify the data sources to use in the query, in this case the RDF/XML derived from Jane, David and Robin's original documents. The WHERE section provides a pattern which can match three events. The first block of FILTERs match up identical start and stop dates as well as locations between the three events. These values, which may be differently typed, are simplified to simple literals with the str() operator. The final two FILTER lines are idiomatic expressions which prevent multiple results returning due to the interchangeability of the variables.

So Jane discovers her friends Robin and David are both in town with her in Edinburgh on January 8th through 10th for the Web Design Conference. Since this is such as useful SPARQL script, she considers bundling it up as a web service so her friends can use it easily without writing SPARQL from scratch.

Mashing-Up Microformats: Booking a Hotel

In this example, we will combine data dialects as different as reviews and social networks in order to guarantee the booking a hotel with a high review from a trusted friend. This process of booking a hotel highlights the role of GRDDL in aggregating data from a variety of different formats and of using RDF as a common format to "mashup" all sorts of data, not just calendar data. We can of course write code in our favorite language to extract and combine these calendar data formats without using RDF. This ability to combine and query multiple kinds of microformat data from different web-pages shows functionality that RDF delivers that simple extraction of microformats to a custom data format can not. This example is similar to the guitar review use case.

Jane is pleased that she has found out all her friends can finally meet up in Edinburgh. However, she is not sure of where to stay in Edinburgh, so she decides to check reviews. There are various special interest publications online which feature hotel reviews, and blogs which contain reviews by individuals. The reviewers include friends and colleagues of Jane and people whose opinion Jane values (e.g. friends and people whose reviews Jane has found useful in the past). There may also be reviews planted by hotel advertisers which offer biased views in an attempt to attract customers.

First, Jane needs to get a list of people she considers trusted sources into some sort of machine readable document. One choice would be FOAF (Friend of a Friend), a popular RDF vocabulary for describing social networks of friends and personal data. Other choices include a collection of contacts stored in vCard using RDF [VCARD].

Another choice is to use microformats. A microformat that allows for more information about friends to be gleaned from the document is XFN, " XHTML Friends Network". Examples of such relationships are friends, colleagues, co-workers, and so on, as given in this example file.

Since XFN relationships are embedded in anchor (a) elements, they can be expressed in RDF in a variety of ways. Given Jane's HTML document uses the XFN microformat, a GRDDL transformation can extract RDF data. These descriptions would allow a RDF spider (a "scutter") to follow links to additional RDF content that may include more XFN, vCard, or FOAF descriptions. Jane's XFN list, is given as:

With this combined "mashed-up" data we can find Jane's friends and find the hotel reviews that those friends created. Using GRDDL we can glean information, including the ratings, about the hotels. Once we have this data as RDF we can "mash-up" the data of the friends and the hotel reviews.

In order to find hotels with specific ratings or higher from a group of her trusted friends, we can now query the "mashed-up" data with SPARQL. SPARQL (The SPARQL Protocol and RDF Query Language) is a query language for RDF that can automatically "mash-up" data from multiple sources.

rating	name	region	hotelname
"5"	"RexR"	"Edinburgh"	"McRae Palace, Edinburgh"
"5"	"MaryV"	"Philadelphia"	"Franklin Hotel Philadelphia"
"5"	"JohnD"	"Helsinki"	"Elena Plaza Hotel"
"5"	"PeterS"	"Amsterdam"	"Enlightenment Amsterdam Hotel"
"4"	"PeterS"	"Cambridge"	"Fano Hotel"
"5"	"PeterS"	"Edinburgh"	"Witch's Caldron Hotel, Edinburgh"
"3"	"JennyR"	"Atlanta"	"Merton Atlanta"
"5"	"RexR"	"LEIDEN"	"Pilgrim Hostel"
"5"	"Simon"	"Edinburgh"	"Forest Cafe Youth Hostel, Edinburgh"
"5"	"PeterS"	"Cambridge"	"Royal Moon Hotel Boston"
"3"	"RexR"	"Washington"	"Bond Plaza Hotel"
"5"	"RexR"	"Edinburgh"	"Ritchie Centre, Edinburgh"
"4"	"JohnD"	"Edinburgh"	"Walter Scot Hotel, Edinburgh"
"5"	"PeterS"	"New York"	"Maximus New York Hotel & Towers"

The query unfortunately gets us all hotels from anywhere in the world with more than 2 stars, so we need to further restrict the results to only hotels in Edinburgh, as we do in this improved query.

Now the results will be hotels with a rating of 2 stars or higher that are located in Edinburgh. The problem with the possible list of results is that there could be biased reviews. The next step is to further restrict the results to only reviews by our trusted list of contacts. Using the XFN links in Jane's page which identifies the URIs of people Jane trusts, by matching URIs we can select only those reviewers who are Jane's friends, as done in this further improved query.

We finally get the result we want: A hotel with a ranking of 5 reviewed by a trusted friend.

SPARQL results can be obtained as XML or JSON and can easily be consumed by another application. This can display the results on screen, email them to Jane or it can be pulled into another application to search the web for the best prices on the short list of hotels.

GRDDL and XML: Integrating Spreadsheets

GRDDL is also useful for integrating data from general-purpose XML dialects produced by everyday applications. A trove of accumulated information is stored in spreadsheets, and spreadsheets can be saved using a general-purpose XML format. Integrating, reusing, and "mashing-up" information stored in spreadsheets can be valuable, and GRDDL provides a mechanism for accessing this information as RDF in order to accomplish this. In this example, we will specifically consider the problem of gleaning information from Microsoft® Excel spreadsheets, although other spreadsheet-like XML dialects would be able to take advantage of the same basic mechanism.

Jane serves as the secretary for a small group with her two friends, David and Robin, that meets once a month. She tracks the attendance at these meetings using a simple Excel spreadsheet, and she starts a new spreadsheet each year. She wants the members of this group to be able to query these accumulated statistics freely, and she recognizes that RDF would support this kind of merging and querying functionality. She decides to use GRDDL to allow any of the members of the group to glean RDF from any of these attendance records and query the data along with any other RDF that may be available.

Jane intends to use a GRDDL transformation called xcel-mf2rdf.xsl, which requires the Excel spreadsheet to conform to a particular profile. She first must identify which cells in her spreadsheet are data cells. In the case of an attendance spreadsheet, the data cells are the attendance indicators, and she identifies these cells by giving them the name "Data". She must also identify the header cells. In this case, the header cells are the cells containing names and dates; Jane identifies these cells by giving them the name "Header". Next, Jane gives each data and header cell an additional name, which serves as the local name of the property for that cell. She names the date cells "date", the member name cells "name", and the attendance cells "present". Finally, Jane must set two custom properties globally on the spreadsheet. The first property is called "profile", and this particular profile has profile URI http://www.mnot.net/2005/08/xcel-mf. The second property is called "namespace", and provides the namespace to be used for RDF properties in the GRDDL results; Jane chooses the namespace URI http://example.org/attendance/.

Attendance spreadsheet with header cells selected

Since GRDDL operates on XML documents, she saves her Excel files using the XML dialect that Excel provides. After saving them as XML, she adds the reference to this transformation to the root element of each attendance document. Following the directives of the Excel profile, and including the appropriate GRDDL reference, this is a slice of the resulting spreadsheet document:



xmlns:grddl="http://www.w3.org/2003/g/data-view#"
  grddl:transformation="xcel-mf2rdf.xsl">
 
 
  http://www.mnot.net/2005/08/xcel-mf
  http://example.org/attendance/
 
 
 
  
    
    2006-04
    
   
    Robin
    
    n

When processed by a GRDDL-aware agent, a document such as this will be transformed into RDF that preserves the meaning of the spreadsheet:



 
 
  Robin
  2006-04
  n

Jane and the other members of the group can now use this data in a variety of situations. For example, suppose there exist other records of decisions that were made at these meetings, and the record of one of those meeting was also stored in a spreadsheet that was converted to RDF.

Merging these triples with the GRDDL results from the attendance record spreadsheets, Jane can now ask questions such as "who attended the meeting at which we decided to choose the new meeting location?" In SPARQL, the corresponding spreadsheet query is:


PREFIX att: 
PREFIX ev: 

SELECT ?name 

FROM 
FROM 

WHERE
{
  ?event ev:label "choose new meeting location" .
  ?event ev:date ?date .
  ?attendance att:date ?date .
  ?attendance att:name ?name .
  ?attendance att:present "y" .
}

Which would give the following results:

name

Jane

David

This indicates that Jane and David were present at the meeting where that decision was made.

In this example, the link to the GRDDL transformation was added by hand. However, as shown in detail in the GRDDL specification [GRDDL] for XML Schema, RDF, and HTML namespace documents may also have links to transformations for XML dialects; so a GRDDL-aware agent can also retrieve the namespace document of an XML dialect to find a GRDDL transformation by "following its nose" from the namespace on the root element of the GRDDL source document to the namespace document. The use of a namespace on the root element represents a declaration that the document conforms to the authoritative definition of that namespace as defined by the namespace owner, which may include a transformation from that XML dialect into RDF using GRDDL.

There are a few rules of thumb for XML namespace owners wanting to make GRDDL transformations available for their particular dialect of XML. Given an XML document representation, a GRDDL-aware agent that wishes to determine namespace or profile transformations may resolve the namespace or profile URI to obtain a representation. Because of content negotiation and other factors, different GRDDL-aware agents resolving the same namespace or profile URI could receive different representations, which could in turn specify different namespace or profile transformations, which could in turn produce different GRDDL results. In particular, a GRDDL-aware agent that receives a namespace or profile representation that specifies GRDDL transformations may not even be aware that some other representation, specifying more or different transformations, is available. This may pose problems to users that intend to retrieve all of the available GRDDL results associated with the original XML document representation.

To help prevent this problem, namespace and profile document authors that choose to serve representations that indicate namespace or profile transformations are advised to ensure that all such representations specify the same namespace or profile transformations.

GRDDL and Inference: Solving Health Care Problems

GRDDL can not only be used for combining HTML data, but for XML data as well. This section uses HL7 CDA, a widely deployed XML vocabulary for use in clinical data, as an example of how an XML dialect can be gleaned for RDF. This part of the primer walks through step-by-step the Health Care: Querying XML-based clinical data use-case.

Kayode wants to write software components which can extract RDF descriptions from XML HL7 CDA documents transmitted from various devices in a healthcare system using a clinical ontology so that he can merge together clinical reports and use inferences to detect possible problems. CDA is a very well-designed information model and heavily optimized for messaging between computerized hospital systems, and an example CDA document is given. Below is a section of this document that describes the author of a clinical document and the patient that the document describes.

In a manner similar to enabling the use of GRDDL with HTML, we can add a glean:transformation attribute to the root of the document in order for a GRDDL-aware agent to interpret an HL7 CDA message transmitted using widely-used and interoperable ontologies.

start1	stop1	loc1	summ1
"2007-01-08	"2007-01-11"	Edinburgh, UK"	Web Design Conference"

image	sign
_:foo	"Chest hyperinflated"

GRDDL Primer

W3C Working Group Note 28 June 2007

Abstract

Status of this Document

Table of Contents

Introduction

GRDDL and XHTML: Scheduling a Meeting

Referencing Through Profile Documents

Mashing-Up Microformats: Booking a Hotel

Jane's XFN List

GRDDL and XML: Integrating Spreadsheets

GRDDL and Inference: Solving Health Care Problems

Further Information

References

Acknowledgements

Change Log

rating	name	hotelname	region
"5"	"RexR"	"Ritchie Centre, Edinburgh"	"Edinburgh"
"5"	"PeterS"	"Witch's Caldron Hotel, Edinburgh"	"Edinburgh"
"5"	"Simon"	"Forest Cafe Youth Hostel, Edinburgh"	"Edinburgh"
"5"	"RexR"	"McRae Palace, Edinburgh"	"Edinburgh"
"4"	"JohnD"	"Walter Scott Hotel, Edinburgh"	"Edinburgh"