Redland :
Free Software / Open Source RDF

Dave Beckett

This talk is personal opinion.
I am not speaking on behalf of my employer.

The slides are presented via S5

Overview

RDF in a Box
Redland
APIs and Features
Components and Modules
Demos

What Is Redland?

Redland RDF Application Framework Set of Libraries Toolkit Whatever!

Redland is a set of mature RDF open source libraries written in C providing a foundation layer of technology for semantic web applications with language bindings to Perl, PHP, Python, Ruby and others.

Vision: RDF "in a box" for developers

Reliable, portable, flexible, ...ible
Fast, efficient, small, ...
Easy to use and program with
Available in (insert your favourite language here)
Web standards compliant
Easy to use license

Aiming for: lowest possible barrier to using RDF technology

RDF in a box — Two Key Requirements

Available (Portable) to your system
Available in your programming language

Key Requirement 1 — Portable

Answer: C
Riposte: You are crazy, man!
GNU autotools and libtool
gcc -Weverything. g++ as a stricter C compiler
Memory leak/overflow checkers such as valgrind(1), dmalloc
Unit tests, graph tests, parsing/serializing syntax tests, query tests

Key Requirement 2 — Language Availability

Use Simplified Wrapper and Interface Generator (SWIG)
Language bindings to the main web development languages:
Java, Perl, PHP, Python, Ruby, Tcl
SWIG works well with C and C++
Non-answers: IDLs (Mozilla, CORBA, KDE/QT)
Improve raw SWIG bindings with native metaphors for usability
Direct bindings: C++, C# (Mono and .Net), Objective-C (OSX)

Other Important Requirements

Fast
Small: Not creating functionality in advance of needing it
Modular: Use whatever system functionality is available or select at build
Embeddable: Can be linked into a low-memory system for e.g. phones
Configurable: GNU autotools to configure modules and parts
Minimal dpendencies: Just an XML parser (if you do not want to read URIs)
Memory use: No leaks, overflows. Handles low memory use (embedded)
Web standards compliant
Friendly licence(s): LGPL 2.1+ / GPL 2.1+ / Apache 2+

Redland from Semantic Web @Bristol

The name: Redland is an area of Bristol, UK (wikipedia)
Not Redlands, Los Angeles or Redland, Florida or ...

FOAF, RDQL and Turtle were created here.

5 RDF W3C Recommendation editors work(ed) here.

Jena Java Semantic Web toolkit was created here at HP Labs.
(Actually Jena was created the same month as Redland, June 2000).

... and I lived there, at the time working at the Institute for Learning and Research Technology, University of Bristol

Semantic Web Stack

Tim Berners-Lee
2005

`#include <stdweb.h>`









	`stdrdf`


	`stdxml`
	`stduri`

Redland Scope in Semantic Web Stack

Semantic Web Layer Cake with Redland Scope

Redland Components

Raptor RDF Parser Toolkit (libraptor)
Parsing, serialising, URIs, WWW retrieval, Unicode and XML
Rasqal RDF Query Library (librasqal)
Query language support and query execution engines
(uses Raptor)
Redland (librdf)
RDF API to graph. Triple stores.
(uses Raptor and Rasqal)
Redland Bindings
Language bindings to Redland librdf
(uses Redland)

What you get in the Redland box

RDF Input and Output to/from syntaxes (Raptor):
RDF/XML, N-Triples, Turtle, RSS 1.0, RSS tag soup, Atom
RDF triple stores (Redland):
Simple & Indexed Memory, File, URI, Sleepycat/BDB, MySQL, SQLite, (Virtuoso)
RDF API for data access and manipulation (Redland):
Graph and Triple APIs
Language bindings (Bindings)
C#, Java, Obj-C, Perl, PHP, Python, Ruby, Tcl
NOTE: Not all of the above are supported and maintained
Query languages and query engine (Rasqal):
SPARQL and RDQL
Command line utilities for RDF with Unix filters:
rdfproc (RDF), rapper (parsing) and roqet (query)

API Style — Object-based C

Object-based not Object-Orientated (OO)

Constructors, destructors and methods reflected into C functions

  librdf_foo* librdf_new_foo(void);
  void librdf_free_foo(librdf_foo* foo);
  int librdf_foo_set_thing(librdf_foo* foo, char *thing);
  char *librdf_foo_get_thing(librdf_foo* foo);

No garbage collecting at C level. Bindings handle this.

Classes and factories for multiple implemenations

  librdf_storage* a=librdf_new_storage("memory", ...);
  librdf_storage* b=librdf_new_storage("uri", ...);

Data flow and constructs

Application initiates actions
Lazy evaluation during parsing and querying
No while loops in the code (except query engine)
Iterators for operating over sequences of objects, results rows
Map to Perl @array, Python generators, C# Iterator ...
(Low-level parser API generates triples via callbacks)

Redland Documentation

Detailed release notes, installation documentation, API reference manuals for each library
Comprehensive tutorial for Raptor
Manual pages for each command utility
Language bindings installation documentation for each language plus
POD for Perl; PyDoc for Python and RDoc for Ruby
Missing: Redland and Rasqal Tutorials
Missing: Redland Book

Web development language bindings

Expectation is to use libraries with one line of code

$ python
Python 2.5.4 (r254:67916, Feb 17 2009, 20:16:45) 
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
<<< import RDF
<<<

Or none:

$ perl -MRDF::Redland '...'

You can always use the command line:

$ rdfproc
Redland RDF processor utility 1.0.9
Copyright 2000-2008 David Beckett. Copyright 2000-2005 University of Bristol
Try `rdfproc --help' for more information.

More of which shortly...

RDF Graph Query

SPARQL Protocol and RDF Query Language

A Query Language ...
Find names and websites of contributors to PlanetRDF:

PREFIX foaf: <http://xmlns.com/foaf/0.1/>
SELECT ?name ?website
FROM   <http://planetrdf.com/bloggers.rdf>
WHERE { ?person foaf:weblog ?website ;
                foaf:name ?name .
        ?website a foaf:Document
      }

... and a Protocol

http://.../qps?
query-lang=http://www.w3.org/TR/rdf-sparql-query/
&graph-id=http://planetrdf.com/bloggers.rdf
&query=PREFIX foaf: <http://xmlns.com/foaf/0.1/...

Run this query (example.rq)
Also supports RDQL, the pre-standardisation deployed language

Demos Part 1 — Parsing in Raptor

Parsing in raptor

# online: count triples
$ rapper -c http://librdf.org/redland.rdf

# online: get triples from home page
$ rapper -g -q -o turtle http://www.dajobe.org/

# online: What was it doing?
$ rapper -g -q -c --trace http://www.dajobe.org/

# offline: parse the slides
$ rapper -g -q -o turtle index.html

# offline: what just happened?
rapper -g -c --trace index.html

or Triplr

rapper parsing — What just happened?

Demos Part 2 — Querying in Rasqal

Querying in rasqal

# online: use planetrdf query
$ roqet example.rq

or run the query via the web (example.rq)

roqet querying — What just happened?

Get Redland

Source! http://librdf.org/
debs for libraries and supported bindings
Linux: Debian, Ubuntu, Fedora, SUSE, ... and several others
BSD ports
Gentoo Linux emerge ports
OSX Objective-C packages (3rd party)
Used with some commercial applications
Win32: not available (need help)
Perl CPAN: old versions. Use newer versions from source/packages

Ongoing Work

Rasqal 1.next: new query engine
Rasqal 1.0: Complete SPARQL support
Raptor 1.next: about a year's worth of bug fixes
Raptor 2.0: API break and improvement
Redland 1.0.next: Virtuoso store
Bindings 1.0.next: nothing much
...

For anything else, I prefer patches in diff -urN form :)

Thanks

Redland: http://librdf.org/

Slides (to appear at):
http://www.dajobe.org/talks/200905-redland/

Questions?

Emergency Command Line Results

$ rapper -c http://librdf.org/redland.rdf
rapper: Parsing URI http://librdf.org/redland.rdf with parser rdfxml
rapper: Serializing with serializer ntriples
rapper: Parsing returned 40 triples

$ rapper -g -q -o turtle http://www.dajobe.org/

results: home.ttl

$ rapper -g -q -c --trace http://www.dajobe.org/
rapper: Processing URI http://www.dajobe.org/
rapper: Processing URI http://www.dajobe.org/foaf.rdf
rapper: Processing URI http://journal.dajobe.org/journal/comments.rdf
rapper: Processing URI http://www.dajobe.org/xslt/grokXFN.xsl

$ rapper -g -c --trace index.html 
rapper: Parsing URI file:///Users/dajobe/websites/www.dajobe.org/talks/200905-redland/index.html with parser guess
rapper: Serializing with serializer ntriples
rapper: Processing URI file:///Users/dajobe/websites/www.dajobe.org/talks/200905-redland/slides.xslt
rapper: Guessed parser name 'guess'
rapper: Parsing returned 56 triples

Redland : Free Software / Open Source RDF — http://librdf.org/

Dave Beckett — www.dajobe.org