Alex Popescu recently reviewed a post by Mikayel Vardanyan on Picking the Right NoSQL Database Tool and was puzzled about the following of Mikayel’s statement:

[Relational database systems] allow versioning or activities like: Create, Read, Update and Delete. For databases, updates should never be allowed, because they destroy information. Rather, when data changes, the database should just add another record and note duly the previous value for that record.

I don’t find it puzzling at all. As Pat Helland rightly says:

In large-scale systems, you don’t update data, you add new data or create a new version.

OK, I guess arguing this on an abstract level serves nobody. Let’s get our hands dirty and have a look at a concrete example. I pick an example from the Linked Data world, but there is nothing really specific to it – it just happens to be the data language I speak and dream in

Look at the following piece of data:

… and now let’s capture the fact that my address has changed …

This looks normal at first sight, but there are two drawbacks attached with it:

1. If I ask the question: ‘Where has Michael been living previously?’, I can’t get an answer anymore once the update has been performed, unless I have a local copy of the old data piece.
2. Whenever I ask the question: ‘Where does Michael live?’ I need to implicitly add ‘at the moment’, as the information is not scoped.

There are few ways one can deal with it, though. And as a consequence, here is what I demand:

• Never ever DELETE data – it’s slow and lossy; also updating data is not good, as UPDATE is essentially DELETE + INSERT and hence lossy as well.
• Each piece of data must be versioned – in the Linked Data world one could, for example, use quads rather than triples to capture the context of the assertion expressed in the data.

Oh, BTW, my dear colleagues from the SPARQL Working Group – having said this, I think SPARQL Update is heading in the wrong direction would benefit from adding an appendix that discusses ‘large-scale deployment considerations’ on a system-level. Can we still change this, pretty please?

PS: disk space is cheap these days, as nicely pointed out by Dorian Taylor

One important aspect of data is “on the Web vs. in the Web” as Rick Jelliffe already highlighted in 2002:

To explain my POV, let me make a distinction between a resource being “on” the Web or “in” the Web. If it is merely “on” the Web, it does not have any links pointing to it. If a resource is “in” the Web, it has links from other resources to it. [...] A service that has no means of discovery (i.e. a link) or advertising is “on” the Web but not “in” the Web, under those terms. It just happens to use a set of protocols but it
is not part of a web. So it should not be called a web service, just an unlinked-to resource.

In 2007 Tom Heath repeated this essential statement in the context of Linked Data.

So, I thought it makes sense to revisit some (more or less) well-known data formats and services and try to pin down what “in the Web” means – a first step to measure how well-integrated they are with the Web. I’ll call the degree of how “much” they are in the Web the Link factor in the following. I suggest that the Link factor ranges from -2 (totally “on the Web”) to +2 (totally “in the Web), with the following attempt of a definition for the scale:

-2 … proprietary, desktop-centric document formats
-1 … structured data that can be exposed and accessed via Web
 0 … standardised, Web-aligned (XML-based) formats or Web services
 1 … open, standardised (document) formats
 2 … full REST-compliant, open (data) standards natively supporting links

Here is what I’ve so far – feel free to ping me if you disagree or have some other suggestions:

So, coming back to the title of this post: do we have a Linked Data research agenda? The answer is a clear: it depends

Looking at the ‘Topics of Interest’ of this year’s Linked Data on the Web (LDOW2010) workshop at WWW2010, and contrasting it with the TOP10 list we produced a year ago, my impression is that (at least in the next couple of months) we should focus on the following topics:

• Interlinking algorithms (beside entity-identity-focused frameworks such as Silk, there is not much there, anyway)
• Provenance & Trust – I see potential outreach possibilities through W3C’s Provenance Incubator, however, lots of legwork to be done, still. Web of Trust? Anyone?
• Dataset Dynamics (alternative/related keywords: change sets, logs, history, temporal tracking of datasets)

What do you see upcoming? What are important issues to be resolved in the Linked Data world (both from a research perspective and concerning open development tasks)?

Anyway, the point is really: every now and then one deploys a Web application (such as ve2; see, that’s why I needed the pitch) and likely wants to also tell the world out there a bit about the application. Some things you want to share with the Web at large that come immediately to mind are:

• who created the app and who maintains it (creator, legal entity, etc.)
• which software it has been created with (Java, PHP, jQuery, etc.)
• where the source code of the app is
• on which other services it depends on (such as Google calendar, flickr API, DBpedia lookup, etc.)
• acknowledgments
• usage conditions

Now, for most of the stuff one can of course use DOAP, the Description of a Project vocabulary, as we did (using RDFa) in the riese project, but some of the metadata goes beyond this, in my experience.

To save myself time (and hopefully you as well) I thought it might not hurt to put together an RDFa template for precisely this job: Announcing Application Metadata on the Web of Data . So, I put my initial proposal, based on Dublin Core and DOAP, at:

http://lab.linkeddata.deri.ie/2010/res/web-app-metadata-template.html

Note: The WebApp metadata template is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License. You may want to include a reference to this blog post.

To support this effort, I’m currently compiling a technical report in and for DERI’s Linked Data Research Centre (LiDRC) that looks at current examples of linked data-driven Web applications, gathers good practices and discusses the anatomy of a typical application (in the last part of the report issues and challenges are discussed, as well). So, one of the central contributions is a proposed concept for linked data-driven Web applications, which renders as follows:

The proposed components read as follows:

• A local RDF store, able to cache results and act as a permanent storage device to track users, etc.. We note that an RDF store such as ARC2 or Virtuoso is not a strict requirement, though often it makes sense to manage the RDF data in a native environment.
• Some logic (a controller) and UI modules implementing the business logic, the User Interface (UI) and the interaction parts of the application. These components are not specific to linked data-driven Web applications, however typically required and found in the wild.
• A data integration component, focusing on fetching linked data from the Web of Data, either directly from the LOD cloud or via Semantic Indexer such as Sindice or Falcons.
• A republishing component that eventually exposes parts of the application’s (interlinked) data on the Web of Data. It is a good practice to republish the application’s data, hence providing again input to the LOD cloud.

I’d be happy to hear from you what you think about this proposal. Any architectural feedback is welcome!

Basically, Daniel asked if it was not possible for every linked data publisher to follow a simple convention, that is to have the SPARQL endpoint available at common location such as /sparql – as Daniel found out, this is the case with most LOD datasets anyway these days.

Though the proposed good practice currently would be to publish a robots.txt that points to a semantic sitemap which in turn is the entry point for a voiD description, Daniel’s proposal is appealing. Simple and effective.

Additionally, it somehow reminds me on the /host proposal by Mark Nottingham and Eran Hammer-Lahav.

Thoughts, anyone?

Web Discovery Stack, v1, 2009-02-14

The Web Discovery Stack as shown here is an attempt to get the many proposals and ideas around discovery into some shape or scheme. The WD stack is basically a layering of specifications (or proposals for specifications) that mainly serve the discovery of resources on the Web.

Very happy to hear you comments (either here, via email or on IRC #swig channel).

Bernhard Haslhofer recently raised an issue at the public-lod@w3.org mailing list regarding Broken Links in LOD Data Sets, which unfortunately didn’t yield big and deep discussions. I intend to rehash the thread here, come up with a straw-man proposal and ask you for your comments.

Here is Bernhard’s core message:

If we assume that the consumers of LOD data are not humans but applications, broken links/references are not only “annoying” but could lead to severe processing errors if an application relies on a kind of “referential integrity”.

Today I gave it a quick thought after reviewing the recent TAG discussion how they intent to deal with broken links in their documents: the straw-man proposal is called ‘repairing vintage link values’ (revival) and may look as follows.

1. A human (e.g. through a built-in feature in a Web of Data browser such
   as Tabulator) encounters a broken link an reports it to the respective
   dataset publisher (the authoritative one who ‘owns’ it)
   OR
   A machine encounters a broken link (should it then directly ping the dataset publisher or first ‘ask’ its master for permission?)
2. The dataset publisher acknowledges the broken link and creates according triples as done in the case for documents (cf. TAG’s proposal)

We note that there are two important assumptions: someone who uses the data reports it and the dataset publisher (rather than a centralised service) fixes it.

Comments, anyone?