On description languages, REST, the Web, MEST, SSDL, and ‘declarative distributed computing’

This is very interesting! The W3C has started a mailing list (subscribed) to discuss Web description formats.

Mark Nottingham is asking in his “Web Description at the W3C” post whether some Web Services folks are probably “seeing the light” about the Web. However, what if it is the other way around? What if it is the Web community realising the power of protocol description languages for interactions with resources/services?

I would like to consider myself as someone who ‘gets’ the Web and REST (obviously not at the same level as the real experts) and actually really likes the architectural simplicity, extensibility, and uniformity of the model. However, I believe in an evolution of the REST and Web ideas; this is the reason the term ‘MEST’ resembles ‘REST’ so much. By coming up with such a name Jim and I wanted to show how influential the thinking behind REST has been in our description of the MEST ideas. Stefan says that MEST is nothing new but just a new name for the old idea of message-orientation. This may indeed be the case but I personally think of MEST as more than that; I hope this post will show why.

I would like to note that this entry does not aim to fuel the Web vs Web Services or the REST vs MEST or REST vs Service-Orientation arguments (as I have previously suggested, there is no such thing as REST vs SOAP). Also, it’s not my intention to start a new argument on description formats. Instead, I am using this opportunity to write about my recent thinking and my views about the Web and where I see distributed computing going.

REST and the Web

Although the Web does not follow the architectural principles of REST throughout, it is still the largest distributed system in existence today which is closest to those principles. Does this make the Web a good platform for all types of distributed applications? My view is ‘no’.

REST promotes the idea of a 1-1 association between a unique identifier (e.g. URI) and a resource. The Web heavily uses URLs which are not only used as identifiers but also as conveyors of the application protocol semantics. As a result, the same resource (e.g. file1) may have multiple identifiers depending on the application protocol being used to operate on it (e.g. http://example.org/file1, ftp://example.org/pub/file1). So, there is a coupling between the identifier, application protocol semantics, and the resource’s state (or representation of state). Applications are usually built by binding directly to this state through the identifier.

Now, let’s briefly revisit distributed object-oriented systems. There, we have a network pointer to some state which is accessible through an interface – specific to that application. What REST tells us is that we can reduce that interface to only a small set of verbs with uniform semantics across all applications. However, the problems of binding directly to state still exist. The problem of having to deal with resources at the architectural level of a distributed application also exists. For a client-server interaction and the browser abstraction this may not be a problem but for multi-participant, long-running interactions it becomes complicated to have to explicitly deal with addressable and bindable state.

The Web is mainly built around HTTP as its application protocol and a wide agreement on a set of media document formats. The Web is built using a resource-oriented paradigm where operations with uniform semantics (e.g. those defined by HTTP, FTP, SMTP, etc.) are used to operate on the media documents (i.e. the Web is a hypermedia application). Applications mostly utilise the browser abstraction for delivering information to users or providing access to services through rich clients. This is all good and it works for browser-style applications (apart for the 404 problem). However, what about computer-to-computer interactions and cross-organisation integration? What about multiple participants in interactions? What about application functionality that doesn’t fit well with the “browser of information” concept?

I always thought of Web Services technologies as trying to fill in this gap. The Web Services suite of technologies work well together, they are extensible and composable, and make integration easy. Ok, we are not exactly in this idealised situation from the word ‘go’ but neither was the Web when it started. One would rightly argue that SOAP, WSDL, and the WS-* technologies can be used according to different architectural principles (WS-RF and WS-Transfer are super examples of using Web Services technologies to implement other styles). Setting the architectural foundation for Internet-scale Web Services was the primary driver for MEST.

MEST

Jim and I wanted to capture the set of architectural principles that make sense for service-oriented distributed computing in the same way REST described the principles for resource-oriented computing. This is how MEST (‘Message Transfer’ for those who haven’t come across the term before) was created. We noted that since REST told us to move away from the application-specific interfaces of the object-oriented world to a limited set of verbs with uniform semantics across all applications, it may be a natural step to move to no operations at all. Instead, make only one assumption at the architectural level: one-way messages can be sent and received. Since we reduce our assumptions about our distributed environment to the absolute minimum, we are really building applications with loose-coupling baked-in from the start.

Here are few points about MEST:

• There are no directly bindable resources and hence no bindable state (or state representation). This, of course, does not mean that there is no state in our distributed application. It’s just that state is not bindable;
• There are no ‘clients’, ‘servers’, ‘consumers’, etc.; only services that exchange one-way messages;
• There are no concepts like ‘object’, ‘resource’, ‘interface’, ‘operation’, ‘inheritance’;
• Messages are combined in interesting ways to create protocols;
• Application semantics are associated with protocols (descriptions of how messages with well-defined structure can be combined).
• It is possible and desirable to associate semantics with the data transferred as payload of messages (e.g. using RDF or a specification document for a particular namespace written in English)

MEST promotes an architecture where there is no coupling between interfaces and endpoints; in fact, there are no interfaces whatsoever. We believe that such a coupling exists in object- and resource-oriented systems irrespective of whether application-specific or uniform interfaces exist. Since the behaviour of a distributed application can be described using messages, there is no need to bind behaviour to endpoints in the form of interfaces. Also, since there is no directly bindable state, there are no issues related to state synchronisation, referential integrity, etc. These issues would probably emerge at some point but they will be then dealt by protocols, which may even be application-specific.

By completely abandoning the notion of an interface, MEST proposes an architecture where there are no shared assumptions at endpoints; only the ability to send/receive messages is assumed. So where does behaviour go? Well, it’s moved on to the network, between services. Application behaviour is captured through protocol and semantic descriptions, through the association of metadata with the application messages. We use metadata to describe the structure of messages, the semantics of the information in those messages, the interesting ways in which these messages can be combined to make up interactions (the protocols), the policies, the quality-of-service requirements/capabilities, etc. Effectively we bring to distributed computing the benefits of declarative programming, where we describe our intentions (the ‘what’) of what we want to achieve rather than dealing with the implementation details (the ‘how’).

Describing message-based interactions

In the same way HTTP is the technology of choice for realising the REST principles (refer to my “WS-Web (The Web using SOAP:-)” for a different approach), SOAP and the WS-* protocols are the best technologies today for building MEST systems. The agreement on a common document processing modelling (the SOAP processing model) and the move away from application protocols in favour of message-based integration are a good match for the MEST ideas.

There have been many discussions on the usefulness of WSDL, on the relationship of WSDL with objects, on the need for description languages, etc. We always advocated for the treatment of WSDL as a description of language for message formats and exchange patterns. Others like to treat WSDL as a way to describe application interfaces and application-specific operations.

WSDL 2.0 has become too complex and too focused on the ‘interface’ and ‘operation’ abstractions. It’s no surprise that the community has been recently discussing alternatives. Dion Hinchcliffe has a survey of description vocabularies (btw, there shouldn’t be a distinction between REST and SOAP since one is an architectural style while the other is an implementation technology). We have proposed SSDL which focuses on messages and protocols. Realistically, it’s going to be very hard to move away from WSDL but the discussions are still worth having.

Now, the Web community is realising that the Web is in need of a language to describe the format of the documents exchanged as a result of HTTP operations. How is this different from what WSDL was supposed to achieve (before it became complicated) or what SSDL does? An XML document is received by a software agent and an XML document comes out (in HTTP land it’s always a request-response). In Web Services, that XML document has a particular structure (the SOAP envelope) with well defined processing semantics. A description language like SSDL allows us to concentrate on the format of these documents and the way they are combined to form protocols.

It seems that the current proposals for a Web description language maintain the request-response pattern for interacting with a resource and the ‘operation’ abstraction. Also, there is no attempt to capture multi-message exchanges. SSDL does all that while remaining simple (I am preparing a separate post about SSDL and the misunderstandings around it). It seems to me that the Web folks are slowly re-inventing the idea of Web Services around HTTP and XML. There is nothing wrong with that. I think we’ll have to wait and see whether the ‘operations with uniform semantics across applications’ approach will catch on.

Conclusion

I would personally like to see the discussions and intellectual effort on everything related to the architecture of Internet-scale applications move away from operations and uniform interfaces to how we can use metadata, semantics, protocols, contracts, etc. in order to build distributed applications in a declarative manner. I am keen to see distributed computing applications being built through the description of intentions based on a semantically rich vocabulary rather than the notion of identifiers + state + operations. Perhaps we could call such an approach ‘declarative’ or ‘intention-based distributed computing’ 🙂

Acknowledgement

Thanks to Jim for the sanity check on an earlier version of the above. So, you can blame him as well if this post doesn’t make sense 🙂