[Wsf-general] Messaging project concept

[James Clark]

Let's start with the easier point:

> It seems to me its key to figure out who the audience is. If its for 
> "users" to move files around, then we shouldn't expose things like 
> transport selection, endpoint selection etc. to them (as much as 
> possible). This feels like a solution that we'd be doing over our stuff 
> (like the identity solution): WSO2 Secure Messaging Solution. (BTW this, I 
> believe, is what we're doing with Zend in France.)
> 
> If the target is for developers to use to exchange data between machines 
> reliably then the UI would be different but not by much. Again I still see 
> it as a simple python script that uses wsclient underneath. Server side 
> would another python script that uses wsf/c with a thttpd type embedded 
> transport.

The main product would be a command-line program.  Like most
properly-designed Unix command-line tools, it would be usable both by
end-users from the command-line and by developers from scripts or C
programs.  Think of something like the "cvs" program or the client part
of the "sendmail" program.

You could do a variety of GUIs (GNOME, Web), but in the Linux world
these are frosting.  As an alternative interface for programs, you could
also do a very thin C library that would talk some simple protocol to a
Unix socket. But again, this is frosting. A solid, command-line
interface is the main thing, and plenty to start with.

I see the key point about the positioning is that it's something
designed to appeal to the mainstream Linux, open source hacker world,
and in that world, I don't see the user/developer dichotomy that you do.

> First of all, this would be a straightforward set of python scripts that 
> uses wsclient underneath. Obviously that won't support adding transports 
> with python etc. but it'll solve business problem: provide a tool to move 
> data from one machine to another. The intermediary would be a 
> straightforward configuration of the ESB. Are you actually suggesting that 
> we re-write SOAP/WS-Sec*/RM etc. all in Python?? I must be mis-reading it.

>From a WS-* perspective, the way to think of Telegon is that it's one
specific Web service.  Implementing one, particular service is a very
different kind of thing from implementing a library that can deal with
arbitrary services.  For example, the client can just print the
appropriate XML directly.  The WSDL implementation could consist of just
printing a fixed WSDL file (perhaps substituting a few variables).
Implementing one fixed service from scratch in Python is not at all the
same kind of effort as rewriting the current C stuff in Python.

Another important point to bear in mind is the productivity/expressivity
gain that you get with Python over C (provided you're not too worried
about performance).  In my experience, the gain with Python over Java is
at least as much as Java over C. (Ruby would give the same benefits as
Python here.)

Overall doing a Python implementation of enough of SOAP Core,
WS-Security and MTOM to support just Telegon is not in my view a big
deal. I think the main difficulty is in the nitty-gritty interop
details: that's where what we've done so far would help a lot.

Your suggested architecture is not anything like what I think is a
reasonable architecture for this application.  The right architecture in
my view would be similar to an MTA (qmail or postfix).  Here's a good
paper on qmail's architecture:

  http://hillside.net/plop/2004/papers/mhafiz1/PLoP2004_mhafiz1_0.pdf

The main goal on the receiver side needs to be security.  Unless we can
demonstrate a good security architecture, it won't become popular.  (For
a start it won't get included in major distributions, which do a
security audit before they include new applications.)  The key to a
high-security architecture for something like Telegon is
compartmentalizing it into independent processes, which are small and
simple and each have limited security permissions.  This is not
something you can retrofit to Axis2/C.  Indeed I don't think it's
possible to design a general purpose WS-* architecture that would give
the same level of security as an architecture optimized just to handle a
specific Telegon-like service.

On the sender side, security is less of an issue.  However, I don't
think a python script on top of wsclient will do it.  The problem is our
memory-based single process architecture, particularly as regards RM.
The command line client should add the message in an queue, so that it
ends up in a file somewhere, and then exit. All the work should be done
by a daemon that picks up messages from the queue and sends them on.
This should happen whether RM is being used or not.  The RM
implementation would I believe be radically different, because it would
share the message queue infrastructure with the rest of the server. Even
here there are security issues because there should be a single daemon
to handle messages from all users.

In fact, I believe the sender-side daemon would end up being the same
program as the receiver-side daemon (and probably as the intermediary as
well).

But perhaps the most important point is a non-technical one. At the
moment I would say that the majority of the mainstream (non-Java,
non-enterprise) technical open source community thinks that WS-* is, to
put it bluntly, crap. They see it as far too complex, too RPC-oriented.
They'll use it if they're forced to, or if they have to interop with
something else that's already using it, but they wouldn't use it by
choice.  This represents a huge lost opportunity for us an open-source
company that is building on WS-*, because it dramatically reduces the
pool of potential contributors, which is the main competitive advantage
that we get by going the open source route.  Perhaps the main reason I
find Telegon exciting is that I believe it that it has the potential to
positively impact this perception.

However, there is no way, in my view, that is potential will be achieved
if the only implementations are based on a full WS-* stack, whether it's
Axis2/C, Axis2/Java or WCF.  A full WS-* stack (even without any modules
-- just the core architecture) of necessity brings along too much
complexity and baggage.  A WS-* sceptic will take one look at the
hundreds of thousands of lines of code in the complete WS-* stack and
say "This is waaaay to complex; there's no way I can trust this". If
instead you could provide an implementation that was say 5,000 lines of
Python without any dependencies on a WS-* stack, such that they could
easily read and understand the code, then I think there's a chance of
convincing people that WS-* is actually rather better than they think.

Another key part of winning people over is overcoming the perception
that WS-* is a indivisible blob, that they have to take or leave as a
whole.  In particular, it's important that people realize that they can
take advantage of the relatively small and simple parts related to
particular message instances (SOAP core, MTOM, WS-Security), without
buying into all the complexity and controversy surrounding the parts
that relate to types/contracts and message payloads (XSD, WSDL,
WS-Policy).  It's hard to do that if your implementation is based on a
full WS-* stack.

To summarize, I'm all in favor of one of our implementations of Telegon
being on top of WSF/C (particularly the sender side, and particularly if
we can find a way to use just Axiom and parts of Rampart without
bringing in everything else): I think having multiple, independent
implementations interoperate significantly enhances the credibility of
the project, especially from the protocol perspective. However, I think
the project has a hugely better chance of achieving widespread
popularity in the open-source community if there's an implementation
that

- does not depend on a WS-* stack, and

- is written mostly in a high-level scripting language (Python or Ruby)
with C only where necessary for performance.

James