HTTP in XProc (updated again)

UPDATE: Norm Walsh, the editor of the XProc spec and the developer of Calabash commented that PUT and DELETE are supported by XProc. See at the end of the article.
Apple Leopard server's Wiki and Blog software throws away links to anchors in the same page, so you need to scroll down by yourself.

Normally when I need to figure out how some RESTlike webservices are working, I take a lazy approach and just fire up some Firefox extension such as RestClient or Poster.

A request in Poster


A response in Poster


And a view on RestClient



Based on a recent blog post of Norman Walsh on Wiki editing using XProc, I decided to do some experimenting with the http facilities of XProc.

POSTING raw XML to a service checking XML character encoding

I have on following address http://www.proxml.be:8080/check/encoding/ a service running,
which allows any XML binary representation to be POSTed to it and performing a best effort to determine its encoding.
The client to the service must issue an HTTP POST request with a body containing the XML resource to be tested.
The posted XML needs to have an XML declaration with encoding specified.
A report on the determined encoding will be returned as a UTF-8 encoded XML response.
The documentation of this service can be found at http://www.proxml.be:1060/book/view/book:urn:be:proxml:apps:xml:encoding:validator/

This service is running on the latest version of Netkernel, is open-sourced and can be downloaded from http://resources.1060research.com/packages/2010/1/proxml-check-encoding-1.1.0.nkp.jar.

A simple XProc doing this posting with the XML inline:

<?xml version="1.0" encoding="UTF-8"?>
<p:declare-step xmlns:p="http://www.w3.org/ns/xproc" 
  xmlns:c="http://www.w3.org/ns/xproc-step"
  version="1.0">
    <p:output port="result"/>
    <p:http-request omit-xml-declaration="false" encoding="UTF-8">
        <p:input port="source">
            <p:inline>
                <c:request href="http://www.proxml.be:8080/check/encoding/" 
                    method="POST">
                    <c:body content-type="application/xml" >
                        <a>Ăąƥʥϖℬ≝</a>
                    </c:body>
                </c:request>
            </p:inline>
        </p:input>
    </p:http-request>
</p:declare-step>

I think the code is rather straightforward.
In XProc an HTTP request is represented by a c:request element.


And the body needs to look like:

In our case:

<c:request href="http://www.proxml.be:8080/check/encoding/" method="POST">
     <c:body content-type="application/xml" >
         <a>Ăąƥʥϖℬ≝</a>
      </c:body>
</c:request>

Remark: the request only works for this webservice if the omit-xml-declaration attribute has been explicitly set to 'false', since the posted XML needs to contain the xml declaration to be checked.

An alternative where the xml has been put in a separate file.

<p:declare-step xmlns:p="http://www.w3.org/ns/xproc" 
    xmlns:c="http://www.w3.org/ns/xproc-step"
    version="1.0">
    <p:output port="result"/>
    <p:identity name="id">
        <p:input port="source">
            <p:document href="input.xml"/>
        </p:input>
    </p:identity>
    <p:insert name="ins" match="/c:request/c:body" position="first-child">
        <p:input port="source">
            <p:inline>
                <c:request href="http://www.proxml.be:8080/check/encoding/" 
                           method="POST">
                    <c:body content-type="application/xml"/>
                </c:request>
            </p:inline>
        </p:input>
        <p:input port="insertion">
            <p:pipe port="result" step="id"/>
        </p:input>
    </p:insert>
    <p:http-request name="request" omit-xml-declaration="false" encoding="UTF-8"/>
</p:declare-step>

First an external xml file is read and then the insert step is used to insert this xml as first-child into the c:body element, see the XSLTMatch pattern on the match attribute.

The result of this insert step is:

<c:request
    xmlns:c="http://www.w3.org/ns/xproc-step"
    href="http://www.proxml.be:8080/check/encoding/"
    method="POST">
    <c:body
        xmlns:c="http://www.w3.org/ns/xproc-step"
        content-type="application/xml">
        <a>Ăąƥʥϖℬ≝</a>
    </c:body>
</c:request>

POSTING XML as a x-www-form-urlencoded parameter

Important: The url used in this example is a fake one.

We need to construct a request that looks like:

<c:request method="POST" href="http://www.example.com/form-action" 
           xmlns:c="http://www.w3.org/ns/xproc-step">
  <c:body content-type="application/x-www-form-urlencoded">
      name=W3C&amp;spec=XProc
  </c:body>
</c:request>

where the body contains name=value pairs separated by an &

The p:www-form-urlencode step encodes a set of parameter values as a x-www-form-urlencoded string and injects it into the source document at the XSLTMatch pattern of the match attribute; in our case at /c:request/c:body/text(). If a string '@@HERE@@' is not placed inside the c:body the code will not work.

Since our xml needs to be placed in an attribute value on the c:param element, it needs to be escaped; hence our preliminary escape-markup step. The p:escape-markup step applies XML serialization to the children of the document element and replaces those children with their serialization. This is the reason we use a wrapper element (wrap) around the xml top be escaped.

<?xml version="1.0" encoding="UTF-8"?>
<p:declare-step xmlns:p="http://www.w3.org/ns/xproc" name="main"
    xmlns:cx="http://xmlcalabash.com/ns/extensions" 
    xmlns:c="http://www.w3.org/ns/xproc-step"
    xmlns:html="http://www.w3.org/1999/xhtml" version="1.0">
    <p:output port="result"/>
    <p:escape-markup name="escape">
        <p:input port="source">
            <p:inline>
                <wrap><a>Hello: Ăąƥʥϖℬ≝</a>
                </wrap>
            </p:inline>
        </p:input>
    </p:escape-markup>
    <p:www-form-urlencode match="/c:request/c:body/text()">
        <p:input port="source">
            <p:inline>
                <c:request method="POST"
                           href="http://www.testservice.org/">
              <c:body content-type="application/x-www-form-urlencoded">@@HERE@@</c:body>
                </c:request>
            </p:inline>
        </p:input>
        <p:with-param name="uid" select="'test'"/>
        <p:with-param name="pwd" select="'test'"/>
        <p:with-param name="xml" select=".">
            <p:pipe port="result" step="escape"/>
        </p:with-param>
    </p:www-form-urlencode>
    <p:http-request/>
</p:declare-step>

The resulting request:

<?xml version="1.0" encoding="UTF-8"?>
<c:request
    xmlns:c="http://www.w3.org/ns/xproc-step"
    xmlns:cx="http://xmlcalabash.com/ns/extensions"
    xmlns:html="http://www.w3.org/1999/xhtml"
    method="POST"
    href="http://www.testservice.org">

    <c:body
        xmlns:c="http://www.w3.org/ns/xproc-step"
        xmlns:cx="http://xmlcalabash.com/ns/extensions"
        xmlns:html="http://www.w3.org/1999/xhtml"
        content-type="application/x-www-form-urlencoded">uid=test&amp;pwd=test&amp;xml=%3Ca%3EHello:%20%C3%84%E2%80%9A%C3%84%E2%80%A6%C3%86%C2%A5%C3%8A%C2%A5%C3%8F%E2%80%93%C3%A2%E2%80%9E%C2%AC%C3%A2%E2%80%B0%EF%BF%BD%3C/a%3E%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20</c:body>
</c:request>

Alternative 2

<?xml version="1.0" encoding="UTF-8"?>

<p:declare-step xmlns:p="http://www.w3.org/ns/xproc" name="main"
    xmlns:cx="http://xmlcalabash.com/ns/extensions" 
    xmlns:c="http://www.w3.org/ns/xproc-step"
    xmlns:html="http://www.w3.org/1999/xhtml" version="1.0">
    <p:output port="result"/>
    <p:escape-markup name="escape">
        <p:input port="source">
            <p:inline>
                <wrap><a>Hello: Ăąƥʥϖℬ≝</a>
                </wrap>
            </p:inline>
        </p:input>
    </p:escape-markup>
    <p:add-attribute match="c:param" name="xml">
        <p:input port="source">
            <p:inline>
                <c:param name="xml"/>
            </p:inline>
        </p:input>
        <p:with-option name="attribute-name" select="'value'"/>
        <p:with-option name="attribute-value" select="wrap/text()">
            <p:pipe port="result" step="escape"/> 
        </p:with-option>
    </p:add-attribute>
    <p:wrap-sequence name="wrap" wrapper="c:param-set">
        <p:input port="source">
            <p:inline>
                <wrap>
                    <c:param name="uid" value="test"/>
                    <c:param name="pwd" value="test"/>
                </wrap>
            </p:inline>
           <p:pipe port="result" step="xml"/>
        </p:input>
    </p:wrap-sequence>
    <p:unwrap match="wrap" name="unwrap"/>
    <p:www-form-urlencode match="/c:request/c:body/text()">
        <p:input port="source">
            <p:inline>
                <c:request method="POST"
                    href="http://www.testservice.org/">
              <c:body content-type="application/x-www-form-urlencoded">@@HERE@@</c:body>
                </c:request>
            </p:inline>
        </p:input>
        <p:input port="parameters">
            <p:pipe port="result" step="unwrap"/>
        </p:input>
    </p:www-form-urlencode>
  <p:http-request/>
</p:declare-step>

All of the pipelines have been tested and with the latest versions of Calumet and Calabash both run from inside OxygenXML.

Conclusion

Is this a good replacement for my usual approach?
One is tempted to say yes, but I'm afraid not completely,
since two HTTP methods which are used heavily in a REST environments, being PUT and DELETE are not supported by XProc (yet?).

UPDATE

Norm Walsh, the editor of the XProc spec and the developer of Calabash commented that PUT and DELETE are supported by XProc.

This is good news. I have been put on the wrong leg by following sentence from the spec.

The method attribute specifies the method to be used against the IRI specified by the href attribute, e.g. GET or POST (the value is not case-sensitive).

Using following XProc:

<?xml version="1.0" encoding="UTF-8"?>
<p:declare-step xmlns:p="http://www.w3.org/ns/xproc" 
                xmlns:c="http://www.w3.org/ns/xproc-step"
                version="1.0">
    <p:output port="result"/>
    <p:http-request omit-xml-declaration="false" encoding="UTF-8">
        <p:input port="source">
            <p:inline>
                <c:request href="http://localhost:8888/exist/rest/db/fruits/Almonds.xml" method="PUT">
                    <c:body content-type="application/xml" >
                        <product>
                            <category>fruits</category>
                            <item>Almonds</item>
                            <inventory>
                                <sku>AlmofruiIV75Lm</sku>
                                <price>2</price>
                                <inventory>915</inventory>
                            </inventory>
                            <vendor>TriCounty Produce</vendor>
                        </product>
                    </c:body>
                </c:request>
            </p:inline>
        </p:input>
    </p:http-request>
</p:declare-step>

I get following response from Calumet from EMC.

SystemID: C:\Users\Paul\OxygenWorkspace\XProc\simplePUT.xpl
Engine name: Calumet XProc
Severity: error
Description: XPROC_ERROR: Unsupported request method: PUT
Original message: XPROC_ERROR: Unsupported request method: PUT

UPDATE: Good news coming our way: HTTP PUT will be supported in the upcoming 1.0.11 release.

And following response from Calabash version 0.9.20 from Norm Walsh

<c:body content-type="application/octet-stream" encoding="base64">
</c:body>

And the PUT has been carried out in eXist-db.

<product xmlns:c="http://www.w3.org/ns/xproc-step">
    <category>fruits</category>
    <item>Almonds</item>
    <inventory>
        <sku>AlmofruiIV75Lm</sku>
        <price>2</price>
        <inventory>915</inventory>
    </inventory>
    <vendor>TriCounty Produce</vendor>
</product>

Does anyone have a hint how to get rid of the namespace declaration 'http://www.w3.org/ns/xproc-step'

in the PUTTED file?

Yes, credits go to Vojtech Toman of EMC.

Changing

<p:inline>

to

<p:inline exclude-inline-prefixes="c">

does the trick.