My Public Notepad: XML

Showing posts with label XML. Show all posts

Tuesday, 14 February 2012

XML Data Binding - Part 3: CodeSynthesis XSD example

In my previous article about XML Data Binding, I demonstrated how to use gSOAP in order to convert data from XML document into in-memory C++ objects and vice versa. Today I will show how to use another tool, CodeSynthesis XSD, to perform the same task.

CodeSynthesis XSD depends on Apache Xerces-C++ XML parser so you need to download and set up Xerces in your development environment first. Setup of both tools is described in README.txt file you can find after unpacking downloaded CodeSynthesis XSD archive.

In order to compare gSOAP and CodeSynthesis Data Binding process, let's create a project that does the same XML processing, like gSOAP one: loads XML, reads and displays data, adds new element, displays data again and saves chnages to XML.

We are going to use the same XML schema - library.xsd:

If XML documents use XML schema grammars Xerces parser requires them to specify location of their XML schemas (by using an xsi:schemaLocation attribute if they use namespaces, and an xsi:noNamespaceSchemaLocation if not).

NOTE: Make sure xml document and its schema are in the same directory (application's working directory).

Similar to gSOAP case, we need to compile schema into C++ classes. CodeSynthesis schema compiler is xsd.exe and can be found in the bin directory of the package (e.g. ..\xsd-3.3.0-i686-windows\bin\). This directory should be added to Path environment variable. We want to generate C++/Tree mapping and code for serialization (object to XML; this code is not generated by default) so call XSD compiler with following parameters:

c:\test\XercesCodeSynthesis_Test1>xsd cxx-tree --generate-serialization library.xsd

It creates two files, library.hxx and library.cxx, and we need to include them into our project as they contain definitions of proxy classes.

Again, like we had with gSOAP, CodeSynthesis XSD generates classes that match XML document elements, by name and structure. We can see that in the header, library.hxx (some parts are omitted):

Schema complier has generated and Library_ functions that serialize/deserialize data to/from XML file.

main.cpp contains code that loads XML document into Library object, traverses through its member (vector) Books and displays all Book elements; it then adds a new Book to the collection, displays it again and serializes back to XML document in file:

Output:

Displaying all books in the library:

Book:
Title:Clean Code
Author:Robert C. Martin
ISBN: 0132350882
Copies available: 2

Book:
Title:The Pragmatic Programmer
Author:Andrew Hunt
ISBN: 020161622X
Copies available: 0

Book:
Title:Design patterns
Author:Erich Gamma
ISBN: 0201633612
Copies available: 1

Adding a new book:
Title: Effective C++
Author: Scott Meyers
ISBN:0321334876
Copies available: 50

Displaying all books in the library:

Book:
Title:Clean Code
Author:Robert C. Martin
ISBN: 0132350882
Copies available: 2

Book:
Title:The Pragmatic Programmer
Author:Andrew Hunt
ISBN: 020161622X
Copies available: 0

Book:
Title:Design patterns
Author:Erich Gamma
ISBN: 0201633612
Copies available: 1

Book:
Title:Effective C++
Author:Scott Meyers
ISBN: 0321334876
Copies available: 50

library.xml is changed - a new Book element has been added:

Links and References:
Boris Kolpackov: An Introduction to XML Data Binding in C++

Monday, 6 February 2012

XML Data Binding - Part 1: Why do we need it?

Your application uses some XML parsing tool (Xerces, libxml, TinyXML, TinyXML++, RapidXml, PugiXML,...) in order to load XML file into a document object, then you traverse through a DOM tree structure, look for nodes, their children, search their attributes so can modify its content (add, modify or delete elements or attributes)...and all this usually requires lots of loops and string comparisons which creates huge, hard to maintain code. Wouldn't be better if you could load XML document into some object in memory which attributes match elements in your XML? You would then be dealing with (C++) objects instead of the complex DOM tree which is much quicker, easier, type safe and less error prone.

Let's say we have some XML that keeps track of the state of the local library. To keep model simple, we can say that library comprises books and staff. Each book has its title, author and ISBN number. Each member of the staff, librarian, has a name. XML document could look like this:

library.xml

We can map each class of nodes into a C++ class where class attributes are node's attributes and its children; siblings are stored in a vector. We need a single instance of the class which represents a root node. Its constructor loads XML document from a file on disk and its destructor saves (eventually modified) XML document back to the file. Our class model and use case might look like this:

main.cpp:

XML document was loaded into object, modified by adding a new book and saved back into the file in just three lines of code! Awesome! But this code is unfinished and actually doesn't work properly in the real life as I omitted the hardest bit: loading and parsing XML in library's constructor and serializing/marshalling object back to the file in the destructor. All I wanted to show was how quick and easy is to manipulate XMLs when representing them through objects - a concept which is known as XML Data Binding.

Another problem steams from the fact that for each new XML we would need to write completely new classes - doing this manually is a no way to go but luckily there are tools that do this automatically. They compile XML schema (a document which specifies XML document itself) into a set of classes, following precise rules of mapping XML into objects (Object/XML Mapping or O/X Mapping). In next two articles I will show how to use gSOAP and CodeSynthesis XSD for this.

Links and References:
XML Data Binding Tools

Thursday, 15 September 2011

How to create a SOAP request

So you know there's a web service deployed on some web server and you want to make a web service call. One way of achieving this is to create SOAP message, put it in a body of HTTP POST command and send it (via TCP/IP) to that web server. If you are using some of web service frameworks (e.g. Apache Axis), you don't need to know the structure of SOAP message; you'll just be adding its elements through dedicated framework functions. But if you don't want to rely on such frameworks, be prepared for creating and parsing SOAP messages by yourself! You can use some HTTP framework (e.g. Microsoft's WinHTTP) to create HTTP POST request and then insert your SOAP request in HTTP message body. Or, you can go one level down and create TCP socket, manually build SOAP and HTTP messages and send them over TCP connection...In this article I'll focus on SOAP messages only.

SOAP (Simple Object Access Protocol) is defined by World Wide Web Consortium (W3C) and the specification of its latest version can be found here.

Paragraph 5 ("SOAP Message Construct") states that SOAP message is represented with one element, named Envelope, from a namespace "http://www.w3.org/2003/05/soap-envelope". XML namespace can be referred via its prefix which is like its alias (abbreviation) and can be any string, soapenv in this case:

<soapenv:envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope">
...
</soapenv:envelope>

Prefix and local name of an element together make its qualified name (QName). XML parser uses the prefix to find the actual namespace URI.

Following examples are using different namespace prefixes but they all refer to the same URI where namespace is defined:

<soap:envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope">
...
</soap:envelope>

<soap-env:envelope xmlns:soap-env="http://www.w3.org/2003/05/soap-envelope">
...
</soap-env:envelope>

<env:envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope">
...
</env:envelope>

Apart from its local name and namespace, Envelope element contains:

other namespaces, from other schemas (optional)
namespace-qualified attributes (optional)
Header element (optional child)
Body element (mandatory child)

namespace-qualified means that node's name is made of namespace prefix and a local name.

One of the possible attributes determines which encoding should be used to deserialize this message. Its name must contain namespace part (envelope's namespace) and its local name - encodingStyle. Its value is URI of schema that defines encoding:

<soapenv:envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope"
soapenv:encodingStyle="http://www.w3.org/2003/05/soap-encoding">
...
</soapenv:envelope>

SOAP Header element contains:

its qualified name made up of envelope's namespace and local name Header (mandatory)
namespace-qualified attributes (optional)
namespace-qualified SOAP Header Blocks - children elements (optional)

Each Header Block contains:

local name and namespace (mandatory)
attributes (optional); any of all of following attributes defined in soapenv namespace:

encodingStyle
role
mustUnderstand
relay

SOAP Body element contains:

its qualified name made up of envelope's namespace and local name Body (mandatory)
namespace-qualified attributes (including encodingStyle) (optional)
namespace-qualified children elements (optional)

Let us assume that web service has method add which accepts two arguments - two integers - and returns their sum. It would expect to receive XML element like this:

Client should send this element within Body part of SOAP message which could be, in its simplest form (containing only mandatory nodes), like this:

<?xml version="1.0"?>
<soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope">
<soapenv:Body>
  <add>
   <op1>3</op1>
   <op2>4</op2>
  </add>
</soapenv:Body>
</soapenv:Envelope>

Web server receives HTTP POST message, extracts SOAP message from its body and forwards it to Web Service which analyses its headers (if any) and extracts XML from its body. It parses XML and extracts web method name and its arguments.

Obviously, both client and service must use the same function and argument names. This can be enforced if they both use XML documents that are associated to (and are validated against) the same XML schema. How to create one for our XML? If we know the valid structure of it (defined here; see W3C XML Schema page), we could do it manually (which is a kind of reverse engineering) but this is not a way to go: it's better using some tool which will generate it for us from a given XML. Microsoft Visual Studio (2008/2010) could be used as such tool: open your XML file in it and select XML->Create Schema from a main menu. It will create XML Schema document file, named after XML file and with extension xsd.

For wstest.xml:

<?xml version="1.0" encoding="utf-8"?>
<add>
<op1>3</op1>
<op2>4</op2>
</add>

wstest.xsd is output:

<?xml version="1.0" encoding="utf-8"?>
<xs:schema attributeformdefault="unqualified" elementformdefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="add">
<xs:complextype>
<xs:sequence>
<xs:element name="op1" type="xs:unsignedByte"/>
<xs:element name="op2" type="xs:unsignedByte"/>
</xs:sequence>
</xs:complextype>
</xs:element>
</xs:schema>

In the example above we are not using namespaces which is a bad practice as element/attribute name clashes can occur. Let us write an XML with namespace-qualified names:

<?xml version="1.0" encoding="utf-8"?>
<ns1:add xmlns:ns1="http://www.bk.com/webservices/wstest">
<ns1:op1>3</ns1:op1>
<ns1:op2>4</ns1:op2>
</ns1:add>

Its wstest.xsd:

<?xml version="1.0" encoding="utf-8"?>
<xs:schema xmlns:ns1="http://www.bk.com/webservices/wstest" attributeFormDefault="unqualified" elementFormDefault="qualified" targetNamespace="http://www.bk.com/webservices/wstest" xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="add">
<xs:complextype>
<xs:sequence>
<xs:element name="op1" type="xs:unsignedByte"/>
<xs:element name="op2" type="xs:unsignedByte"/>
</xs:sequence>
</xs:complextype>
</xs:element>
</xs:schema>

XML schema above defines namespace "http://www.bk.com/webservices/wstest" (look for targetNamespace attribute) and other metadata related to our XML: valid element names ("add", "op1", "op2"), element and value types. It is worth mentioning here that namespace names are just arbitrary strigs, not locations on the web that XML parsers use in order to retrieve any information. We usually use URIs though (in form of URNs or URLs) as that decreases the chance of two namespaces having the same name. XML parsers treat them just like simple strings.

SOAP request which regards this new namespace looks like this:

<?xml version="1.0"?>
<soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope">
<soapenv:Body>
  <ns1:add xmlns:ns1="http://www.bk.com/webservices/wstest">
   <ns1:op1>3</ns1:op1>
   <ns1:op2>4</ns1:op2>
  </ns1:add>
</soapenv:Body>
</soapenv:Envelope>

Having local names within namespaces still does not make this document valid. XML document is valid only if it meets requirements of the schema with which it has been associated.

XML Schema can be referenced in an XML document by using schemaLocation attribute in document's root element. This attribute is defined in "http://www.w3.org/2001/XMLSchema-instance" namespace (which is traditionally aliased with xsi name) so we need to include this namespace as well.

We can reference our schema in our SOAP request:

<?xml version="1.0"?>
<soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.bk.com/webservices/wstest wstest.xsd">
<soapenv:Body>
  <ns1:add xmlns:ns1="http://www.bk.com/webservices/wstest">
   <ns1:op1>3</ns1:op1>
   <ns1:op2>4</ns1:op2>
  </ns1:add>
</soapenv:Body>
</soapenv:Envelope>

There are two forms of referencing schema:

using relative URL and schema file name which says that the XSD file is in the same directory as the XML document:

xsi:schemaLocation="http://www.bk.com/webservices/wstest wstest.xsd" (used in the example above)

xsi:schemaLocation="wstest.xsd"

using absolute URL of XSD:

xsi:schemaLocation="http://www.bk.com/webservices/wstest/wstest.xsd"

If a schema processor finds schema reference in a document, it should try to retrieve the schemas at the locations indicated. When no information is provided at all the schema processor is free to try any method to find a schema.

If our XML document uses simple types, we can use and reference W3C XML Schema in our document. It is bounded to "http://www.w3.org/2001/XMLSchema" namespace whose aliases are traditionally xs (used in this example) and xsd (often used for Microsoft schemas).

<?xml version="1.0"?>
<soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xs="http://www.w3.org/2001/XMLSchema">
<soapenv:Body>
  <ns1:add xmlns:ns1="http://www.bk.com/webservices/wstest">
   <ns1:op1 xsi:type="xs:int">3</ns1:op1>
   <ns1:op2 xsi:type="xs:int">4</ns1:op2>
  </ns1:add>
</soapenv:Body>
</soapenv:Envelope>