|understanding security issues surrounding XML data|
Years ago, programmers would create customized systems for storing and exchanging data. Often these systems were optimized according to the needs of the program. Unfortunately, this meant that every data file was incompatible, and programs serving different purposes generally couldn’t intercommunicate over a network. Databases were invented to solve the data file problem so that programmers could rely on a consistent and generally fast storage system. Systems such as CORBA and RMI were created to allow arbitrary programs to work together over a network. For a little while, things worked pretty well.
Then one day, someone decided that a new way of doing the same thing was necessary. Everything was being re-designed to run over the web. Applications that worked perfectly well as traditional client-server systems were “webified”, often with rather horrid results. In the dot.com frenzy, technologists looked for ways of improving the web – redesigning it as a true distributed application framework. The result was XML, and a heck of a lot of other acronyms too (XSLT, sXML, DXML, XDML, WDDI, UDDI, SOAP… and so on).
Isn’t it funny how you can be incredibly familiar with something yet actually know nothing about it? Take any three-letter US government agency. We hear about the CIA, the FBI and the NSA all the time. They’re in movies, the papers, on TV… yet few people have the faintest idea as to what they really do or how they really operate.
XML is another one of those three-letter words that is familiar, yet unknown. It pops up everywhere – especially since all new software products are XML compatible. Of course, nobody knows what that means, or why they should care. Oh wait – they should care because XML is better somehow, right?
A long time ago (1985), the International Standards Organization put its stamp of approval (ISO 8879) on a generic document and data representation language called – drum roll please – the Standard Generalized Markup Language a.k.a. SGML. It’s actually a language for defining other document languages, and as such is incredibly powerful and flexible. It was used in various industrial and government computing applications, but received little public attention. This was partially because the general public was still figuring out how to make PCs do something other than “Abort, Fail or Retry?”
Then Tim-Burners-Lee used SGML to define a really simple language called the HyperText Markup Language (HTML). Within a short period of time, everyone knew about HTML. A lot of people even knew how to write HTML. But still, nobody knew much about SGML.
Tim Burners-Lee and his team hit the drawing board again and dramatically simplified SGML. Their goal was to create a replacement for HTML that would allow application developers to add whatever extensions they needed in a consistent manner. They called the result the eXtensible Markup Language – which was thereafter known as XML.
The benefit to XML is that it provides a common, well-organized structure for representing data. Any application that can understand XML can share data with other XML-capable applications. More importantly, any specific extensions that one application needs to make are described in a way that lets other applications take advantage of the extensions without any application-specific coding. The extensions to the language are defined in a special XML document called the Document Type Definition (DTD).
XML has no innate security. It’s actually built on the opposite principle: making data easily available to any application that can effectively use it. XML information is designed to be mostly human-readable. Applications were expected to provide their own security, which was often accomplished by encrypting the entire document whenever it was exposed or in transit.
The problem with encrypting the entire document is that all of the document structure gets encrypted. This means that a program must be able to decrypt the entire document before it can make decisions as to how the data should be handled. This is not very efficient. Ideally, only the bits that are truly important need to be secured.
The solution was to create a system called XML Encryption. This is a standardized extension to XML that allows documents to encrypt individual data items or regions of data on a case-by-case basis. It’s an incredibly flexible system that allows a single document to offer some information to everybody, and other information based upon who’s reading.
For example, a marketing document for a product could have multiple prices. The list price would be un-encrypted. Vendors would be able to decrypt their own cost, and preferred vendors would be able to unlock an even better cost. This is a win for a sales department, because they don’t have to create three separate documents, or worry that the wrong document will fall into the wrong hands. A single document would serve the purposes of everybody, provide customized information, and protect all other specialized information.
But wait, why can’t this just be done using a dynamic database-driven website? Vendors would get the appropriate price by logging into the website. Furthermore, the price could be changed at any time. Compare this to the document, which has to be recreated whenever the price changes.
The advantage to the XML solution is that you get much of the functionality of a database-driven application, but without actually needing to create and operate a service. This can be a big cost savings, since dynamic applications are expensive to create.
How It Works
In order to understand XML encryption, we first need to look at some XML basic terminology:
Document: the entire XML file.
Tag: The fundamental structure unit for XML. A tag is a word in angle brackets. For example, <name> is a tag. Tags come in pairs – an opening and a closing tag. If the opening tag is <name>, the closing tag is </name>. In between the opening and closing tags is data and possibly more tags. For example:
In this example, the tag contains a <name> tag and <surname> tag. The name tag has the data “John”, while the surname tag is “Doe”. Tags are also called Elements.
The XML Encryption specification allows encryption to happen on multiple levels:
Document Level: The entire XML document can be enclosed in a <EncryptedData> tag.
Data Level: A single tag’s data can be enclosed in an <Encrypted Data> tag. For example, a tag might be <creditcardnum>. The actual number could be encrypted.
Tag/Element Level: An entire element can be enclosed in an <Encrypted Data> tag. This protects against hackers gaining knowledge by just analyzing the tags used. The downside is that the tags are no longer available for the application to use unless decryption is first performed.
Meta-Level: The DTD can specify elements that require encryption. This eliminates the need for putting “<encrypted data>” tags all over the document, which makes transmitting and processing the document more efficient.
XML Encryption is a relatively recent addition to the XML specification. Many applications that use XML don’t support XML Encryption. Some don’t encrypt the data at all.
Data Structure: XML is a highly structured format. The structure can give a hacker valuable information, beyond the data itself. For example, a snippet of XML could look like this:
Although the actual data in our example is encrypted, the structure tells a hacker that important information is being exchange. This could be just as useful as knowing the value of the data itself. Why? Because now the hacker can look closely at the source and destination machines for weaknesses. Instead of breaking the encryption, the hacker might just break into either end of the communication and look at everything before it gets encrypted!
The structure of XML makes analyzing captured data trivial for even the novice hacker. Therefore XML data should never travel over a network unencrypted, especially if the transmitted data will ultimately reside in a file on a storage device. Furthermore, XML Encryption should be used to encrypt structural data that might help a hacker figure out the type of information exchanged. This might very well mean “everything”. The “onion skin” approach mentioned in the best practices section provides a way around this.
Making The Connection
Accessing Information: Many client-server applications use XML as the data format for communication when flexibility is preferred over performance. Many web-based applications use XML internally because it’s easy to convert XML to HTML when it’s time to display results to a web browser. Some applications even send XML to the browser, taking advantage of the XML support in modern browsers.
Synchronizing Information/Collaboration Tools: XML is often used to exchange data in collaborative environments. These environments often mix a number of different applications together. XML provides a standard, common format for exchange that different applications can use. The ability to provide structured metadata is also a big win.
Databases: XML is often seen as an alternative storage system to databases. It isn’t. It’s a good way of generically representing structured information, but the format is not designed for efficient look up and storage. It’s effective in moving data between systems while preserving structure. It’s often used to get information from database system into another.
Central Monitoring System: these often generate reports in XML format. Some of the systems they monitor also communicate using XML. This would be a good application for XML Encryption, as it would efficiently protect the monitoring information as it moves across the network. Few systems currently take advantage of XML Encryption though. This may change.
Collaboration Control: One of the most useful applications of XML Encryption is for document management in a collaborative environment. In certain cases multiple people might be working on a single document. For example, two teams of attorneys might be preparing a contract. Company A’s attorneys use a revision system while creating the document to demark each contributor’s text and changes. They don’t want Company B to see these revisions, so they use XML Encryption to protect their revisions. The attorneys for Company B get the contract and make their own notes and changes. They encrypt their notes, protecting them from Company A. When the A attorneys get the document back, they have all of their original revision notes, as well as the changes from Company B. What they don’t have are B’s notes. As the document passes back and forth, each team can keep notes in the document.
Now for reality. No attorney would ever work this way. Notes are kept separately, only final text is sent to the other side. The possibility of a slip-up (forgetting to encrypt the notes, or the software not properly working) would be enough to discourage such a system, especially since the existing practice of keeping notes separate works just fine. Furthermore, even if Company A used this system, Company B might not have the same software. Even if they did, they would probably just create a new document with their own changes, in the format that worked best for them. They’d be even more inclined to do this if they knew Company A had hidden notes in the original document.
While attorneys are an extreme case, many types of document collaboration situations have similar characteristics. The same concerns would prevent many people from taking advantage of this type of opportunity. Collaboration currently happens without this functionality. For most cases it doesn’t add enough value to be worth the risks. But if this type of environment is used, it could be very effective from a productivity point of view.
Onion Skinning, a.k.a. Super Encryption: The problem with XML Encryption is that there’s constantly a trade off between maintaining enough structure to be useful and securing the structure against analysis. One solution is to encrypt the sensitive information, then encrypt the structure, encrypt the entire XML document, and send it through an encrypted channel. The result is an onion of encryption.
The onion creates a flexible trust model. The information in transit is the least trusted – it’s the tough, outer skin of the encrypted communication channel. Once it gets to the destination the document is still encrypted. This protects against hackers who are on the receiving machine. The proper application will be able to decrypt the document structure. Some users may be able to decrypt additional document structure but not the actual data. This allows for processing that requires the additional structure but doesn’t need the additional data. Finally, the appropriate user will be able to get at the data core.
The only downside is that all of the extra encryption/decryption processing overhead might make your computer cry. But modern computers are generally fast enough to handle a lot of layers before bursting into tears.
Trust with Multiple Parties: Let’s say you have three parties that need to exchange data. It may be easier for all three parties to trust the security provided by XML encryption than to implement a database-driven application for controlling access to the data.
The truth is that XML, encrypted or not, is one of many options. It may or may not be the best option for any given situation. The encryption makes it a more flexible, and therefore there will be cases where it is the preferred choice.
The above information is the start of a chapter in "Network Security Illustrated," published by McGraw-Hill and available from amazon.com, as well as your local bookstore. The book goes into much greater depth on this topic. To learn more about the book and what it covers, click here.
Below, you'll find links to online resources that supplement this portion of the book.