XiBIT

XML in the browser interoperability tests

Balisage 2012

Montréal, 9 August 2012

http://www.blackmesatech.com/2012/08/xibit/

Overview

Background
Deliverables
Challenges
Current status
Invitation

Background

XML was “SGML on the Web”: true or false?
XML is “SGML on the Web”: true or false?

XML won

Every major browser supports XML and XSLT.

And yet ...

XML won

Every major browser supports XML and XSLT.

And yet ...

Between the idea

And the reality

Between the motion

And the act

Falls the Shadow

(When did T.S. Eliot hear about XML interoperability?)

A simple practical demo

Make an XML document (Here's one we made earlier).
Link it to an XSL stylesheet (Here's one).
Load the XML document in a browser (Handy starting point). Try different browsers:
- Safari,
- Firefox,
- Chrome.
Try different ways of loading:
- from file system
- from (local) HTTP server

What's going on here?

War on local use of XML?

Possibly.

Or same-origin policy.

Maybe there's a spec somewhere ...

It's not about the same-origin policy

Other sources of variation:

valid? invalid?
DTD location?
well-formed? not well-formed?
internal DTD subset? external subset?
CSS? XSLT? both? single? multiple?
namespace axis
document('')
unparsed-entity-uri()
...

It's not about conformance

At least, not just about conformance.

failures of conformance
implementation-defined practice
implementation-dependent practice
out-of-scope issues

What is to be done?

Evolve our thinking? From ...

What is to be done? (2)

Evolve our thinking? To ...

If the only thing that matters is that it works with existing software, how is standards-based practice different?

What is to be done? (3)

Evolve our thinking? To ...

What is to be done? (3′)

Evolve our thinking? To ...

Push the documents into that lens in the middle.

I need a map

Every useful code of good practice is a record of past failures.

We have the failures.

Let's make use of it.

XiBIT

XML-in-the-browser interoperability tests.

XML
in the browser
interoperability
tests

Goal: investigate and document the behavior of current Web browsers in the processing and display of XML.

Related work

conformance tests
- XML (Oasis, W3C)
- XSLT 1.0
HTML5
W3C Web Applications WG
Alex Milowski [2009], [2011]

Interoperability

Client/server concept. What is interoperability for other software?

The capability to communicate, execute programs, or transfer data among various functional units in a manner that requires the user to have little or no knowledge of the unique characteristics of those units

-ISO/IEC 2381-01

Interoperability and conformance tests

Interoperability tests, not conformance tests:

coverage
telling right from wrong
implementation-defined behavior
implementation-dependent behavior
spec-scope issues

XiBIT deliverables

test cases
test reports: test × browser version × OS × configuration
analysis, discussion
automated diagnostic tool?
input mechanisms

Challenges

what to test? issues / axes of variation / causes of failure
keeping tests tractable / avoiding combinatorial explosion
keeping tests consistent
running tests
presenting raw(ish) results
understanding results
diagnosing risks

What to test

Identify axes of variation:

review of literature / experience reports
introspection
public input
review of specs
* review of conformance test suites

For example

Consider documents and DTDs:

valid? invalid?
internal subset? external subset? both?
form of external URI:
- relative URI downward
- relative URI upward
- relative URI pseudo-upward
- absolute URI same host, scheme, port
- absolute URI different host, same scheme, port
- absolute URI same host, different scheme or port

Avoiding combinatorial explosion

In theory, axes are independent.

But Cartesian products are big.

So assume independent until proven intertwingled.

For example

With or without DTD?

Stylesheet: XSLT? CSS? none?

2 (valid or invalid?) × 2 (internal subset yes or no?) × 6 (URI forms for external subset) × 2 (CSS or XSLT?) × 6 (URI forms for stylesheet) = 288 cases for external DTD + one stylesheet

+ 2 (valid or invalid?) × 2 (internal subset yes or no?) × 6 (URI forms for external subset) = 24 cases for external DTD + no stylesheet

+ 2 (valid or invalid?) cases for internal DTD, no external DTD, no stylesheet

+ 2 (CSS or XSLT?) × 6 (URI forms for stylesheet) = 12 cases for no DTD + one stylesheet

+ 1 case for no DTD, no stylesheet

= 327 cases DTD × one stylesheet

Cutting it down

Initially assume axes are independent.

Test interactions when it seems they aren't.

Consistency

XQuery to generate Cartesian product
... and filter it
XSLT to generate actual files

Running tests

one ... by ... one
or using iframes

[Demo]

Some initial results

No browsers failed on invalid input?!
Location of DTD makes no difference.
All browsers tested enforce same-origin policy on XSLT stylesheets.
No browsers tested enforce same-origin policy on CSS stylesheets.
Some browsers treat all file://... and file://... pairs as different host / violation of same-origin policy.
Some browsers treat some but not all file://... and file://... pairs as different host / violation of same-origin policy.

So ...

So if you want

to read XML from your file system
and use a common library directory

the browsers are not working hard to help.

Invitation

Pitch in!