Download XQuery Unit Testing

XQuery Unit Testing
Mark Helmstetter, Senior Consultant
NOVA MUG - June 29, 2010
Slide 1
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
A Story about Testing
Slide 2
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Agenda





Unit Testing Basics
Unit Testing XQuery via XQuery
Unit Testing XQuery via Java
End-to-End Testing using Selenium
Testing Tips
Slide 3
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Unit Tests
 Test individual functions or modules (units)
 A unit is the smallest testable part (function)
 White-Box/Glass-Box – derived from internal structure
 Test each unit separately, in a controlled environment
 Repeatable with the same results
 Facilitate automation
 Facilitate change, code confidence, agility
 Allows for “modular” development – build backend before UI
Slide 4
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Steps of a Unit Test
1.
2.
3.
4.
Slide 5
setup()
Call function(s) under test
Verify (assert) results from each function call
teardown()
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
TDD Basics
 Add functionality in very small chunks
 Write the test before we implement the unit
1.
2.
3.
4.
5.
6.
Decide how each unit will be tested
Write the unit test code
Implement (code) the unit
Run the test, ideally any related unit tests, or even all tests
Refactor to make unit cleaner, more efficient
Repeat
Slide 6
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
XQuery Unit Testing
 Common approaches to XQuery Unit Testing
 Write the test code in XQuery
 Leverage full capability of XQuery within test code
 Write the test code in some other language (Java)
 Run tests directly from Eclipse
 Leverage JUnit and related frameworks
 Declarative test script using XML
 Framework (XQuery, Java, etc.) reads XML, executes tests as described
 Ideally test environment mimics execution environment
as closely as possible
Slide 7
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
TDD Process
http://en.wikipedia.org/wiki/Test-driven_development
Slide 8
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
XQuery Unit Testing – What’s
different?




Very similar to testing other languages
Techniques similar to testing other databases
Connecting to a database, considered integration test (?)
Not trivial to mock MarkLogic Server or an XQuery environment
 Better to use MarkLogic Server
 Keep your datasets small
Slide 9
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Agenda





Unit Testing Basics
Unit Testing XQuery via XQuery
Unit Testing XQuery via Java
End-to-End Testing using Selenium
Testing Tips
Slide 10
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
xqunit – common functions for
testing
module namespace xq = "http://marklogic.com/xqunit";
declare function xq:assert-equal($name, $actual as
item()*, $expected as item()*) {
if (fn:deep-equal($expected, $actual)) then
<pass test="{$name}"/>
else
<fail test="{$name}">
<expected>{$expected}</expected>
<actual>{$actual}</actual>
</fail>
};
...
Slide 11
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 1: search-lib-test.xqy
xquery version "1.0-ml";
import module namespace search="http://marklogic.com/search"
at "search-lib.xqy";
import module namespace xq = "http://marklogic.com/xqunit"
at "/lib/xqunit.xqy";
declare function local:test-get-query() {
let $query as schema-element(cts:query)? :=
search:get-query("foobar")
let $expected := document{cts:word-query("foobar")}/node()
return xq:assert-equal("test-get-query", $query, $expected)
};
<results>
{local:test-get-query()}
</results>
Slide 12
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 1: search-lib.xqy
xquery version "1.0-ml";
module namespace search="http://marklogic.com/search";
declare function get-query($query-str as xs:string)
as schema-element(cts:query)? {
if ($query-str eq "") then
()
else
document{cts:word-query($query-str)}/node()
};
Slide 13
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Execute test via browser
Slide 14
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 1 (Continued)
...
declare function local:test-get-query-empty() {
let $query as schema-element(cts:query)? :=
search:get-query(())
return xq:assert-equal("test-get-query-empty", $query, ())
};
declare function local:test-get-query-empty-string() {
let $query as schema-element(cts:query)? :=
search:get-query("")
return xq:assert-equal("test-get-query-empty-string",
$query, ())
};
<results>
{local:test-get-query(), local:test-get-query-empty(),
local:test-get-query-empty-string()}
</results>
Slide 15
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 1: Test PASSED!
Slide 16
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 1: Many more test
cases to consider






test-get-query-stemmed()
test-get-query-case-sensitive()
test-get-query-phrased()
test-get-query-diacritic-sensitive()
test-get-query-boolean-and()
test-get-query-boolean-or()
 …
Slide 17
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Loading Test Data
 Previous test example did not require any database data
 Many XQuery functions require some data
 Unit tests should insert test data in set-up()
 Framework invokes set-up() before each test function
 Unit tests should remove all test data in tear-down()
 Framework invokes tear-down() after each test function
Slide 18
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Test Environment Provisioning
 One database per developer
 Guarantee known state of database
 In a shared server environment, need 1 unit test database per
developer
 Or ideally, each developer has a local MLS instance for testing
 Use a separate database used exclusively for unit tests
 Typically have a database with a larger set of data for ad-hoc
application testing, keep that separate
 Use MarkLogic Admin API
 Script database configuration, make test setup easier
 Ensure consistent configuration across environments
Slide 19
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Agenda





Unit Testing Basics
Unit Testing XQuery via XQuery
Unit Testing XQuery via Java
End-to-End Testing using Selenium
Testing Tips
Slide 20
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Unit Testing XQuery from Java
 Why test from Java?
 Many frameworks to help facilitate
 JUnit, XMLUnit, HtmlUnit, XQueryUnit
 Tests can be run directly from the IDE
 Eclipse, IntelliJ, NetBeans
 Tools have good support for JUnit
 Ant, Maven, CruiseControl, Hudson, Anthill
 Can combine Java unit tests with XQuery unit tests
 Ideal for cases where the “front-end” is Java – mimic execution
environment
Slide 21
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
XQueryUnit
 JUnit extension classes and support classes to help facilitate
testing XQuery
 Developed by Mark Logic Professional Services, used by
numerous customers
 Supports both HTTP and XDBC interfaces
 Includes a number of convenience utilities:
 Inserting test data, clean up at tear down
 Generates wrapper code for invoking library modules
 XML diff utilities for assertions
 Open Source, Apache 2 license
 Available for download at http://xquery-unit.googlecode.com/
Slide 22
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Creating a simple XQueryUnit
Test Case
1. Create a new class, subclass XQueryTestCase or
XQueryXmlHttpTestCase
2. Override setUp()(optional) to insert any test data
3. Implement 1 or more test methods that invoke XQuery
modules/functions under test
Slide 23
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Testing XQuery Modules
 2 types of modules:
 Main
 Query body
 Library
 No query body
 Variable and function declarations
 XQueryTestCase provides support for testing main and
library modules, invoked via XCC
Slide 24
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 1 – Library Module Test
public class SearchLibTest extends XQueryTestCase {
private String modulePath = "/example1/search-lib.xqy";
private String moduleNamespace =
"http://marklogic.com/search";
public void testGetQuery() throws Exception {
XdmValue[] params = new XdmValue[] {
ValueFactory.newXSString("foobar") };
Document doc = executeLibraryModuleAsDocument(modulePath,
moduleNamespace, "get-query", params);
assertNotNull(doc);
}
}
Slide 25
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 1: search-lib.xqy
xquery version "1.0-ml";
module namespace search="http://marklogic.com/search";
declare function get-query($query-str as xs:string)
as schema-element(cts:query)? {
if ($query-str eq "") then
()
else
document{cts:word-query($query-str)}/node()
};
Slide 26
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 1: Run via Eclipse
Slide 27
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
HTTP POX Services
 POX = “Plain Old XML” – Like POJO
 POX is complementary to REST
 REST refers to a communication pattern, while POX refers to an
information format style.
 POX Services are easy to test
 Browser, wget, curl, xquery-unit, JUnit, HttpUnit
 XQueryXmlHttpTestCase provides support for testing Http based
services, including POX
http://en.wikipedia.org/wiki/Plain_Old_XML
Slide 28
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example POX Service
 Request:
<request>
<search>cardiac</search>
</request>
 Response:
<response>
<search:response total="1" start="1" pagelength="10">
<search:result index="1" uri="/medline1.xml" ...>
...
</search:result>
</search:response>
</response>
Slide 29
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 3
public class SearchTest extends XQueryXmlHttpTestCase {
...
protected
void setUp() throws Exception {
public
void testSearch() throws Exception {
this.setServicePath("example3/search.xqy");
String req = "<request><search/></request>";
super.setUp();
} Document doc = executeQueryAsDocument(req);
XPath xpath = XPath.newInstance("/response");
Element e = (Element) xpath.selectSingleNode(doc);
assertNotNull(e);
}
public void testInvalidRequest() throws Exception {
String req = "<request><XXX/></request>";
Document doc = executeQueryAsDocument(req);
XPath xpath = XPath.newInstance("/error");
Element e = (Element) xpath.selectSingleNode(doc);
assertNotNull(e);
}
}
Slide 30
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 3: search.xqy
xdmp:set-response-content-type("text/xml"),
try {
let $request := cm:get-request-element(())
let $action := $request/element()
let $check := if ($action) then () else
fn:error($NO-REQ, "No request...")
let $result :=
typeswitch ($action)
case element(search) return search:search($action)
default return fn:error($INV-REQ, "Invalid request...")
let $complete := cm:request-complete()
return $result
} catch($e) {
(xdmp:log(xdmp:quote($e)),
<error>400 Bad Request {$e}</error>)
}
Slide 31
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 3: Test results
Slide 32
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Some problems to fix
The test case passed, but we have some problems
1. search.xqy does not set the http response code when an
error occurs
2. We probably want an exception would be thrown for the
invalid request
3. Test does not handle exceptions
Slide 33
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 4: Fix SearchTest
public class SearchTest extends XQueryXmlHttpTestCase {
...
public void testInvalidRequest() throws Exception {
String req = "<request><XXX/></request>";
int statusCode = 0;
try {
Document doc = executeQueryAsDocument(req);
} catch (HttpResponseException expected) {
statusCode = expected.getStatusCode();
}
assertEquals(400, statusCode);
}
}
* We could also use JUnit4, which uses annotations for expected exceptions
Slide 34
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 4: Exception handling
fixed
Slide 35
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 4: Fix response code in
search.xqy
try {
let $request := cm:get-request-element(())
let $action := $request/element()
let $check := if ($action) then () else
fn:error($NO-REQ, "No request...")
let $result :=
typeswitch ($action)
case element(search) return search:search($action)
default return fn:error($INV-REQ, "Invalid request...")
let $complete := cm:request-complete()
return $result
} catch($e) {
(xdmp:set-response-code(400,"Bad Request"),
xdmp:log(xdmp:quote($e)),
<error>400 Bad Request {$e}</error>)
}
Slide 36
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 4: Now the test passes
legitimately
Slide 37
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 5: Inserting Test Data
 Gather 1 or more test data files (XML, text, or binary)
 Place the test data file(s) in a “resource” directory under the test
package
 Add a setUp() method to the TestCase
 Call one of the insertContent() methods in setUp()
protected void setUp() throws Exception {
this.setServicePath("example5/search.xqy");
super.setUp();
insertTestContent("resource/medline1.xml",
"/medline1.xml");
insertTestContent("resource/medline2.xml",
"/medline2.xml");
}
Slide 38
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 5: Testing Search
public void testSearch() throws Exception {
// Build request and execute
String req = "<request><search>cardiac</search></request>";
Document doc = this.executeQueryAsDocument(req);
// Use XPath to retrieve the response body, assert not null
XPath xpath = XPath.newInstance("/response/s:response");
xpath.addNamespace("s", "http://marklogic.com/appservices/search");
Element response = (Element) xpath.selectSingleNode(doc);
assertNotNull(response);
// Use JDOM to assert the total number of results
int total = response.getAttribute("total").getIntValue();
assertEquals(1, total);
// Execute more XPath to assert the highlighted keyword match
xpath = XPath.newInstance("s:result/s:snippet/s:match/s:highlight/text()");
xpath.addNamespace("s", "http://marklogic.com/appservices/search");
Text highlightText = (Text)xpath.selectSingleNode(response);
assertEquals("cardiac", highlightText.getText());
}
Slide 39
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 5: Test results
Slide 40
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 6: Testing a Main
module
xquery version "1.0-ml";
declare variable $uri as xs:string external;
declare variable $new-title as xs:string external;
let $doc := fn:doc($uri)
let $title := $doc/MedlineCitation/Article/ArticleTitle
return xdmp:node-replace($title,
<ArticleTitle>{$new-title}</ArticleTitle>)
Slide 41
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 6: Unit test
public class UpdateTest extends XQueryTestCase {
private String docUri = "/medline1.xml";
protected void setUp() throws Exception {
super.setUp();
insertTestContent("resource/medline1.xml", docUri);
}
…
Slide 42
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 6: Unit test (continued)
public void testUpdate() throws Exception {
String newTitle = "NEW_TITLE";
XdmVariable[] variables = new XdmVariable[] {
ValueFactory.newVariable(new XName("uri"),
ValueFactory.newXSString(docUri)),
ValueFactory.newVariable(new XName("new-title"),
ValueFactory.newXSString(newTitle))};
executeMainModule("example6/update.xqy",
null, variables);
// Verify update by running a query
String q = "fn:doc('/medline1.xml')"
+ "/MedlineCitation/Article/ArticleTitle/text()";
ResultSequence rs = executeQuery(q, null, null);
String updatedTitle = rs.asString();
assertEquals(newTitle, updatedTitle);
}
Slide 43
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 6 Results
Slide 44
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Agenda





Unit Testing Basics
Unit Testing XQuery via XQuery
Unit Testing XQuery via Java
End-to-End Testing using Selenium
Testing Tips
Slide 45
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 7: Simple Web
Application
Slide 46
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Example 7: Search Results
Slide 47
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Launching Selenium IDE
Slide 48
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Selenium assertText
Slide 49
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Selenium: Execute Suite
Slide 50
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Selenium: JUnit generated
source
Slide 51
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Agenda





Unit Testing Basics
Unit Testing XQuery via XQuery
Unit Testing XQuery via Java
End-to-End Testing using Selenium
Testing Tips
Slide 52
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Testing Tips
 “Whenever you are tempted to type something into a print statement
or a debugger expression, write it as a test instead.”1
 When to write tests:
 During Development - Write the tests first, you’re done when the test
passes
 During Debugging – When a defect is found, write a test for that defect,
debug/fix until the test passes
 Find bugs once2
 First time a human tester finds a bug, it should be the last time
1
http://junit.sourceforge.net/doc/testinfected/testing.htm
2
The Pragmatic Programmer, Andy Hunt, Dave Thomas
Slide 53
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Summary
 Developer testing should be an integral part of the development
process
 Incremental steps of parallel development and testing lead to a
quality end product
 Testing tools and frameworks can help with test development,
execution, and automation
 Developer testing will result in fewer application defects, higher
code confidence, greater agility
 Examples from this presentation available for download at
http://xquery-unit.googlecode.com/
Slide 54
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Thank You
Mark Helmstetter
MarkLogic Corporation
mark.helmstetter@marklogic.com
Slide 55
Copyright © 2010 MarkLogic® Corporation. All rights reserved.
Questions?
Slide 56
Copyright © 2010 MarkLogic® Corporation. All rights reserved.