XQueryHacker

The search interface I built to the Mark Logic API (see Searching the Mark Logic API function reference with Mark Logic) is now hosted on SourceForge for anyone to download and run locally.

http://mlapi.sourceforge.net

Try it out here: http://api.xqueryhacker.com

It requires Mark Logic Server v4 (community edition is fine, it’s only small). There’s a few steps to the installation, instuctions are in a readme file in the download. Please let me know if there are any problems and I’ll try to help.

I’ve always thought it odd that the Mark Logic API function reference isn’t searchable. It seems obvious that it should be loaded into Mark Logic itself. Most of the time I use it for looking up the signature of a particular function, take cts:uris() for example – can anyone remember what order the parameters go in? Finding a function detail page from the home page is a mission, so I always just hit F3 to search the home page.

So anyway, I loaded the content into Mark Logic and built a quick interface using ExtJS. We’ve been using it at work for about 6 months, and it’s now got a new home at api.xqueryhacker.com. You can choose to search just function names or all content, and browse by namespace. There’s a link in the header to switch between version 4.0 and 4.1 of the API.

In the next few weeks I’ll tidy up the code and put it up on SourceForge so if anyone’s interested they can install it locally – I’m not really intending to publicly host it permanently.

As always, all feedback welcome..

api.xqueryhacker.com

Here’s a very early version of a new interface to CQ that I’m slowly working on, imaginatively named DQ. It aims to address a number of shortcomings in CQ, the web XQuery interface provided by Mark Logic.

Current features
Tabbed code editor (based on EditArea) with XQuery syntax highlighting, search and replace, and support of tab key
Save queries and results to file system
Auto-save of all query tabs using browser local storage (no more session confusion)
Highlighting a section of code and executing will run just the selected code (inspired by SQL Query Analyzer!)

Future features
Integration of XQuery API reference
Function auto-complete
More suggestions welcome!

Requirements
Recent version of CQ installed (I’m using 4.1.2)
Firefox 3.5 or IE8 (Chrome doesn’t style XML output, Safari and Opera not yet tested)

Installation
Unzip DQ folder in root of your CQ directory
Browse to: http://SERVER:CQPORT/DQ

The project is up on sourceforge:
https://sourceforge.net/projects/mldq/

Please give it a go and let me know what you think. I’d be interested in suggestions on improvements, extra features etc.

After it spent about 2 hours restoring a forest I got an annoying but slightly amusing error:

500 Internal Server Error

XDMP-EXTIME: xdmp:sleep(3000) -- Time limit exceeded
in /forest-backup-go.xqy, on line 56 [0.9-ml]

Now what do I do? Try again and cross fingers this time?

And why is restore not async in the admin ui? Backup is..

This is just the first forest, there’s another 4 to go. Not what I need on Friday afternoon

Rant over.

UPDATE… Rookie mistake believing the error message meant the restore had failed – turns out it has worked!

Oracle have published a PDF comparing their new XML DB to Mark Logic 4.1.

It’s amazing how openly hostile they are. Instead of comparing the two products on any meaningful basis, they resort to making vague, unsubstantiated claims about their superiority. It really comes across as desperation and highlights that Oracle clearly feel threatened by Mark Logic. It’ll be interesting to see how Mark Logic respond – CEO Dave Kellogg has said he’ll post a response on his blog in the next few days.

I’ve just improved the code snippets in my posts by using the excellent SyntaxHighlighter by Alex Gorbatchev.

I had to write a new brush for XQuery, but this was pretty straightforward (it has a much better extension system than I found in Notepad++ for example!). As well as XQuery 1.0 functions, keywords etc. I’ve included the MarkLogic 4.1 API.

Here’s an example..

xquery version '1.0-ml';
declare variable $URI as xs:string external;

declare function local:document-move-forest($uri as xs:string, $forest-ids as xs:unsignedLong*)
{
	xdmp:document-insert(
		$uri,
		fn:doc($uri),
		xdmp:document-get-permissions($uri),
		xdmp:document-get-collections($uri),
		xdmp:document-get-quality($uri),
		$forest-ids
	)
};

let $xml :=
  
    sdasdasdasd
  

If you want to use it download SyntaxHighlighter, and then get the XQuery brush files:

shBrushXQuery.js
shThemeXQuery.css

Example usage:

<script type="text/javascript" src="scripts/shCore.js"></script>
<script type="text/javascript" src="scripts/shBrushXQuery.js"></script>
<link type="text/css" rel="stylesheet" href="styles/shCore.css"/>
<link type="text/css" rel="stylesheet" href="styles/shThemeDefault.css"/>
<link type="text/css" rel="stylesheet" href="styles/shThemeXQuery.css"/>
<script type="text/javascript">
	SyntaxHighlighter.config.clipboardSwf = 'scripts/clipboard.swf';
	SyntaxHighlighter.all();
</script>

<pre class="brush: xquery;">xquery version '1.0-ml';</pre>

Next job is to change the layout of this blog to make the content column wider…

Y74C9EM8ZP43

Good news! MarkLogic has been listening to it’s customers and are working on adding XSLT 2.0 support. Norman Walsh announced the news at a MarkLogic user group meeting in New York – see his blog post for details. In the past the argument has always been XSLT is unnecessary because XQuery is enough, so it’s nice to see acceptance that XQuery isn’t the best tool for every job.

I can see this being of great benefit at our company where we have far more experience with XSLT compared to XQuery. The separation of query and output will allow us to write cleaner, more focussed XQueries and leave the specifics of different output formats to XSLT. This should make debugging and performance tuning XQuery much simpler.

Only downside is we can’t start using it straight away, and as yet there’s no information on when it will be released.

Our development database had been allowed to grow unchecked and the single forest was well over the recommended 200GB max size. So I created 2 new forests to move all the documents into (I think that’s faster than moving half into one new forest and then having loads of deleted fragments to clean up in the initial forest).

The technique is to first set the initial forest as delete-only and then re-insert all the documents without changing them. Inserting a document at a URI that already exists will normally update it, but because the forest is delete-only, it will be saved in one of the new forests instead.

Here’s the code to move a single document

xquery version '1.0-ml';

declare function local:document-move-forest($uri as xs:string)
{
  local:document-move-forest(
    $uri,
    xdmp:database-forests(xdmp:database())
  )
};

declare function local:document-move-forest($uri as xs:string,
  $forest-ids as xs:unsignedLong*)
{
  xdmp:document-insert(
    $uri,
    fn:doc($uri),
    xdmp:document-get-permissions($uri),
    xdmp:document-get-collections($uri),
    xdmp:document-get-quality($uri),
    $forest-ids
  )
};

This will keep the document’s permissions, collections and quality. And the properties document is maintained and moved with it.

Call this function using Corb on all the documents in the original forest, and then just remove the now empty forest.

Although MarkLogic isn’t officially supported on Ubuntu, it will run fine – after jumping through some hoops to get in installed.

Thankfully someone has already detailed the process:
“MarkLogic install on Ubuntu 9.04 and libbteuclid and libbtunicode

A port of common spreadsheet functions to XQuery by the FLWOR foundation:
http://sourceforge.net/apps/mediawiki/zorba/index.php?title=EXcel_Function_Library

Contains loads of useful functions, especially for statistics. It says tested with Zorba, Saxon, and eXist. By the looks of it they don’t use any proprietary XQuery, so should be fine in MarkLogic too.