So it would be really useful if you could also call a specific function directly. One way to do this would be if xdmp:apply accepted the same options node:

xdmp:apply(
    $function,
{xdmp:database("another-db")},
    $params
)

  2012-03-01



  2012-03-01



  01 March 2012

We could handle multiple element QNames by using a field, but the non xs:date format in doc 3 is a bigger problem.

Current Solution

The usual approach is to perform some normalisation, or “enrichment” at or before ingestion to create a value specifically for indexing:

  2012-01-01
  2012-01-01



  2012-02-01
  2012-02-01



  01 March 2012
  2012-03-01

This is fine until our query requirements change. We then have to change the enrichment script and either re-ingest everything or run a batch job with corb or similar to update every document.

Computed Index Idea

Instead of having to enrich the data at ingestion, why not do the work at index time instead? Similar to computed columns in relational dbs. I imagine something like extending the range index options to allow specifying a function to be executed on each fragment to get the index value:

declare function compute-date-index($fragment as node()) as xs:date?
{
  (: logic to get the date - same as used to enrich documents with index:date :)
};

We’d then need some new functions to access the index, new cts query constructors like cts:computed-range-query() etc. The trade off would be increased index time – and I’ve got no idea what sort of effect even the most simple function would have. The benefit would be far greater flexibility and ease of creating/managing indexes. It would also allow an index to be based on an XPath expression, useful when the schema has repeated element QNames whose meaning is dependant on context.

  2012-01-01
  
  	2
  	2012-02-01
  

I’m sure there are loads of implications I’m unaware of, so would be great to get people’s thoughts.

Check out the GitHub repository for more details.

The code looked something like this. The facets are defined in XML and there is a function to resolve a single facet which is called on each using function mapping. The total time was around 0.5 seconds.

The cts:element-values function has an option called ‘concurrent’. The description from the API documentation:

Perform the work concurrently in another thread. This is a hint to the query optimizer to help parallelize the lexicon work, allowing the calling query to continue performing other work while the lexicon processing occurs. This is especially useful in cases where multiple lexicon calls occur in the same query (for example, resolving many facets in a single query).

Unfortunately simply adding the concurrent option to the facet definitions doesn’t make any difference because the function mapping is blocked waiting for the response of each call. The same would happen if we looped over the facets in a FLWOR statement.

One solution is to rewrite the resolve-facet function to recurse through the facets, only accessing the results of cts:element-values after the recursive call. This way it will loop through the entire sequence of facets first and the expensive calls won’t be blocked. Total time is now 0.2 seconds – the time of the slowest facet.

By the way, if you’re using the MarkLogic search API then there’s no need to do anything, it already computes facets concurrently – sorry for wasting your time :)

It’s also worth noting that this technique isn’t limited to cts:element-values, other lexicon functions also support the concurrent option.

Tab renaming – double-click on a tab

Explore – new button in toolbar to explore current database

Auto-save – queries are now saved on execute instead of on exit

http://github.com/robwhitby/DQ

See it in action:
http://api.xqueryhacker.com

Download/Fork:
http://github.com/robwhitby/MarkLogic-API-Interface

The search interface I built to the Mark Logic API (see Searching the Mark Logic API function reference with Mark Logic) is now hosted on SourceForge for anyone to download and run locally.

http://mlapi.sourceforge.net

Try it out here: http://api.xqueryhacker.com

It requires Mark Logic Server v4 (community edition is fine, it’s only small). There’s a few steps to the installation, instuctions are in a readme file in the download. Please let me know if there are any problems and I’ll try to help.

So anyway, I loaded the content into Mark Logic and built a quick interface using ExtJS. We’ve been using it at work for about 6 months, and it’s now got a new home at api.xqueryhacker.com. You can choose to search just function names or all content, and browse by namespace. There’s a link in the header to switch between version 4.0 and 4.1 of the API.

In the next few weeks I’ll tidy up the code and put it up on SourceForge so if anyone’s interested they can install it locally – I’m not really intending to publicly host it permanently.

As always, all feedback welcome..

api.xqueryhacker.com

DQ v0.2 now available

Here’s a very early version of a new interface to CQ that I’m slowly working on, imaginatively named DQ. It aims to address a number of shortcomings in CQ, the web XQuery interface provided by Mark Logic.

Current features
Tabbed code editor (based on EditArea) with XQuery syntax highlighting, search and replace, and support of tab key
Save queries and results to file system
Auto-save of all query tabs using browser local storage (no more session confusion)
Highlighting a section of code and executing will run just the selected code (inspired by SQL Query Analyzer!)

Future features
Integration of XQuery API reference
Function auto-complete
More suggestions welcome!

Requirements
Recent version of CQ installed (I’m using 4.1.2)
Firefox 3.5 or IE8 (Chrome doesn’t style XML output, Safari and Opera not yet tested)

Installation
Unzip DQ folder in root of your CQ directory
Browse to: http://SERVER:CQPORT/DQ

The project is up on sourceforge:
https://sourceforge.net/projects/mldq/

Please give it a go and let me know what you think. I’d be interested in suggestions on improvements, extra features etc.

500 Internal Server Error

XDMP-EXTIME: xdmp:sleep(3000) -- Time limit exceeded
in /forest-backup-go.xqy, on line 56 [0.9-ml]

Now what do I do? Try again and cross fingers this time?

And why is restore not async in the admin ui? Backup is..

This is just the first forest, there’s another 4 to go. Not what I need on Friday afternoon :(

Rant over.

UPDATE… Rookie mistake believing the error message meant the restore had failed – turns out it has worked!