eXist Native XML Database (NXD)

Edited by Lim Siong Boon, last dated 22-Sep-2012.

email:    contact->email_siongboon  

website: http://www.siongboon.com



 

Introduction

I was looking for implementing a database for my project and come across Apache Xindice. Ii took me half a day to understand something about Xindice. When I am just about to deploy it, I realised that it has retired in 2011. It was then I search again for other native XML database. eXist and BaseX were the two database I cannot decide on. eXist seems very nice and have a nice website. I decided to try it.

 





1. Installing eXist

eXist supports many operating system and is fairly simple to install.

For Window user, download the *.jar, and double click it to begin the installation process.
You can download from eXist website.
Here is a version that I was using eXist-setup-2.0-tech-preview.jar

You can also refer to the eXist quick start guide available online.
or http://xml.sys-con.com/node/155664

- Install the eXist in the folder "D:\eXist". The installer does not allow me to install in the C:\. You need to have your JDK installed before installing eXist.
Choose a password for your user admin. I use "password", for testing purposes.
- eXist is actually a server running on port 8080 (default). My Tomcat is already using port 8080, so this eXist will have to change to another port.
I have assigned eXist to use port 8899. To change the port settings, open and edit the file "/tools/jetty/etc/jetty.xml" with the following,
<Set name="port"><SystemProperty name="jetty.port" default="8899"/></Set>
eXist is also using another port 8443 for communication. If change is required, edit the following,
<Set name="confidentialPort">8443</Set>
<Set name="Port">8443</Set>
- Click on the program "Database Startup" or execute "D:\eXist\bin\startup.bat" to launch eXist.

A console will be launched, running some log output. The eXist server is now ready listening to the port 8899.

2. Test the eXist running server

- Open a web browser and key in the following URL,
"http://localhost:8899/exist/index.xml"
This is the address (locahost:8899) where eXist is running.
You should see the following screen shot, which is very much like the eXist own website..

3. Log in to the administration page

To login, go to the list of menu on the left side of the page. Under the section named Administration, click on the admin link.
User: "admin"
Password: "password" (as set during the installation process)

5. Create a new collection Go to Browse>Collection, enter in a new name for your own database collection under the text field labelled as "New collection".
New collection: "MyData"
Click on the button "Create Collection" to create. A collection is like a folder in eXist.

4. Upload an existing *.xml data file into the collection.

Click on the newly created "MyData" collection/folder.

Download the following test file.
test file: db_study.xml

Browse the location of the above db_study.xml file, and click upload.
The XML file database is now loaded onto the eXist server "/db/MyData/db_study.xml".

5. XQuery. Sending query to the database.

To retreive/store information to the XML, we have to learn about XQuery. XQuery is a standardise statement to help programmer interact with XML database. It is something like making a SQL query call, different in syntax.

We can try out some command by using the XQuery Sandbox which is provided by the eXist hosting server.
The webserver address is as follows, http://localhost:8899/exist/sandbox/sandbox.xql
You can also local Sandbox from the Homepage>Eaamples>XQuery Sandbox.
XQuery Sandbox provides a text box for you to enter and send the query. It is very useful for testing or learning the query that you have made.


Besides retreiving/storing of information to database, the system can also do simple task like calculation.

reference for XQuery Language Path Expressions:
http://www.w3.org/TR/2001/WD-xquery-20010215/

Send out our first simple query.
Type>5+2
then click send. Notice that the result returns 7
Type>"Hello"
click send. Notice that the result returns Hello. The XQuery echos back

To get the timeof the server running the database,
Type>current-time()
result>11:59:50.38+08:00

To execute multiple XQuery statement, put a comma between your statement. The XQuery returns 3 results.
Type>5+2, "Hello", current-time()
Result 1>7
Result 2 >Hello
Result 3 >11:59:50.38+08:00

The XQuery that were present above is not very useful, but they provide simple illustration about the syntax of XQuery.
Now we shall proceed to understand the manupulation of data for a xml database. Using the db_study.xml that we have loaded in our previous step.

-----------------------------------------------------------------
A sample of the content in db_study.xml
<root>
  <element1 id="111">
    <element2 key="key1">1-item1</element2>
    <element2 key="key2">1-item2</element2>
    <element2 key="key3">1-item3</element2>
  </element1>
  <element1 id="222">
    <element2 key="key1">2-item1</element2>
    <element2 key="key2">2-item2</element2>
    <element2 key="key3">2-item3</element2>
  </element1>
  <element1 id="333">
    <element2 key="key1">3-item1</element2>
    <element2 key="key2">3-item2</element2>
    <element2 key="key3">3-item3</element2>
  </element1>
</root>

-----------------------------------------------------------------

1) This is the first simple query to extract the whole xml file.

XQuery:
doc('MyData/db_study.xml')

Return 1 Result:
<root>
  <element1 id="111">
    ....
  </element1>
  ....
  ....
  <element1 id="333">
    ....
  </element1>
</root>
   
2) This query returns the immediate root element  
XQuery:
doc("MyData/db_study.xml")/root
 Return 1 Result:
<root>
  <element1 id="111">
    ....
  </element1>
  ....
  ....
  <element1 id="333">
    ....
  </element1>
</root>
   
3) This query returns the next element of root/element1. If the previous statement has already select the database file, the next statement can omit the doc.
XQuery:
/root/element1
Return 3 Result:
  <element1 id="111">
    ....
  </element1>

  <element1 id="222">
    ....
  </element1>

  <element1 id="333">
    ....
  </element1>
   
4) If you are not sure which level (the path of the element) for the element <tagnext>, simply place "//".
The query will search deep into the hierarchy, and return you the result. Result will be the same as the previous query.
XQuery:
//element1
 Return 3 Result:
  <element1 id="111">
    ....
  </element1>

  <element1 id="222">
    ....
  </element1>

  <element1 id="333">
    ....
  </element1>
   
5) A search for all the element2 tags.
XQuery:
//element2
Return 9 Result:
    <element2 key="key1">1-item1</element2>
    <element2 key="key2">1-item2</element2>
    <element2 key="key1">1-item1</element2>
  <element2 key="key2">1-item2</element2>
    <element2 key="key1">1-item1</element2>
   <element2 key="key2">1-item2</element2>
    <element2 key="key1">3-item1</element2>
   <element2 key="key2">1-item2</element2>
   <element2 key="key1">1-item1</element2>
   
6) To select the particular unique attribute of id=111
XQuery:
//element1[@id="111"]
 Return 1 Result:
  <element1 id="111">
    <element2 key="key1">1-item1</element2>
    <element2 key="key2">1-item2</element2>
    <element2 key="key3">1-item3</element2>
  </element1>
   
7) To list all element2 tag of a particular unique attribute of id=111
XQuery:
//element1[@id="111"]/element2
Return 3 Result:
    <element2 key="key1">1-item1</element2>
    <element2 key="key2">1-item2</element2>
    <element2 key="key3">1-item3</element2>
   
8) To list a particular attribute of element2 tag of a particular unique attribute of id=111. Append /text() to the end of the query to extract out the text.

XQuery:
//element1[@id="111"]/element2[@key="key1"]

 Return 1 Result:
<element2 key="key1">1-item1</element2>

XQuery:
//element1[@id="111"]/element2[@key="key1"]/text()

 Return 1 Result:
1-item1
   
   
9) To add an element2 (float value) to all element1 object. The first statement open the database "MyData/db_study.xml". The second statement after the comma execute the insert instruction to the last opened/access database.
XQuery:
doc("MyData/db_study.xml"),
update insert <element2 key="key4">8.5</element2> into /root/element1

Return 1 Result:
display the full xml file
   
10) To update the element1(id=222), element2(key="key4") with 5.5

XQuery:
update value /root/element1[@id="222"]/element2[@key="key4"] with '5.5'

 Return 1 Result:
   
10) To delete the element1(id=333), element2(key="key4")
XQuery:
update delete /root/element1[@id="333"]/element2[@key="key4"]
 Return 1 Result:
   
11) To update the value if element1(id=333), element2(key="key4") is available, insert the element2(key="key4") is not available. Additional bracket around the expression makes it easier to read.

XQuery:
if (exists(/root/element1[@id="333"]/element2[@key="key4"])) then update value (/root/element1[@id="333"]/element2[@key="key4"]) with ('2.5') else update insert (<element2 key="key4">3.5</element2>) into (/root/element1[@id="333"])

 Return 1 Result:
   
12) To list the item id with key4 > 5.0 using comparison operator. Full XQuery statement is written here. This is call a FLWOR Expressions, which is like a Select statement in SQL. for $v databind variable $v to the path.

XQuery:
for $p in doc("MyData/db_study.xml")/root/element1
where $p/element2[@key="key4"] > 5.0
order by $p
return $p

Return 2 Result:
  <element1 id="111">
    <element2 key="key1">1-item1</element2>
    <element2 key="key2">1-item2</element2>
    <element2 key="key3">1-item3</element2>
    <element2 key="key4">8.5</element2>
  </element1>
  <element1 id="222">
    <element2 key="key1">2-item1</element2>
    <element2 key="key2">2-item2</element2>
    <element2 key="key3">2-item3</element2>
    <element2 key="key4">5.5</element2>
  </element1>
   
13) Extracting Attribute Values. (Can return the number of result, but no display in eXist Sandbox)

XQuery:
for $p in //element1
where contains($p, "1-item1")
return $p

Return 1 Result:
<element1 id="111">
    <element2 key="key1">1-item1</element2>
    <element2 key="key2">1-item2</element2>
    <element2 key="key3">1-item3</element2>
    <element2 key="key4">8.5</element2>
  </element1>
   
14) Extracting Attribute Values. (Can return the number of result, but no display in eXist Sandbox)

XQuery:
/root/element1/@id

Return 2 Result:
 
   
15) Assigning variables

let $var := 3000
let $old := $users_doc/user[userid=$userid]/@trusted="oldvalue"

 
   
   
   


if (exists(/p/pl[id=6]/name))  then update insert <name>thenname</name> into /p/pl[id=6]  else update value /p/pl[id=6]/name with 'elsename'

 

6. Make XQuery through web browser URL

Location of the *.xml file
http://localhost:8899/exist/servlet/db/MyData/db_study.xml

note:
What I realised is that there is no security on the database file. Even if it is not admin login, anyone can still key in the path to peek at the *.xml database file.

 

XQuery through URL
http://localhost:8899/exist/servlet/db/MyData/db_study.xml?_query=//element1[@id="111"]
or
http://localhost:8899/exist/servlet/db/MyData/db_study.xml?_query=//element1[@id=%22111%22]
where %22 is ".
or
http://localhost:8899/exist/servlet/db/MyData/?_query=//element1[@id=%22111%22]
or
http://localhost:8899/exist/servlet/?_query=doc("MyData/db_study.xm")//element1[@id="111"]
http://localhost:8899/exist/servlet/db?_query=doc('MyData/db_study.xml')//element1[@id="111"]

return:

<exist:result exist:hits="1" exist:start="1" exist:count="1">
   <element1 id="111" a="hello">
      <element2 key="key1">1-item1</element2>
      <element2 key="key2">1-item2</element2>
      <element2 key="key3">1-item3</element2>
      <element2 key="key4">8.5</element2>
   </element1>
</exist:result>

-----------------------------------------------------------------------------------

update data through URL
localhost:8899/exist/servlet/MyData/db_study.xml?_query=update value /root/element1[@id="222"]/element2[@key="key4"] with "5.2"

return the following error because the admin/password is required to update information to the database,

HTTP ERROR 500

Problem accessing /exist/servlet/MyData/db_study.xml. Reason:

    An error occurred: getDocumentURI not implemented on class org.exist.dom.DocumentImpl

-----------------------------------------------------------------------------------

 



7. Sending XQuery with Java.

Download Java Library for XQuery, exist-xqj-api-1.0.1.zip