From Q for Mortals:
"Recall that a query against a partitioned table executes sequentially 
across the required partitions. 
I/O bandwidth limits performance on large partitioned tables because 
retrieving data from physical storage is much slower 
that processing that data once it is in memory. 
Simply put, the q process will spend time waiting for data, 
do a flurry of calculations, only to wait on the next chunk of data. 
The way to address this is to take advantage of 
parallel I/O and concurrent processing. 
To this end, we aim to spread a table across multiple I/O 
channels to facilitate parallel retrieval."
http://code.kx.com/wiki/JB:KdbplusForMortals/segments

Segmentation sets up partitioned tables across multiple storage devices.
Thus each storage device contains a partitioned table.
Recall that each partitioned table is both vertically and horizontally
partitioned.

q segmentsetup.q

Look at the structure:

/am -- top level directory
 /2009.01.01
  /t
  /q
 /2009.01.02
  /t
  /q

/nz -- another top level directory
 /2009.01.01
  /t
  /q
 /2009.01.02
  /t
  /q
 
  
q /db
can do queries from most restrictive to least

select from t where date=2009.01.01, sym=`ibm

Chapter 2 (for assignments 2 and 3)
Move segments around and adjust par.txt

Chapter 3:

Let's see how to duplicate a database.

cp -r /db /db2
cp -r /am /am2
cp -r /nz /nz2

q /db2

select from t where date=2009.01.01, sym=`ibm

Identical results from before.

Round-robin broker to route queries to different servers.
So, client communicates with broker which communicates with servers.

Thus, server performs a 
q /db
and waits.

server2 performs
q /db2
and waits.

The only thing we change is that the servers each can respond
to an interprocess procedure call that includes an sql statement.

Chapter 4: One server has an error and sends back an error signal.
Broker goes to another server.


Broker repairs a problem by copying in databases.
At this point, copy in whole database.
At some point in the future, copy in just the partition that is at issue.

Chapter 5: Use the broker to update all databases.
Broker keeps order among queries and updates, so if an update hits
the broker before a query, the query will see the effect of that update.


Now that we know how to do segmentation and how to query,
we can start replicating.
The effect we want is to have one process run on /db
but if there is a problem then go somewhere else.
Then try to copy things.

So, we have a client and several servers.
Each server controls a different database.
Client does a query an that routes to a server class.
The server class goes to one of several redundant servers.


Spread data over multiple data by just replicating the data
and adjusting par.txt.

Moving stuff is just OS level mv/cp.

@[h;"presval[10; 100 200 100 300]";"function doesn't exist"]

