https://code.kx.com/trac/wiki/QforMortals/tables
https://code.kx.com/trac/browser/kx/kdb%2B/sp.q


Let's begin by generating a random table having 100,000 rows from
the stocks `ibm`hp`amaz`goog`aapl,
prices in the range 20 to 400,
and volumes in the range 100 to 100000 but multiples of 100,
times arbitrary in the range from 10 AM to 4 PM.
	q gendata.q

n: 100000;
stocks: `ibm`hp`amaz`goog`aapl;
rantrade:([]stock: n?stocks; price: 20 + n?380.0;amount: 100*(1+n?1000);
  time: 10:00:00.000 + n?06:00:00.000);

Points to note:
a) n ? stocks
b) 20 + n ? 380.0
c) n ? 06:00:00.000

save `:rantrade.csv


Use the table generated and 
select the name and last price of
each stock into a new table pricecol.

endofdayprices: select last price by stock from (`time xasc rantrade)


Notes:
a. You do not have to put "stock" in the select clause since
already in the by. In fact, you shouldn't do it.
b. Because the columns aren't ordered in
the original rantrade, this does not give us the
last stock by time.
That is why we use xasc.


A few notes on inserting data.


Can insert into a table using columns:
newstocks: n?stocks;
newprice: 20 + n?380.0;
newamount: amount: 100*(1+n?1000);
newtime: 10:00:00.000 + n?06:00:00.000;

`rantrade  insert (newstocks; newprice; newamount; newtime);

Can also insert one table into another:

insert[`rantrade; rantrade2];


Exercise 4. (Easy) Generate a second table having the same schema as in 
the random table
and bulk insert it into the random table.
Then form columns corresponding to this schema, each of the same
length as the columns of the original random table. Bulk insert those columns.
Which is faster? Table insert or column insert? (Use \t before
the statement).
	q bulkinsert.q





Spreading Load:

When queries are expensive, it's useful to spread them around
on different servers.
The idea here is that we'll have a generic server that will receive
requests from application-specific clients 
and send them to application-specific slaves.

Please look at 
https://code.kx.com/trac/wiki/Cookbook/LoadBalancing

Here might be a client:

h: hopen `:localhost:5001

neg[h]"select avg price*amount by stock from rantrade"; h[]
neg[h]"select max price*amount by stock from rantrade"; h[]
neg[h]"select min price*amount by stock from rantrade"; h[]
neg[h]"select var price*amount by stock from rantrade"; h[]


Here might be a slave server:

n: 100000;
stocks: `ibm`hp`amaz`goog`aapl;
rantrade:([]stock: n?stocks; price: 20 + n?380.0;amount: 100*(1+n?1000);
  time: 10:00:00.000 + n?06:00:00.000);

The generic router is mserve.q, copied from the website
and that we will treat as a black box.


Exercise 8. (Middle) Communicate a set of read only requests to a master
asynchronously and then they are routed to slaves.
Import into each slave the trade.csv table and then have the 
client send several requests, each one about a single stock.
        q mserve.q -p 5001 3 slaveserver.q
	q masterclient.q 

That's all you do. No need to start slaveserver.q.
mserve.q does this for you.
Note that the client issues async calls and then waits for the response
with h[].


It is sometimes useful to access a website, scrape the result,
and present the result as a table.
https://code.kx.com/svn/cookbook_code/yahoo.q
presents a nice example.
Let's analyze it.
I've added in some print statements that begin with 0N!

Even more useful is a statement that creates
a deliberate type error 1 + `x;
This causes an error, but from this error, one can
inspect variables and even change them.
Then you resume by typing :



Exercise 9. (Debugging and modification) Web access.
You want to build a function that accesses the yahoo site
and gives the last n days worth of close prices for a list of stocks.
	q yahoomod.q

Example:
q)yahoo[10;`GOOG`AMZN]
Sym  Date       Close 
----------------------
AMZN 2011.07.20 215.55
GOOG 2011.07.20 595.35
AMZN 2011.07.21 213.21
GOOG 2011.07.21 606.99
AMZN 2011.07.22 216.52
GOOG 2011.07.22 618.23
AMZN 2011.07.25 213.49
GOOG 2011.07.25 618.98
AMZN 2011.07.26 214.18
GOOG 2011.07.26 622.52
AMZN 2011.07.27 222.52
GOOG 2011.07.27 607.22
AMZN 2011.07.28 223.9 
GOOG 2011.07.28 610.94
AMZN 2011.07.29 222.52
GOOG 2011.07.29 603.69


