
Distributing Data to Realtime Customers

The first thing to take care of is getting data into the right format
so it can be processed.
We start with gather.q which takes large csv files cnd accumulates them
into a memory structure.

How does gather  work with very big csv input files?
https://code.kx.com/trac/wiki/Cookbook/LoadingFromLargeFiles
Assume that setup.q has already run and set up a csv representation
of some data
and gather.q already has some data.
We can append the new data into the rantrade table that gather
already has accessed.
	q setup.q
	q gather.q


Now the next issue is distributing the data.
For this, we use a file that both gathers data from csv files in 
a chunking fashion (using .Q.fs) and sends it to customers.
gatherfeed.q (the client) is just like gather.q but it dispenses
data to a customer called
realtime.q (the server that receives the data)
which is on port 1111 (chosen arbitrarily)



/ client inserts into foo

	q realtime.q 
 	q gatherfeed.q

Now we add in logging.
https://code.kx.com/trac/wiki/Cookbook/Logging
	q realtime.q -l
	q gatherfeed.q

Now kill realtime.q by escaping.
When reinvoked by 
	q realtime -l
you get the state back.

Note that there is a file with the data called realtime.log.
That has the actual data.
If you remove it, you get nothing left.
This collects the log over time.



Callbacks see
https://code.kx.com/trac/wiki/Cookbook/Callbacks
Callbacks are a way for a client to call a server in such a way
to cause the server to call the client back when the server is done
with some task.

The callback uses asynchronous calls in both directions.
The server uses the .z.w to get the handle of the caller.
	q servercallback.q
	q clientcallback.q

What happens here is that the client calls the function foo on the server
with arguments 4 and `echoclient.
foo on the server takes two arguments, calls
its local function echoserver to get 16, retrieves the handle
of the caller and then calls the caller with `echoclient  and 4^3.
