/ From ~/q/network.d/src/network.a
/ First thing to do is to see how fast we can input data.
/ Second thing to do is to look for similar profiles and based on those
/ predict the next one.
/ Look at a handset and calculate the recent
/ variance of quality and from that, predict the future
/ variance.
/ In general, we are tracking statistics on particular entities (ships, packets,
/ base station to handset) and looking for patterns.
/ Here are three basic queries:
/ 1. We have many different sources of data (e.g. different network elements
/ through which packets flow), but we want to track particular entities
/ (e.g. particular packets).
/ 2. We want to predict some property (e.g. quality) by looking
/ for similar patterns in the past.
/ 3. At other times, we want to perform load balancing based on recent quality
/ from different servers and load on those servers.
/ I think the above are the three basic problems.
/ The packet-tracking application algorithmically looks like a join based on
/ packet id and a computation of statistics on packet ids.
/ The property prediction application is a time series prediction,
/ extrapolating the future from the past.
/ The load balancing problem gets some statistics (e.g. average quality
/ and confidence interval) from each server and the load and adjusts.
/ If I just want to perform data-driven improvement of quality, I take
/ previous handsets and look at their quality profiles to different
/ base stations and then correlate them over time.
/ The correlation can be fast or slow. That will tell me who will have
/ the best quality a few moments later. That will in turn suggest
/ whom to swap to.
/ Want to track a packet to see where the time is going.
/ Base station can't store much, but knows how long it takes
/ a packet to get to the base station.
/ Reports this up the chain.
/ Can do sampling by having a common hash function at the base station
/ and at the gateway.
numhandsets: 5;
numtimepoints: 10;
numbasestations: 4;
myhandsets: ` $ (numhandsets # "h") ,' string(1 + til numhandsets);
mytimepoints: 1 + til numtimepoints;
mybasestations: ` $ (numbasestations # "b") ,' string(1 + til numbasestations);
myqualities: (numhandsets * numtimepoints * numbasestations) ? 100
mycross: flip mybasestations cross myhandsets cross mytimepoints cross myqualities;
basestation: mycross[0];
handset: ();
quality: ();
time: ();
i: 0;
t:([]basestation:mycross[0];handset:mycross[1];time:mycross[2];quality:mycross[3])
`:network.csv 0:csv 0:t
delete t from `.
CREATE TABLE network(basestation STRING, handset STRING, time INT, quality INT)
LOAD DATA INFILE "network.csv" INTO TABLE network FIELDS TERMINATED BY ","
// INSERT INTO qualtable(basestation, handset, qualityvec)
// INSERT INTO qualtable
SELECT basestation, handset, quality FROM network ASSUMING ASC time GROUP BY basestation, handset
x: SELECT basestation, handset, avgs(3,quality) as movavg FROM network ASSUMING ASC time GROUP BY basestation, handset
count x