Dennis Shasha, Professor, Courant Institute, New York University.
Course Description
This three day 20 hour course is designed for programmers who (i) are strong in some language (e.g. C, C++, Java, Scheme, R, Matlab, SQL) (ii) but are new or beginners to Q. The goal is to make them proficient Q programmers. The course is structured as a hands-on course in which students are introduced to elements of Q and then work on exercises some of which might be puzzles. The class consists of the following parts:
-
Module 1: Computational Primitives.
- Fundamental primitives of q including list, string, linear algebra, aggregation, grouping and data generation operators.
- User-written functions.
- Integration of adverbs with built-in and user-written functions.
- Functions that can read from the command line and read and write to files.
- Hands-on programming exercises from statistics, basic finance calculations, Monte Carlo simulation, and date arithmetic on in-memory data. This module will be necessary for anyone doing serious scripting in Q.
-
Module 2: Input/Output and Inter-process Communication
- File input and output for small and large files.
- Inter-process communication
- Single client-multiple server communication.
-
Module 3: Databases and Basic Database Operators
- Creating tables using SQL semantics such as computing volume weighted average price.
- Ordered table semantics enabling moving averages with grouping.
- Integration of user-defined functions within select clauses.
- Generating large random data for simulation, bulk insert, adding columns, update, and delete performance communication.
- Foreign key joins, other equijoin types, asof, window joins.
- Stored procedures.
- Simulated trade histories and data warehouses
-
Module 4: Scaling Up
- Indexing (grouping/parting) and performance considerations
- Query tuning.
- Handling large tables through splaying, partitioning, compression, and segmentation.
- Analytical programming including machine learning, domain-specific data access languages, and finance-specific exercises.
The course is available in open (public) and closed (in-house) formats. Please contact me at shasha@cs.nyu.edu for further details. The next public course begins June 4, 2012 in Greenwich Village New York. (Days are June 4, 5, and 7 with the 6th as a homework day.)
About the instructor
I am a professor of computer science at the Courant Institute of New York University where I work with biologists on pattern discovery for microarrays, combinatorial design, network inference, and protein docking; with physicists, musicians, and financial people on algorithms for time series; and on database applications in untrusted environments. Other areas of interest include database tuning as well as tree and graph matching. Because I like to type, I have written six books of puzzles about a mathematical detective, a biography about great computer scientists, and technical books about database tuning, biological pattern recognition, time series, and statistics. For fun, I have written the puzzle column for various publications including Dr. Dobb's Journal and Scientific American.
If you're curious, here is my
CV
and a review paper on some our time series work.
