Feed handlers sound scary. They are often written in some language other than q (or at least the ones I was exposed to initially) such as java which is a pain to look at once you get used to a compact language like q. I always avoided feed handlers until the day I read a white paper on them written by Nathan Perrem from First Derivative as part of their lecture series. Suddenly, they didn’t seem so frightening. They were like any other piece of code that I dealt with. Surprisingly, there wasn’t much to setting a sample feed going. In fact, it was really easy.
In this post, I present to you my code for a feed handler (inspired by the white paper mentioned earlier).
What does a feed handler do?
Before we get started, we have to understand what it is that a feed handler does. A feed handler’s job is to parse the data and push it to a subscriber (ticker plant mostly). For example, the Reuters’ SSL feed handler captures the data from the source, parses the data as configured, and then pushes it to a ticker plant to be distributed to multiple subscribers.
How does it do that?
Let’s break down the three purposes of a feed handler:
Capturing/parsing data – this can be done however you please. This post doesn’t cover this as we are interested in just generating some random data rather than actually capturing data from a source. One way to capture data is to simply read from a text file every day (necessary for some data sets). In this post, we are going to generate a table with random values using the code below:
`syms set `MSFT.O`IBM.N`GS.N`BA.N`VOD.L; /stocks `ex set `NASDAQ`NYSE`ICE`BATS; `n set 5; /number of rows per update ([]date:.z.d;sym:n?syms;time:`time$.z.t;exch:n?ex;exchDate:.z.d;exchTime:`time$.z.p;price:n?130.00;size:n 100000;cond:`;seqNum:0j;globalSeqNum:0j)
Generating data continuously – A feed handler doesn’t just read and push data once (it could but that’s just boring). It is more interesting and realistic when it is handling tick by tick real-time data. For that to happen, you need to set a timer so that the random table generates regularly.
Lucky for us, q has a built-in function called .z.ts. The function is simple…it is empty by default. You can redefine it as anything. You can even redefine it to publish "All hail Himanshu!" all day long. And you can set the timer to configure how often to run this function.
system"t ",string 10000; /trigger timer every 10000ms
.z.ts:{[]
/ define whatever you want to define here.
};
Pushing data to ticker plant – q has a built in function called .u.upd which takes two arguments: table name and data. We can use it to update the target table.
In the code below, we first open a handle to ticker plant and then set the timer before defining the timer function. The timer is set to trigger every 10 seconds. The .z.ts function only contains one line. All it does is run a remote query using the handle h. It runs the function .u.upd and passes it the name of the table to be updated and the data with which to update the table.
`h set hopen`::5010; /connect to tickerplant
system"t ",string 10000; /trigger timer every 10000ms
.z.ts:{[]
h(".u.upd";`trade;([]date:.z.d;sym:n?syms;time:`time$.z.t;exch:n?ex;exchDate:.z.d;exchTime:`time$.z.p;price:n?130.00;size:n?100000;cond:`;seqNum:0j;globalSeqNum:0j));
};
And that’s it, folks! It’s that simple. Just open a handle to ticker plant, set timer and define the .z.ts function and you have got yourself a feed handler.