Limit Order Book Visualisation
Visualising High Frequency Trading te Bitcoin
Sat Nov 22 02:26:07 UTC 2014
Some time ago, I had a look at the seasonality of traded volume on Bitcoin exchanges, up until December 2013. My objective wasgoed to determine approximate trading sessions for Three popular exchanges. I found that the intra-day volume followed a zuigeling of sinusoidal pattern, which I attributed spil the tell tale sign of the presence of humans on the exchanges. This article attempts to visually explore the extent of algorithmic trading ter Bitcoin, with a concentrate specifically on the Bitstamp exchange and limit order book gegevens.
Limit Order Books
Feel free to skip this part if you are already familiar with the internal workings of a limit order book and exchanges te general.
An exchange/bourse is a marketplace where agents can buy and sell _things_ to each other. There are many ways for an exchange to facilitate this, however the most popular mechanism, and the subject of this article, is the concept of a Limit Order Book.
The very first main component of the exchange, serving parties interested te buying or selling units of some object (a stock, contract, currency, etc.) is the Limit Order Book. The Limit Order Book is a type of auction mechanism for recording the _passive_ trading intentions of individuals (people, organisations, algorithms..). A passive intention to buy an asset, is a bid to buy that asset at a price which is less than, or equal to, the current best bid for the asset ter question. Similarly, a passive intention to sell an asset, is a (asking) price which is greater than, or equal, to the current best asking price. The price of the bid or ask order is known spil the Limit Price. Te the case of a Bid, the limit price is the maximum amount a party is ready to pay to buy the asset, and ter the case of an Ask, the limit price is the ondergrens amount the party is ready to sell the asset for.
The order book consists of Two sides. The Bid side, which contains limit orders for parties interested ter buying, and the Ask side, which contains limit orders for parties interested ter selling. Each side of the book is a priority queue: orders are ordered by their limit prices, such that on the bid side of the book, parties willing to pay more for the asset are placed at the top of the book. Similarly, parties willing to sell for less are placed at the top of the ask side of the book. The top of the bid and ask sides of the book are known spil the current best bid and ask. The difference inbetween the best bid and ask is known spil the market spread. If Two or more parties insert orders at the same bid or ask price, then the orders are ordered according to arrival time. Specifically, each side of the order book is a price priority FIFO queue.
The 2nd main component of the exchange is the matching engine. The role of the matching engine is to match incoming orders to buy or sell against limit orders resting at the various levels ter the order book. The resting limit orders are said to be market makers because they are providing liquidity to the market. Ter order for a trade to occur, a trader (market taker) vereiste cross the market spread and pay the asking price if the trader is buying, or the bid price if the trader is selling. Specifically, limit orders are liquidated from the order book according to their price/time ordering, by impatient traders who require their orders to be executed instantaneously. The arrival rates and volume of thesis impatient orders (request) vs the amount of resting volume, and the rate at which it is replenished (supply), is the mechanical essence of the exchange.
To illustrate the Two components of the limit order book and exchange matching engine, the following table represents the top Ten best bid and ask orders to buy and sell Bitcoin on the Bitstamp exchange at Ten:13am on the 2nd November 2014. The left forearm side contains the top Ten best bids from parties interested ter buying a specific amount at a specific price. The right arm side contains the top Ten best asks from parties interested te selling. The very first row represents the current best bid and ask prices along with the amount of volume available at those prices. The Liquidity katern is the cumulative sum of volume available at all price levels.
Now, suppose that an order to instantly buy Ten Bitcoin arrives. The exchange matching engine will honor this request by removing the very first Four ask orders and then partially removing the 5th order, resulting ter Five trades, and widening the spread, such that the fresh best ask te the order book after packing the order request will be 12.89324700 @ $325.68. Similarly, if an order to instantly sell Five Bitcoin arrives, the order will consume (succesnummer/lift) the very first Four bid orders and partially consume the 5th, leaving the best bid at 13.25048100 @ $324.56. The initiator/aggressor of the buy trade (market order) pays the VWAP (Volume Weighted Average Price) for Ten Bitcoin:
((325.38*1.12400000)+(325.39*0.61464703)+(325.45*1.20200000)+(325.67*0.45260000)+(325.68*6.60675297)) / Ten = $325.60
It is te the rente of the aggressor who placed the market order to buy Ten Bitcoin to receive the lowest possible VWAP. The VWAP received depends on the amount of volume available at each price level ter the order book. And since the exchange matching engine matches incoming market orders to resting limit orders by price/time priority, the VWAP depends on the Liquidity: the cumulative sum of volume at each price level required to pack the incoming order.
The amount of volume/liquidity available on each side of the order book determines the order book “form”. The following graph shows the form of the order book for the same point te time spil above (Ten.13am). The y-axis shows the cumulative sum of volume available at each price level +-5% from the current best bid/ask. Higher amounts of volume closer to the current best bid/ask represent better liquidity (market orders receive better VWAPs and leave less influence). Te a “balanced” market, the form of the book would show up to be symmetrical. Ter this example, there is slightly more volume available on the ask side of the book than there is on the bid side (within +-5% of the current best bid/ask). Wij might say that there is an imbalance te this order book: parties are more interested te selling than ter buying.
The dynamics of limit order books is a complicated subject, and I only shortly describes some elementary mechanical aspects of them here ter the hope to keep this article self contained. The following papers opoffering excellent insights into the subject:
Order Book Event Gegevens
I collected order book event gegevens from the Bitstamp exchange overheen a Four month period, inbetween July and October (2014), resulting te a dataset of
33 million individual events, A miniscule dataset ter comparison to the throughput on “conventional” exchanges, see (Nanex: High Frequency Quote Spam) for example.
The gegevens contains individual order book events describing the life cycle/state of individual limit orders. An order event may be one of Add, Modify, Delete. An Add event corresponds to the injection of a limit order into the order book, Modify corresponds to a partial pack of an order, and Delete corresponds to the removal of an order from the book (either from a accomplish pack or cancellation). A “raw” event is structured spil goes after:
[ id, timestamp, price, volume, activity, side ]
Where id = Unique identifier for the limit order. Timestamp = the time at which the initial Add order arrived at the exchange, Price = the limit price, Volume = the amount of volume for the order te Bitcoin, activity = Add, Modify, or Delete, and eventually, side = Bid or Ask, depending on the direction (buy/sell) of the order.
Te reference to the observation that the average traded volume by time of day goes after a sinusoidal pattern, the average number of events arriving every 15 minutes goes after the same pattern, spil shown ter the following graph:
The graph shows the average number of order book events occurring averaged overheen each 24 hour period (white line). The regression (crimson) shows a clear sinusoidal pattern. Trading activity is lowest at
3am and peaks at
3pm (UTC). The most active continuous period with respect to the mean number of events (blue line) is inbetween 9.30am and 10pm.
While the event dataset consists of
33 million events, thesis events can be violated down into individual orders and their types. Te total, of the identifiable order types, there were 14,619,019 individual “flashed orders” (orders added and zometeen deleted without being succesnummer) signifying 93% of all order book activity, 707,113 “resting orders” (orders added and not deleted unless kasstuk) and 455,825 “marketable orders” (orders that crossed the book resulting ter 1 or more reported trades).
Visualising Order Book Events
Spil mentioned previously, all orders have a life cycle. An order is very first added, it may then be updated, if it is partially packed, and eventually will be deleted if it has bot cancelled by the trader or fully packed. The following visualisation shows 1 hour of limit order events on the 27th October. The y- axis represents the limit price for both bid and ask orders which can be distinguished by the colour crimson for ask and blue for bid. The circles are an approximate guide to the amount of volume te the order. And furthermore, if the volume is being added to the order book, the circle is opaque, whereas if the order is being eliminated (cancelled), the circle is empty. The point at which the bid orders meet the ask orders corresponds to the top of the order book, such that higher priced bids and lower priced asks are closer to the current market price. Here, the limit price has bot restricted to the $342:$375 interval.
Ter reference to the above graph, it is interesting to note the apparent regularity and order of the event gegevens. I think that given this evident regularity, not to mention quantity, it would be reasonable to assume that most, if not all, of the activity ter this time period is the result of the systematic activity from automated market participants. Zooming te closer (shown below), to a 30 minute range, further highlights the systematic activity surrounding the market midprice. At this time resolution, it is possible to start to interpret the type of activity occurring ter the graph.
At each price level, orders are being added and then liquidated on a very periodic poot. This is the result of position re-allocation: Ter response to perceived switching conditions, trading agents are re-positioning their orders. The switching conditions could be due to a switch te midprice, order book depth, or some outer event. Whatever the reason, it is interesting to observe that the cyclic patterns could be the result of agents re-positioning their orders te response to other agents re-positioning. Ter the below graph, zoomed into a 15 minute time period, the order re-allocation is slightly more evident. The apparent X-X-X pattern could demonstrate the presence of numerous orders at different price levels originating from the same strategy being adjusted.
Zooming te further, to a Five minute interval and closer to the midprice, the re-allocation is clear. Looking at the 2nd “X” pattern on the bid side of the order book (blue), wij can see that orders are more or less at the same time being added (opaque circles) and liquidated (empty circles). The individual X’s te this case seem to vertoning a battle inbetween Two processes: The very first process adds orders (by descending price), while ter response, a 2nd process seems to cancel orders (by ascending price). The 2nd X ter the series, perhaps shows the very first process deleting the orders it added ter the very first X (this time te ascending price order) and the 2nd process adding back it’s previously deleted orders.
The processes seem to be feeding off one another, yielding this X-X-X effect. For mij, this highlights the most interesting opzicht of order book event gegevens: If the market participants are event driven, then the market dynamics are the result of (agents responding to (agents responding to (agents responding to (events)))) and so on. At this level, everything can be diminished to mechanical interactions. The market can be viewed spil a Ingewikkeld Adaptive System, which is a enormous subject, well beyond the scope of this article. For anyone who may be reading this with a robotics/cybernetics/ai background, I recommend Eugene A. Durenard’s Professional Automated Trading which contains some excellent insights.
The X-X-X order re-allocation pattern shown above is due to the environment te which the traders/agents are operating ter and can be reproduced with some plain rules. Given that a limit order book is a (price,time) queue, the only way to leap queue position (with respect to a bid order) is to increase the limit price. By placing an order some distance from the current best bid, spil is the case here, there is a chance that the order will be kasstuk. The likelihood of the order being succesnummer, essentially, decreases spil a function of distance (ter volume) from the current best bid, the market order arrival rates (flow), and the rate at which liquidity is replenished (how resilient the order book is to market impacts). Spil such, if an tuut has a bid order placed at say, $347 for Ten Bitcoin, the juut placing the order has some view on the likelihood of their order being kasstuk. Most likely, they are looking to samenvatting value from infrequent market influence events. If another tuut comes along, with a similar view and places an order at $347.01, then abruptly the original view of the very first tuut switches, since there is now more volume “te vooraanzicht” of the order at $347. This may cause the very first smeris to re-position the order to $347.02, which ter turn may cause the 2nd smeris to re-position even higher and so forward. Te this case, the contesting processes are likely to be reacting to both the Add and Delete events of the other party: When the one tuut sees the other delete, it defaults to it’s previous position. This might explain the horizontal sequence.
This is of course, subjective and based on an initial interpretation of the above visualisation. If anything, it is at least clear that there are predominantly systematic processes involved.
Visualising Order Cancellations
Rapidly deleted orders, or “fleeting orders”, seem to get a loterijlot of negative attention te the media. Te conventional markets, some have associated rapidly cancelled orders spil a manipulative tactic employed by high frequency traders. My (discreet) belief is that very likely most (but not all) of the time, high cancellation rates can be attributed to various strategies feeding off one another/re-adjusting positions, spil described above. The fact that limit order books are price/time priority based (there are also other schemes), ter a way, compels participants to permanently re-position their orders: the market participants are most certainly not collaborating ter any way, so most of the time the activity seen te the order book is most likely the result of some giant reactive programming exercise. Naive side note: Maybe there exists an alternative (spel theoretical) scheme, ter which Two market participants could somehow (at least temporarily) agree to “share” a queue position, instead of hopping in-front of one another.
Having said that, there are indeed various ways that high cancellation rates can be associated with more “predatory” high frequency trading. Given that trading ter Bitcoin, on Bitstamp, is entirely unregulated, and basically anything goes, I would expect (and hope) to see evidence of manipulative activity. Spil an initial step, the following Four visualisations showcase every single “fleeting” order event overheen a Four month period. The intention of the visualisations are not to expose any “suspicious” order cancellation activity, but more to further highlight the predominance of systematic trading on this particular exchange.
Each graph shows the amount of deleted volume (vanaf event) along the horizontal axis, vs time, ascending on the vertical axis. Each point ter the graph corresponds to a cancelled order event, where the colour blue differentiates the cancelled Bid events from the cancelled Ask events. I define “flashed volume” spil an order that is placed te the order book and straks cancelled. Spil such, thesis visualisations exclude any partially packed, then cancelled orders. Eventually, the horizontal axis is plotted te log-scale, since the distribution of order size decays exponentially: puny orders are much more frequent (This is another subject altogether).
The very first observation is again the striking regularity/geometric appearance of the gegevens. Simply plotting the cancellations by volume shows demonstrable systematic activity (there are other dimensions, this is the most demonstrable to explore). The 2nd observation, is that the cancellations emerge to toebijten ter regimes or clusters, with cancelled bid volume sometimes far greater than cancelled ask volume, and vice-versa. This may lead to the interpretation that orders are being re-allocated either te response to some outward news, or to pursue the market.
Zooming ter on a single day (this time with volume shown on a linear scale), it is overduidelijk that most of the volume falls under 100 Bitcoin vanaf order, at least for this particular day (2014-11-02) shown below. Which makes some sense, since on that particular day the VWAP wasgoed $323.95 and $32,395 is fairly a loterijlot of money to place te a single order.. Te fact, 95% of the volume on this day is <,= 60.89 with the median volume being Five Bitcoin.
Filtering the graph to include cancelled volume <,= 100 shows the density of cancelled volume for this day (260,524 deleted order events). The next few visualisations voorstelling the same time period, however with a progressively lower limit on the size of volume displayed.
Visualising Order Book Depth and Liquidity
The available volume at any given price level is the sum of the volume from the individual orders enqueued at that price. Spil such, the available volume at a price level can be viewed spil an individual time series that switches when a fresh order is enqueued (Add), liquidated (Cancel), or packed (Modified). The cumulative sum of the volume available at price levels above and below the market midprice corresponds to the level of liquidity available ter the market.
A common treatment to viewing the order book volume is to plot the cumulative sum of the volume on either side of the book (spil shown ter the introduction). This treatment shows available liquidity, order book imbalance and volume size at each level spil a type of step function, more generally, the order book “form”. The problem with this treatment is that it is limited to displaying instantaneous information: it shows no persistent information, instead showcasing a snapshot of the order book for 1 particular instant te time.
Order Book Price Level Volume
The following visualisations voorstelling how the order book volume evolves through time, and showcase a finish picture of all limit order activity across the day. The very first example, shown below, shows the volume available at every price level, te 1 cent ticks, filtered to voorstelling the 2100 levels >,= $313 and <,= $334 for a 24 hour period on the 2nd November. The white time series corresponds to the market midprice, which is defined spil the average price inbetween the current best bid and ask prices: (best.bid+best.ask)/Two. All price levels above the midprice correspond to the Ask side of the book, while levels below correspond to the Bid side. The colours differentiate inbetween the amount of volume at each level: Blue means little volume, through to Green, Yellow, and eventually, Crimson to indicate a large quantity of volume. Since the majority of price levels contain very little volume, the colour scheme has bot rescaled, so that most of the colour differentiation occurs below
Five Bitcoin and anything above (zonderling) is te the red-end of the spectrum.
Unlike the order book event visualisations shown previously, this visualisation shows the actual life cycle of limit orders. Te the setting of this article, the sheer extent of limit order activity is shown, demonstrating the ebb and flow of activity via the day.
Order Book Liquidity
I borrowed the idea for visualising liquidity/depth (and the spectral colour scheme) from Nanex: I’m a big fan of the visualisations resulting from their research.
The below chart (which is aligned with the above) shows the cumulative sum of order book depth through time. The amount of gegevens I am dealing with here comes nowhere near spil close spil the amount of gegevens processed by Nanex. Furthermore, this exchange (Bitstamp) is very unliquid. Spil such, instead of showcasing the available depth at actual price levels, I have grouped the volume into 40 percentile buckets above (20) and below (20) the current best bid and ask price. This example, shows the amount of volume available via the day on the bid side of the book (negative liquidity on the y-axis) vs the volume available on the ask side of the book (positive). Commencing at the 0 line, the very first crimson region above it shows the amount of volume available at the best ask price up until 0.0025% (25 BPS) above. The plakband above that, shows the amount of volume for all prices >,= 25 BPS and <, 50 BPS. The top relatie (blue), shows the amount of volume at price levels >,= 475 BPS and <, 500 BPS (5%). The same percentile ranges are repeated for price levels below the best bid price.
Using the same day spil an example, zooming ter to a Three hour period, centered at 12pm, the following chart and accompanying depth opbergmap shows the order book activity at the end of a minor (te Bitcoin terms) sell off. It is interesting to note the midprice movement te this chart ter relation to the order book depth. Firstly, note the ribbons of declining volume above the midprice. This shows fairly a large amount of volume being periodically cancelled and then shifted down a few levels. 2nd, the depth opbergmap below the chart, shows the total volume above and below, note how instantaneously before the last sell off there is an increase te liquidity above the midprice, and instantaneously after, below.
Zooming ter to a 15 minute interval, centered on the same region, it becomes adequate to voorstelling the current best bid and ask along with trade events instead of the market midprice. Here, visible on both the order book price level chart and the depth schrijfmap, wij can see a number of limit orders being cancelled instantly before a sell off.
Using the price level and depth schrijfmap visualisations above, it is possible to start to interpret some of the activity. Looking however some random slices of the gegevens, some re-occurring themes start to emerge. Spil mentioned before, given that this market is totally unregulated (I’m not implying this is a bad thing), and is essentially a free-for-all ter terms of the potential for challenging trading algos, I hope to find some interesting activity to visualise. So far I have identified a few patterns reminiscent of the “Negative HFT” primarily reported by and categorised by Nanex.
Ter the next few examples, I will attempt to interpret a few “suspicious” looking patterns. Generally, it wasgoed not very hard to find thesis examples, albeit I am disappointed a little te terms of the sophistication of the apparent tactics. I may add more examples at a straks date.
Layering is reported ter conventional markets spil a negative HFT tactic. Related to fleeting orders (unavoidable order cancellations), laying involves adding volume at various price levels with the foot intention to influence other market participants into believing (observing) an order book imbalance or strong buying/selling pressure. Ter other words, laying attempts to coerce the market into a direction that will benefit the initiators position.
I have identified Trio types of layering so far, which pretty much occur all of the time. Having said that, some laying will simply be market pursuing strategies: placing orders always a certain level below or above the market te the hope to build up value from zonderling events. Te enhancing order of aggressiveness, the types of layering found so far are: 1) Basic resting order layering: orders, usually very large, are placed te the book and left there. The intent may be to give a false impression of “support”. Two) Variable layering: A large amount of volume is added for a brief period of time, deleted, and then added back at a higher (te case of bid) price. The intent may be to stir the market. Trio) Eventually, the most aggressive form of layering occurs at, or very close to the best bid or ask price. Te this case the layering instantaneously affects the midprice and can step by step budge the market.
The very first type of layering is not very titillating, so I will not display it ter isolation, but instead commence with the 2nd type. The very first example, below, shows a 6 hour snapshot inbetween 5pm and 11pm on 2014-10-31. Here, the bid side of the book emerges to display consistent layering activity with high volume (orange/crimson orders). It could be argued that this strategy is following the price. However I selected this example since the “climbing” activity seems to persist regardless of market direction.
This climbing activity is best visualised te terms of order events. Below, te a 1 hour example, the price climbing pattern of add/delete/add, at incremental price levels on the bid side is fairly visible.
The next example, from the same day, shows the same pattern. However this time, on both sides of the order book.
The next example, along with the associated depth chart, taken from 2014-10-03, shows some very interesting activity. The void underneath the midprice around 12pm is the result of fairly a large market influence, the results of which are best viewed ter the aligned depth chart. Just after 1pm, a large resting order (
300 Bitcoin) emerges below the bid and then above, illustrating the very first type of layering.
On the right palm side of the above visualisation, there is some tightly packed layering occurring. Closer examination, below, shows a number of orders placed very close to the best bid. I’m not sure if there is anything suspicious happening here (besides the climbing bid), however it is interesting to see that all of the orders are kasstuk at 15.40.
The next example, from the same day, shows layering on the ask side of the book. This time, the orders are eliminated before they can be kasstuk.
The next day, 2014-10-04, shows a loterijlot of layering activity, particularly on the ask side of the book. On the bid side, the volume at $326.01 is continuously “flashed”, commencing at 17:42:57s 68ms, averaging 356.8 Bitcoin, up until 775.Four Bitcoin ($252,788). Spil soon spil the best bid is within 80 BPS range, it vanishes at Nineteen:54:07s 099ms.
Zooming te to just before 8pm and restricting the maximum amount of displayed volume to 100, shows the layering above the midprice ter some more detail.
The same time period and price level range shown spil order book events clarifies the situation. The layering on the ask side of the book seems to be stacked te blocks. The entire block is shifted down incrementally, spil can be seen by the add/cancel pattern below.
The final level of layering aggressiveness occurs at the market spread. The example below, from 2014-10-03, shows the best bid leisurely enlargening. It could be argued that this is some kleuter of pegged-order, however there are times when the best ask is not enlargening (or ter fact decreasing) while the best bid is still enhancing periodically.
This example, from 2014-10-14, shows a more active version. Note how the market shows up to be being compelled upward.
Looking at the event gegevens for this time period shows the climbing effect ter more detail.
The final example, with corresponding event visualisation, shows the phenomenon occurring on 2014-10-18.
Quote stuffing, which is referred to spil “one of the most visually evident forms of HFT” ter Credit Suisse: High Frequency Trading вЂ“ Measurement, Detection and Response and documented extensively by Nanex, for example here: Nanex: Quote Stuffing and Strange Sequences, occurs when bid or ask orders are added and then/deleted ter rapid succession. Whether or not quote stuffing is malicious or not seems to be the subject of heated debate. I’m not an experienced, so I’m not going to take a one sided position on this. I’m simply interested to see if similar observations can be made ter Bitcoin, and whether or not it is possible to make sense of any observed activity. If I had to guess, I would say livelock like conditions are very likely to occur te event driven code: I’ve had to overeenkomst with all manor of different situations with my own trading code. I would also say that while this kleintje of activity may most of the time be a side effect of event driven processes, I think it is very plausible that it could be used te a targeted/creative way. Either way, the fact that this activity occurs at all is interesting te its own right.
The following examples voorstelling some activity on the Bitstamp exchange that show up to mirror quote stuffing (malicious or otherwise) found te conventional markets. The very first example, below, shows a 15 minute time period on 2014-10-01. Note that 15 minutes is a very long time ter comparison to where this phenomena are usually observed. Nevertheless, the asking price exhibits what is referred to spil a square-wave ter the Credit Suisse report.
This next example, taken from 2014-10-03, the best ask oscillates fairly frantically overheen a long period of time (
30 minutes). If this is the result of a bug, then it is a pretty serious one: the fluctuation is big.
Zooming te to a Five minute period shows the anomaly ter finer detail. Te addition, the depth opbergmap shows the fluctuating liquidity spil the order is continuously added and liquidated.
The next example, taken from 2014-10-17, shows a more interesting pattern. The ask te this case is decreasing at an irregular rate. I’m guessing that this is simply Two or more selling algos leaping overheen each other. The bid side however seems to exhibit a more regular pattern: the bid increases to some level te puny steps, and then resets.
Looking at the bid event event gegevens, restricted to the current best bid for the same period, shows this climbing and then reseting effect ter greater detail.
A slightly more interesting observation occurring on 2014-10-18. This time, reminiscent of the “sawtooth” pattern.
Another square wave pattern. This time occurring on 2014-10-20.
The sawtooth pattern occurring below, on 2014-10-22 is fairly interesting. If wij look at the corresponding order events, not only is this pattern enlargening te price, but also ter volume. The aligned cancelled volume chart shows the actual amounts.
The remaining visualisations demonstrate various other formations.