Timelion: The time series composer for Kibana

Update: The install commands towards the end of this post have been updated to include Kibana 5. You can also try Timelion through Elastic Cloud, our hosted & managed Elasticsearch and Kibana service.

So I’m putting together a demo of Kibana, and thinking: I wish I could do more with time series data in Kibana. At the same time, I’d been mulling a new expression syntax for Kibana: What if there was a way to describe queries, transformations and visualization in one place, in one line. I had a trans-atlantic flight the next day, and I had a plan: A tiny, in-flight, one-person, hack-a-thon to build an experiment that scratched both the time series and expressions itches.

By the time I landed in Houston, I had a syntax, a grid of charts and a few simple functions. A few more hours of hacking and I was in Phoenix answering questions I’d never been able to ask before:

• How many pages does each unique user hit over time?
• Whats the difference between this Friday and last Friday?
• What % of Japan’s population came to my site today?
• What’s the 10 day moving average of the S&P 500?
• How what is the cumulative sum of all searches made in the last 2 years?

Timelion makes all that possible, and more. Timelion is an Elastic {Re}search project into time series, but its more than just an experiment: Its completely usable. By you. Right now.

Elastic {Re}search?

{Re}search is Elastic’s lab for cutting edge technology. {Re}search is our playground, the sandbox in which we build our wildest sand castles. {Re}search projects are launched by Elastic engineers that want to grab an idea, run with it, and share it. The projects launched under the {Re}search tag highlight our next-big-things, in packages you can use (and abuse) today. Some are standalone, but many are plugins and extensions for our existing products. We invite you to install, experiment, and communicate your experiences with our {Re}search: Thats why they exist, to gather feedback on our ideas.

This is Timelion

Timelion, pronounced "Timeline", brings together totally independent data sources into a single interface, driven by a simple, one-line expression language combining data retrieval, time series combination and transformation, plus visualization. Every Timelion expression starts with a data source function. For example .elasticsearch(*) (or .es(*) for short). That’s about as simple as it gets right? Count everything in Elasticsearch over time. Easy peasy. As you can see, functions always start with a '.' (dot). What if I want to answer that first question above: “How many pages does each unique user hit over time”. Well, I could plot the two parts together on the same chart, by separating them with a comma. But there's a better way ...

We can do more than individual functions, we can chain functions! What I really want is to divide total hits by unique users. WE CAN DO THAT. SCORE. What we're doing here is saying: Get everything, then divide every point in that series, by every point in this cardinality-of-user-field series I'm passing to .divide()

We can do that, we can do more: Timelion can reach out to other data sources, using the exact same syntax. For example, the Worldbank’s Data API. Series even can be grouped together into lists with parenthesis and a function applied to the grouping. All data sources can receive an offset argument, eg offset=-1M to compare last month as if it was happening now. Timeline can even fit series with dissimilar intervals to a reference, then enabling you to divide you by-the-minute Elasticsearch series with say, yearly Worldbank data.

That means we can mix and match these sources, even within the same expression.Thus we can ask crazy questions like “What percentage of the US Gross Domestic Product is my company personally responsible for year-to-date?” Also, if that number is big, how about sharing with your old buddy Rashid? Just kidding. Or am I?

Funk-shun Al

There’s 25 different functions, from simple arithmetic like addition and division to moving averages, cumulative sums and derivatives. That said, Timelion functions and data sources are totally pluggable and super easy to write. We’d love your help rounding out the offering, so get hacking!

Go get it

I won't give it all away, there’s more to it than just this post. Timelion will launch a tutorial to step you through configuration and some simple starter functions, the rest is up to you to discover.

Installing it is easy, run this, bounce the Kibana server. and refresh your browser: ./bin/kibana plugin -i elastic/timelion. Or you can try it on Found, the best hosted Elasticsearch in the history of the universe, for free: Found by Elastic.

Update: If you're on 5.x the correct command is: ./bin/kibana-plugin install timelion

Now use it. Abuse it.

Once you've installed it, you'll have a new icon in Kibana, which opens the app switcher and allows you to enter other apps

If you've joined us on Elastic{ON} tour you've already seen this video. If you didn't join us on Elastic{ON}, well, I guess you've learned your lesson? Maybe you should come to Elastic{ON}¹⁶?

Keep an eye on this blog for tips and tricks. Keep an eye on Elastic's Twitter space for sweet sweet eye candy. Have a great idea? Wrote a new awesome timelion function? Git over here.