Flux Language Tutorial with Grafana

Flux Language is the shiny new query language that comes with InfluxDB 2.0. Since it is “the future”, and the OSS (free) version of InfluxDB 2.0 is not getting the old query language support in (at least) the first few versions, I’ve been using Flux Language for some new projects. Meanwhile, the (functional) documentation for both the Flux Language and its support in Grafana does not exist yet. In this post, I’d like to give some examples to quickly address common data processing needs, for both server monitoring and simple BI usages.

Disclaimer: InfluxDB 2.0 is currently in beta, so things might change quickly. All examples are verified against the follow versions of software:

  • InfluxDB Version 2.0.0 (37cc047)
  • Grafana Server v7.3.4 (14c494085e)

This post focuses on the Flux Language grammar, so I’ll assume you have no issue setting up InfluxDB (HTTP API, dashboard) and Grafana. If you need help on connecting InfluxDB to Grafana, see How to Build Grafana Dashboards with InfluxDB, Flux and InfluxQL.

You also need to know the basic terms for InfluxDB (v1 or v2). If not, read InfluxDB data elements now.

Basics

Graph is the default (and most intuitive) panel type in Grafana, so we’ll start with it. On every refresh, a panel sends all the queries to the selected InfluxDB data source, then draws every series returned as one line. (Key take: One series, one line. ) Note that a query might return more than one series. Here’s an example of one query returning 3 series:

Other panel types works largely the same way.

In InfluxDB documentation, a series is called a table which is confusing. I’ll use the term series throughout the post.

Debugging Flux Query

To effectively debug your Flux query, we’ll use the InfluxDB dashboard (it was called Chronograf, and was a separate installation). We’ll be only using the Data Explorer.

After you get to the Data Explorer, you should:

  • Click on the “Script Editor” left of the blue “Submit” button
  • Click on the “Graph” dropdown menu on the top left, and select “Table”

The “Table” display mode will tell you what actual data is returned from your query. Depends on what your query is, you should see one of the two interfaces.

This is what you see when only one series is returned from your query:

This is what you see when multiple series are returned from your query:

Note that the series key is displayed on the left.

Querying Data

Nearly every Flux query start with the following 3 instructions:

Feel free to add more filters so we are only left with exactly what we need. For example, you can use regex ( =~ for matching, !~ for not matching):

Or drop some columns, only leaving useful ones:

Use

to return the current data. You can use it more than once in the query, just set the name argument to something different. If you only need to return once, yield() can be omitted.

Changing Series Display Name

Grafana does not support using tag variables in query series names anymore. Instead, there are 2 simple rules:

  • If the series key only contains one key=value segments, the value is used as the series’ display name
  • If the series key contains more than one key=value segments, its full, JSON-formatted string is appended to the table name as the series’ display name

So if you write your query like this:

You will get series names like this:

Which is very annoying. In order to rename a series, we can inject its name into every row with a map() function call:

Don’t be surprised if there is only one series returned, because the map rule removed your original series key. We need to regroup this large plain table back to different series with another function call:

So our final query will look like this:

Already excited? Let’s have more fun.

More String Operations

String interpolation is a more concise method for achieving the same result.
String interpolation

This claim is actually not correct, and I’ll show you why. We’ve already used a little bit string interpolation.

We can use dot notation to access a dict too:

But what if r.device  might be null? You can’t use string interpolation any more, or you’ll have the following error:

And you can’t use fill()  because tags cannot be filled.

Well, you can use string concentration with a type cast:

but if r.device is null, the full string evaluates to null, This might or might not be what you want. If you want some default value for null strings, write your own function using the unary operator exists .

Selecting and Aggregating Data

There are some selectors and aggregate functions which, when called, is applied to every series independently at the same time. So there’s a common pattern to use them:

  1. Use a group()  function call to divide your data into temporary series
  2. Run the selector or aggregate functions on the temporary series
  3. Use another group()  function call to divide your data back to their logical series for display

For example, you have the following data:

_time cashier item sales_count
2020-01-01 Amy apple 1
2020-01-01 Amy banana 3
2020-01-01 Li Hua apple 4
2020-01-01 Amy orange 6
2020-01-02 Li Hua apple 2
2020-01-02 Amy banana 5
2020-01-02 Li Hua orange 8

and you want to display a graph of how much items is sold per day per person, regardless of the item. There is a function, sum(), which accumulates every value of a column and outputs a new table consisting only one row. So you have to do this:

First group your data by time + anything you want to keep to the end

Then run sum() on the column we want (as a result, the “item” column will be discarded)

Finally re-group the data by person. Grouping may not keep the data order, so you might want to sort it again:

Use group()  to put everything into a single series. You can also use window()  to group your dataset by a time window if required.

Change some of the Fields in a Row

You’ve already know that map() can be used to transform every row. But how to only change some of the fields while leaving other fields intact? There is a shorthand on the {} operator:

Grafana Dashboard Variables

To get a list of tags, use the following query:

If you want multiple selection of tags, use this filter:

Leave a Reply

Your email address will not be published. Required fields are marked *