Batch queries with one missing value lose other values

[leon-barrett]

If I have a kapacitor batch query for two values and the first one is missing, the second one will also be missing. This seems like a major bug: Kapacitor queries from InfluxDB will unexpectedly fail to retrieve data!

Steps:
Insert InfluxDB data with values v1 missing and v2 present. INSERT meas,t1=missing v2=2
Set up a Kapacitor batch query asking for values v1 and v2. batch |query('SELECT v1, v2')
Expected result:
A value of v2=2 is retrieved.
Actual result:
No value is retrieved for v2.
Additional information:
If v2 is requested before v1, then the value v2=2 is retrieved. batch |query('SELECT v2, v1')

Here is a bash script that demonstrates the issue, including setting up an InfluxDB database and a Kapacitor task. I am running InfluxDB v1.1.1 and Kapacitor v1.1.1 on OSX (installed via homebrew).

#!/bin/bash

# Set up InfluxDB with value v1 missing for some measurements.
influxd &>/dev/null &
sleep 1
influx -execute 'CREATE DATABASE kapacitor_repro'
# Tag t1 marks whether v1 is present or missing.
influx -database kapacitor_repro -execute 'INSERT meas,t1=present v1=1,v2=2'
influx -database kapacitor_repro -execute 'INSERT meas,t1=missing v2=2'
influx -execute 'SELECT * FROM kapacitor_repro.autogen.meas'

# Set up Kapacitor with queries.
kapacitord &
sleep 1
# If v1 is present, there is no problem.
cat > working1_batch.tick <<END
batch
    |query('''
        SELECT t1, v1, v2
        FROM kapacitor_repro.autogen.meas
        WHERE t1 = 'present' ''')
        .period(1d)
        .every(1s)
    |log()
        .prefix('working1_batch_query')
    |min('v2')
        .as('minv2')
END
kapacitor define working1_batch -type batch -tick working1_batch.tick -dbrp kapacitor_repro.autogen
kapacitor enable working1_batch
# If v1 is missing, we fail to query data for v2!
cat > broken_batch.tick <<END
batch
    |query('''
        SELECT t1, v1, v2
        FROM kapacitor_repro.autogen.meas
        WHERE t1 = 'missing' ''')
        .period(1d)
        .every(1s)
    |log()
        .prefix('broken_batch_query')
    |min('v2')
        .as('minv2')
END
kapacitor define broken_batch -type batch -tick broken_batch.tick -dbrp kapacitor_repro.autogen
kapacitor enable broken_batch
# If we query in a different order, we get data for v2!
cat > working2_batch.tick <<END
batch
    |query('''
        SELECT t1, v2, v1
        FROM kapacitor_repro.autogen.meas
        WHERE t1 = 'missing' ''')
        .period(1d)
        .every(1s)
    |log()
        .prefix('working2_batch_query')
    |min('v2')
        .as('minv2')
END
kapacitor define working2_batch -type batch -tick working2_batch.tick -dbrp kapacitor_repro.autogen
kapacitor enable working2_batch
kapacitor list tasks
sleep 2
kapacitor list tasks

# Note this output:
#[broken_batch:log2] 2017/01/18 15:49:19 I! broken_batch_query {"name":"meas","tmax":"2017-01-18T15:49:19.341909221-08:00","points":[{"time":"2017-01-18T23:49:17.175909077Z","fields":{"t1":"missing"},"tags":null}]}
#
#[edge:broken_batch|log2->min3] 2017/01/18 15:49:19 I! aborting c: 1 e: 1
#[broken_batch:min3] 2017/01/18 15:49:19 E! invalid influxql func min with field v2: invalid field type: <nil>
#[task_master:main] 2017/01/18 15:49:19 E! Stopped task: broken_batch min3: invalid influxql func min with field v2: invalid field type: <nil>
#[task_store] 2017/01/18 15:49:19 E! task broken_batch finished with error: min3: invalid influxql func min with field v2: invalid field type: <nil>

# Clean up Kapacitor
kapacitor delete tasks working1_batch
kapacitor delete tasks broken_batch
kapacitor delete tasks working2_batch
kill %kapacitord
wait %kapacitord
# Clean up InfluxDB
influx -execute 'DROP DATABASE kapacitor_repro'
kill %influxd
wait %influxd

[leon-barrett]

Can anyone else confirm this bug? This seems like a major bug: batch queries unexpectedly return nil when they should not. I tried to include detailed steps to reproduce it--have they been insufficient?

Thanks.

[MRylander]

I have also seen this issue.

Additional Info: If you query for several fields, all fields after the missing field will not be reported.

[nathanielc]

@leon-barrett This issue looks like it is a duplicate of #1294 which was fixed in the 1.30 release. Can you confirm that this issue has been fixed?

[leon-barrett]

Yeah, it seems to be fixed, thanks!

(Though based on incident numbers, I'd argue which of the two issues is the duplicate :) )

[nathanielc]

(Though based on incident numbers, I'd argue which of the two issues is the duplicate :) )

Agreed :), glad its working.