Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[wso2si-1.1.0-45] "MarkIn consecutively called without calling markOut" Error when Statistics enabled in Siddhi #2208

Closed
charithjayasanka opened this issue Oct 10, 2023 · 1 comment
Closed

[wso2si-1.1.0-45] "MarkIn consecutively called without calling markOut" Error when Statistics enabled in Siddhi #2208

charithjayasanka opened this issue Oct 10, 2023 · 1 comment
Labels
Missing/Component Type/Bug

Comments

@charithjayasanka
Copy link

Description

Consuming of events fail when statistics are enabled in wso2si-1.1.0

Receiving the following error when consuming events from an event source to an RDBMS table

[2023-10-10 14:23:16,636]  INFO {io.siddhi.core.stream.output.sink.LogSink} - StatisticsTest : SweetProductionStream : Event{timestamp=1696927996626, data=[Cake, 20.12], isExpired=false}
[2023-10-10 14:23:25,337] ERROR {io.siddhi.core.stream.StreamJunction} - Error in 'StatisticsTest' after consuming events from Stream 'SweetProductionStream', MarkIn consecutively called without calling markOut in io.siddhi.SiddhiApps.StatisticsTest.Siddhi.Tables.TestTable.insert.latency. Hence, dropping event 'Event{timestamp=1696928005334, data=[Cake, 20.12], isExpired=false}' java.lang.IllegalStateException: MarkIn consecutively called without calling markOut in io.siddhi.SiddhiApps.StatisticsTest.Siddhi.Tables.TestTable.insert.latency
	at org.wso2.carbon.si.metrics.core.LatencyMetric.markIn(LatencyMetric.java:62)
	at io.siddhi.core.table.Table.addEvents(Table.java:221)
	at io.siddhi.core.query.output.callback.InsertIntoTableCallback.send(InsertIntoTableCallback.java:73)
	at io.siddhi.core.query.output.ratelimit.OutputRateLimiter.sendToCallBacks(OutputRateLimiter.java:104)
	at io.siddhi.core.query.output.ratelimit.PassThroughOutputRateLimiter.process(PassThroughOutputRateLimiter.java:45)
	at io.siddhi.core.query.selector.QuerySelector.process(QuerySelector.java:98)
	at io.siddhi.core.query.input.ProcessStreamReceiver.processAndClear(ProcessStreamReceiver.java:182)
	at io.siddhi.core.query.input.ProcessStreamReceiver.process(ProcessStreamReceiver.java:84)
	at io.siddhi.core.query.input.ProcessStreamReceiver.receive(ProcessStreamReceiver.java:127)
	at io.siddhi.core.stream.StreamJunction.sendEvent(StreamJunction.java:203)
	at io.siddhi.core.stream.StreamJunction$Publisher.send(StreamJunction.java:506)
	at io.siddhi.core.stream.input.InputDistributor.send(InputDistributor.java:34)
	at io.siddhi.core.stream.input.InputEntryValve.send(InputEntryValve.java:45)
	at io.siddhi.core.stream.input.InputHandler.send(InputHandler.java:79)
	at io.siddhi.core.stream.input.source.PassThroughSourceHandler.sendEvent(PassThroughSourceHandler.java:35)
	at io.siddhi.core.stream.input.source.InputEventHandler.sendEvent(InputEventHandler.java:83)
	at io.siddhi.extension.map.json.sourcemapper.JsonSourceMapper.mapAndProcess(JsonSourceMapper.java:255)
	at io.siddhi.core.stream.input.source.SourceMapper.onEvent(SourceMapper.java:201)
	at io.siddhi.core.stream.input.source.SourceMapper.onEvent(SourceMapper.java:145)
	at io.siddhi.extension.io.http.source.HttpWorkerThread.run(HttpWorkerThread.java:62)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

Steps to Reproduce

  1. Get a wso2si-1.1.0 pack and Update it to the U2 Level 45
  2. Navigate and open wso2si-1.1.0/conf/server/deployment.yaml
  3. Do the following modifications to the deployment.yaml

  • Under WSO2_PERMISSIONS_DB Datasource settings, replace the jdbcUrl with the follwing URL
    'jdbc:h2:${sys:carbon.home}/wso2/${sys:wso2.runtime}/database/PERMISSION_DB;DB_CLOSE_ON_EXIT=FALSE;LOCK_TIMEOUT=60000;'

  • Under metrics configurations, set the following

# This is the main configuration for metrics
wso2.metrics:
  # Enable Metrics
  enabled: true
  reporting:
    console:
      - # The name for the Console Reporter
        name: Console

        # Enable Console Reporter
        enabled: true

        # Polling Period in seconds.
        # This is the period for polling metrics from the metric registry and printing in the console
        pollingPeriod: 5
  1. Obtain the mysql-connector-java-8.0.18.jar from [2]
  2. Setup a local mysql database and obtain the credentials.
  3. Create the following Siddhi Application in the following directory wso2si-1.1.0-45/wso2/server/deployment/siddhi-files/StatisticsTest.siddhi
@App:name("StatisticsTest")

@App:Statistics(reporter = 'console',interval = '5')
@App:description('Receive events via HTTP transport and view the Statistics output on the console')

@Store(type="rdbms",
       jdbc.url="jdbc:mysql://localhost:3306/production?useSSL=false",
       username="root",
       password="password" ,
       jdbc.driver.name="com.mysql.jdbc.Driver")
define table TestTable (name string, amount double);

@sink(type='log') 
@Source(type = 'http',
        receiver.url='http://localhost:8006/productionStream',
        basic.auth.enabled='false',
        @map(type='json'))
define stream SweetProductionStream (name string, amount double);

from SweetProductionStream
insert into TestTable;
  1. Publish a sample event to the stream by using the following CURL request
curl --location 'http://localhost:8006/productionStream' \
--header 'Content-Type: application/json' \
--data '{"event":{"name":"Cake","amount":20.12}}'
  1. Then it is possible to see the following output reproducing the same.
[2023-10-10 14:23:16,636]  INFO {io.siddhi.core.stream.output.sink.LogSink} - StatisticsTest : SweetProductionStream : Event{timestamp=1696927996626, data=[Cake, 20.12], isExpired=false}
[2023-10-10 14:23:25,337] ERROR {io.siddhi.core.stream.StreamJunction} - Error in 'StatisticsTest' after consuming events from Stream 'SweetProductionStream', MarkIn consecutively called without calling markOut in io.siddhi.SiddhiApps.StatisticsTest.Siddhi.Tables.TestTable.insert.latency. Hence, dropping event 'Event{timestamp=1696928005334, data=[Cake, 20.12], isExpired=false}' java.lang.IllegalStateException: MarkIn consecutively called without calling markOut in io.siddhi.SiddhiApps.StatisticsTest.Siddhi.Tables.TestTable.insert.latency
	at org.wso2.carbon.si.metrics.core.LatencyMetric.markIn(LatencyMetric.java:62)
	at io.siddhi.core.table.Table.addEvents(Table.java:221)
	at io.siddhi.core.query.output.callback.InsertIntoTableCallback.send(InsertIntoTableCallback.java:73)
	at io.siddhi.core.query.output.ratelimit.OutputRateLimiter.sendToCallBacks(OutputRateLimiter.java:104)
	at io.siddhi.core.query.output.ratelimit.PassThroughOutputRateLimiter.process(PassThroughOutputRateLimiter.java:45)
	at io.siddhi.core.query.selector.QuerySelector.process(QuerySelector.java:98)
	at io.siddhi.core.query.input.ProcessStreamReceiver.processAndClear(ProcessStreamReceiver.java:182)
	at io.siddhi.core.query.input.ProcessStreamReceiver.process(ProcessStreamReceiver.java:84)
	at io.siddhi.core.query.input.ProcessStreamReceiver.receive(ProcessStreamReceiver.java:127)
	at io.siddhi.core.stream.StreamJunction.sendEvent(StreamJunction.java:203)
	at io.siddhi.core.stream.StreamJunction$Publisher.send(StreamJunction.java:506)
	at io.siddhi.core.stream.input.InputDistributor.send(InputDistributor.java:34)
	at io.siddhi.core.stream.input.InputEntryValve.send(InputEntryValve.java:45)
	at io.siddhi.core.stream.input.InputHandler.send(InputHandler.java:79)
	at io.siddhi.core.stream.input.source.PassThroughSourceHandler.sendEvent(PassThroughSourceHandler.java:35)
	at io.siddhi.core.stream.input.source.InputEventHandler.sendEvent(InputEventHandler.java:83)
	at io.siddhi.extension.map.json.sourcemapper.JsonSourceMapper.mapAndProcess(JsonSourceMapper.java:255)
	at io.siddhi.core.stream.input.source.SourceMapper.onEvent(SourceMapper.java:201)
	at io.siddhi.core.stream.input.source.SourceMapper.onEvent(SourceMapper.java:145)
	at io.siddhi.extension.io.http.source.HttpWorkerThread.run(HttpWorkerThread.java:62)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

Please check the possibility of fixing this issue

[1]. https://siddhi.io/en/v5.1/docs/query-guide/#statistics
[2]. https://dev.mysql.com/downloads/connector/j/

Thanks & best regards,
Charith.

Affected Component

SI

Version

1.1.0

Environment Details (with versions)

No response

Relevant Log Output

[2023-10-10 14:23:25,337] ERROR {io.siddhi.core.stream.StreamJunction} - Error in 'StatisticsTest' after consuming events from Stream 'SweetProductionStream', MarkIn consecutively called without calling markOut in io.siddhi.SiddhiApps.StatisticsTest.Siddhi.Tables.TestTable.insert.latency. Hence, dropping event 'Event{timestamp=1696928005334, data=[Cake, 20.12], isExpired=false}' java.lang.IllegalStateException: MarkIn consecutively called without calling markOut in io.siddhi.SiddhiApps.StatisticsTest.Siddhi.Tables.TestTable.insert.latency
	at org.wso2.carbon.si.metrics.core.LatencyMetric.markIn(LatencyMetric.java:62)
	at io.siddhi.core.table.Table.addEvents(Table.java:221)
	at io.siddhi.core.query.output.callback.InsertIntoTableCallback.send(InsertIntoTableCallback.java:73)
	at io.siddhi.core.query.output.ratelimit.OutputRateLimiter.sendToCallBacks(OutputRateLimiter.java:104)
	at io.siddhi.core.query.output.ratelimit.PassThroughOutputRateLimiter.process(PassThroughOutputRateLimiter.java:45)
	at io.siddhi.core.query.selector.QuerySelector.process(QuerySelector.java:98)
	at io.siddhi.core.query.input.ProcessStreamReceiver.processAndClear(ProcessStreamReceiver.java:182)
	at io.siddhi.core.query.input.ProcessStreamReceiver.process(ProcessStreamReceiver.java:84)
	at io.siddhi.core.query.input.ProcessStreamReceiver.receive(ProcessStreamReceiver.java:127)
	at io.siddhi.core.stream.StreamJunction.sendEvent(StreamJunction.java:203)
	at io.siddhi.core.stream.StreamJunction$Publisher.send(StreamJunction.java:506)
	at io.siddhi.core.stream.input.InputDistributor.send(InputDistributor.java:34)
	at io.siddhi.core.stream.input.InputEntryValve.send(InputEntryValve.java:45)
	at io.siddhi.core.stream.input.InputHandler.send(InputHandler.java:79)
	at io.siddhi.core.stream.input.source.PassThroughSourceHandler.sendEvent(PassThroughSourceHandler.java:35)
	at io.siddhi.core.stream.input.source.InputEventHandler.sendEvent(InputEventHandler.java:83)
	at io.siddhi.extension.map.json.sourcemapper.JsonSourceMapper.mapAndProcess(JsonSourceMapper.java:255)
	at io.siddhi.core.stream.input.source.SourceMapper.onEvent(SourceMapper.java:201)
	at io.siddhi.core.stream.input.source.SourceMapper.onEvent(SourceMapper.java:145)
	at io.siddhi.extension.io.http.source.HttpWorkerThread.run(HttpWorkerThread.java:62)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

Related Issues

No response

Suggested Labels

si
@charithjayasanka charithjayasanka changed the title [wso2si-1.1.0-45] MarkIn consecutively called without calling markOut when Statistics enabled in Siddhi [wso2si-1.1.0-45] "MarkIn consecutively called without calling markOut" Error when Statistics enabled in Siddhi Oct 10, 2023
@YasasRangika
Copy link

Public fix: siddhi-io/siddhi#1772 has already been merged into the master branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Missing/Component Type/Bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@YasasRangika @charithjayasanka