Sunbird Lern
Replies: 2 comments 10 replies
-
|
Hi @sachinb80 , From the logs, it seems the job fails after zipping the result file without a clear reason in the logs. However, jobs like course progress and user info are successful. I recommend comparing the response-exhaust config with other exhaust jobs. You can find the configurations in model-config.sh of Spark under section of the dataproducts configmap. If there’s any mismatch, please fix it and try again. If the issue persists, share the model-config.sh file from your cluster setup, and I’ll assist you further. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @GayathriSrividya after checking the configs. I have attached the model-config.sh file of ours |
Beta Was this translation helpful? Give feedback.
-
|
Hi @GayathriSrividya the issue still persist. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @GayathriSrividya the issue still persist. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @GayathriSrividya Did you check the file?? |
Beta Was this translation helpful? Give feedback.
-
|
@sachinb80 - The model-config doesn't give the complete context for us to analyse the issue. Please check the complete logs of the execution and look for any errors. Please check the spark logs to understand what is the root cause of the issue. Please share the exception stack trace for us to understand the issue and provide inputs. |
Beta Was this translation helpful? Give feedback.
-
|
@maheshkumargangula sure. Will check again. |
Beta Was this translation helpful? Give feedback.
-
|
@maheshkumargangula Here are the log files. |
Beta Was this translation helpful? Give feedback.
-
|
Please let us know if you need anything |
Beta Was this translation helpful? Give feedback.
-
Hello All
We are using 5.1 version of sunbird.
We are facing issue of Question set reports is failing after submitting it.
Need your help in resolving the issue
We have restarted the response-exhaust flink job from spark server but still no success.
Here are the logs and screenshot for the reference purpose
Please feel free to ask any other details needed for the resolving this issue
{"eid":"JOB_START","ets":1737541749358,"ver":"3.0","mid":"E7E2FC11BE7518DA87F583EDB9B13409","actor":{"id":"","type":"System"},"context":{"channel":"in.ekstep","pdata":{"id":"AnalyticsDataPipeline","ver":"1.0","pid":"ResponseExhaustJob","model":null},"env":"analytics","sid":null,"did":null,"cdata":null,"rollup":null},"object":null,"edata":{"data":{"config":"{"search":{"type":"none"},"model":"org.sunbird.analytics.exhaust.collection.ResponseExhaustJob","modelParams":{"store":"azure","mode":"OnDemand","batchFilters":["TPD"],"searchFilter":{}, "sparkElasticsearchConnectionHost":"10.1.2.12","sparkRedisConnectionHost":"10.1.2.14","sparkUserDbRedisIndex":"12","sparkUserDbRedisPort":"6379", "sparkCassandraConnectionHost":"10.1.2.4", "fromDate":"$(date --date yesterday +%Y-%m-%d)","toDate":"$(date --date yesterday +%Y-%m-%d)"},"parallelization":8,"appName":"Response Exhaust"}","model":"ResponseExhaustJob"},"message":"ResponseExhaustJob started executing - ver3","class":"org.sunbird.analytics.exhaust.collection.ResponseExhaustJob","level":"INFO"},"@timestamp":"2025-01-22T10:29:09+00:00"}
{"eid":"JOB_LOG","ets":1737541766744,"ver":"3.0","mid":"C63C73BD63A515F43A3954AA28BD1E8A","actor":{"id":"","type":"System"},"context":{"channel":"in.ekstep","pdata":{"id":"AnalyticsDataPipeline","ver":"1.0","pid":"ResponseExhaustJob","model":null},"env":"analytics","sid":null,"did":null,"cdata":null,"rollup":null},"object":null,"edata":{"data":{"timeTaken":6096,"count":35723},"message":"Time to fetch enrolment details","class":"org.sunbird.analytics.exhaust.collection.ResponseExhaustJob","level":"INFO"},"@timestamp":"2025-01-22T10:29:26+00:00"}
{"eid":"JOB_LOG","ets":1737541767228,"ver":"3.0","mid":"CD374E0CF786D2CF276F0D8362A2610B","actor":{"id":"","type":"System"},"context":{"channel":"in.ekstep","pdata":{"id":"AnalyticsDataPipeline","ver":"1.0","pid":"ResponseExhaustJob","model":null},"env":"analytics","sid":null,"did":null,"cdata":null,"rollup":null},"object":null,"edata":{"message":"fetched records count1","class":"org.sunbird.analytics.exhaust.collection.ResponseExhaustJob","level":"INFO"},"@timestamp":"2025-01-22T10:29:27+00:00"}
{"eid":"JOB_LOG","ets":1737541767742,"ver":"3.0","mid":"16DA0142A2C82CEE495837BE84D075C5","actor":{"id":"","type":"System"},"context":{"channel":"in.ekstep","pdata":{"id":"AnalyticsDataPipeline","ver":"1.0","pid":"ResponseExhaustJob","model":null},"env":"analytics","sid":null,"did":null,"cdata":null,"rollup":null},"object":null,"edata":{"data":{"jobId":"response-exhaust","totalRequests":1},"message":"Total Requests are ","class":"org.sunbird.analytics.exhaust.collection.ResponseExhaustJob","level":"INFO"},"@timestamp":"2025-01-22T10:29:27+00:00"}
{"eid":"JOB_LOG","ets":1737541767764,"ver":"3.0","mid":"33A9BA390F615A109A8C280EBCF46D11","actor":{"id":"","type":"System"},"context":{"channel":"in.ekstep","pdata":{"id":"AnalyticsDataPipeline","ver":"1.0","pid":"ResponseExhaustJob","model":null},"env":"analytics","sid":null,"did":null,"cdata":null,"rollup":null},"object":null,"edata":{"data":{"Total Requests":1,"filtered Requests":1,"Duplicate Requests":0},"message":"The Request count details","class":"org.sunbird.analytics.exhaust.collection.ResponseExhaustJob","level":"INFO"},"@timestamp":"2025-01-22T10:29:27+00:00"}
{"eid":"JOB_LOG","ets":1737541767769,"ver":"3.0","mid":"420332F39281941F83C70CEA147858B2","actor":{"id":"","type":"System"},"context":{"channel":"in.ekstep","pdata":{"id":"AnalyticsDataPipeline","ver":"1.0","pid":"ResponseExhaustJob","model":null},"env":"analytics","sid":null,"did":null,"cdata":null,"rollup":null},"object":null,"edata":{"data":{"channel":"0137236500887961602","file size":0,"completed batches":0},"message":"Channel details at executeOnDemand","class":"org.sunbird.analytics.exhaust.collection.ResponseExhaustJob","level":"INFO"},"@timestamp":"2025-01-22T10:29:27+00:00"}
{"eid":"JOB_LOG","ets":1737541773448,"ver":"3.0","mid":"A991146BE33DFD9C477375778DECF268","actor":{"id":"","type":"System"},"context":{"channel":"in.ekstep","pdata":{"id":"AnalyticsDataPipeline","ver":"1.0","pid":"ResponseExhaustJob","model":null},"env":"analytics","sid":null,"did":null,"cdata":null,"rollup":null},"object":null,"edata":{"data":{"channel":"0137236500887961602","file size":0,"completed batches":0},"message":"Channel details at processBatches","class":"org.sunbird.analytics.exhaust.collection.ResponseExhaustJob","level":"INFO"},"@timestamp":"2025-01-22T10:29:33+00:00"}
{"eid":"JOB_LOG","ets":1737541780454,"ver":"3.0","mid":"149279E24DF2C98A19E3178C605CDEAD","actor":{"id":"","type":"System"},"context":{"channel":"in.ekstep","pdata":{"id":"AnalyticsDataPipeline","ver":"1.0","pid":"ResponseExhaustJob","model":null},"env":"analytics","sid":null,"did":null,"cdata":null,"rollup":null},"object":null,"edata":{"data":{"timeTaken":6512,"count":1},"message":"Time to fetch batch enrolment","class":"org.sunbird.analytics.exhaust.collection.ResponseExhaustJob","level":"INFO"},"@timestamp":"2025-01-22T10:29:40+00:00"}
{"eid":"JOB_LOG","ets":1737541781255,"ver":"3.0","mid":"B7087B365A7336BC905229F291E240C3","actor":{"id":"","type":"System"},"context":{"channel":"in.ekstep","pdata":{"id":"AnalyticsDataPipeline","ver":"1.0","pid":"ResponseExhaustJob","model":null},"env":"analytics","sid":null,"did":null,"cdata":null,"rollup":null},"object":null,"edata":{"data":{"requestId":"C147444BB5979A4C36432E5A42D76EC3","remainingRequest":1},"message":"The Request is processed. Pending zipping","class":"org.sunbird.analytics.exhaust.collection.ResponseExhaustJob","level":"INFO"},"@timestamp":"2025-01-22T10:29:41+00:00"}
{"eid":"JOB_LOG","ets":1737541781278,"ver":"3.0","mid":"F9C1D17F9191270A2BE47218E4DFEC8C","actor":{"id":"","type":"System"},"context":{"channel":"in.ekstep","pdata":{"id":"AnalyticsDataPipeline","ver":"1.0","pid":"ResponseExhaustJob","model":null},"env":"analytics","sid":null,"did":null,"cdata":null,"rollup":null},"object":null,"edata":{"data":{"requestId":"C147444BB5979A4C36432E5A42D76EC3","timeTakenForZip":20},"message":"Request is zipped","class":"org.sunbird.analytics.exhaust.collection.ResponseExhaustJob","level":"INFO"},"@timestamp":"2025-01-22T10:29:41+00:00"}
{"eid":"JOB_END","ets":1737541781612,"ver":"3.0","mid":"3F92BDA21F308D8ACA0B11EF8DD1422D","actor":{"id":"","type":"System"},"context":{"channel":"in.ekstep","pdata":{"id":"AnalyticsDataPipeline","ver":"1.0","pid":"ResponseExhaustJob","model":null},"env":"analytics","sid":null,"did":null,"cdata":null,"rollup":null},"object":null,"edata":{"data":{"timeTaken":24864,"successRequests":0,"failedRequests":1,"totalRequests":1,"duplicateRequests":0},"status":"SUCCESS","message":"ResponseExhaustJob completed execution","class":"org.sunbird.analytics.exhaust.collection.ResponseExhaustJob","level":"INFO"},"@timestamp":"2025-01-22T10:29:41+00:00"}
@vinukumar-vs @rajeevsathish @rajeshkumaravel @Krishnaj20 @pallakartheekreddy
Beta Was this translation helpful? Give feedback.
All reactions