Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug(sqlite): database locks under load #122

Closed
EsmailELBoBDev2 opened this issue Nov 30, 2024 · 30 comments · Fixed by #125
Closed

bug(sqlite): database locks under load #122

EsmailELBoBDev2 opened this issue Nov 30, 2024 · 30 comments · Fixed by #125
Labels
bug Something isn't working

Comments

@EsmailELBoBDev2
Copy link

Description of the bug

I noticed it appeared lately and not sure why, I changed nothing in my config.

Steps To Reproduce

  1. hook up autopulse with arrs
  2. error

Additional Information

I use debian testing with docker compose
logs:

2024-11-29T20:14:56.323603384Z [migrations] started
2024-11-29T20:14:56.323692935Z [migrations] no migrations found
2024-11-29T20:14:56.454020331Z usermod: no changes
2024-11-29T20:14:56.455804908Z ───────────────────────────────────────
2024-11-29T20:14:56.455822868Z   _____ __ __ _____ _____ _____ _____ 
2024-11-29T20:14:56.455825908Z  |     |  |  |   __|_   _|     |     |
2024-11-29T20:14:56.455828298Z  |   --|  |  |__   | | | |  |  | | | |
2024-11-29T20:14:56.455831079Z  |_____|_____|_____| |_| |_____|_|_|_|
2024-11-29T20:14:56.455833499Z        _____ __ __ _ __    ____  
2024-11-29T20:14:56.455835999Z       | __  |  |  | |  |  |    \ 
2024-11-29T20:14:56.455838449Z       | __ -|  |  | |  |__|  |  |
2024-11-29T20:14:56.455841089Z       |_____|_____|_|_____|____/ 
2024-11-29T20:14:56.455843329Z 
2024-11-29T20:14:56.455845649Z   Based on images from linuxserver.io
2024-11-29T20:14:56.455847979Z ───────────────────────────────────────
2024-11-29T20:14:56.456666907Z 
2024-11-29T20:14:56.456685777Z To support LSIO projects visit:
2024-11-29T20:14:56.456689017Z https://www.linuxserver.io/donate/
2024-11-29T20:14:56.456691517Z 
2024-11-29T20:14:56.456693977Z ───────────────────────────────────────
2024-11-29T20:14:56.456696817Z GID/UID
2024-11-29T20:14:56.456699237Z ───────────────────────────────────────
2024-11-29T20:14:56.459763887Z 
2024-11-29T20:14:56.459776857Z User UID:    911
2024-11-29T20:14:56.459779837Z User GID:    911
2024-11-29T20:14:56.459782277Z ───────────────────────────────────────
2024-11-29T20:14:56.518177018Z [custom-init] No custom files found, skipping...
2024-11-29T20:14:56.532642880Z 2024-11-29T20:14:56.532520959Z  INFO autopulse: 💫 autopulse starting up...
2024-11-29T20:14:56.538947222Z 2024-11-29T20:14:56.538656599Z  INFO autopulse: 🚀 Listening on 0.0.0.0:2875
2024-11-29T20:14:56.538966372Z 2024-11-29T20:14:56.53882361Z  INFO actix_server::builder: starting 16 workers
2024-11-29T21:34:08.934552005Z 2024-11-29T21:34:08.934462734Z  INFO actix_web::middleware::logger: 172.18.0.15 
2024-11-30T13:05:07.174609621Z 2024-11-30T13:05:07.17450698Z  INFO autopulse::service::runner: found 1 new file
2024-11-30T13:05:13.699563484Z 2024-11-30T13:05:13.699495883Z  INFO actix_web::middleware::logger: 172.18.0.15 "POST /triggers/radarr HTTP/1.1" 500 24 "-" "Radarr/5.15.1.9463 (alpine 3.20.3)" 0.027548    
2024-11-30T13:05:27.616378534Z 2024-11-30T13:05:27.616310953Z  INFO autopulse::service::runner: sent 1 file to targets
2024-11-30T13:06:00.565812178Z 2024-11-30T13:06:00.565724967Z  INFO autopulse::service::runner: found 1 new file
2024-11-30T13:06:29.574692523Z 2024-11-30T13:06:29.574593402Z  INFO autopulse::service::runner: found 1 new file
2024-11-30T13:06:57.082620299Z 2024-11-30T13:06:57.082506357Z  INFO autopulse::service::runner: sent 1 file to targets
2024-11-30T13:07:07.202583776Z 2024-11-30T13:07:07.202488564Z  INFO autopulse::service::runner: found 1 new file
2024-11-30T13:07:26.384326134Z 2024-11-30T13:07:26.384204682Z  INFO autopulse::service::runner: sent 1 file to targets
2024-11-30T13:07:58.406739247Z 2024-11-30T13:07:58.406656425Z  INFO autopulse::service::runner: sent 1 file to targets
2024-11-30T13:08:10.747687370Z 2024-11-30T13:08:10.747572288Z  INFO autopulse::service::runner: found 1 new file
2024-11-30T13:08:42.700505552Z 2024-11-30T13:08:42.70040054Z ERROR autopulse::service::manager: unable to add notify event: database is locked
2024-11-30T13:08:42.734899163Z 2024-11-30T13:08:42.734820402Z ERROR autopulse::service::manager: unable to add notify event: database is locked
2024-11-30T13:08:42.735077836Z 2024-11-30T13:08:42.735040056Z ERROR autopulse::service::manager: unable to add notify event: database is locked
2024-11-30T13:08:53.781266216Z 2024-11-30T13:08:53.781134454Z  INFO autopulse::service::runner: found 1 new file
2024-11-30T13:09:08.021860702Z 2024-11-30T13:09:08.02173217Z  INFO autopulse::service::runner: sent 1 file to targets
2024-11-30T13:09:11.460911029Z 2024-11-30T13:09:11.460806047Z  INFO autopulse::service::runner: sent 1 file to targets
2024-11-30T13:09:42.582500474Z 2024-11-30T13:09:42.582381852Z  INFO autopulse::service::runner: found 1 new file
2024-11-30T13:10:14.510485100Z 2024-11-30T13:10:14.510410359Z  INFO autopulse::service::runner: found 1 new file
2024-11-30T13:10:36.452868436Z 2024-11-30T13:10:36.452780344Z  INFO autopulse::service::runner: sent 1 file to targets
2024-11-30T13:11:06.540825942Z 2024-11-30T13:11:06.540764611Z ERROR autopulse::service::manager: unable to run pulse: database is locked
2024-11-30T13:11:09.463826906Z 2024-11-30T13:11:09.463725964Z  INFO autopulse::service::runner: sent 1 file to targets
2024-11-30T13:11:11.584437741Z 2024-11-30T13:11:11.58434621Z ERROR autopulse::service::manager: unable to add notify event: database is locked

config:

app:
 hostname: 0.0.0.0
 port: 2875
 database_url: sqlite://data/autopulse.db

anchors:
  - /mnt/zurg/version.txt
  - /mnt/usenetdrive/version.txt

auth:
 username: admin
 password: password

opts:
 check_path: true # Check if the path exists, assuming the path is available to the container
 max_retries: 5 # Exponential backoff when a target fails

triggers:
 radarr:
   type: radarr
 sonarr:
   type: sonarr
 lidarr:
   type: lidarr
 readarr:
   type: readarr
 whisparr:
   type: notify
   paths:
    - "/mnt/usenetdrive/adult"
#   timer:
#     wait: 30

webhooks:
 discord:
   type: "discord"
   url: "https://discord.com/api/webhooks//"

targets:
 jellyfin:
   type: jellyfin
   url: http://jellyfin:8096
   token: ""
@EsmailELBoBDev2 EsmailELBoBDev2 added the bug Something isn't working label Nov 30, 2024
@EsmailELBoBDev2
Copy link
Author

I tested to nuke everything (aka data folder) and still get error on new created db by the app. I use the sqllite version and not sure as it sometimes work and sometimes not so if its related to the arrs or autopulse (autoscan used to work tho)

@dan-online
Copy link
Owner

Could you share your docker config? Seems like this is a permissions issue possibly

@dan-online
Copy link
Owner

@EsmailELBoBDev2 feel free to try danonline/autopulse:pr-125 which should fix this issue. I replicated your issue with some curl spam, and then verified a fix with some load-testing. Let me know how it goes!

@dan-online dan-online changed the title bug: unable to add notify event: database is locked bug(sqlite): database locks under load Dec 2, 2024
@EsmailELBoBDev2
Copy link
Author

I used it and still noticed 2 errors about it but not same as last time so not sure if its related or no

autopulse  | 2024-12-02T23:52:25.306571541Z [migrations] started
autopulse  | 2024-12-02T23:52:25.306638152Z [migrations] no migrations found
autopulse  | 2024-12-02T23:52:25.610812075Z usermod: no changes
autopulse  | 2024-12-02T23:52:25.613563247Z ───────────────────────────────────────
autopulse  | 2024-12-02T23:52:25.613584527Z   _____ __ __ _____ _____ _____ _____ 
autopulse  | 2024-12-02T23:52:25.613587387Z  |     |  |  |   __|_   _|     |     |
autopulse  | 2024-12-02T23:52:25.613589857Z  |   --|  |  |__   | | | |  |  | | | |
autopulse  | 2024-12-02T23:52:25.613591937Z  |_____|_____|_____| |_| |_____|_|_|_|
autopulse  | 2024-12-02T23:52:25.613593997Z        _____ __ __ _ __    ____  
autopulse  | 2024-12-02T23:52:25.613596307Z       | __  |  |  | |  |  |    \ 
autopulse  | 2024-12-02T23:52:25.613598367Z       | __ -|  |  | |  |__|  |  |
autopulse  | 2024-12-02T23:52:25.613600367Z       |_____|_____|_|_____|____/ 
autopulse  | 2024-12-02T23:52:25.613602337Z 
autopulse  | 2024-12-02T23:52:25.613604287Z   Based on images from linuxserver.io
autopulse  | 2024-12-02T23:52:25.613606317Z ───────────────────────────────────────
autopulse  | 2024-12-02T23:52:25.613828171Z 
autopulse  | 2024-12-02T23:52:25.613838401Z To support LSIO projects visit:
autopulse  | 2024-12-02T23:52:25.613840851Z https://www.linuxserver.io/donate/
autopulse  | 2024-12-02T23:52:25.613842811Z 
autopulse  | 2024-12-02T23:52:25.613844731Z ───────────────────────────────────────
autopulse  | 2024-12-02T23:52:25.613847191Z GID/UID
autopulse  | 2024-12-02T23:52:25.613849181Z ───────────────────────────────────────
autopulse  | 2024-12-02T23:52:25.617359924Z 
autopulse  | 2024-12-02T23:52:25.617372184Z User UID:    911
autopulse  | 2024-12-02T23:52:25.617374374Z User GID:    911
autopulse  | 2024-12-02T23:52:25.617376284Z ───────────────────────────────────────
autopulse  | 2024-12-02T23:52:25.751919099Z [custom-init] No custom files found, skipping...
autopulse  | 2024-12-02T23:52:25.766646793Z 2024-12-02T23:52:25.766547081Z  INFO autopulse: 💫 autopulse starting up...
autopulse  | 2024-12-02T23:52:25.845550102Z 2024-12-02T23:52:25.84543311Z  INFO autopulse: 🚀 Listening on 0.0.0.0:2875
autopulse  | 2024-12-02T23:52:25.845731725Z 2024-12-02T23:52:25.845700224Z  INFO actix_server::builder: starting 16 workers
autopulse  | 2024-12-02T23:52:50.427718555Z 2024-12-02T23:52:50.427641694Z  INFO actix_web::middleware::logger: 172.18.0.17 "POST /triggers/radarr HTTP/1.1" 401 12 "-" "Radarr/5.15.1.9463 (alpine 3.20.3)" 0.000159    
autopulse  | 2024-12-02T23:52:51.528885790Z 2024-12-02T23:52:51.528752308Z  INFO actix_web::middleware::logger: 172.18.0.17 "POST /triggers/radarr HTTP/1.1" 401 12 "-" "Radarr/5.15.1.9463 (alpine 3.20.3)" 0.000168    
autopulse  | 2024-12-02T23:53:03.167369219Z 2024-12-02T23:53:03.167273477Z  INFO autopulse::service::runner: sent 1 file to targets
autopulse  | 2024-12-02T23:53:10.160499293Z 2024-12-02T23:53:10.159730591Z  INFO autopulse::service::runner: found 1 new file
autopulse  | 2024-12-02T23:53:14.468898127Z 2024-12-02T23:53:14.468822386Z  INFO actix_web::middleware::logger: 172.18.0.17 "POST /triggers/radarr HTTP/1.1" 401 12 "-" "Radarr/5.15.1.9463 (alpine 3.20.3)" 0.000138    
autopulse  | 2024-12-02T23:53:40.847212416Z 2024-12-02T23:53:40.847120535Z ERROR autopulse::service::runner: unable to delete not found events: DatabaseError(Unknown, "database is locked")
autopulse  | 2024-12-02T23:53:40.847239017Z 2024-12-02T23:53:40.847165706Z ERROR autopulse::service::runner: unable to delete failed events: DatabaseError(Unknown, "database is locked")
autopulse  | 2024-12-02T23:54:42.081650461Z 2024-12-02T23:54:42.08155798Z  INFO autopulse::service::runner: sent 1 file to targets
autopulse  | 2024-12-02T23:55:02.161220738Z 2024-12-02T23:55:02.161130787Z  INFO actix_web::middleware::logger: 172.18.0.15 "POST /triggers/radarr HTTP/1.1" 401 12 "-" "Radarr/5.15.1.9463 (alpine 3.20.3)" 0.000181    
autopulse  | 2024-12-02T23:55:07.077582542Z 2024-12-02T23:55:07.07746151Z  INFO actix_web::middleware::logger: 172.18.0.15 "POST /triggers/radarr HTTP/1.1" 401 12 "-" "Radarr/5.15.1.9463 (alpine 3.20.3)" 0.000167    
autopulse  | 2024-12-02T23:55:51.651578949Z 2024-12-02T23:55:51.651503828Z  INFO actix_web::middleware::logger: 172.18.0.15 "POST /triggers/radarr HTTP/1.1" 401 12 "-" "Radarr/5.15.1.9463 (alpine 3.20.3)" 0.000167    
autopulse  | 2024-12-02T23:56:23.853373569Z 2024-12-02T23:56:23.853306838Z  INFO autopulse::service::runner: found 1 new file

@dan-online
Copy link
Owner

I think the database pool is both getting cloned somewhere it shouldn't coupled with a notify issue, so I'll look at this further and do some more testing

@EsmailELBoBDev2
Copy link
Author

I think the database pool is both getting cloned somewhere it shouldn't coupled with a notify issue, so I'll look at this further and do some more testing

If it helps I will stop notify one and test again (will post results)

@dan-online
Copy link
Owner

@EsmailELBoBDev2 I think I found the main source of the issue, and hopefully solved it with 87224f1 (#125) which disabled WAL mode on SQLite. danonline/autopulse:pr-125 should update shortly, let me know if that fixes it!

@EsmailELBoBDev2
Copy link
Author

Hello, it's still erroring out :/

autopulse  | 2024-12-03T17:32:39.401302204Z [migrations] started
autopulse  | 2024-12-03T17:32:39.401350095Z [migrations] no migrations found
autopulse  | 2024-12-03T17:32:39.581671963Z usermod: no changes
autopulse  | 2024-12-03T17:32:39.583226216Z ───────────────────────────────────────
autopulse  | 2024-12-03T17:32:39.583238416Z   _____ __ __ _____ _____ _____ _____ 
autopulse  | 2024-12-03T17:32:39.583240746Z  |     |  |  |   __|_   _|     |     |
autopulse  | 2024-12-03T17:32:39.583242706Z  |   --|  |  |__   | | | |  |  | | | |
autopulse  | 2024-12-03T17:32:39.583244606Z  |_____|_____|_____| |_| |_____|_|_|_|
autopulse  | 2024-12-03T17:32:39.583247026Z        _____ __ __ _ __    ____  
autopulse  | 2024-12-03T17:32:39.583249426Z       | __  |  |  | |  |  |    \ 
autopulse  | 2024-12-03T17:32:39.583251886Z       | __ -|  |  | |  |__|  |  |
autopulse  | 2024-12-03T17:32:39.583254226Z       |_____|_____|_|_____|____/ 
autopulse  | 2024-12-03T17:32:39.583256606Z 
autopulse  | 2024-12-03T17:32:39.583258766Z   Based on images from linuxserver.io
autopulse  | 2024-12-03T17:32:39.583261116Z ───────────────────────────────────────
autopulse  | 2024-12-03T17:32:39.583531311Z 
autopulse  | 2024-12-03T17:32:39.583539351Z To support LSIO projects visit:
autopulse  | 2024-12-03T17:32:39.583542231Z https://www.linuxserver.io/donate/
autopulse  | 2024-12-03T17:32:39.583555821Z 
autopulse  | 2024-12-03T17:32:39.583557901Z ───────────────────────────────────────
autopulse  | 2024-12-03T17:32:39.583560551Z GID/UID
autopulse  | 2024-12-03T17:32:39.583562451Z ───────────────────────────────────────
autopulse  | 2024-12-03T17:32:39.586796850Z 
autopulse  | 2024-12-03T17:32:39.586809650Z User UID:    911
autopulse  | 2024-12-03T17:32:39.586812250Z User GID:    911
autopulse  | 2024-12-03T17:32:39.586814531Z ───────────────────────────────────────
autopulse  | 2024-12-03T17:32:39.642367804Z [custom-init] No custom files found, skipping...
autopulse  | 2024-12-03T17:32:39.659662086Z 2024-12-03T17:32:39.659574385Z  INFO autopulse: 💫 autopulse starting up...
autopulse  | 2024-12-03T17:32:39.661857430Z 2024-12-03T17:32:39.661801739Z ERROR r2d2: database is locked    
autopulse  | 2024-12-03T17:32:39.661879090Z 2024-12-03T17:32:39.661838529Z ERROR r2d2: database is locked    
autopulse  | 2024-12-03T17:32:39.662035312Z 2024-12-03T17:32:39.662001172Z ERROR r2d2: database is locked    
autopulse  | 2024-12-03T17:32:39.662082833Z 2024-12-03T17:32:39.662053713Z ERROR r2d2: database is locked    
autopulse  | 2024-12-03T17:32:39.662202845Z 2024-12-03T17:32:39.662181585Z ERROR r2d2: database is locked    
autopulse  | 2024-12-03T17:32:39.662274956Z 2024-12-03T17:32:39.662254806Z ERROR r2d2: database is locked    
autopulse  | 2024-12-03T17:32:39.662407618Z 2024-12-03T17:32:39.662386748Z ERROR r2d2: database is locked    
autopulse  | 2024-12-03T17:32:39.662473859Z 2024-12-03T17:32:39.662456499Z ERROR r2d2: database is locked    
autopulse  | 2024-12-03T17:32:39.662598771Z 2024-12-03T17:32:39.662575591Z ERROR r2d2: database is locked    
autopulse  | 2024-12-03T17:32:39.662681672Z 2024-12-03T17:32:39.662659112Z ERROR r2d2: database is locked    
autopulse  | 2024-12-03T17:32:39.662808294Z 2024-12-03T17:32:39.662789424Z ERROR r2d2: database is locked    
autopulse  | 2024-12-03T17:32:39.662874035Z 2024-12-03T17:32:39.662857325Z ERROR r2d2: database is locked    
autopulse  | 2024-12-03T17:32:39.663010167Z 2024-12-03T17:32:39.662986637Z ERROR r2d2: database is locked    
autopulse  | 2024-12-03T17:32:39.663074398Z 2024-12-03T17:32:39.663055638Z ERROR r2d2: database is locked    
autopulse  | 2024-12-03T17:32:39.663165370Z 2024-12-03T17:32:39.663147489Z ERROR r2d2: database is locked    
autopulse  | 2024-12-03T17:32:40.065193202Z 2024-12-03T17:32:40.065116211Z  INFO autopulse: 🚀 Listening on 0.0.0.0:2875
autopulse  | 2024-12-03T17:32:40.065375255Z 2024-12-03T17:32:40.065278134Z  INFO actix_server::builder: starting 16 workers
autopulse  | 2024-12-03T17:32:56.232102295Z 2024-12-03T17:32:56.232039294Z  INFO autopulse::service::runner: found 1 new file
autopulse  | 2024-12-03T17:33:39.097849670Z 2024-12-03T17:33:39.097754168Z  INFO autopulse::service::runner: found 1 new file
autopulse  | 2024-12-03T17:34:33.031378841Z 2024-12-03T17:34:33.031306909Z  INFO autopulse::service::runner: sent 1 file to targets
autopulse  | 2024-12-03T17:36:40.891923338Z 2024-12-03T17:36:40.891791986Z  INFO autopulse::service::runner: sent 1 file to targets
autopulse  | 2024-12-03T17:37:42.093223274Z 2024-12-03T17:37:42.093151713Z  INFO autopulse::service::runner: found 1 new file
autopulse  | 2024-12-03T17:38:46.313562710Z 2024-12-03T17:38:46.313457968Z  INFO autopulse::service::runner: found 1 new file
autopulse  | 2024-12-03T17:39:42.641524257Z 2024-12-03T17:39:42.641405775Z  INFO autopulse::service::runner: sent 1 file to targets
autopulse  | 2024-12-03T17:40:47.504478628Z 2024-12-03T17:40:47.504360116Z  INFO autopulse::service::runner: sent 1 file to targets
autopulse  | 2024-12-03T17:40:59.098769351Z 2024-12-03T17:40:59.098699Z  INFO autopulse::service::runner: found 1 new file
autopulse  | 2024-12-03T17:42:18.255444668Z 2024-12-03T17:42:18.255298986Z  INFO autopulse::service::runner: found 1 new file
autopulse  | 2024-12-03T17:42:51.813810988Z 2024-12-03T17:42:51.813691206Z  INFO autopulse::service::runner: sent 1 file to targets
autopulse  | 2024-12-03T17:43:28.104360066Z 2024-12-03T17:43:28.104140863Z ERROR autopulse::service::runner: failed to process target 'jellyfin': error sending request for url (http://jellyfin:8096/Items?Recursive=true&Fields=Path&EnableImages=false&IncludeItemTypes=&ParentId=0a7fd8175719d8f7ebfb93874e55a2d5&EnableTotalRecordCount=false)
autopulse  | 2024-12-03T17:43:28.104431967Z 
autopulse  | 2024-12-03T17:43:28.104436357Z Caused by:
autopulse  | 2024-12-03T17:43:28.104439807Z     operation timed out
autopulse  | 2024-12-03T17:43:28.145153675Z 2024-12-03T17:43:28.144990723Z  INFO autopulse::service::runner: sent 1 file to targets
autopulse  | 2024-12-03T17:43:50.346796980Z 2024-12-03T17:43:50.346225062Z  INFO autopulse::service::runner: found 1 new file
autopulse  | 2024-12-03T17:44:51.623135625Z 2024-12-03T17:44:51.623006294Z  INFO autopulse::service::runner: sent 1 file to targets

I'm now testing without whisparr/inotify and will tell you if i still face same issue

@dan-online
Copy link
Owner

Hello, it's still erroring out :/


autopulse  | 2024-12-03T17:32:39.401302204Z [migrations] started

autopulse  | 2024-12-03T17:32:39.401350095Z [migrations] no migrations found

autopulse  | 2024-12-03T17:32:39.581671963Z usermod: no changes

autopulse  | 2024-12-03T17:32:39.583226216Z ───────────────────────────────────────

autopulse  | 2024-12-03T17:32:39.583238416Z   _____ __ __ _____ _____ _____ _____ 

autopulse  | 2024-12-03T17:32:39.583240746Z  |     |  |  |   __|_   _|     |     |

autopulse  | 2024-12-03T17:32:39.583242706Z  |   --|  |  |__   | | | |  |  | | | |

autopulse  | 2024-12-03T17:32:39.583244606Z  |_____|_____|_____| |_| |_____|_|_|_|

autopulse  | 2024-12-03T17:32:39.583247026Z        _____ __ __ _ __    ____  

autopulse  | 2024-12-03T17:32:39.583249426Z       | __  |  |  | |  |  |    \ 

autopulse  | 2024-12-03T17:32:39.583251886Z       | __ -|  |  | |  |__|  |  |

autopulse  | 2024-12-03T17:32:39.583254226Z       |_____|_____|_|_____|____/ 

autopulse  | 2024-12-03T17:32:39.583256606Z 

autopulse  | 2024-12-03T17:32:39.583258766Z   Based on images from linuxserver.io

autopulse  | 2024-12-03T17:32:39.583261116Z ───────────────────────────────────────

autopulse  | 2024-12-03T17:32:39.583531311Z 

autopulse  | 2024-12-03T17:32:39.583539351Z To support LSIO projects visit:

autopulse  | 2024-12-03T17:32:39.583542231Z https://www.linuxserver.io/donate/

autopulse  | 2024-12-03T17:32:39.583555821Z 

autopulse  | 2024-12-03T17:32:39.583557901Z ───────────────────────────────────────

autopulse  | 2024-12-03T17:32:39.583560551Z GID/UID

autopulse  | 2024-12-03T17:32:39.583562451Z ───────────────────────────────────────

autopulse  | 2024-12-03T17:32:39.586796850Z 

autopulse  | 2024-12-03T17:32:39.586809650Z User UID:    911

autopulse  | 2024-12-03T17:32:39.586812250Z User GID:    911

autopulse  | 2024-12-03T17:32:39.586814531Z ───────────────────────────────────────

autopulse  | 2024-12-03T17:32:39.642367804Z [custom-init] No custom files found, skipping...

autopulse  | 2024-12-03T17:32:39.659662086Z 2024-12-03T17:32:39.659574385Z  INFO autopulse: 💫 autopulse starting up...

autopulse  | 2024-12-03T17:32:39.661857430Z 2024-12-03T17:32:39.661801739Z ERROR r2d2: database is locked    

autopulse  | 2024-12-03T17:32:39.661879090Z 2024-12-03T17:32:39.661838529Z ERROR r2d2: database is locked    

autopulse  | 2024-12-03T17:32:39.662035312Z 2024-12-03T17:32:39.662001172Z ERROR r2d2: database is locked    

autopulse  | 2024-12-03T17:32:39.662082833Z 2024-12-03T17:32:39.662053713Z ERROR r2d2: database is locked    

autopulse  | 2024-12-03T17:32:39.662202845Z 2024-12-03T17:32:39.662181585Z ERROR r2d2: database is locked    

autopulse  | 2024-12-03T17:32:39.662274956Z 2024-12-03T17:32:39.662254806Z ERROR r2d2: database is locked    

autopulse  | 2024-12-03T17:32:39.662407618Z 2024-12-03T17:32:39.662386748Z ERROR r2d2: database is locked    

autopulse  | 2024-12-03T17:32:39.662473859Z 2024-12-03T17:32:39.662456499Z ERROR r2d2: database is locked    

autopulse  | 2024-12-03T17:32:39.662598771Z 2024-12-03T17:32:39.662575591Z ERROR r2d2: database is locked    

autopulse  | 2024-12-03T17:32:39.662681672Z 2024-12-03T17:32:39.662659112Z ERROR r2d2: database is locked    

autopulse  | 2024-12-03T17:32:39.662808294Z 2024-12-03T17:32:39.662789424Z ERROR r2d2: database is locked    

autopulse  | 2024-12-03T17:32:39.662874035Z 2024-12-03T17:32:39.662857325Z ERROR r2d2: database is locked    

autopulse  | 2024-12-03T17:32:39.663010167Z 2024-12-03T17:32:39.662986637Z ERROR r2d2: database is locked    

autopulse  | 2024-12-03T17:32:39.663074398Z 2024-12-03T17:32:39.663055638Z ERROR r2d2: database is locked    

autopulse  | 2024-12-03T17:32:39.663165370Z 2024-12-03T17:32:39.663147489Z ERROR r2d2: database is locked    

autopulse  | 2024-12-03T17:32:40.065193202Z 2024-12-03T17:32:40.065116211Z  INFO autopulse: 🚀 Listening on 0.0.0.0:2875

autopulse  | 2024-12-03T17:32:40.065375255Z 2024-12-03T17:32:40.065278134Z  INFO actix_server::builder: starting 16 workers

autopulse  | 2024-12-03T17:32:56.232102295Z 2024-12-03T17:32:56.232039294Z  INFO autopulse::service::runner: found 1 new file

autopulse  | 2024-12-03T17:33:39.097849670Z 2024-12-03T17:33:39.097754168Z  INFO autopulse::service::runner: found 1 new file

autopulse  | 2024-12-03T17:34:33.031378841Z 2024-12-03T17:34:33.031306909Z  INFO autopulse::service::runner: sent 1 file to targets

autopulse  | 2024-12-03T17:36:40.891923338Z 2024-12-03T17:36:40.891791986Z  INFO autopulse::service::runner: sent 1 file to targets

autopulse  | 2024-12-03T17:37:42.093223274Z 2024-12-03T17:37:42.093151713Z  INFO autopulse::service::runner: found 1 new file

autopulse  | 2024-12-03T17:38:46.313562710Z 2024-12-03T17:38:46.313457968Z  INFO autopulse::service::runner: found 1 new file

autopulse  | 2024-12-03T17:39:42.641524257Z 2024-12-03T17:39:42.641405775Z  INFO autopulse::service::runner: sent 1 file to targets

autopulse  | 2024-12-03T17:40:47.504478628Z 2024-12-03T17:40:47.504360116Z  INFO autopulse::service::runner: sent 1 file to targets

autopulse  | 2024-12-03T17:40:59.098769351Z 2024-12-03T17:40:59.098699Z  INFO autopulse::service::runner: found 1 new file

autopulse  | 2024-12-03T17:42:18.255444668Z 2024-12-03T17:42:18.255298986Z  INFO autopulse::service::runner: found 1 new file

autopulse  | 2024-12-03T17:42:51.813810988Z 2024-12-03T17:42:51.813691206Z  INFO autopulse::service::runner: sent 1 file to targets

autopulse  | 2024-12-03T17:43:28.104360066Z 2024-12-03T17:43:28.104140863Z ERROR autopulse::service::runner: failed to process target 'jellyfin': error sending request for url (http://jellyfin:8096/Items?Recursive=true&Fields=Path&EnableImages=false&IncludeItemTypes=&ParentId=0a7fd8175719d8f7ebfb93874e55a2d5&EnableTotalRecordCount=false)

autopulse  | 2024-12-03T17:43:28.104431967Z 

autopulse  | 2024-12-03T17:43:28.104436357Z Caused by:

autopulse  | 2024-12-03T17:43:28.104439807Z     operation timed out

autopulse  | 2024-12-03T17:43:28.145153675Z 2024-12-03T17:43:28.144990723Z  INFO autopulse::service::runner: sent 1 file to targets

autopulse  | 2024-12-03T17:43:50.346796980Z 2024-12-03T17:43:50.346225062Z  INFO autopulse::service::runner: found 1 new file

autopulse  | 2024-12-03T17:44:51.623135625Z 2024-12-03T17:44:51.623006294Z  INFO autopulse::service::runner: sent 1 file to targets

I'm now testing without whisparr/inotify and will tell you if i still face same issue

Ah, I think I'll need to add this as a migration... if you remove the existing db then it should work

@EsmailELBoBDev2
Copy link
Author

oh yeah sure no problem, I deleted whole data folder and will test again and let you know!

@EsmailELBoBDev2
Copy link
Author

Seems working 🥳 but off topic issue is:

autopulse  | 2024-12-03T21:54:54.563149495Z 2024-12-03T21:54:54.563063923Z ERROR autopulse::service::webhooks::manager: failed to send webhook 'discord': failed to send webhook: {"embeds": ["0"]}
autopulse  | 2024-12-03T21:54:56.322331850Z 2024-12-03T21:54:56.32227106Z  INFO autopulse::service::runner: sent 9 files to targets
autopulse  | 2024-12-03T21:55:04.572864878Z 2024-12-03T21:55:04.572758976Z ERROR autopulse::service::webhooks::manager: failed to send webhook 'discord': failed to send webhook: {"embeds": ["1"]}

even tho my config seems fine and it indeed send some notifications fine


webhooks:
 discord:
   type: "discord"
   url: "https://discord.com/api/webhooks/RANDOM-19-NUMBERS/RANDOM-68-CHARS"

@dan-online
Copy link
Owner

Yep that would be linked to #105, I'll change the PR to auto-migrate the database then merge which will close this

@EsmailELBoBDev2
Copy link
Author

Alright, good luck and thank you so much for the help!

@EsmailELBoBDev2
Copy link
Author

EsmailELBoBDev2 commented Dec 4, 2024

Hello @dan-online, I found some entries but thats after longer run

autopulse  | 2024-12-04T16:17:08.386882322Z 2024-12-04T16:17:08.386545836Z ERROR autopulse::service::manager: unable to run pulse: database is locked
autopulse  | 2024-12-04T16:17:13.392002391Z 2024-12-04T16:17:13.39192432Z ERROR autopulse::service::manager: unable to run pulse: database is locked
autopulse  | 2024-12-04T16:17:18.395649263Z 2024-12-04T16:17:18.395583862Z ERROR autopulse::service::manager: unable to run pulse: database is locked

and

autopulse  | 2024-12-03T23:37:54.274173452Z 2024-12-03T23:37:54.274099631Z ERROR r2d2: database is locked    
autopulse  | 2024-12-03T23:37:54.674446034Z 2024-12-03T23:37:54.674373413Z ERROR r2d2: database is locked    

I'm using inotify but removed the db as you recommended before

@dan-online
Copy link
Owner

dan-online commented Dec 4, 2024

Hmmm, that's odd, I write/del 20 files/s for 15mins and I didn't see any database locking on danonline/autopulse:pr-125, did you make sure you pulled the new image?

Edit: corrected pr number

@EsmailELBoBDev2
Copy link
Author

uhh i was on 125 still 😅 -- for 105 I get manifest for danonline/autopulse:pr-105 not found: manifest unknown: manifest unknown

@dan-online
Copy link
Owner

Whoops I mean 125, but since it pushes a new image to the same tag you may be on an old version

@EsmailELBoBDev2
Copy link
Author

oh yes I made sure to pull it before I reply above and pulling now does not yield any new updates

@EsmailELBoBDev2
Copy link
Author

and for me it took over some hours to notice the error so first 15m were fine indeed

@dan-online
Copy link
Owner

Hm, okay interesting, I'll link diesel-rs/diesel#2365 to keep track

@EsmailELBoBDev2
Copy link
Author

if you want me to test anything please let me know and to be fair I don't get it as often as used to

@dan-online
Copy link
Owner

dan-online commented Dec 4, 2024

I've pushed 1539335 (#125) which seemed not to lock while using WAL, so perhaps this fixes it, I'll run some 100 file/s spam over night and see if that works

@EsmailELBoBDev2
Copy link
Author

okay i pulled it too and gonna test and let you know later! (PS I have so much files so yeah... 😂 )

@EsmailELBoBDev2
Copy link
Author

autopulse  | 2024-12-04T23:03:24.723970305Z [migrations] started
autopulse  | 2024-12-04T23:03:24.724029506Z [migrations] no migrations found
autopulse  | 2024-12-04T23:03:24.823584973Z usermod: no changes
autopulse  | 2024-12-04T23:03:24.825081737Z ───────────────────────────────────────
autopulse  | 2024-12-04T23:03:24.825092377Z   _____ __ __ _____ _____ _____ _____ 
autopulse  | 2024-12-04T23:03:24.825095137Z  |     |  |  |   __|_   _|     |     |
autopulse  | 2024-12-04T23:03:24.825097647Z  |   --|  |  |__   | | | |  |  | | | |
autopulse  | 2024-12-04T23:03:24.825100167Z  |_____|_____|_____| |_| |_____|_|_|_|
autopulse  | 2024-12-04T23:03:24.825102527Z        _____ __ __ _ __    ____  
autopulse  | 2024-12-04T23:03:24.825104927Z       | __  |  |  | |  |  |    \ 
autopulse  | 2024-12-04T23:03:24.825107837Z       | __ -|  |  | |  |__|  |  |
autopulse  | 2024-12-04T23:03:24.825110317Z       |_____|_____|_|_____|____/ 
autopulse  | 2024-12-04T23:03:24.825112687Z 
autopulse  | 2024-12-04T23:03:24.825115087Z   Based on images from linuxserver.io
autopulse  | 2024-12-04T23:03:24.825117657Z ───────────────────────────────────────
autopulse  | 2024-12-04T23:03:24.825396162Z 
autopulse  | 2024-12-04T23:03:24.825402052Z To support LSIO projects visit:
autopulse  | 2024-12-04T23:03:24.825404822Z https://www.linuxserver.io/donate/
autopulse  | 2024-12-04T23:03:24.825407202Z 
autopulse  | 2024-12-04T23:03:24.825409442Z ───────────────────────────────────────
autopulse  | 2024-12-04T23:03:24.825412122Z GID/UID
autopulse  | 2024-12-04T23:03:24.825414392Z ───────────────────────────────────────
autopulse  | 2024-12-04T23:03:24.828617063Z 
autopulse  | 2024-12-04T23:03:24.828625263Z User UID:    911
autopulse  | 2024-12-04T23:03:24.828627643Z User GID:    911
autopulse  | 2024-12-04T23:03:24.828629613Z ───────────────────────────────────────
autopulse  | 2024-12-04T23:03:24.871756131Z [custom-init] No custom files found, skipping...
autopulse  | 2024-12-04T23:03:24.881601658Z 2024-12-04T23:03:24.881512696Z  INFO autopulse: 💫 autopulse starting up...
autopulse  | 2024-12-04T23:03:24.883341925Z 2024-12-04T23:03:24.883292515Z ERROR r2d2: database is locked    
autopulse  | 2024-12-04T23:03:25.284034433Z 2024-12-04T23:03:25.283928181Z  INFO autopulse: 🚀 Listening on 0.0.0.0:2875
autopulse  | 2024-12-04T23:03:25.284134724Z 2024-12-04T23:03:25.284101664Z  INFO actix_server::builder: starting 16 workers

not great start but hopefully its one time thing

@dan-online
Copy link
Owner

dan-online commented Dec 4, 2024

Huh, when it's the first boot on a new database- it seems to fail setting the pragmas (probably because it runs on each connection which is set to 16 for now) so yea should be okay past there

@EsmailELBoBDev2
Copy link
Author

welp, im running it now and testing so i will let you know later! and thank you

@dan-online
Copy link
Owner

26c8688 (#125) should fix that in future, I only need to set journal mode once as its persistent, and the other options are set for any future multi-threaded connections. This should lead to no more database lock errors

@EsmailELBoBDev2
Copy link
Author

I pulled it and gonna gonna let you know

@EsmailELBoBDev2
Copy link
Author

Hello, I think this fixes it! 🥳 I got no more issues about db or even discord. I get some errors about jellyfin itself but i will assume it's something related to my setup

autopulse  | 2024-12-05T18:12:29.294693414Z 2024-12-05T18:12:29.294597102Z ERROR autopulse::service::runner: failed to process target 'jellyfin': error sending request for url (http://jellyfin:8096/Items?Recursive=true&Fields=Path&EnableImages=false&IncludeItemTypes=&ParentId=0a7fd8175719d8f7ebfb93874e55a2d5&EnableTotalRecordCount=false)

@dan-online
Copy link
Owner

Awesome! Merged to latest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants