Gitea crashes on startup after running "Delete all repository archives" (view logs) #11754

Nathan13888 · 2020-06-03T23:04:36Z

Gitea version (or commit ref): v1.13.0+dev-117-ga3b185d54
Git version: 2.20.1 (for host)
Operating system: Docker (Debian Buster)
Database (use [x]):
- PostgreSQL
- MySQL
- MSSQL
- SQLite
Can you reproduce the bug at https://try.gitea.io:
- Yes (provide example URL)
- No
- [x ] Not relevant
Log gist:

Description

Gitea worked perfectly fine before I changed some settings under the server section in app.ini. After I noticed this crashing, I immediately changed all the settings back to the original settings but the same problem would arise. I'm not sure but I believe that this would have happened even if I didn't change these settings.

app.ini:
app.zip

Log from inside the Gitea container:
gitea.log

Log before crashing (in docker)
:

03 17:24:08 routers/init.go:120:GlobalInit() [T] AppWorkPath: /app/gitea


2020/06/03 17:24:08 routers/init.go:121:GlobalInit() [T] Custom path: /data/gitea


2020/06/03 17:24:08 routers/init.go:122:GlobalInit() [T] Log path: /data/gitea/log


2020/06/03 17:24:08 routers/init.go:127:GlobalInit() [I] Delete all repository archives


2020/06/03 17:24:08 ...dules/setting/log.go:233:newLogService() [I] Gitea v1.13.0+dev-117-ga3b185d54 built with GNU Make 4.3, go1.14.4 : bindata, sqlite, sqlite_unlock_notify


panic: runtime error: index out of range [0] with length 0



goroutine 1 [running]:


code.gitea.io/gitea/modules/setting.DBConnStr(0xff55927, 0x55934c0, 0xc0005a54f0, 0x6d0a47)


	/go/src/code.gitea.io/gitea/modules/setting/database.go:108 +0x93c


code.gitea.io/gitea/models.getEngine(0x0, 0x0, 0x0)


	/go/src/code.gitea.io/gitea/models/models.go:137 +0x26


code.gitea.io/gitea/models.SetEngine(0x2, 0x2)


	/go/src/code.gitea.io/gitea/models/models.go:168 +0x34


code.gitea.io/gitea/models.NewEngine(0x3f4e8c0, 0xc002da9dc0, 0x3aae5b8, 0x2b, 0xc0001f3000)


	/go/src/code.gitea.io/gitea/models/models.go:190 +0x26


code.gitea.io/gitea/routers.initDBEngine(0x3f4e8c0, 0xc002da9dc0, 0x21b6d0c, 0x2)


	/go/src/code.gitea.io/gitea/routers/init.go:72 +0x1f2


code.gitea.io/gitea/routers.GlobalInit(0x3f4e8c0, 0xc002da9dc0)


	/go/src/code.gitea.io/gitea/routers/init.go:135 +0x81f


code.gitea.io/gitea/cmd.runWeb(0xc0000d8f20, 0x0, 0x0)


	/go/src/code.gitea.io/gitea/cmd/web.go:116 +0x2c1


github.com/urfave/cli.HandleAction(0x1e507c0, 0x3aae538, 0xc0000d8f20, 0xc001738400, 0x0)


	/go/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/app.go:490 +0xc8


github.com/urfave/cli.Command.Run(0x21bdcd6, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x22d6d5c, 0x16, 0x0, ...)


	/go/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/command.go:210 +0x9e8


github.com/urfave/cli.(*App).Run(0xc000c6b6c0, 0xc00000e080, 0x2, 0x2, 0x0, 0x0)


	/go/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/app.go:255 +0x741


main.main()


	/go/src/code.gitea.io/gitea/main.go:109 +0x8a3


Received signal 15; terminating.
...

The text was updated successfully, but these errors were encountered:

Nathan13888 · 2020-06-05T00:59:58Z

Does anyone have any idea of perhaps resetting something that way I could get gitea back up and running?? I am running the Gitea instance in Docker and I also have complete access to the external MySQL database which Gitea uses.

Nathan13888 · 2020-06-05T01:10:19Z

It seems that Gitea crashes after running "Delete all repository archives". Is there a way to stop that?

Nathan13888 · 2020-06-10T15:02:32Z

I'll close this as no one seems to see this... ;''/

techknowlogick · 2020-06-10T15:04:02Z

I may have an idea of what is happening. As such I will re-open. When starting Gitea, can you look at memory consumption?

techknowlogick · 2020-06-10T15:06:34Z

Edit: This assumption I made was incorrect.

Nathan13888 · 2020-06-10T15:19:03Z

@techknowlogick Thanks for reopening the issue.

I've applied the settings you've just talked about.

Since I'm running Gitea in docker, I couldn't view the memory usage as Gitea would just crash right away when I start it; meaning I couldn't view the memory usage.

Side Note: I'm running this docker container on a home server running Debian Buster that only has 16GB of ram.

Below is what the server and queue settings are in the app.ini config.

[server]
APP_DATA_PATH    = /data/gitea
SSH_DOMAIN       = 192.168.10.200
HTTP_PORT        = 3000
ROOT_URL         = http://192.168.10.200:888
DISABLE_SSH      = false
SSH_PORT         = 22
SSH_LISTEN_PORT  = 22
LFS_START_SERVER = true
LFS_CONTENT_PATH = /data/git/lfs
LFS_JWT_SECRET   = ZQE8MORf4mb-gkZUQWv9GY_dSR5CM95g9bpuisYT4v0
DOMAIN           = localhost
OFFLINE_MODE     = false

[queue]
TYPE=channel
MAX_WORKERS=2
BOOST_WORKERS=1

Nathan13888 · 2020-06-10T15:20:51Z

Also, would it be possible to potentially transplant the data from the current Gitea container to a new one? I don't mind if I lose some data as I have a copy of all the repositories except for the settings.

zeripath · 2020-06-11T08:47:17Z

@Nathan13888 are you still getting the same panic?

panic: runtime error: index out of range [0] with length 0

goroutine 1 [running]:
code.gitea.io/gitea/modules/setting.DBConnStr(0xff55927, 0x55934c0, 0xc0005a54f0, 0x6d0a47)
	/go/src/code.gitea.io/gitea/modules/setting/database.go:108 +0x93c
code.gitea.io/gitea/models.getEngine(0x0, 0x0, 0x0)
	/go/src/code.gitea.io/gitea/models/models.go:137 +0x26
code.gitea.io/gitea/models.SetEngine(0x2, 0x2)
	/go/src/code.gitea.io/gitea/models/models.go:168 +0x34
...

have you actually looked at the code for: /go/src/code.gitea.io/gitea/modules/setting/database.go:108 ?

gitea/modules/setting/database.go

Line 108 in 4a4977a

if Database.Host[0] == '/' { // looks like a unix socket

Looking at your provided app.ini:

...
[database]
PATH     = /data/gitea/gitea.db
DB_TYPE  = mysql
USER     = gitea
PASSWD   = **OMITTED**
SCHEMA   = 
SSL_MODE = disable
CHARSET  = utf8mb4

[indexer]
...

I see that HOST is not set. We don't provide a default:

gitea/modules/setting/database.go

Line 73 in 4a4977a

Database.Host = sec.Key("HOST").String()

Therefore your problem is that you haven't set a HOST

Nathan13888 · 2020-06-11T14:28:51Z

Hi @zeripath, you were right about the HOST property as I seem to have accidentally removed this line along with another line NAME in [database].

After these changes, the container no longer crashes instantly but after half a minute.

There are no details in the Docker log but I found this in the log within the container (view the end of the message).

I also compared an old working version of the config and there seems to have no more differences from the current config.

From the logs, it seems that Gitea is unable to communicate with MYSQL but the mysql container has been running non-stop for a month or two already without problems and haven't been changed by me either.

2020/06/11 10:03:40 ...rm/session_schema.go:25:Ping() [I] PING DATABASE mysql
2020/06/11 10:03:40 ...m.io/xorm/core/db.go:154:QueryContext() [I] [SQL] SELECT `TABLE_NAME` from `INFORMATION_SCHEMA`.`TABLES` WHERE `TABLE_SCHEMA`=? and `TABLE_NAME`=? [ version] - 445.593µs
2020/06/11 10:03:40 ...m.io/xorm/core/db.go:277:ExecContext() [I] [SQL] CREATE TABLE IF NOT EXISTS `version` (`id` BIGINT(20) PRIMARY KEY AUTO_INCREMENT NOT NULL, `version` BIGINT(20) NULL) DEFAULT CHARSET utf8mb4 ROW_FORMAT=DYNAMIC [] - 92.163µs
2020/06/11 10:03:40 routers/init.go:77:initDBEngine() [E] ORM engine initialization attempt #9/10 failed. Error: migrate: sync: Error 1046: No database selected
2020/06/11 10:03:40 routers/init.go:78:initDBEngine() [I] Backing off for 3 seconds
2020/06/11 10:03:43 routers/init.go:71:initDBEngine() [I] ORM engine initialization attempt #10/10...
2020/06/11 10:03:43 ...rm/session_schema.go:25:Ping() [I] PING DATABASE mysql
2020/06/11 10:03:43 ...m.io/xorm/core/db.go:154:QueryContext() [I] [SQL] SELECT `TABLE_NAME` from `INFORMATION_SCHEMA`.`TABLES` WHERE `TABLE_SCHEMA`=? and `TABLE_NAME`=? [ version] - 483.875µs
2020/06/11 10:03:43 ...m.io/xorm/core/db.go:277:ExecContext() [I] [SQL] CREATE TABLE IF NOT EXISTS `version` (`id` BIGINT(20) PRIMARY KEY AUTO_INCREMENT NOT NULL, `version` BIGINT(20) NULL) DEFAULT CHARSET utf8mb4 ROW_FORMAT=DYNAMIC [] - 119.583µs
2020/06/11 10:03:43 routers/init.go:138:GlobalInit() [F] ORM engine initialization failed: migrate: sync: Error 1046: No database selected

Nathan13888 · 2020-06-11T14:29:43Z

Btw, I retried multiple times and have come to the same error in the log

zeripath · 2020-06-11T15:03:51Z

Gitea cannot connect to your database. You need to check the rest of your config.

I suspect the hint is here: PING DATABASE mysql - implying that you've set NAME=mysql

But that's very unlikely to the name of the Gitea database. (The default is NAME=gitea)

I suggest you check the results of SHOW DATABASES from mysql.

Nathan13888 changed the title ~~Gitea Crashes on starup: "panic: runtime error: index out of range [0] with length 0"~~ Gitea crashes on startup after running "Delete all repository archives" (view logs) Jun 5, 2020

Nathan13888 closed this as completed Jun 10, 2020

techknowlogick reopened this Jun 10, 2020

zeripath mentioned this issue Jun 11, 2020

Prevent panic on empty HOST for mysql #11850

Merged

techknowlogick closed this as completed in #11850 Jun 11, 2020

go-gitea locked and limited conversation to collaborators Nov 24, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Gitea crashes on startup after running "Delete all repository archives" (view logs) #11754

Gitea crashes on startup after running "Delete all repository archives" (view logs) #11754

Nathan13888 commented Jun 3, 2020

Nathan13888 commented Jun 5, 2020

Nathan13888 commented Jun 5, 2020

Nathan13888 commented Jun 10, 2020

techknowlogick commented Jun 10, 2020

techknowlogick commented Jun 10, 2020 •

edited

Loading

Nathan13888 commented Jun 10, 2020

Nathan13888 commented Jun 10, 2020

zeripath commented Jun 11, 2020 •

edited

Loading

Nathan13888 commented Jun 11, 2020

Nathan13888 commented Jun 11, 2020

zeripath commented Jun 11, 2020 •

edited

Loading

Gitea crashes on startup after running "Delete all repository archives" (view logs) #11754

Gitea crashes on startup after running "Delete all repository archives" (view logs) #11754

Comments

Nathan13888 commented Jun 3, 2020

Description

Nathan13888 commented Jun 5, 2020

Nathan13888 commented Jun 5, 2020

Nathan13888 commented Jun 10, 2020

techknowlogick commented Jun 10, 2020

techknowlogick commented Jun 10, 2020 • edited Loading

Nathan13888 commented Jun 10, 2020

Nathan13888 commented Jun 10, 2020

zeripath commented Jun 11, 2020 • edited Loading

Nathan13888 commented Jun 11, 2020

Nathan13888 commented Jun 11, 2020

zeripath commented Jun 11, 2020 • edited Loading

techknowlogick commented Jun 10, 2020 •

edited

Loading

zeripath commented Jun 11, 2020 •

edited

Loading

zeripath commented Jun 11, 2020 •

edited

Loading