WIP: Reimplement dump and implment restore command #2917

lunny · 2017-11-15T09:22:31Z

This PR would like to replace #1637. It reimplemented dump command and old dump file will not be used anymore. And also added a restore command that could restore dumped zip file to some machine. You have to use the same version Gitea with dump when restore.

The dump databases will be stored as <table_name>.yml in one directory. This format is expected to be compatible with Gitea's test fixtures. So that it's easy to export test data from a real system.

Currently it's blocked because yaml package didn't support Gonic name mapping, so I have forked yml package to https://github.com/go-gitea/yaml. Before that package supports that feature, this PR is not really work.

codecov-io · 2017-11-15T09:37:09Z

Codecov Report

Merging #2917 into master will decrease coverage by 19.29%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master    #2917      +/-   ##
==========================================
- Coverage    39.4%   20.11%   -19.3%     
==========================================
  Files         393      145     -248     
  Lines       53264    29252   -24012     
==========================================
- Hits        20990     5884   -15106     
+ Misses      29287    22474    -6813     
+ Partials     2987      894    -2093

Impacted Files	Coverage Δ
models/models.go	`21.15% <0%> (-35.26%)`	⬇️
routers/api/v1/misc/version.go	`0% <0%> (-100%)`	⬇️
routers/api/v1/misc/swagger.go	`0% <0%> (-100%)`	⬇️
modules/util/time_stamp.go	`0% <0%> (-90.91%)`	⬇️
modules/sync/exclusive_pool.go	`0% <0%> (-89.29%)`	⬇️
models/graph.go	`0% <0%> (-81.04%)`	⬇️
modules/sync/unique_queue.go	`0% <0%> (-78.58%)`	⬇️
models/status.go	`1.73% <0%> (-68.96%)`	⬇️
models/repo_activity.go	`0% <0%> (-66.28%)`	⬇️
routers/repo/activity.go	`0% <0%> (-64.71%)`	⬇️
... and 369 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d771769...c85526d. Read the comment docs.

bkcsoft

I would keep the gitea dump command for a while and create a gitea backup command instead.

bkcsoft · 2017-12-03T20:27:55Z

cmd/dump.go

@@ -1,5 +1,5 @@
 // Copyright 2014 The Gogs Authors. All rights reserved.
-// Copyright 2016 The Gitea Authors. All rights reserved.
+// Copyright 2017 The Gitea Authors. All rights reserved.


Don't change the copyright year ;)

But in fact all contents changed.

No it has not 🤔

elvarb · 2018-02-28T10:43:01Z

Any ETA on this?

techknowlogick · 2018-04-29T01:31:58Z

models/models.go

+
+	var bufferSize = 100
+	v := reflect.MakeSlice(table.Type, 0, bufferSize)
+	// BLOCK: need yaml support gonic name mapper


There have been 61 commits to go-yaml/yaml since it was forked into the go-gitea org. Is this still blocked?

I will check that.

techknowlogick · 2018-05-08T20:35:15Z

cmd/restore.go

@@ -0,0 +1,153 @@
+// Copyright 2017 The Gitea Authors. All rights reserved.


As this file is entirely new, I know you wrote it in 2017, however it is likely to get merged in 2018 should this date be updated to 2018?

The copyright year should be the time first created I think.

techknowlogick · 2018-05-08T20:47:07Z

@lunny It looks like you've made some recent commits to this. Can the blocked label be removed?

lunny · 2018-05-09T01:48:10Z

@techknowlogick It still cannot work fine when restore.

sphrak · 2018-06-19T05:02:45Z

Is there a possibility this will be merged into 1.5.0?

lunny · 2018-06-19T05:08:04Z

@sphrak it's impossible since v1.5 has been freezed. I would like this could be in v1.6.

sphrak · 2018-06-19T05:22:38Z

@lunny I understand, thanks for communicating that :-)

jlelse · 2018-09-12T11:38:21Z

Any updates on this? I would like to migrate from SQLite to Postgres and therefor need this feature.

sphrak · 2018-10-23T16:38:59Z

Any chance this will be part of 1.7.0?

cryptix · 2018-12-07T11:03:01Z

@lunny I'd love to try and test this but can share a little light on the go-yaml/yaml fork situation?

lunny · 2018-12-08T03:10:52Z

@cryptix you are welcome we need yaml support Gonic name mapping, maybe you could send a PR to yaml at first.

dodedodo · 2019-07-11T10:13:16Z

Any updates on this? Like jlelse I'm trying to do a database migration, and this should make my life so much easier.

dev-dsp · 2020-07-21T07:44:17Z

Any updates on this? I would like to migrate from SQLite to Postgres and therefore need this feature.

dev-dsp · 2020-07-21T16:09:37Z

For those who are curious about how to migrate data from sqlite to postgresql, see this link: #5651 (comment)

singuliere · 2022-01-02T21:16:29Z

It looks like this PR is still relevant because the restore process still is manual.

yardenshoham · 2023-05-02T09:16:58Z

This is in progress for a while now, I'm closing it. Please reopen when it's ready for review.

lunny added type/feature Completely new functionality. Can only be merged if feature freeze is not active. status/blocked This PR cannot be merged yet, i.e. because it depends on another unmerged PR labels Nov 15, 2017

lunny added this to the 1.x.x milestone Nov 15, 2017

tboerger added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Nov 15, 2017

bkcsoft suggested changes Dec 3, 2017

View reviewed changes

lafriks mentioned this pull request Dec 6, 2017

Backup/Restore commands #1637

Closed

5 tasks

bkcsoft approved these changes Apr 29, 2018

View reviewed changes

techknowlogick reviewed Apr 29, 2018

View reviewed changes

bkcsoft added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Apr 29, 2018

lunny force-pushed the lunny/dump_restore branch 2 times, most recently from 6b392aa to 9a9a0bc Compare May 8, 2018 08:59

techknowlogick reviewed May 8, 2018

View reviewed changes

sphrak mentioned this pull request Jan 6, 2019

Possible bug when migrating from sqlite to postgres [with workaround] #5651

Closed

7 tasks

lunny added 5 commits March 29, 2019 10:54

Reimplement dump and implment restore command

5014204

fix name convert

4fdb5fa

fix lint

7302de8

fix typo

dcb0811

fix restore

c85526d

lunny force-pushed the lunny/dump_restore branch from 9434bc7 to c85526d Compare March 29, 2019 02:54

GitMensch mentioned this pull request Jul 8, 2020

Migrate backup/restore from Gogs #1292

Closed

lunny changed the title ~~Reimplement dump and implment restore command~~ WIP: Reimplement dump and implment restore command Dec 23, 2020

lunny removed this from the 1.x.x milestone Dec 30, 2022

yardenshoham closed this May 2, 2023

lunny deleted the lunny/dump_restore branch May 2, 2023 13:28

go-gitea locked as resolved and limited conversation to collaborators Aug 1, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: Reimplement dump and implment restore command #2917

WIP: Reimplement dump and implment restore command #2917

lunny commented Nov 15, 2017

codecov-io commented Nov 15, 2017 •

edited

Loading

bkcsoft left a comment

bkcsoft Dec 3, 2017

lunny Dec 4, 2017

bkcsoft Jan 25, 2018

elvarb commented Feb 28, 2018

techknowlogick Apr 29, 2018

lunny Apr 29, 2018

techknowlogick May 8, 2018

lunny May 9, 2018

techknowlogick commented May 8, 2018

lunny commented May 9, 2018

sphrak commented Jun 19, 2018

lunny commented Jun 19, 2018

sphrak commented Jun 19, 2018

jlelse commented Sep 12, 2018

sphrak commented Oct 23, 2018

cryptix commented Dec 7, 2018

lunny commented Dec 8, 2018

dodedodo commented Jul 11, 2019

dev-dsp commented Jul 21, 2020

dev-dsp commented Jul 21, 2020

singuliere commented Jan 2, 2022

yardenshoham commented May 2, 2023

		@@ -0,0 +1,153 @@
		// Copyright 2017 The Gitea Authors. All rights reserved.

WIP: Reimplement dump and implment restore command #2917

WIP: Reimplement dump and implment restore command #2917

Conversation

lunny commented Nov 15, 2017

codecov-io commented Nov 15, 2017 • edited Loading

Codecov Report

bkcsoft left a comment

Choose a reason for hiding this comment

bkcsoft Dec 3, 2017

Choose a reason for hiding this comment

lunny Dec 4, 2017

Choose a reason for hiding this comment

bkcsoft Jan 25, 2018

Choose a reason for hiding this comment

elvarb commented Feb 28, 2018

techknowlogick Apr 29, 2018

Choose a reason for hiding this comment

lunny Apr 29, 2018

Choose a reason for hiding this comment

techknowlogick May 8, 2018

Choose a reason for hiding this comment

lunny May 9, 2018

Choose a reason for hiding this comment

techknowlogick commented May 8, 2018

lunny commented May 9, 2018

sphrak commented Jun 19, 2018

lunny commented Jun 19, 2018

sphrak commented Jun 19, 2018

jlelse commented Sep 12, 2018

sphrak commented Oct 23, 2018

cryptix commented Dec 7, 2018

lunny commented Dec 8, 2018

dodedodo commented Jul 11, 2019

dev-dsp commented Jul 21, 2020

dev-dsp commented Jul 21, 2020

singuliere commented Jan 2, 2022

yardenshoham commented May 2, 2023

codecov-io commented Nov 15, 2017 •

edited

Loading