[Feature Request] Option to output to folder path instead of immich

[Dleewee]

Summary: I would like an option to scan a Google Takeout folder, fix and export images with metadata to a given fold/path, retaining album folders but skipping the upload into immich.

Benefit: This would allow users to take the output of immich-go and add to immich as an external library.

Why? I have over 100,000 images in my Google Takeout, and feel there is some risk to run this large of dataset through immich-go and import directly to immich. Exporting to a folder would simplify remediation if any files are found to be problematic or lacking metadata such as date.

Example: If, for example, my dataset contains 10,000 files with bad or missing date, I can simply disconnect the external library, solve any metadata issues, and then re-attach. On the other hand, if I import into immich directly, I am not sure how it would be possible to remediate such a large set (10k) of images.

[kelter-antunes]

I think that would be a completely different tool, but let see what simulot has to say. But there is already a tool that seems to be what you're looking for:
https://github.com/TheLastGimbus/GooglePhotosTakeoutHelper

[paulcollinsiii]

I'll add a couple use cases to this, as I would really like this as well

• The external library is slightly "safer" in immich as it's effectively treated as a read-only photo source
• Photo portability, immich is amazing but having the export pushed to a external library makes it easier to import all those photos into some other management tool.

Code wise immich-go already appears to be parsing a lot of the metadata from the json files, feeding that through this immich client parser which ultimately when uploaded is being sent in the asset.go.

Given a lot of the pipeline is here, writing the file to some destination and embedding the json data in the image file directly would be a pretty massive win. Heck just cleaning up the json file-naming chllenges of a Takeout export by embedding the entire json into the EXIF would make it easier for future parsers. That said, refactoring the pipeline for this without looking like a complete kludge is likely non-trivial 😅

[simulot]

The feature is there, but still in release candidate.
Check the https://github.com/simulot/immich-go/releases/tag/v0.23.0-RC4
and the related readme file

I have completely restructured the code to enable almost all combinations like:

• from google photos takeout to folder
• from immich to folder
• from folder to immich
• from immich to immich

[paulcollinsiii]

Oh nice! Next time I need to check the branches too😂
Looking forward to the release of 0.23 then and thank you 🍻

[paulcollinsiii]

Ran my takeout files through RC4 and then rebuilt on next as of this morning and ran into the same panic on both.

panic: time: missing Location in call to Date

goroutine 34 [running]:
time.Date(0xc00002c209?, 0x2?, 0x17?, 0xc0048a3a80?, 0xc004e67860?, 0x472dc9?, 0x10?, 0x997a40?)
        /usr/lib/go-1.23/src/time/time.go:1518 +0x4c5
github.com/simulot/immich-go/internal/filenames.TakeTimeFromName({0xc00002c1f8?, 0x17?}, 0x0)
        /home/paulcollins/coding/immich-go/internal/filenames/namesdate.go:41 +0x10f
github.com/simulot/immich-go/internal/filenames.TakeTimeFromPath({0xc00002c1f8, 0x17}, 0x0)
        /home/paulcollins/coding/immich-go/internal/filenames/namesdate.go:21 +0x78
github.com/simulot/immich-go/internal/filenames.InfoCollector.GetInfo({0x0?, 0xc0000aca80?}, {0xc00002c1f8, 0x17})
        /home/paulcollins/coding/immich-go/internal/filenames/infoCollector.go:40 +0x291
github.com/simulot/immich-go/adapters/googlePhotos.(*Takeout).makeAsset(0xc0001121e0, {0xc00030e780?, 0xc0020c4363?}, {0xc000bd0090?, 0x782f50ded1b8?}, 0xc0002a4050, 0xc0000a8500)
        /home/paulcollins/coding/immich-go/adapters/googlePhotos/googlephotos.go:554 +0x385
github.com/simulot/immich-go/adapters/googlePhotos.(*Takeout).solvePuzzle(0xc0001121e0, {0xb51a20, 0xc0002a4000})
        /home/paulcollins/coding/immich-go/adapters/googlePhotos/googlephotos.go:350 +0x73f
github.com/simulot/immich-go/adapters/googlePhotos.(*Takeout).Browse.func1()
        /home/paulcollins/coding/immich-go/adapters/googlePhotos/googlephotos.go:135 +0xe9
created by github.com/simulot/immich-go/adapters/googlePhotos.(*Takeout).Browse in goroutine 1
        /home/paulcollins/coding/immich-go/adapters/googlePhotos/googlephotos.go:125 +0xc9
root@blade-runner:/home/paulcollins/coding/immich-go#2024-12-29 16:21:35 WRN unsupported file file=takeout-20241221T055026Z-003:Takeout/archive_browser.html reason=unsupported file type
panic: time: missing Location in call to Date

goroutine 34 [running]:
time.Date(0xc00002c209?, 0x2?, 0x17?, 0xc0048a3a80?, 0xc004e67860?, 0x472dc9?, 0x10?, 0x997a40?)
        /usr/lib/go-1.23/src/time/time.go:1518 +0x4c5
github.com/simulot/immich-go/internal/filenames.TakeTimeFromName({0xc00002c1f8?, 0x17?}, 0x0)
        /home/paulcollins/coding/immich-go/internal/filenames/namesdate.go:41 +0x10f
github.com/simulot/immich-go/internal/filenames.TakeTimeFromPath({0xc00002c1f8, 0x17}, 0x0)
        /home/paulcollins/coding/immich-go/internal/filenames/namesdate.go:21 +0x78
github.com/simulot/immich-go/internal/filenames.InfoCollector.GetInfo({0x0?, 0xc0000aca80?}, {0xc00002c1f8, 0x17})
        /home/paulcollins/coding/immich-go/internal/filenames/infoCollector.go:40 +0x291
github.com/simulot/immich-go/adapters/googlePhotos.(*Takeout).makeAsset(0xc0001121e0, {0xc00030e780?, 0xc0020c4363?}, {0xc000bd0090?, 0x782f50ded1b8?}, 0xc0002a4050, 0xc0000a8500)
        /home/paulcollins/coding/immich-go/adapters/googlePhotos/googlephotos.go:554 +0x385
github.com/simulot/immich-go/adapters/googlePhotos.(*Takeout).solvePuzzle(0xc0001121e0, {0xb51a20, 0xc0002a4000})
        /home/paulcollins/coding/immich-go/adapters/googlePhotos/googlephotos.go:350 +0x73f
github.com/simulot/immich-go/adapters/googlePhotos.(*Takeout).Browse.func1()
        /home/paulcollins/coding/immich-go/adapters/googlePhotos/googlephotos.go:135 +0xe9
created by github.com/simulot/immich-go/adapters/googlePhotos.(*Takeout).Browse in goroutine 1
        /home/paulcollins/coding/immich-go/adapters/googlePhotos/googlephotos.go:125 +0xc9
root@blade-runner:/home/paulcollins/coding/immich-go#```

Let me know if there's anything additional info you need.

[simulot]

I just used the RC4 on a linux system. I didn't got this one.

Have you tried to set the TZ on the command line?
Ex:
--time-zone=America/Argentina/Buenos_Aires

The readme for the latest RC: https://github.com/simulot/immich-go/tree/next

[tatablack]

I've tried the RC5, from Google Takeout zips to a folder, and I got a panic as well (not the same).

I've tried adding the --time-zone parameter, but I get an unknown flag error (to be fair, the docs for the archive command do not mention that parameter).

Here's my command:

immich-go archive from-google-photos --write-to-folder=./output ./takeout/takeout-*.zip

And here's the panic:

 panic: time: missing Location in call to Time.In

goroutine 33 [running]:
time.Time.In(...)
        time/time.go:1195
github.com/simulot/immich-go/adapters/googlePhotos.googTimeObject.Time({{0x1400018f350?, 0x1010c0580?}})
        github.com/simulot/immich-go/adapters/googlePhotos/json.go:184 +0xd4
github.com/simulot/immich-go/adapters/googlePhotos.GoogleMetaData.AsMetadata({{0x1400018f340, 0xe}, {0x140001a00e0, 0x1d}, {0x0, 0x0}, 0x0, 0x1400019e440, 0x140001b6168, 0x140001b6150, ...}, ...)
        github.com/simulot/immich-go/adapters/googlePhotos/json.go:91 +0x174
github.com/simulot/immich-go/adapters/googlePhotos.(*Takeout).passOneFsWalk.func1({0x1400103b090, 0x41}, {0x1084278e0, 0x140000c1600}, {0x0?, 0x0?})
        github.com/simulot/immich-go/adapters/googlePhotos/googlephotos.go:197 +0x5f8
io/fs.walkDir({0x1011115a0, 0x14000198018}, {0x1400103b090, 0x41}, {0x1084278e0, 0x140000c1600}, 0x1400003ff08)
        io/fs/walk.go:73 +0x5c
io/fs.walkDir({0x1011115a0, 0x14000198018}, {0x14000ae6060, 0x2d}, {0x101116368, 0x14001300060}, 0x1400003ff08)
        io/fs/walk.go:95 +0x24c
io/fs.walkDir({0x1011115a0, 0x14000198018}, {0x140001b6018, 0x15}, {0x101116368, 0x14001300020}, 0x1400003ff08)
        io/fs/walk.go:95 +0x24c
io/fs.walkDir({0x1011115a0, 0x14000198018}, {0x1400018e379, 0x7}, {0x101116368, 0x14001300000}, 0x1400003ff08)
        io/fs/walk.go:95 +0x24c
io/fs.walkDir({0x1011115a0, 0x14000198018}, {0x100ff4948, 0x1}, {0x101115fe8, 0x1400019e030}, 0x1400003ff08)
        io/fs/walk.go:95 +0x24c
io/fs.WalkDir({0x1011115a0, 0x14000198018}, {0x100ff4948, 0x1}, 0x140000d5f08)
        io/fs/walk.go:122 +0x88
github.com/simulot/immich-go/adapters/googlePhotos.(*Takeout).passOneFsWalk(0x0?, {0x101115e60?, 0x140001a4000?}, {0x1011115a0?, 0x14000198018?})
        github.com/simulot/immich-go/adapters/googlePhotos/googlephotos.go:147 +0x64
github.com/simulot/immich-go/adapters/googlePhotos.(*Takeout).Browse.func1()
        github.com/simulot/immich-go/adapters/googlePhotos/googlephotos.go:129 +0xb8
created by github.com/simulot/immich-go/adapters/googlePhotos.(*Takeout).Browse in goroutine 1
        github.com/simulot/immich-go/adapters/googlePhotos/googlephotos.go:125 +0xc8

[simulot]

Thanks for the details.
I have fixed the issue. It will be available soon

[tatablack]

Hello 👋🏻
I've seen the new release and went on to test it. Unfortunately, I still have a time-related panic, with the same image (the panic occurs soon after a certain "scanned image file" line in the log).

panic: time: missing Location in call to Time.In

goroutine 49 [running]:
time.Time.In(...)
        time/time.go:1195
github.com/simulot/immich-go/adapters/googlePhotos.googTimeObject.Time({{0x1400011b350?, 0x1051d4780?}})
        github.com/simulot/immich-go/adapters/googlePhotos/json.go:184 +0xd4
github.com/simulot/immich-go/adapters/googlePhotos.GoogleMetaData.AsMetadata({{0x1400011b340, 0xe}, {0x140001240e0, 0x1d}, {0x0, 0x0}, 0x0, 0x14000122440, 0x14000140168, 0x14000140150, ...}, ...)
        github.com/simulot/immich-go/adapters/googlePhotos/json.go:91 +0x174
github.com/simulot/immich-go/adapters/googlePhotos.(*Takeout).passOneFsWalk.func1({0x1400105f090, 0x41}, {0x10c44c160, 0x140000c3600}, {0x0?, 0x0?})
        github.com/simulot/immich-go/adapters/googlePhotos/googlephotos.go:197 +0x5f8
io/fs.walkDir({0x105225940, 0x14000118018}, {0x1400105f090, 0x41}, {0x10c44c160, 0x140000c3600}, 0x1400003ff08)
        io/fs/walk.go:73 +0x5c
io/fs.walkDir({0x105225940, 0x14000118018}, {0x14000aee060, 0x2d}, {0x10522a6e8, 0x140010f8060}, 0x1400003ff08)
        io/fs/walk.go:95 +0x24c
io/fs.walkDir({0x105225940, 0x14000118018}, {0x14000140018, 0x15}, {0x10522a6e8, 0x140010f8020}, 0x1400003ff08)
        io/fs/walk.go:95 +0x24c
io/fs.walkDir({0x105225940, 0x14000118018}, {0x1400011a379, 0x7}, {0x10522a6e8, 0x140010f8000}, 0x1400003ff08)
        io/fs/walk.go:95 +0x24c
io/fs.walkDir({0x105225940, 0x14000118018}, {0x10510a970, 0x1}, {0x10522a368, 0x14000122030}, 0x1400003ff08)
        io/fs/walk.go:95 +0x24c
io/fs.WalkDir({0x105225940, 0x14000118018}, {0x10510a970, 0x1}, 0x140000d5f08)
        io/fs/walk.go:122 +0x88
github.com/simulot/immich-go/adapters/googlePhotos.(*Takeout).passOneFsWalk(0x0?, {0x10522a1e0?, 0x1400012a000?}, {0x105225940?, 0x14000118018?})
        github.com/simulot/immich-go/adapters/googlePhotos/googlephotos.go:147 +0x64
github.com/simulot/immich-go/adapters/googlePhotos.(*Takeout).Browse.func1()
        github.com/simulot/immich-go/adapters/googlePhotos/googlephotos.go:129 +0xb8
created by github.com/simulot/immich-go/adapters/googlePhotos.(*Takeout).Browse in goroutine 1
        github.com/simulot/immich-go/adapters/googlePhotos/googlephotos.go:125 +0xc8

Let me know if you need more info to debug. I can also provide the image file which I think triggers it.

[simulot]

I just tested it on my linux box.
What is the command line and your platform?

[tatablack]

I'm on a Mac (Sequoia 15.2 arm64). Command line:

immich-go archive from-google-photos --write-to-folder=./output ./takeout/takeout-*.zip

The logfile shows I'm on RC6 (immich-go·version:dev,··commit:550dac7fe0c192444a49ab7cc2fa5919378a7375).

[simulot]

Well, different on the mac... Unfortunately, I can't test on a mac.
So, I'm checking the code, to see what could be wrong.
Can you test an interim version?

[simulot]

Here it is
immich-go.gz

[tatablack]

Thanks for the build 🙇🏻
This one worked fine, from what I can see. 👍

A few pieces of feedback (which you may or may not want to consider for this iteration):

• I was expecting an overview at the end of the log, similar to what the upload command does—at least the "input analysis" part. I think it would be a useful feature (besides making the output more consistent).

• I noticed that the disk space used after the extraction was roughly double the size of the pictures. I went looking around, and I realised that $TMPDIR contained the files extracted from the takeout, so it wasn't cleaned up after running immich-go. I think MacOS will clean that folder after three days, but seeing that immich-go often deals with a lot of files, it would be nicer if it removed those files automatically (I say this without looking at the source code, so I don't know how much control you have on this part of the process).

• The README says, regarding Google Takeouts, that "Restarting an interrupted import won't cause any problems and will resume where it left off". This led me (wrongly) to believe that running the command again would skip any files already extracted, but that turned out not to be the case (files were recreated instead, with a suffix appended). This may be worth a change in the docs, or an interactive warning during the program's execution (something like "The file xxx exists already at PATH. What do you want to do? [Overwrite/Duplicate/Skip]).

Anyway, THANKS for implementing the archive command! I use only external libraries in Immich, so this was essential for me to abandon Google Photos 👏🏻

[simulot]

Thanks, those are valid remarks!

1. easy, meaningful...

2. this is a bug... there is always one more bug...

3. I don't know what is the good procedure
   The problem comes from the possibility to have different images with the same name. We can thank Apple for this feature. Google Photos store them with (1), (2)....
   As an easy solution, I did the same for the archive command. But this suppose the write folder to be empty.

How do you intend to use this command? Would you accumulate files in the folder, or you will rebuild the folder?

What if the files were named with after their HASH?
The original name is anyway stored in the JSON. The command archive checks if a file with the hash doesn't exists before saving the file. Otherwise the file is skipped.

[tatablack]

Happy I can contribute with testing and feedback, at least 😅

1. Good to hear!
2. 😆 Yes, there is.
3. I'm not sure either. To be honest, it may be enough to describe in the README how the program behaves right now, and then see if anyone suggests valid use cases for a different behaviour. I don't plan to run the archive command more than once (once I'm out of Google Photos, I'm gonna stay out), and in any case I would only run it with an empty destination folder. In this case I ran it twice mainly to see what would happen, fully expecting that I would have to wipe and recreate the folder if things were messed up 🤷🏻

[tatablack]

Sorry, one more thing... I've noticed that my output contains trashed files (i.e. files prefixed with .trashed-), but the README says that the default for Google Takeouts is NOT to include trashed files (and I definitely did not use the --include-trashed parameter). Am I misunderstanding something?

[simulot]

I'm using the trashed field from the JSON file, not the name.

1 and 2 are fixed and merged to the next branch.
I have updated the archive document to describe the current behavior.

[pyro2927]

Chiming in here incase it helps, I'm also dealing with issues on Sequoia 15.2.

@simulot I downloaded the interim version you posted yesterday and ran it, getting this error message:

unknown time zone usr/share/zoneinfo.default/America/Chicago

It looks like whatever is pulling the timezone on macOS is returning a full path to the data. Passing in --timezone America/Chicago cleared up the issues for me (got past the panic seen in #595)

[simulot]

I can't test on macOS... Sorry
Is it just a warning, and the program continues, or is a crash with stack dump?

[pyro2927]

It doesn't crash, it just prints that error message and then exits.