OAuth2 method deprecated

[oliof]

Describe the bug
New setups of gphotos-sync fail with an error during the response token generation phase

To Reproduce

1. Follow the instructions in Oauth2 for gphotos-sync.

2. after saving client_secret.json, run any gphotos-sync command that triggers the out of band authentication flow

3. This fails with an error in the browser stating the authentication failed because the app doesn't satisfy security requirements

The reason for this is that Google deprecated the OAuth flow used on February 28th, 2022

[gilesknap]

@oliof thanks for the report.

I'm afraid that I'm now considering deprecating this project.

First, it cannot create a true backup due to a number of issue with the Google API that have been left unresolved for years now (see #119)

Second, it is becoming increasingly difficult to generate Oauth tokens.

There may be a way to work around this latest restriction on tokens but I'm not sure I have the motivation to look into it. I don't believe Google wants this project to exist.

[oliof]

@gilesknap I see your point, I believe your assessment to be correct.

[adewolfe]

I just ran into this problem - but I found a manual work around – here’s what I did:

• in my client_secrets.json file, removed the redirect_uri “urn:ietf:wg:oauth:2.0:oob”
• after confirming access and pressing the Allow button, my browser redirects to localhost and displays a ‘This site cannot be reached” page
• in the URL of the page that failed to load, there are three query params: state, code, and scope. Copy the code value – it is the token
• paste the value into the SSH session that is waiting for the token
• syncing should begin

[gilesknap]

@adewolfe thanks for posting the workaround.

[oliof]

I'll give that a go.

[teookie]

The workaround worked for me.

[synman]

I'm afraid that I'm now considering deprecating this project.

Say it isn't so. There is nothing else out there even remotely close to what gphotosync does.

[gilesknap]

Update. Not deprecating yet! If I can get @marcocaberletti's fix to work then that is enough incentive to put a bit of effort into refreshing this project.

[tjsd]

Will that be pushed as a new docker image as well? I see last update to the docker was 2 years ago (I'm running this in docker).

[gilesknap]

@tjsd yes I will do a docker image. I'm making progress on a refresh of the project. Updating the CI to github actions, doing some proper docs and integrating docker image generation into the CI.

I'm personally switching to using gphotos-sync in kubernetes so will be a customer of the docker image.

However, this is going to take a few weeks so watch this space.

[christophershad]

Hi, I'm using pip install to get gphotos-sync, and I've just had a message from Google about this. When will a new version be published that I can use to avoid this issue, or is there a file I can edit or replace to fix it? I'm not really a developer but I understand a lot of coding and scripting. This software is my lifeline to having a safe copy of 100K+ photos, and I would seriously like pay you guys to keep it running, there's nothing else like it that works.

[gilesknap]

@christophershad I'm actively working on a new release which will be a couple of weeks, probably. In the mean time if you wish to try out the new authentication flow then the master branch has this change and all tests are passing.

[ericjgreen]

I just wanted to say, THANK YOU! This a unique tool without any real substitutes. I realize you put your time into this and just to let you know, there are many of us our there that both use it and are very appreciative of it (and your time). THANK YOU, @gilesknap (and others).

[christophershad]

@christophershad I'm actively working on a new release which will be a couple of weeks, probably. In the mean time if you wish to try out the new authentication flow then the master branch has this change and all tests are passing.

Thank you! I already had the .json files, do I need to edit those as well?

[gilesknap]

You'll need to go through creating the app token JSON file.I have not documented the steps yet but I believe they are not much different to the current documented process.

[synman]

I just ran into this problem - but I found a manual work around – here’s what I did:

• in my client_secrets.json file, removed the redirect_uri “urn:ietf:wg:oauth:2.0:oob”
• after confirming access and pressing the Allow button, my browser redirects to localhost and displays a ‘This site cannot be reached” page
• in the URL of the page that failed to load, there are three query params: state, code, and scope. Copy the code value – it is the token
• paste the value into the SSH session that is waiting for the token
• syncing should begin

I was asked to setup another sync account for someone in my family today and this work-around appeared to work for me as well.

Oddly, there was nothing like noted above in the redirect_uris node. I went ahead and proceeded, got the 404 result, and did as instructed and used the code query string item as my key.

[gilesknap]

@synman A fixed version is nearing release. Will include documentation of a full walkthrough for generating the application oauth token.

[gilesknap]

And new client_secret.json files will have a localhost redirect_uri.

[synman]

Sounds good. In the meantime, it wasn't all that difficult to work around things.

Pulling down a 300k photo account now.

[gilesknap]

OK I have made a release with the OAuth fix in it https://github.com/gilesknap/gphotos-sync/releases/tag/3.0.0.

There is documentation for creating the new client id that you will need to do here: https://gilesknap.github.io/gphotos-sync/main/tutorials/oauth2.html

@tjsd The container image is also hosted on Github Here ghcr.io/gilesknap/gphotos-sync.

Please reopen this if you still have issues (or maybe create a new ticker)

[ericjgreen]

Thank.you for your effort on this.

[aaccioly]

Hi Folks, just posting some hints here if someone stumble in the same problems that I did while upgrading to the new docker version.

1. If you are deleting your old OAuth credentials and creating new ones don't forge to delete .gphotos.token at the root of the storage folder or you will get a message about invalid credentials.

2. This is properly document at https://gilesknap.github.io/gphotos-sync/main/tutorials/installation.html#execute-in-a-container, but if you are using docker you will need --net=host and -it for the auth URL to show, otherwise the app will be stuck in the following message:

   05-15 22:32:30 INFO version: 3.0.0, database schema version 5.7

   After your new .gphotos.token file is generated the flags above are no longer necessary.

3. Speaking of it, version 3.0.0 is not in Docker Hub (will it be published there as well?), you'll need the full GitHub Container Registry URL to run the latest version:

   docker run --rm -v $CONFIG:/config -v $STORAGE:/storage --net=host -it ghcr.io/gilesknap/gphotos-sync /storage
   

4. This may be pretty specific, but port 8080 is already in use on my system and It was not practical to stop the software that is using it. I've tried to remove the --net=host flag and use something like -p 98080:8080, But port 8080 seems to be hard coded in the auth logic and simply replaying the request meant for port 8080 on a different port didn't work for me. Luckily I figured out that I could generate .gphotos.token using another machine and simply copy the file over.

I hope that this is useful to other people.

[gilesknap]

@aaccioly Thanks for this, I'm going to open a new issue with your points for the benefit of others making the upgrade.

1st regarding Dockerhub, I wasn't planning on doing this but it's something that could be added into the CI so if people feel strongly about it then I'll add it.

2nd regarding your issue with the port. This is not nice and its a common port. I did have a plan to provide a command line argument to specify the redirect URL so that you could even do it on a different machine as long as the container is routable from your browser. If I did this then we could also configure the port number.