Failing to upload draft, uses GitHub dummy e-mail

[CxRes]

I was trying to upload a draft today, and it failed on me because the automation picks up my dummy GitHub email. Now, I have submitted a draft below using this tool before, and it worked for me then. And the correct email is still set as before on my latest draft. So I suspect that this is a regression, though I have no understanding of what really happened. The repo is https://github.com/CxRes/prep/ if you want to check!

I am attaching the operational portion of the CLI dump below:

warning: lib/default-branch.py: correcting the default branch locally:
warning: lib/default-branch.py:     git symbolic-ref refs/remotes/origin/HEAD refs/remotes/origin/main
make: '.targets.mk' is up to date.
warning: lib/default-branch.py: correcting the default branch locally:
warning: lib/default-branch.py:     git symbolic-ref refs/remotes/origin/HEAD refs/remotes/origin/main
git show "draft-gupta-httpbis-per-resource-events-01:draft-gupta-httpbis-per-resource-events.md" | sed -e 's/draft-gupta-httpbis-per-resource-events-latest/draft-gupta-httpbis-per-resource-events-01/g' >versioned/draft-gupta-httpbis-per-resource-events-01.md
cat versioned/draft-gupta-httpbis-per-resource-events-01.md  | kramdown-rfc --v3 | lib/trace.sh versioned/draft-gupta-httpbis-per-resource-events-01.xml -s venue python3 lib/add-note.py | lib/trace.sh versioned/draft-gupta-httpbis-per-resource-events-01.xml -s v2v3 xml2rfc -q --rfc-base-url https://www.rfc-editor.org/rfc/ --id-base-url https://datatracker.ietf.org/doc/html/ --cache=/github/workspace/.refcache --v2v3 /dev/stdin -o /dev/stdout >versioned/draft-gupta-httpbis-per-resource-events-01.xml
rm versioned/draft-gupta-httpbis-per-resource-events-01.md
set -ex; tag="draft-gupta-httpbis-per-resource-events-01"; \
email=""; \
[ -z "$email" ] && email="$(git tag --list --format '%(taggeremail)' "$tag" | sed -e 's/^<//;s/>$//')"; \
[ -z "$email" ] && email=$(xmllint --xpath '/rfc/front/author[1]/address/email/text()' versioned/draft-gupta-httpbis-per-resource-events-01.xml 2>/dev/null); \
[ -z "$email" ] && ! echo "Unable to find email to use for submission." 1>&2; \
replaces() { \
  [ "${1##*-}" = "00" ] || return; \
  file="$(git ls-files "${1%-[0-9][0-9]}.*")"; \
  for last in $(git log --follow --name-only --format=format: -- "${file%-[0-9][0-9]}" | sed -e '/^$/d' | grep -v draft-todo-yourname-protocol | cut -f 2 | uniq | tail +2); do \
    if [ -n "$(git tag -l "${last%.*}")" ]; then \
      echo -F; echo "replaces=${last%.*}"; break; \
    fi; \
  done; \
}; \
lib/trace.sh versioned/draft-gupta-httpbis-per-resource-events-01.xml -s upload-request curl -sS -D "versioned/.draft-gupta-httpbis-per-resource-events-01.upload" \
    -F "user=$email" -F "xml=@versioned/draft-gupta-httpbis-per-resource-events-01.xml" $(replaces "$tag") \
    "https://datatracker.ietf.org/api/submission" && echo && \
  (head -1 "versioned/.draft-gupta-httpbis-per-resource-events-01.upload" | grep -q '^HTTP/\S\S* [20](https://github.com/CxRes/prep/actions/runs/9848489275/job/27190751952#step:8:21)[01]\b' || lib/trace.sh versioned/draft-gupta-httpbis-per-resource-events-01.xml -s upload-result ! cat "versioned/.draft-gupta-httpbis-per-resource-events-01.upload" 1>&2)
+ tag=draft-gupta-httpbis-per-resource-events-01
+ email=
+ '[' -z '' ']'
++ git tag --list --format '%(taggeremail)' draft-gupta-httpbis-per-resource-events-01
++ sed -e 's/^<//;s/>$//'
+ [email protected]
+ '[' -z [email protected] ']'
+ '[' -z [email protected] ']'
++ replaces draft-gupta-httpbis-per-resource-events-01
++ '[' 01 = 00 ']'
++ return
+ lib/trace.sh versioned/draft-gupta-httpbis-per-resource-events-01.xml -s upload-request curl -sS -D versioned/.draft-gupta-httpbis-per-resource-events-01.upload -F [email protected] -F xml=@versioned/draft-gupta-httpbis-per-resource-events-01.xml https://datatracker.ietf.org/api/submission
+ echo
{"error": "No such user: [email protected]"}
+ head -1 versioned/.draft-gupta-httpbis-per-resource-events-01.upload
+ grep -q '^HTTP/\S\S* 20[01]\b'
+ lib/trace.sh versioned/draft-gupta-httpbis-per-resource-events-01.xml -s upload-result '!' cat versioned/.draft-gupta-httpbis-per-resource-events-01.upload

[martinthomson]

Ah, I'm guessing that I'm going to get a rash of these reports now. GitHub seems to have changed something. That is, I'm guessing that they are using annotated tags for releases now. This is a good thing, but the dummy email is a real pain to handle.

I'm going to guess that you used the GitHub UI to make a release. Is that right?

[CxRes]

I'm going to guess that you used the GitHub UI to make a release. Is that right?

No. I just pushed the tag from my local environment on the first occasion.

Only when I did not get an e-mail, did I go into the GitHub UI to check and re-ran the Action again!

EDIT: I did create an annotated tag for the release on my local main. Does that make a difference?

[martinthomson]

The key problem here is that you have not set your git user.email anywhere or you have something that is redacting it somehow (which could be in any number of places). The result is that the code that detects who is making the submission gets "[email protected]", which doesn't work.

(Your repository has a mix of tagged annotations and not, which is what made me think that something had changed.)

So I've made some changes that should help you. In future. Nothing to be done this time around, of course. The deadline passed.

[CxRes]

That's weird. My user.email is and has been from the start set to the GitHub dummy address on my local environment (Just rechecked). In fact, I started with the dummy address at the time I initialized the template and I had changed the e-mail to a real one on the commit before my -00 submission tag/submission. The -00 successful submission indeed was a lightweight tag, but that goes against the instructions. It somehow does not add up!

In any case, thanks for the fixes! Hopefully, I'll have better luck next time.

Sucks about the submission. FYI, I opened a ticket immediately with IETF support for a manual submission (that has some bug too which prevents the XML from being accepted, which both the CI here and online checker say are OK), but I don't know if they will be able to help.

[CxRes]

Sorry, to be a bother, but I ran the new code you wrote a few minutes ago on my local system, I got this error (scroll to the bottom):

VERSIONED="versioned" lib/build-targets.sh draft-gupta-httpbis-per-resource-events >>.targets.mk
set -ex; tag="draft-gupta-httpbis-per-resource-events-01"; \
email="$(lib/get-email.sh "$tag" "versioned/draft-gupta-httpbis-per-resource-events-01.xml")"; \
[ -z "$email" ] && exit 1; \
replaces() { \
  [ "${1##*-}" = "00" ] || return; \
  file="$(git ls-files "${1%-[0-9][0-9]}.*")"; \
  for last in $(git log --follow --name-only --format=format: -- "${file%-[0-9][0-9]}" | sed -e '/^$/d' | grep -v draft-todo-yourname-protocol | cut -f 2 | uniq | tail +2); do \
    if [ -n "$(git tag -l "${last%.*}")" ]; then \
      echo -F; echo "replaces=${last%.*}"; break; \
    fi; \
  done; \
}; \
curl -sS -D "versioned/.draft-gupta-httpbis-per-resource-events-01.upload" \
    -F "user=$email" -F "xml=@versioned/draft-gupta-httpbis-per-resource-events-01.xml" $(replaces "$tag") \
    "https://datatracker.ietf.org/api/submission" && echo && \
  (head -1 "versioned/.draft-gupta-httpbis-per-resource-events-01.upload" | grep -q '^HTTP/\S\S* 20[01]\b' || lib/trace.sh versioned/draft-gupta-httpbis-per-resource-events-01.xml -s upload-result ! cat "versioned/.draft-gupta-httpbis-per-resource-events-01.upload" 1>&2)
+ tag=draft-gupta-httpbis-per-resource-events-01
++ lib/get-email.sh draft-gupta-httpbis-per-resource-events-01 versioned/draft-gupta-httpbis-per-resource-events-01.xml
Unable to find email to use for submission.
Tried:
    $UPLOAD_EMAIL environment variable
    git committeremail
    git authoremail
    draft author
+ email=
make[1]: *** [lib/upload.mk:27: versioned/.draft-gupta-httpbis-per-resource-events-01.upload] Error 1
make: *** [lib/targets.mk:28: versioned/.draft-gupta-httpbis-per-resource-events-01.upload] Error 2

I understand the script failing on the first three, but in the fourth case "draft author", the email is there in the xml files. It does not leave the .upload artifact for me to examine. I am surprised because the path here:

if [ -n "$draft" ]; then
    emailok "draft author" "$(xmllint --xpath '/rfc/front/author[1]/address/email/text()' "$draft" 2>/dev/null)"
fi

matches up with the correct email in xml file.

[MikeBishop]

Yeah, I've been poking at this code as well. #443 makes it pick up the annotated tag for me, at least in local testing. If I force it to fall back by putting a lightweight tag, though, it finds [email protected] and declares success. (Presumably that will then fail for not having a datatracker account.)

If it had continued on to authoremail, it would have gotten it correct.

[fenner]

I got bitten by a related email address problem; in this case, I commit on github with [email protected] but that's not my Datatracker account. (Previous releases have worked ok, maybe because it used the draft author list before.). I see there's an UPLOAD_EMAIL override, but it's not quite clear how to use it. Can this override be documented?

[martinthomson]

You can set UPLOAD_EMAIL in your Makefile. You'll have to export it:

export UPLOAD_EMAIL := [email protected]

You can also trigger a build manually and set the email in the Web UI there.

[martinthomson]

See 638eeb4

[fenner]

You can set UPLOAD_EMAIL in your Makefile.

Thanks! Is there a suggested mechanism that survives make update-files?

[martinthomson]

Right now, that process is more destructive than it should be. That's something that I need to work on. I've opened #445 to track that. My goal is to preserve most of the Makefile, so that you don't have to worry about that stuff getting wiped.

[fenner]

My goal is to preserve most of the Makefile, so that you don't have to worry about that stuff getting wiped.

Great, thanks!

[bc-pi]

I've come here somewhat after discovering that UPLOAD_EMAIL in the Makefile will hopefully help with a situation where a not acceptable email address was being scrobbled oauth-wg/oauth-sd-jwt-vc#255

[martinthomson]

I'm going to call this one done. This is likely never really the case, but we'll continue to work on it.