Feature request: save body of GET result to file

[steinarb]

I need to do a GET with basic auth to get a token.

The body of the GET response is the token.

I would like to save that body to a file, and I haven't found a way to do this.

I had hoped that "> ~/.authToken" in the body would do the trick, similar to how a body can be fetched from a file.

But that didn't work.

It's also possible that "->" with some lisp code, could do the trick, but I looked at restclient-jq-json-var-function and I wasn't able to figure out where/how the body enters the function.

[steinarb]

I can possibly borrow code from some of the examples in #149 ?

But it would be nice to have a simpler way do do things.

And it would be nice to have the handling located in the query, instead of having to add a hook, and test for the url in the hook.

[steinarb]

I thought something similar to this, in my .emacs, could do the trick:

;; restclient from git to get restclient-jq
(add-to-list 'load-path (expand-file-name "~/git/restclient.el"))
(when (locate-library "restclient")
  (require 'restclient)
  (require 'restclient-jq)
  (restclient-register-result-func "save-body" #'restclient-save-body "Save body to filename given as arg"))

(defun restclient-save-body (args args-offset)
  (let ((filename) args)
    (with-current-buffer restclient-same-buffer-response-name
      (write-region (point-min) (restclient-jq-result-end-point) filename))))

And use it like this:

# Fetch token
GET https://stinky-appsikk.utv.mycompany.com/api/authorize
Authorization: Basic bXl1c2VyOm15cGFzc3dvcmQ=
X-CSRF: 1
-> save-body "~/.stinkyAuthtoken"

But it fails because the HTTP response buffer can't be found (race condition?):

Debugger entered--Lisp error: (error "No buffer named *HTTP Response*")
  set-buffer("*HTTP Response*")
  (save-current-buffer (set-buffer restclient-same-buffer-response-name) (write-region (point-min) (restclient-jq-result-end-point) filename))
  (let ((filename) args) (save-current-buffer (set-buffer restclient-same-buffer-response-name) (write-region (point-min) (restclient-jq-result-end-point) filename)))
  restclient-save-body("\"~/.stinkyAuthtoken\"" 178)
  restclient-parse-hook("save-body" 178 "\"~/.stinkyAuthtoken\"")
  restclient-http-parse-current-and-do(restclient-http-do nil nil)
  restclient-http-send-current()
  funcall-interactively(restclient-http-send-current)
  call-interactively(restclient-http-send-current nil nil)
  command-execute(restclient-http-send-current)

Also, I can't borrow restclient-jq-result-end-point to find the end of the response, because a GET response looks like this:

sha256~czTeujYgDQ-7zuSZ4khsOBRb-QhTT13toRYNl02blSY
GET https://stinky-appsikk.utv.mycompany.com/api/authorize
HTTP/1.1 200 
content-type: text/plain;charset=UTF-8
content-length: 50
date: Tue, 22 Feb 2022 08:12:19 GMT
set-cookie: 7ab05648246fe8d68c317cabd6c4cbf9=3aa5804c54752903e7c4b7233d160dbf; path=/; HttpOnly; Secure; SameSite=None
cache-control: private
Request duration: 0.249098s

I would have liked to save everything from "sha256" and up to, and including, the final "Y" (i.e. not save the line shift).

[steinarb]

I've made this work, with a little help from the help-gnu-emacs mailing list.

The first enlightenment I had was that the evaluation result of restclient-jq-json-var-function was a lambda.

The second enlightenment, was that I learned about emacs lisp lexical scope.

I tried returning a lambda from the hook function, with the following content in ~/.emacs:

;; restclient from git to get restclient-jq
(add-to-list 'load-path (expand-file-name "~/git/restclient.el"))
(when (locate-library "restclient")
  (require 'restclient)
  (require 'restclient-jq)
  (restclient-register-result-func "save-body" #'restclient-save-body "Save body to filename given as arg"))

(defun restclient-save-body (args args-offset)
  (save-match-data
    (let ((filename (expand-file-name args)))
      (lambda ()
        (with-current-buffer restclient-same-buffer-response-name
          (write-region (point-min) (restclient-jq-result-end-point) filename))))))

But that didn't work, and I eventually figured out that this was because filename was undefined when the lambda was executed.

However I could see restclient-jq-json-var-function do the very thing I was trying to do: define a variable in a let clause, and use that variable in a lambda.

So I asked on the help-gnu-emacs mailing list, and was told that I needed to add a magical cookie to the file to get lexical scoping.

So I created a file called sb-restclient.el in a directory in the load-path, with the following content:

;;; -*- lexical-binding: t; -*-

(require 'restclient-jq)

(defun restclient-save-body (args args-offset)
  (save-match-data
    (let ((filename (expand-file-name args)))
      (lambda ()
        (with-current-buffer restclient-same-buffer-response-name
          (write-region (point-min) (restclient-jq-result-end-point) filename))))))

(provide 'sb-restclient)

and in my ~/.emacs put the following:

;; restclient from git to get restclient-jq
(add-to-list 'load-path (expand-file-name "~/git/restclient.el"))
(when (locate-library "restclient")
  (require 'restclient)
  (require 'restclient-jq)
  (require 'sb-restclient)
  (restclient-register-result-func "save-body" #'restclient-save-body "Save body to filename given as arg"))

A little to my surprise, restclient-jq-result-end-point worked on emacs 27 on Windows and on debian GNU/linux, because in both places, the headers of a GET response were prefixed with "//".

But on emacs 27 on mac I needed to write a different function to find the end of the body, because there the HTTP headers in the response were not prefixed with "//" (as shown in the previous comment).

[steinarb]

The sb-restclient.el file on mac looks like this

;;; -*- lexical-binding: t; -*-

(defun restclient-get-result-end-point ()
  (save-excursion
    (goto-char (point-max))
    (or (progn (re-search-backward "^GET.*" nil t)
               (previous-line)
	       (line-end-position))
	(point-max))))

(defun restclient-save-body (args args-offset)
  (save-match-data
    (let ((filename (expand-file-name args)))
      (lambda ()
        (with-current-buffer restclient-same-buffer-response-name
          (write-region (point-min) (restclient-get-result-end-point) filename))))))

(provide 'sb-restclient)

I haven't tested, but this will probably work for a GET on non-macs as well.

However, the version that uses restclient-jq-result-end-point, works on all methods, not just GET.

No idea why the GET response on mac is different from the POST response.

emacs-version on the mac reports: GNU Emacs 27.1 (build 1, x86_64-apple-darwin18.7.0, NS appkit-1671.60 Version 10.14.6 (Build 18G95)) of 2020-08-12

The restclient.el version is the current git master HEAD.