Access to Browser API webRequest #397
Comments
|
The thing is, A viable solution might be declarative syntax e.g. provide an array of rules that specify what to do. |
|
On the other hand, maybe Tampermonkey's sandbox can be applied to userscript code in the background page too... |
|
@tophf I understand it and certainly agree with you what is required declarative syntax for rules. If I knew exactly how Tampermonkey works, I would try to implement it. Agree that this option would give a strong advantage before similar extensions? |
|
Ok, I've created a preview that shows how GM_webRequest and @webRequest could possibly work. After downloading and drag and dropping it to the extension page you can install this script: // ==UserScript==
// @name GM_webRequest testing
// @namespace http://tampermonkey.net/
// @version 0.1
// @description try to take over the world!
// @author You
// @match *://*/*
// @include http://new_static.url/*
// @include *//redirected.to/*
// @grant GM_webRequest
// @webRequest [{"selector":"*cancel.me/*","action":"cancel"},{"selector":{"include":"*","exclude":"http://exclude.me/*"},"action":{"redirect":"http://new_static.url"}},{"selector":{"match":"*://match.me/*"},"action":{"redirect":{"from":"([^:]+)://match.me/(.*)","to":"$1://redirected.to/$2"}}}]
// ==/UserScript==
var currently_active_webrequest_rule = JSON.stringify(GM_info.script.webRequest); // == @webRequst header from above
GM_webRequest([
{ selector: '*cancel.me/*', action: 'cancel' },
{ selector: { include: '*', exclude: 'http://exclude.me/*' }, action: { redirect: 'http://new_static.url' } },
{ selector: { match: '*://match.me/*' }, action: { redirect: { from: '([^:]+)://match.me/(.*)', to: '$1://redirected.to/$2' } } }
], function(info, message, details) {
console.log(info, message, details);
});
/*
Notes:
Action:
The final redirect: URL needs to be included into the scripts @match or @include header
Selector:
If just a string is given, it is interpreted as include: property.
include:, exclude: and match: properties are supported and can have an array or a string value - the syntax equals @include, @match and @exclude
@webRequest may have a human readable format in the future, but for now it's the stringified first argument of GM_webRequest and allow request
manipulation even if the script didn't run yet. The only downside atm is that you need to re-register the rules (which overrides all previous ones)
in order to get manipulation events. GM_webRequest(currently_active_webrequest_rule, function...)
*/This all is not heavily tested so expect bugs here and there. :) |
|
@derjanb If I understood right, the callback function will only receive information about the WebRequest once it is done and is not able to manipulate it. I know it is not a good idea to let random function to run in privileged extension context, but there can be times when that would be handy. What is your opinion on this? Should I just create another extension to do this or we can have a special header for it? Also, as a side thought, shouldn't it be named |
|
@derjanb I try to play with https://ssl.gstatic.com/gb/images/v1_b3735dd8.png, but not one of the rules does not work for me. // ==UserScript==
// @name GM_webRequest testing
// @namespace http://tampermonkey.net/
// @version 0.1
// @description try to take over the world!
// @author You
// @match *://*/*
// @include *gstatic.com/*
// @include https://google.com/*
// @include *//google.com/*
// @grant GM_webRequest
// @webRequest [{"selector":"*cancel.me/*","action":"cancel"},{"selector":{"include":"*","exclude":"http://exclude.me/*"},"action":{"redirect":"http://new_static.url"}},{"selector":{"match":"*://match.me/*"},"action":{"redirect":{"from":"([^:]+)://match.me/(.*)","to":"$1://redirected.to/$2"}}}]
// ==/UserScript==
console.log("GM_webRequest start");
var currently_active_webrequest_rule = JSON.stringify(GM_info.script.webRequest); // == @webRequst header from above
GM_webRequest([
//{ selector: '*gstatic.com/*', action: 'cancel' },
{ selector: { include: '*gstatic.com/*', exclude: 'http://exclude.me/*' }, action: { redirect: 'https://google.com/' } },
//{ selector: { match: '*://*.gstatic.com*' }, action: { redirect: { from: '([^:]+)://(.+)gstatic.com/(.*)', to: '$1://google.com/$2' } } }
], function(info, message, details) {
console.log(info, message, details); // This also does not work
}); |
|
Ah sorry, I forgot to add this information. Since intercepting requests makes things slower Tampermonkey only handles the following request types at the moment: 'sub_frame', 'script', 'xmlhttprequest' and 'websocket'. All other are considered to be replaceable by a userscript, even after they were loaded, but that's discussable. |
|
Hm, can be done something like? /* here, any calculations of javasscript are not related to the page (setting vars, functions) */
GM_webRequest([
{ selector: '*gstatic.com/*', action: 'replace' },
], function(info, message, details) {
console.log(info, message, details);
details.url = details.url.replace('gstatic', 'gstatic-2');
return details;
});We change |
Atm only the URL via: { selector: { match: '*://match.me/*' }, action: { redirect: { from: '([^:]+)://match.me/(.*)', to: '$1://redirected.to/$2' } } }
Shouldn't make a difference.
This can't work because all communication is asynchronous. This means when the request is inspected by the background page, then there is no synchronous way to contact the userscript to make a decision. That's why all rules need to be defined when the requests happens. |
|
@derjanb
So will it work? |
The background page has all extension permissions. It's not possible to run foreign JavaScript code there without leaking these permissions. Sorry. That's why a defined rule set is needed to tell the background page what needs to be done with a request. |
|
@derjanb var rules = [
{ event: 'request', key: 'url', action: replace: { from: '([^:]+)://match.me/(.*)', to: '$1://redirected.to/$2'}, },
]; // Execution in order
GM_webRequest(rules, function(info, message, events) {
console.log(info, message, events); // ex. events.request.url or events.response.body
}); // Reload background page and load rules with GM_webRequest into array rules at background page.Help:
UPDATE 03.06.17 |
|
I don't think only intercepting certain request types is a good idea. There might be instances where one wouldn't want some request to even be made in the first place. Also in general replacing things that already have been loaded by something that too has to first be loaded seems pretty slow as well. How about being able to configure which requests types will be intercepted? |
|
@LennyPenny You mean, XHR and Location types of query handlers, as well as the query types like POST, GET and others? |
|
I'm just responding to @derjanb and this #397 (comment) |
|
Isn't it still the case that the soonest Tampermonkey can run scripts is asynchronously after document-start? If so, then wouldn't it be possible to miss resources (scripts, etc) loaded in the head? Don't get me wrong, I appreciate the effort, but it seems kind of silly to me to have an API that can 1. not dynamically handle requests, 2. can only catch certain resource types, and 3. may or may not actually work. |
That's true. (Even though it's much less likely if the experimental "Inject Mode" is set to "Fast".) That's why you can use
For now. As I said, that's discussable and may be subject to change.
That's doable, but adds a lot of complexity. So I'll implment it if really needed by will try to go without as long as possible. |
|
I was able to use this to cancel a request for a single static javascript file, however refreshing the page repeatedly, both with CTRL+R and also CTRL+SHIFT+R (clear cache), debug console sometimes doesn't show log output from: console.log(info, message, details); In Chromium, I see the red text line for "GET file.js net::ERR_BLOCKED_BY_CLIENT" but sometimes the console.log text is missing. Why is this? |
|
Because the script was not injected/executed yet, but Tampermonkey blocked the request based on the |
|
@derjanb i need block script loading only from "script" type request, allow loading from XHR. Is not possible to do this? |
|
@augustoresende You can initially block all by using |
|
@derjanb webRequest is not human readable. You can easily fix this allowing multiple @webRequest, like multiple @include |
|
And... A suggestion: webRequest Header/Body content intercept and editor GM_xmlhttpRequest already have header request manipulation, is like that: like xhook, but xhook only works with XHR and fetch. And... include "main_frame", is not replaceable by a userscript |
|
@derjanb BUG: |
|
@derjanb ? |
Yes, this would be one way to make it better readable.
This is AFAIK only possible at Firefox:
I know. All you get is that it's loaded from cache at the onResponseStarted listener: |
|
header intercept working on chrome: |
|
@youk, as was mentioned above the communication between userscript and background page is asynchronous so there is no way to dynamically analyze the request and then take different actions. You only can define static rules and listeners for rule triggering. @derjanb, looks like GM_webRequest returns object with |
// ==UserScript==
// @name Test GM.webRequest
// @namespace http://tampermonkey.net/
// @version 0.1
// @description try to take over the world!
// @author 7nik
// @match https://httpbin.org/*
// @grant GM.webRequest
// @connect httpbin.org
// ==/UserScript==
(async () => {
const rules = [
// error on the background page
{ selector: "https://httpbin.org/anything?key=1" },
// works but never calls the listener
{ selector: "https://httpbin.org/anything?key=2", action: {} },
// always redirects to the static URL
{ selector: "https://httpbin.org/anything?key=3", action: {
redirect: {
from: "https://httpbin.org/anything\\?(.*)",
to: "https://httpbin.org/anything/redirect?mode=dynamic&$1",
url: "https://httpbin.org/anything/redirect?mode=static",
},
} },
];
console.log("start");
for (let i = 0; i < rules.length; i++) {
try {
let [action, message, details] = await new Promise((resolve, reject) => {
GM.webRequest([rules[i]], (...args) => resolve(args))
.then(() => fetch(`https://httpbin.org/anything?key=${i+1}`))
.finally(() => setTimeout(reject, 500, "timeout"));
});
console.log(i+1, details.description || message, details.redirect_url || details.url);
} catch (ex) {
console.log(i+1, ex, rules[i]);
}
}
console.log("done");
})(); |
|
I've stumbled upon this ticket while figuring out how to intercept request headers (specifically, auth token) in a user script. I tried a simple match-and-forward rule with For posterity, the workaround I ended up with relies on overriding |
|
@rukletsov, the You can use |
|
why it just works when i press ctrl + f5? |
|
@imrelo, we aren't psychics to say what's going on without seeing your code. |
|
Can I listen for the request being made and get notified when it is done and also get response data? #397 (comment) Here it is stated that we can download, modify and forward it. How this forwarding works then? |
|
@MikeDabrowski
Static redirect: {
selector: "https://httpbin.org/anything?redirect=1",
action: { redirect: "https://httpbin.org/anything/redirect?mode=static1" },
}"Dynamic" redirect (made on regexp): {
selector: "https://httpbin.org/anything?redirect=3&key=*",
action: {
redirect: {
from: "https://httpbin.org/anything\\?redirect=3&key=(\\d+)",
to: "https://httpbin.org/anything/redirect?mode=dynamic&key=$1",
},
},
}, |
|
Ok, I get it now. All I get is that the request was done and I know its url. Then I could do regular xhr to get details and so on. But in my case endpoint response is random and breaks the whole concept :( |
|
Endpoints cannot be absolutely random. You can use multiple selectors with patterns: |
|
Endpoint url is constant but its response varies. Second call returns
different data.
I need to find some other way
…On Thu, 7 Jan 2021 at 16:09, 7nik ***@***.***> wrote:
Endpoints cannot be absolutely random. You can use multiple selectors with
patterns:
selector: ["https://example.com/api/*", "https://api.example.com/v1/*",
"*.example.com/api/*]
Or listen for another endpoint that returns the address of the target
endpoint.
And make something similar to #1086 (comment)
<#1086 (comment)>
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#397 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEICW3Z6MPM4UJOA5YYUQ4DSYXFCNANCNFSM4DKMY64Q>
.
|
|
Is there another version than the preview from 2017? Firefox says it's corrupt and chrome says CRX_HEADER_INVALID. |
|
@tylkomat, now webRequest is presented in both stable and beta versions. |
|
@7nik thanks for the info. I didn't notice it was already inside. Canceling requests works. but redirect does not. Must the redirect url be from the same domain then the selector url? |
|
|
Here the docs I have for this function: GM.webRequest(rules, listener)(Re-)registers rules for web request manipulations and the listener of triggered rules. Parameters
And here are some of my notes. |
Is this still the case? I need to modify an image that can be created and destroyed many times, and so far the only way I can find to do this is a redirect. I can do this using a separate browser extension (Request Interceptor), but really need to consolidate all changes I do to this webapp into one script. |
|
I settled on overwriting fetch. I just assigned custom function to fetch, in it I did my intercept and at the end called original fetch. Maybe this will help someone. |
|
I can't seem to do this at all in tampermonkey any more. Has anyone else figured out a way to do this? EDIT I solved it by overriding Also if this is considered a "Bug" in tampermonkey please please do not fix it unless there's a workaround. |
|
oh wow how did i not find out about this sooner... can't even count how many times i've wanted it. but to be useful it will need more juice. if not arbitrary code execution, then at least regex search/replace in the response, and be applicable to the main html request too. this way we can really prepend a script tag to the html and solve the "guarantee to run before website scripts" hassles, among other things. |
|
btw the api to manipulate most things is blocking but specifically for changing the body it is async, so maybe having user code for that part will be ok? edit: actually, reading more docs, it can be async for headers too.
edit 2: oh, you already knew, but it's ff only. i should've read the thread more carefully. sorry. though personally, this might be a good enough reason to switch to ff. amazing how chrome made no progress in the 5 years this has been open. |
|
@derjanb thanks!! -DJ |
|
so far, script BLOCKING and REDIRECT seem very promising ... thank you for where you have it so far! -DJ |
and others
Is it possible to make access to Browser API from the User-Script code? At least for Chrome-like browser.
Sometimes this really is really not enough and I do not want to write everytime extra extensions for the sake of a couple of lines of code.
Similar to:
The text was updated successfully, but these errors were encountered: