4.0.0.b1 fails when testing file uploads

[stevepiercy]

🐛 Bug Report

4.0.0.b1 fails when testing file uploads. This worked in 4.0.0a7.

Test script or set of commands reproducing this issue

Baseline successful build on 2021-01-28 with selenium-4.0.0a7:
https://github.com/Pylons/deformdemo/runs/1783717797?check_suite_focus=true

Failed build on 2021-02-22 with selenium-4.0.0b1:
https://github.com/Pylons/deformdemo/runs/1949937935?check_suite_focus=true

Final successful build on 2021-02-22 with selenium-4.0.0a7:
https://github.com/Pylons/deformdemo/runs/1950522237?check_suite_focus=true

Stacktrace of one of four similar test failures:

______________________ FileUploadTests.test_submit_filled ______________________

self = <deformdemo.test.FileUploadTests testMethod=test_submit_filled>

    def test_submit_filled(self):
        # submit one first
        path, filename = _getFile()
>       findcss("input[type=file]").send_keys(path)

deformdemo/test.py:1466: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.tox/py/lib/python3.8/site-packages/selenium/webdriver/remote/webelement.py:520: in send_keys
    self._execute(Command.SEND_KEYS_TO_ELEMENT,
.tox/py/lib/python3.8/site-packages/selenium/webdriver/remote/webelement.py:664: in _execute
    return self._parent.execute(command, params)
.tox/py/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py:344: in execute
    self.error_handler.check_response(response)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <selenium.webdriver.remote.errorhandler.ErrorHandler object at 0x7f8c94fb6850>
response = {'status': 400, 'value': '{"value":{"error":"invalid argument","message":"File not found: /home/runner/work/deformdemo...//marionette/content/error.js:310:5\\ninteraction.uploadFiles@chrome://marionette/content/interaction.js:509:13\\n"}}'}

    def check_response(self, response):
        """
        Checks that a JSON response from the WebDriver does not have an error.
    
        :Args:
         - response - The JSON response from the WebDriver server as a dictionary
           object.
    
        :Raises: If the response contains an error message.
        """
        status = response.get('status', None)
        if not status or status == ErrorCode.SUCCESS:
            return
        value = None
        message = response.get("message", "")
        screen = response.get("screen", "")
        stacktrace = None
        if isinstance(status, int):
            value_json = response.get('value', None)
            if value_json and isinstance(value_json, str):
                import json
                try:
                    value = json.loads(value_json)
                    if len(value.keys()) == 1:
                        value = value['value']
                    status = value.get('error', None)
                    if not status:
                        status = value["status"]
                        message = value["value"]
                        if not isinstance(message, str):
                            value = message
                            message = message.get('message')
                    else:
                        message = value.get('message', None)
                except ValueError:
                    pass
    
        if status in ErrorCode.NO_SUCH_ELEMENT:
            exception_class = NoSuchElementException
        elif status in ErrorCode.NO_SUCH_FRAME:
            exception_class = NoSuchFrameException
        elif status in ErrorCode.NO_SUCH_WINDOW:
            exception_class = NoSuchWindowException
        elif status in ErrorCode.STALE_ELEMENT_REFERENCE:
            exception_class = StaleElementReferenceException
        elif status in ErrorCode.ELEMENT_NOT_VISIBLE:
            exception_class = ElementNotVisibleException
        elif status in ErrorCode.INVALID_ELEMENT_STATE:
            exception_class = InvalidElementStateException
        elif status in ErrorCode.INVALID_SELECTOR \
                or status in ErrorCode.INVALID_XPATH_SELECTOR \
                or status in ErrorCode.INVALID_XPATH_SELECTOR_RETURN_TYPER:
            exception_class = InvalidSelectorException
        elif status in ErrorCode.ELEMENT_IS_NOT_SELECTABLE:
            exception_class = ElementNotSelectableException
        elif status in ErrorCode.ELEMENT_NOT_INTERACTABLE:
            exception_class = ElementNotInteractableException
        elif status in ErrorCode.INVALID_COOKIE_DOMAIN:
            exception_class = InvalidCookieDomainException
        elif status in ErrorCode.UNABLE_TO_SET_COOKIE:
            exception_class = UnableToSetCookieException
        elif status in ErrorCode.TIMEOUT:
            exception_class = TimeoutException
        elif status in ErrorCode.SCRIPT_TIMEOUT:
            exception_class = TimeoutException
        elif status in ErrorCode.UNKNOWN_ERROR:
            exception_class = WebDriverException
        elif status in ErrorCode.UNEXPECTED_ALERT_OPEN:
            exception_class = UnexpectedAlertPresentException
        elif status in ErrorCode.NO_ALERT_OPEN:
            exception_class = NoAlertPresentException
        elif status in ErrorCode.IME_NOT_AVAILABLE:
            exception_class = ImeNotAvailableException
        elif status in ErrorCode.IME_ENGINE_ACTIVATION_FAILED:
            exception_class = ImeActivationFailedException
        elif status in ErrorCode.MOVE_TARGET_OUT_OF_BOUNDS:
            exception_class = MoveTargetOutOfBoundsException
        elif status in ErrorCode.JAVASCRIPT_ERROR:
            exception_class = JavascriptException
        elif status in ErrorCode.SESSION_NOT_CREATED:
            exception_class = SessionNotCreatedException
        elif status in ErrorCode.INVALID_ARGUMENT:
            exception_class = InvalidArgumentException
        elif status in ErrorCode.NO_SUCH_COOKIE:
            exception_class = NoSuchCookieException
        elif status in ErrorCode.UNABLE_TO_CAPTURE_SCREEN:
            exception_class = ScreenshotException
        elif status in ErrorCode.ELEMENT_CLICK_INTERCEPTED:
            exception_class = ElementClickInterceptedException
        elif status in ErrorCode.INSECURE_CERTIFICATE:
            exception_class = InsecureCertificateException
        elif status in ErrorCode.INVALID_COORDINATES:
            exception_class = InvalidCoordinatesException
        elif status in ErrorCode.INVALID_SESSION_ID:
            exception_class = InvalidSessionIdException
        elif status in ErrorCode.UNKNOWN_METHOD:
            exception_class = UnknownMethodException
        else:
            exception_class = WebDriverException
        if not value:
            value = response['value']
        if isinstance(value, str):
            raise exception_class(value)
        if message == "" and 'message' in value:
            message = value['message']
    
        screen = None
        if 'screen' in value:
            screen = value['screen']
    
        stacktrace = None
        st_value = value.get('stackTrace') or value.get('stacktrace')
        if st_value:
            if isinstance(st_value, str):
                stacktrace = st_value.split('\n')
            else:
                stacktrace = []
                try:
                    for frame in st_value:
                        line = self._value_or_default(frame, 'lineNumber', '')
                        file = self._value_or_default(frame, 'fileName', '<anonymous>')
                        if line:
                            file = "%s:%s" % (file, line)
                        meth = self._value_or_default(frame, 'methodName', '<anonymous>')
                        if 'className' in frame:
                            meth = "%s.%s" % (frame['className'], meth)
                        msg = "    at %s (%s)"
                        msg = msg % (meth, file)
                        stacktrace.append(msg)
                except TypeError:
                    pass
        if exception_class == UnexpectedAlertPresentException:
            alert_text = None
            if 'data' in value:
                alert_text = value['data'].get('text')
            elif 'alert' in value:
                alert_text = value['alert'].get('text')
            raise exception_class(message, screen, stacktrace, alert_text)
>       raise exception_class(message, screen, stacktrace)
E       selenium.common.exceptions.InvalidArgumentException: Message: File not found: /home/runner/work/deformdemo/deformdemo/deformdemo/test.py
E       Stacktrace:
E       WebDriverError@chrome://marionette/content/error.js:181:5
E       InvalidArgumentError@chrome://marionette/content/error.js:310:5
E       interaction.uploadFiles@chrome://marionette/content/interaction.js:509:13

Environment

OS: Ubuntu 18.04 and 20.04 LTS
Browser: Firefox
Browser version: Selenium/Firefox
Browser Driver version: GeckoDriver 0.29.0
Language Bindings version: Python 3
Selenium Grid version (if applicable): See above

Deform and Deformdemo are open source projects. I am happy to share whatever information may be helpful to isolate the cause.

[AutomatedTester]

I've just added a test for this in 4750e40 and can't recreate the issue. Are you sure the file is available where you think it is?

[stevepiercy]

The file is present. The file that gets uploaded is the Python file that contains all the tests.

I don't know if it is readable by Selenium, perhaps due to permissions or something in GitHub Actions? How can I check that?

[hxhappy]

I've just added a test for this in 4750e40 and can't recreate the issue. Are you sure the file is available where you think it is?

I meet the same issue and resolve it by update ''.join(str(value)) to ''.join(value).
The variable value is tuple type variable so that it should be ''.join(value) when convert to string.

[AutomatedTester]

@stevepiercy Are the tests running using remote or directly to geckodriver?

[stevepiercy]

@AutomatedTester remote via GitHub Actions. Please see above stacktrace for exactly where it fails.

Locally with geckodriver, tests pass successfully.

FWIW, I tried to install my GitHub fork of selenium with the proposed fix from @hxhappy, but I was not able to successfully install it through GitHub Actions.

[hxhappy]

@AutomatedTester remote via GitHub Actions. Please see above stacktrace for exactly where it fails.

Locally with geckodriver, tests pass successfully.

FWIW, I tried to install my GitHub fork of selenium with the proposed fix from @hxhappy, but I was not able to successfully install it through GitHub Actions.

@stevepiercy you can update webelement.py in your local as below:
D:\Python37\Lib\site-packages\selenium\webdriver\remote\webelement.py

[stevepiercy]

@hxhappy yes, locally, that works fine. My problem was not being able to successfully install my fork of Selenium as a package into GitHub Actions, then run my project's tests. Sorry if that was not clear.

[AutomatedTester]

Looking at this again I think I need a bit more info. I've added another test for uploading multiple files in 6e44d37 and that appears to be working for me.

[AutomatedTester]

I've just added a test for this in 4750e40 and can't recreate the issue. Are you sure the file is available where you think it is?

I meet the same issue and resolve it by update ''.join(str(value)) to ''.join(value).
The variable value is tuple type variable so that it should be ''.join(value) when convert to string.

How are you getting it to be a tuple? It should be at worst a string or a list.

If you can give me an example of how you are using it I will look into it further.

[hxhappy]

@AutomatedTester
Refer my debug info.

[stevepiercy]

Actually I think I found something. Here's an example. Note the path to the sought file is /home/runner/work/deformdemo/deformdemo/deformdemo/test.py, but the file actually exists at /home/runner/work/deformdemo/deformdemo/test.py.

[AutomatedTester]

@AutomatedTester
Refer my debug info.

@hxhappy I am not sure why you're passing in a tuple but it should be a string. I think your issue is very different to @stevepiercy so let's stop muddying the water here.

[stevepiercy]

I need to verify the value of path as a sanity check. One way to do that for both local testing and GitHub Actions would be to insert a test that is designed to fail, for example comparing path to an explicit bogus value.

self.assertEqual("",path)

This should output the actual value of path in pytest, unless there is a better way to debug in GitHub Actions.

I could also compare other package versions and dependencies, but given that the current tests pass with selenium-4.0.0a7 but fail in later versions, I reckon that would be a waste of time.

Is there anything else I could possibly check? I'm open to ideas.

[AutomatedTester]

@stevepiercy let's try that. I can't currently think of a different way.

One thing actually...

Are you locally and in github actions running the tests form the same level within the git tree? It shouldn't matter since we're asking for the full path before we pass it in.

[stevepiercy]

Are you locally and in github actions running the tests form the same level within the git tree? It shouldn't matter since we're asking for the full path before we pass it in.

Yes.

Here's the test result on GHA:

E AssertionError: '' != '/home/runner/work/deformdemo/deformdemo/deformdemo/test.py'

Here is the test result locally:

E       AssertionError: '' != '/Users/stevepiercy/projects/deform/deform[36 chars]t.py'
E       + /Users/stevepiercy/projects/deform/deformdemo_functional_tests/deformdemo/test.py

The paths differ between GHA and local, of course, but the important part is that the directory from which the tests are run is:

GHA:
deformdemo/deformdemo/deformdemo/test.py

Local:
deformdemo_functional_tests/deformdemo/test.py

Somewhere in the stack for GHA, the path segment deformdemo gets doubled up.

[hxhappy]

@AutomatedTester I have reference function definitions in python document as below. If the form “*identifier” is present, it is initialized to a tuple receiving any excess positional parameters, defaulting to the empty tuple.
The file is uploaded by function send_keys(self, *value) so that the variable value will be initialized to a tuple. Thanks!
https://docs.python.org/3.7/reference/compound_stmts.html#function-definitions

[stevepiercy]

I completed more testing. The filepath is correct on GHA. I did two runs, where the differences were only in the requirements for Python 3.9:

# Run #1
testing_extras.extend(["selenium >= 4.0a, < 4.0.0.b1"])

# Run #2
testing_extras.extend(["selenium >= 4.0a"])

There are 4 file upload tests, and 1 test that is designed to always fail and return the filepath of the file.

self.assertEqual("", path)

For both runs, the returned file path from the deliberately failing test is the following.

/home/runner/work/deformdemo/deformdemo/deformdemo/test.py

• In Run 1, only the 1 forced failure test fails. See https://github.com/Pylons/deformdemo/runs/2147838081?check_suite_focus=true#step:7:181
• In Run 2, all 4 upload tests fail with file not found, as well as the 1 forced failure test. See https://github.com/Pylons/deformdemo/runs/2147862771?check_suite_focus=true#step:7:369

I conclude that, since all other things are equal, the problem must be within Selenium >= 4.0.0.b1 or its dependencies. Those are the only differences between these two runs.

Run 1 relevant dependencies

-e git+https://github.com/Pylons/deformdemo@8183f420523a0bc1b35310a78c1a801eda16dab3#egg=deformdemo
selenium==4.0.0a7
trio==0.18.0
trio-websocket==0.9.2
urllib3==1.26.4

Run 2 relevant dependencies

-e git+https://github.com/Pylons/deformdemo@1db6273eae81ea10f2527e132488d590bc59d18c#egg=deformdemo
selenium==4.0.0b2.post1
trio==0.17.0
trio-websocket==0.9.0
urllib3==1.26.2

Below is the stacktrace of just one of the file upload test failures, as they are repetitive. The only thing I don't understand in the stacktrace relates to @chrome://marionette/content.

deformdemo/test.py:1466: AssertionError
______________________ FileUploadTests.test_submit_filled ______________________

self = <deformdemo.test.FileUploadTests testMethod=test_submit_filled>

    def test_submit_filled(self):
        # submit one first
        path, filename = _getFile()
>       findcss("input[type=file]").send_keys(path)

deformdemo/test.py:1471: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.tox/py/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py:539: in send_keys
    self._execute(Command.SEND_KEYS_TO_ELEMENT,
.tox/py/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py:693: in _execute
    return self._parent.execute(command, params)
.tox/py/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py:369: in execute
    self.error_handler.check_response(response)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <selenium.webdriver.remote.errorhandler.ErrorHandler object at 0x7fb65a54e5e0>
response = {'status': 400, 'value': '{"value":{"error":"invalid argument","message":"File not found: /home/runner/work/deformdemo...//marionette/content/error.js:310:5\\ninteraction.uploadFiles@chrome://marionette/content/interaction.js:541:13\\n"}}'}

    def check_response(self, response):
        """
        Checks that a JSON response from the WebDriver does not have an error.
    
        :Args:
         - response - The JSON response from the WebDriver server as a dictionary
           object.
    
        :Raises: If the response contains an error message.
        """
        status = response.get('status', None)
        if not status or status == ErrorCode.SUCCESS:
            return
        value = None
        message = response.get("message", "")
        screen = response.get("screen", "")
        stacktrace = None
        if isinstance(status, int):
            value_json = response.get('value', None)
            if value_json and isinstance(value_json, str):
                import json
                try:
                    value = json.loads(value_json)
                    if len(value.keys()) == 1:
                        value = value['value']
                    status = value.get('error', None)
                    if not status:
                        status = value["status"]
                        message = value["value"]
                        if not isinstance(message, str):
                            value = message
                            message = message.get('message')
                    else:
                        message = value.get('message', None)
                except ValueError:
                    pass
    
        if status in ErrorCode.NO_SUCH_ELEMENT:
            exception_class = NoSuchElementException
        elif status in ErrorCode.NO_SUCH_FRAME:
            exception_class = NoSuchFrameException
        elif status in ErrorCode.NO_SUCH_WINDOW:
            exception_class = NoSuchWindowException
        elif status in ErrorCode.STALE_ELEMENT_REFERENCE:
            exception_class = StaleElementReferenceException
        elif status in ErrorCode.ELEMENT_NOT_VISIBLE:
            exception_class = ElementNotVisibleException
        elif status in ErrorCode.INVALID_ELEMENT_STATE:
            exception_class = InvalidElementStateException
        elif status in ErrorCode.INVALID_SELECTOR \
                or status in ErrorCode.INVALID_XPATH_SELECTOR \
                or status in ErrorCode.INVALID_XPATH_SELECTOR_RETURN_TYPER:
            exception_class = InvalidSelectorException
        elif status in ErrorCode.ELEMENT_IS_NOT_SELECTABLE:
            exception_class = ElementNotSelectableException
        elif status in ErrorCode.ELEMENT_NOT_INTERACTABLE:
            exception_class = ElementNotInteractableException
        elif status in ErrorCode.INVALID_COOKIE_DOMAIN:
            exception_class = InvalidCookieDomainException
        elif status in ErrorCode.UNABLE_TO_SET_COOKIE:
            exception_class = UnableToSetCookieException
        elif status in ErrorCode.TIMEOUT:
            exception_class = TimeoutException
        elif status in ErrorCode.SCRIPT_TIMEOUT:
            exception_class = TimeoutException
        elif status in ErrorCode.UNKNOWN_ERROR:
            exception_class = WebDriverException
        elif status in ErrorCode.UNEXPECTED_ALERT_OPEN:
            exception_class = UnexpectedAlertPresentException
        elif status in ErrorCode.NO_ALERT_OPEN:
            exception_class = NoAlertPresentException
        elif status in ErrorCode.IME_NOT_AVAILABLE:
            exception_class = ImeNotAvailableException
        elif status in ErrorCode.IME_ENGINE_ACTIVATION_FAILED:
            exception_class = ImeActivationFailedException
        elif status in ErrorCode.MOVE_TARGET_OUT_OF_BOUNDS:
            exception_class = MoveTargetOutOfBoundsException
        elif status in ErrorCode.JAVASCRIPT_ERROR:
            exception_class = JavascriptException
        elif status in ErrorCode.SESSION_NOT_CREATED:
            exception_class = SessionNotCreatedException
        elif status in ErrorCode.INVALID_ARGUMENT:
            exception_class = InvalidArgumentException
        elif status in ErrorCode.NO_SUCH_COOKIE:
            exception_class = NoSuchCookieException
        elif status in ErrorCode.UNABLE_TO_CAPTURE_SCREEN:
            exception_class = ScreenshotException
        elif status in ErrorCode.ELEMENT_CLICK_INTERCEPTED:
            exception_class = ElementClickInterceptedException
        elif status in ErrorCode.INSECURE_CERTIFICATE:
            exception_class = InsecureCertificateException
        elif status in ErrorCode.INVALID_COORDINATES:
            exception_class = InvalidCoordinatesException
        elif status in ErrorCode.INVALID_SESSION_ID:
            exception_class = InvalidSessionIdException
        elif status in ErrorCode.UNKNOWN_METHOD:
            exception_class = UnknownMethodException
        else:
            exception_class = WebDriverException
        if not value:
            value = response['value']
        if isinstance(value, str):
            raise exception_class(value)
        if message == "" and 'message' in value:
            message = value['message']
    
        screen = None
        if 'screen' in value:
            screen = value['screen']
    
        stacktrace = None
        st_value = value.get('stackTrace') or value.get('stacktrace')
        if st_value:
            if isinstance(st_value, str):
                stacktrace = st_value.split('\n')
            else:
                stacktrace = []
                try:
                    for frame in st_value:
                        line = self._value_or_default(frame, 'lineNumber', '')
                        file = self._value_or_default(frame, 'fileName', '<anonymous>')
                        if line:
                            file = "%s:%s" % (file, line)
                        meth = self._value_or_default(frame, 'methodName', '<anonymous>')
                        if 'className' in frame:
                            meth = "%s.%s" % (frame['className'], meth)
                        msg = "    at %s (%s)"
                        msg = msg % (meth, file)
                        stacktrace.append(msg)
                except TypeError:
                    pass
        if exception_class == UnexpectedAlertPresentException:
            alert_text = None
            if 'data' in value:
                alert_text = value['data'].get('text')
            elif 'alert' in value:
                alert_text = value['alert'].get('text')
            raise exception_class(message, screen, stacktrace, alert_text)
>       raise exception_class(message, screen, stacktrace)
E       selenium.common.exceptions.InvalidArgumentException: Message: File not found: /home/runner/work/deformdemo/deformdemo/deformdemo/test.py
E       Stacktrace:
E       WebDriverError@chrome://marionette/content/error.js:181:5
E       InvalidArgumentError@chrome://marionette/content/error.js:310:5
E       interaction.uploadFiles@chrome://marionette/content/interaction.js:541:13

.tox/py/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py:236: InvalidArgumentException

[pythobot]

I think this issue began here:

85ae06e#diff-167af39a13c2c1c011a621b7b0f0ff5a76a6bd965a0d077e611d02de2992945cL539

I was also having problem when uploading a file and found that was the issue. I fixed it and it works OK now.

[stevepiercy]

@arnaldoanez would you please submit a PR that fixes the issue for you?

@diemol I am eager to run any fix in our repo's GitHub Actions to share results with y'all. Thank you!

[AutomatedTester]

@stevepiercy if you checkout the selenium repo and build the wheel with bazel build //py:selenium-wheel it would be good to know if #9330 has fixed your issue.

[stevepiercy]

@AutomatedTester I have no clue how to do that in GitHub Actions. Do you have an example configuration I can reuse or adapt? If not, I'll just wait for the next beta release that includes this patch.

[stevepiercy]

Selenium 4.0.0.b4 was released today, but the file upload tests still do not pass in deformdemo's GitHub Actions. It looks like this issue is still on the To-Do column for the Selenium 4 project board and did not make the cut for this release.

[GeyseR]

@stevepiercy, as I could see the test that fails (FileUploadTests.test_getfile), should fail as it was created for debugging purposes.

One way to do that for both local testing and GitHub Actions would be to insert a test that is designed to fail, for example comparing path to an explicit bogus value.

I don't see any other tests failures for this particular build

[stevepiercy]

@GeyseR what would I do without you? Thank you! I completely forgot I added that deliberately failing test. Removing that test yields a fully successful build. 🎉

It guess the fix for this issue actually did make it into the release. I didn't see a change history file in the root of the repo, and I am not familiar with the release process. Apologies for my misunderstanding.

Closing as resolved.

[GeyseR]

@stevepiercy I glad that my comment helped you :)
btw, here is the file where the changes are listed for the python lib - https://github.com/SeleniumHQ/selenium/blob/trunk/py/CHANGES