[🐛 Bug]: [java] Actions.scrollToElement() throws exception if WebElement is outside the current viewport

[jtnord]

Description

Actions.scrollToElement() is throwing a MoveTargetOutOfBoundsException when the element is outside the current viewport rather than scrolling it into view.

org.openqa.selenium.interactions.MoveTargetOutOfBoundsException: Move target (927, 1012) is out of bounds of viewport dimensions (1678, 978)

If pause before the attempted scroll and in the javascript console of firefox grab the same elements and do element.scrollIntoView() then the action does not fail.  Almost as if the scroll is trying to do a move to rather than a scroll to?

logs from marionette have the following:

Scrolling...
1772052770315   webdriver::server       DEBUG   -> POST /session/c4f517d3-876f-45e9-8d7b-d618ea1742f9/actions {
  "actions": [
    {
      "type": "wheel",
      "actions":  ...  },
          "x": 0,
          "y": 0,
          "type": "scroll"
        }
      ],
      "id": "default wheel"
    }
  ]
}
1772052770317   Marionette      DEBUG   0 -> [0,667,"WebDriver:PerformActions",{"actions":[{"actions":[{"deltaX":0,"deltaY":0,"duration":250,"origin":{"element-6066-11e4-a52e-4f735466cecf":"6c95de65-b06e-4c89-a343-0b44ea45d33e"},"type":"scroll","x":0,"y":0}],"id":"default wheel","type":"wheel"}]}]
1772052770324   RemoteAgent     TRACE   Dispatching tick 1/1
1772052770330   Marionette      DEBUG   0 <- [1,667,{"error":"move target out of bounds","message":"Move target (923, 1016) is out of bounds of viewport dimensions (1665, 998)","stacktrace":"RemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8\nWebDriverError@chrome://remote/content/shared/webdriver/Er ... eCommandsChild.sys.mjs:67:17\nreceiveMessage@chrome://remote/content/marionette/actors/MarionetteCommandsChild.sys.mjs:187:42\n"},null]
1772052770330   webdriver::server       DEBUG   <- 500 Internal Server Error {"value":{"error":"move target out of bounds","message":"Move target (923, 1016) is out of bound ... andsChild.sys.mjs:67:17\nreceiveMessage@chrome://remote/content/marionette/actors/MarionetteCommandsChild.sys.mjs:187:42\n"}}

To prove I have the right element and it is not unattached from the dom in anyway if I replace new Actions(driver).scrollToElement(placeholder).perform(); with placeholder.click(); then the element is infact scrolled into view per the webdriver click spec.. 

as far as I can understand from the webdriver spec the error is correct - which means the javadoc for https://www.selenium.dev/selenium/docs/api/java/org/openqa/selenium/interactions/Actions.html#scrollToElement(org.openqa.selenium.WebElement) is wrong or the code is broken

javadoc

If the element is outside the viewport, scrolls the bottom of the element to the bottom of the viewport.

Yet the spec says (part 4 onwards)

4. Let (x, y) be the result of trying to get coordinates relative to an origin with source, x offset, y offset, origin, browsing context, and actions options.

5. If x is less than 0 or greater than the width of the viewport in CSS pixels, then return error with error code move target out of bounds.

6. If y is less than 0 or greater than the height of the viewport in CSS pixels, then return error with error code move target out of bounds.

So in 4 the element is offscreen so the resulting X,Y is outside the current viewport.

Then the check in 5/6 will say this is outside the current viewport and an error will be returned.

Reproducable using selenium/standalone-firefox:4.41.0 (Linux) and also a local firefox (on windows)

Reproducible Code

import java.time.Duration;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.Dimension;
import org.openqa.selenium.NoSuchElementException;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.interactions.Actions;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.FluentWait;
import org.openqa.selenium.support.ui.Wait;

public class WebDriverScrollIssueTest {

    public static WebDriver createDriver() {
        // docker run --name selenium --rm --shm-size=2g -d -p 127.0.0.1:4444:4444 -p 127.0.0.1:5900:5900 -e
        // SE_NODE_GRID_URL=http://127.0.0.1:4444 -e SE_SCREEN_WIDTH=1680 -e SE_SCREEN_HEIGHT=1090
        // selenium/standalone-firefox:4.41.0
        String u = System.getenv("REMOTE_WEBDRIVER_URL");
        if (u == null || u.isBlank()) {
            System.out.println("REMOTE_WEBDRIVER_URL not set - using http://localhost:4444/wd/hub");
            u = "http://localhost:4444/wd/hub";
        }

        WebDriver driver = RemoteWebDriver.builder()
                .address(u)
                .addAlternative(new FirefoxOptions())
                .build();
        driver.manage().timeouts().pageLoadTimeout(Duration.ofSeconds(30));
        driver.manage().timeouts().implicitlyWait(Duration.ofMillis(0));

        return driver;
    }

    @Test
    public void demonstrateScrollingIssue() {
        WebDriver driver = null;
        try {
            driver = createDriver();
            // set a specific window size.
            driver.manage().window().setSize(new Dimension(800, 600));

            Wait<WebDriver> wait = new FluentWait<WebDriver>(driver)
                    .withTimeout(Duration.ofSeconds(30L))
                    .pollingEvery(Duration.ofSeconds(5L))
                    .ignoring(NoSuchElementException.class);

            driver.get("https://www.selenium.dev/selenium/docs/api/java/org/openqa/selenium/interactions/Actions.html");
            WebElement webElement = wait.until(ExpectedConditions.presenceOfElementLocated(
                    By.id("scrollToElement(org.openqa.selenium.WebElement)")));

            new Actions(driver).scrollToElement(webElement).perform();
        } finally {
            if (driver != null) {
                driver.close();
            }
        }
    }
}

Debugging Logs

1772052770315   webdriver::server       DEBUG   -> POST /session/c4f517d3-876f-45e9-8d7b-d618ea1742f9/actions {
  "actions": [
    {
      "type": "wheel",
      "actions":  ...  },
          "x": 0,
          "y": 0,
          "type": "scroll"
        }
      ],
      "id": "default wheel"
    }
  ]
}
1772052770317   Marionette      DEBUG   0 -> [0,667,"WebDriver:PerformActions",{"actions":[{"actions":[{"deltaX":0,"deltaY":0,"duration":250,"origin":{"element-6066-11e4-a52e-4f735466cecf":"6c95de65-b06e-4c89-a343-0b44ea45d33e"},"type":"scroll","x":0,"y":0}],"id":"default wheel","type":"wheel"}]}]
1772052770324   RemoteAgent     TRACE   Dispatching tick 1/1
1772052770330   Marionette      DEBUG   0 <- [1,667,{"error":"move target out of bounds","message":"Move target (923, 1016) is out of bounds of viewport dimensions (1665, 998)","stacktrace":"RemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8\nWebDriverError@chrome://remote/content/shared/webdriver/Er ... eCommandsChild.sys.mjs:67:17\nreceiveMessage@chrome://remote/content/marionette/actors/MarionetteCommandsChild.sys.mjs:187:42\n"},null]
1772052770330   webdriver::server       DEBUG   <- 500 Internal Server Error {"value":{"error":"move target out of bounds","message":"Move target (923, 1016) is out of bound ... andsChild.sys.mjs:67:17\nreceiveMessage@chrome://remote/content/marionette/actors/MarionetteCommandsChild.sys.mjs:187:42\n"}}

[selenium-ci]

@jtnord, thank you for creating this issue. We will troubleshoot it as soon as we can.

Selenium Triage Team: remember to follow the Triage Guide

[cgoldberg]

as far as I can understand from the webdriver spec the error is correct - which means the javadoc

From the section of the spec you posted, it sounds like everything is working as expected. I updated the javadoc in #17143 to reflect the real behavior.

Can you verify if Chrome/ChromeDriver also throws an exception?

[jtnord]

as far as I can understand from the webdriver spec the error is correct - which means the javadoc

From the section of the spec you posted, it sounds like everything is working as expected. I updated the javadoc in #17143 to reflect the real behavior.

Can you verify if Chrome/ChromeDriver also throws an exception?

chrome does not throw an exception. (using docker container selenium/standalone-chrome:4.41.0)

[jtnord]

I found mozilla/geckodriver#2078 and that has an interesting comment

After speaking to @jgraham it turns out that the scroll action of the wheel input source is actually never meant to be used to "scroll to an element". Instead it has to be used to "scroll at the location of the element"

So basically whatever scrollToElement would appear to be attempting wrong just be wrong on every count.

Now what should be happening is questionable.

I (and others) need a way to ensure that an element is in the visible viewport. (e.g. akin to javascript's Element.scrollIntoView.
This could probably be somethign like obtain the bounds of the webelement and then "scroll" (position!) the page so that the bottom of the bounds rect is at the bottom of the page (if not already visible).

I'm currently hacking around this by performing WebElement.click() (and using the implicit scoll) which is not great as depending on exactly how the page is layed out the click may get intercepted by something else (meaning surrounding that with try/catch) but also the intention is not clear and it would not mimic user interaction (as something may now be in focus that was not in focus before, or expected to be in focus).

[jtnord]

https://www.selenium.dev/documentation/webdriver/actions_api/wheel/#scroll-to-element is marked as chromium only - but chromium is as we now know not adhering to the spec so is broken and should regress unless the webdriver spec is updated.

So should that page be updated to match the spec and marked firefox only?