Support xpath support for cy.get() and .find()

[fringd]

Current behavior:

.get('//div') doesn't work
.find('p/span') also doesn't work

Desired behavior:

It gets the element using document.evaluate just like get and find currently uses document.querySelector.

or to avoid collisions we add getXpath and findXpath or something.

Motivation

xpath is fairly standard in integration tests, and is much more powerful than css selectors are, and I'm working on a shared library of xpath selectors for use with selenium, puppeteer, and ideally, cypress.

I'm happy to help with writing this code if people agree it's a useful feature that would be likely to get merged.

[jennifer-shehane]

I don't know the full depth of the complexity of the implementation of this, but this is definitely possible.

• cy.get() code is custom to cypress: https://github.com/cypress-io/cypress/blob/issue-895/packages/driver/src/cy/commands/querying.coffee#L93
• .find() code just passes is along to jQuerys implementation basically: https://github.com/cypress-io/cypress/blob/issue-895/packages/driver/src/cy/commands/traversals.coffee#L5

[3d-mac]

+1 for XPath support

[CliffDavis]

+1, this would make migrating from an existing selenium framework significantly easier.

[brian-mann]

These things already exist. You can use off the shelf tools to convert xpath to css selectors.

https://www.google.com/search?q=xpath+to+selector+converter&oq=xpath+to+selector+converter&aqs=chrome..69i57j0l4.6500j0j4&sourceid=chrome&ie=UTF-8

There are a few NPM modules you can just install and require in cypress.

[fringd]

I'm pretty sure xpath is a super-set of css, so any translator from xpath to css would be incomplete. Part of the reason I want xpath is that it's more powerful.

[fringd]

for example, in xpath you can select a node depending upon its children, or its text content.

[ailadson]

+1, although i got what I needed with cy.get('svg').find('id-of-svg-element')

[nusco]

+1. Even in cases where CSS is enough, having Xpath would make some migrations from other frameworks so much easier.

[szymach]

Yeah I've just ran into an issue, where a select() did not work due to a non-braking space in the inner HTML of the option. In XPath I could just normalize spaces and it would work just fine, so definitely a +1 from me.

[saintflow47]

+1. Adding a data-cy attribute is a nice way, but we should be allowed to use xpath as well if we don't want to add any code specifically for testing.

[bryantabaird]

Yes, searching by text content would be great with xpath, since that cannot be done with CSS selectors (it doesn't allow contains, see https://stackoverflow.com/questions/1520429/is-there-a-css-selector-for-elements-containing-certain-text?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa)

Edit: just found this: https://docs.cypress.io/api/commands/contains.html#Content

[kamituel]

I'd love that. I use CSS selectors whenever possible, but sometimes XPath is necessary.

One example of a selector that is hugely useful in XPath, but not possible in CSS, is selecting a table cell based on the header.

For instance, to get a cell in row 3 in column with a label "Age", you can use:

//table
  /tbody
    /tr[position()=3]
      /td[count(//table/thead/tr/th[text()='Age']/preceding-sibling::th) + 1]

Not exactly a trivial selector, but I still prefer it over doing this programmatically.

Here's a corresponding table:

<table>
  <thead>
    <tr>
      <th>Username</th>
      <th>Email address</th>
      <th>Age</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Mary</td>
      <td>[email protected]</td>
      <td>62</td>
    </tr>
    <tr>
      <td>John</td>
      <td>[email protected]</td>
      <td>60</td>
    </tr>
    <tr>
      <td>Kate</td>
      <td>[email protected]</td>
      <td>20</td>
    </tr>
  </tbody>
</table>

[v-mwalk]

Am averse to 'me toos' and +1's but xpath capability it a must. There are a few scenarios where css will not work; the primary one being pointed out by fringd.

Even if it is the XPath 1.0 engine, it would make Cypress fully usable for any webpage...

[brian-mann]

@kamituel in your example above why couldn't you just do this?

// find the cell containing the content
cy.contains('td', '20')

// or you could find a higher up element if you wanted to
cy.contains('tr', '20')

The vast majority of use cases I've seen are for XPath related to an element containing text and you can accomplish that with cy.contains().

[brian-mann]

I just looked it up and apparently there is already a document.evaluate which will take an xpath.

You should already be able to do this with Cypress with a custom command.

https://developer.mozilla.org/en-US/docs/Web/API/Document/evaluate

At the very least, adding this to the query methods of Cypress may not be that difficult afterall.

[brian-mann]

@bryantabaird went back through to review comments. Per my comment here: you can use cy.contains() rather than xpath if you want to select an element or a parent element by its text content.

We've found cy.contains() can achieve what you normally would do with a much more complex xpath selector.

#1274 (comment)

[kamituel]

@brian-mann To follow a real example, I'm working on a web application that has several large tables (for reporting purposes), that can easily have 20-30 columns. In some test cases, I want to assert on contents of just a few out of those 20-30 columns.

Using: cy.contains('td', '20') would work, but:

• it doesn't specify in which column I expect to see the value "20"
• it leads to not very readable test code that will grow to be hard to maintain

Let's assume, still following my example, that I want to assert on contents of the first and third column, and I'm not interested in the contents of the 2nd:

cy.contains('td', '20')
cy.contains('td', 'Kate')

This code will match:

    <tr>
      <td>Kate</td>
      <td>[email protected]</td>
      <td>20</td>
    </tr>

but it'll also match:

    <tr>
      <td>20</td>
      <td>[email protected]</td>
      <td>Kate</td>
    </tr>

So I can start adding some more specific selectors, or use data-cy:

cy.contains('td:nth-of-type(3)', '20')
cy.contains('td:nth-of-type(1)', 'Kate')

But it gets tricky quickly - I'll have to update tests as the table grows, or when I remove a column, etc...

Compare it to (a hypothetical code):

cy.get(cell(3, 'Age')).contains('20');
cy.get(cell(3, 'Username')).contains('Kate');

// With a helper function:

function cell(rowNumber, columnHeader) {
  return `//table
            /tbody
              /tr[position()=${ rowNumber }]
                /td[count(//table/thead/tr/th[text()='${ columnHeader }']/preceding-sibling::th) + 1]`;
}

Sure, the helper function uses a fairly complex XPath selector, but it's not that bad once one gets used to XPath, and it's still simpler than a helper function/command would be without XPath. Plus, it leads to simple, readable and maintainable test code.

In general, I prefer CSS over XPath any day, as it leads to shorter selectors whose syntax everyone is familiar with. But when given a choice of a complex helper function that would need to iterate over a number of DOM nodes, versus a declarative XPath selector, I strongly prefer XPath.

Also, yes, there is document.evaluate I use often to test XPath selectors (when using them with Selenium). I even added a bookmarklet to Chrome which allows me to easily use it on every website:

javascript:(window.find_all = function (query) {     var nodes = [];     var iterator = document.evaluate(query, document);     try {         var node = iterator.iterateNext();         while (node) {             nodes.push(node);             node = iterator.iterateNext();         }     } catch (e) {         console.error("Document tree modified during iteration", e);         return null;     }          return nodes; })();

Usage:

find_all(xpath_selector);

I would imagine in Cypress, we could accept XPath selectors prefixed with some character. Like in cy.get(...), which accepts aliases with @ prefix. Or maybe we could just use /?

[brian-mann]

What I'm saying is that with Cypress you already have native DOM access to your window/document.

Just use the code you wrote just now and put it in a custom command so that it finds you the DOM element and then pass that off to cy.wrap() so you can then assume the element and then continue to chain off of it via Cypress.

[kamituel]

You mean write a custom command that'll accept an XPath selector and yield a DOM element? Sure, we can do that. And I'm pretty sure I eventually will, as we slowly (but surely) go about migrating our Selenium tests to Cypress ;)

Question is - should Cypress' built-in .get() support XPath? (or maybe have support for it in a separate command...)

I'd estimate that maybe 5% of selectors I would write in Selenium are XPath. Apparently, 20+ people who upvoted the OP have somewhat similar experiences. Is it enough to warrant having it built into Cypress itself? Not for me do decide :)

If you, the Cypress' core team, will decide it's not worth the effort, would you accept a PR that adds XPath support to .get(), or you rather think it belongs elsewhere (an external library)? Do you have plans to provide a "commons" library with stuff that is helpful to a large number of people, but doesn't necessarily belong to the set of core API's?

[jennifer-shehane]

@kamituel We would certainly accept a PR that adds support for XPath! We are not against supporting XPath considering the demand from users here.

Unfortunately, we are a small team, so it is not within our roadmap at this moment to work on this - so @brian-mann suggestions were to help you all with a workaround to use now.

We also feature custom commands in our plugins, so if that's an easier option than doing a PR - let us know and we can share it there.

[bahmutov]

No, but you can chain cy.get, cy.contains and find elements by text, no xpath necessary Sent from my iPhoneOn Aug 9, 2023, at 11:37, Moji ***@***.***> wrote: @bahmutov but cy.contains cannot handle xpath or it can? Isn't it easier to bring back cypress-xpath again? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[mojiAh]

@bahmutov It is true when I'm starting a project, but in our case transitioning from xpath in our existing tests means we should rewrite pretty much all the tests which it doesn't make any sense(it does make sense if we get more value but simply we don't).
Can you explain the reason of deprecating the @cypress/xpath while it did make sense to have it in first place?

[Hellsfoul]

What about //parent::div ?
When I remember correctly, I also had issues with chaining too many cypress commands after cy.get
In additon it is bad, when I have to use the locator multiple times in the code. Then, I have to chain the same cypress commands multiple times.

[scorpyto]

contains

First of all "contains" is so useless when you have multiple similar elements and second -> chain commands are kinda broken, especially with .get command.

[luciarodriguez]

+1 from me, even more now that cypress/xpath is no longer supported.

[Sindhuboston]

+1 for xpath please

[satkunas]

+1 for xpath support

[betole]

1+ for xpath support.

[umeetiusbaar]

+1 for xpath support

[jgonzalezalbisu]

A very large part of the Cypress users will appreciate that this is prioritized. Thanks!

[dil-ddasmohapatra]

+1 for xpath support. It's realy helps a lot while integrating with UI elements.

[mateusjsilva]

+1 for xpath support

[kiranHanumantharaya]

+1 for xpath support.

[wieben]

+1
Coming from Selenium to Cypress testing, what seems hard in Cypress is targeting the below button with comment in this piece of DOM

<div>
    <div data-w-component="my-panel">
        <div>
            <div><span>Panel title to ignore</span></div>
        </div>
        <div>
            <div>
                <button>Click here</button>
            </div>
        </div>
    </div>
    <div data-w-component="my-panel">
        <div>
            <div><span>Panel title</span></div>
        </div>
        <div>
            <div>
                <button>do not Click here</button>
            </div>
        </div>
        <div>
            <div>
                <button>Click here</button> <!-- this button should be targeted -->
            </div>
        </div>
    </div>
</div>

The xpath that would do it

//*[@data-w-component="my-panel" and .//*[text()="This panel"]]//*[text()="Click here"]

It allows the button to be targeted leaving the DOM to be able to change in all possible ways, except changes to data-w-component attribute, or changes to the title of the panel, or changes to the text of the button.
If Cypress can do that with as little constraint on changes to the DOM, then xpath would not be missed that badly.

[wieben]

If Cypress can do that

It can do something close. For the example in above post the below code clicks the desired button. This Cypress test will break if another panel is nested around the title, whereas the Selenium/XPath test would keep passing.

cy.get('[data-w-component="my-panel"]')
            .contains('*', /^This panel$/)
            .parents('[data-w-component="my-panel"]')
            .contains('*', /^Click here$/)
            .click()

[dqiubread]

how has this not been prioritized since the xpath package has been deprecated?

[allancostaquality]

+1 for Cypress to be able to support the use of xpath natively. It seems that the Cy team’s thinking is only geared towards development, where only the simplest and fastest is considered. Xpath is undoubtedly a selector infinitely more powerful than the CSS Selector.

Not to mention teams that would like to migrate from other frameworks and already use xpath and won’t break their heads to change all selectors to CSS.

[alexsch01]

As Salesforce Lightning element testing requires xpath (#29294) and the xpath package has been deprecated, I give +1 for this issue
(I would usually not comment just to give +1, but this issue is different)

[dqiubread]

I've been having same issues as @alexsch01 here with testing Salesforce Lightning Elements and sometimes the shadow elements are not recognized so we have to use xpath. @bahmutov, @jennifer-shehane as Salesforce is custom LWC made by salesforce which will not have data-testids and use a ton of shadow containers is there any support for this?

[bahmutov]

Shadow support is different from xpath. On the personal note regarding xpath selectors: don’t use them. Sent from my iPhoneOn Apr 9, 2024, at 10:14, dqiubread ***@***.***> wrote: I've been having same issues as @alexsch01 here with testing Salesforce Lightning Elements and sometimes the shadow elements are not recognized so we have to use Cypress. @bahmutov, @jennifer-shehane as Salesforce is custom LWC made by salesforce which will not have data-testids and use a ton of shadow containers is there any support for this? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[dqiubread]

@bahmutov, the problem for us is that in our salesforce application its really hard to tell what is and isnt a shadow dom element since salesforce does not explicitly tell you if the element type is inside a shadow element. Instead of for me requesting for a feature on shadow dom elements where idk where the issue is coming from, it would be easier to get xpath support as developer tools already supports it so to me it feels more of a universal solution.

[bahmutov]

Can’t you set cypress option to traverse the shadow borders by default?Sent from my iPhoneOn Apr 9, 2024, at 10:41, dqiubread ***@***.***> wrote: @bahmutov, the problem for us is that in our salesforce application its really hard to tell what is and isnt a shadow dom element since salesforce does not explicitly tell you if the element type is inside a shadow element. Instead of for me requesting for a feature on shadow dom elements where idk where the issue is coming from, it would be easier to get xpath support as developer tools already supports it so to me it feels more of a universal solution. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[dqiubread]

@bahmutov, I turned the includeShadowDom to true to try to resolve the issue, but for some reason for some of the elements like these elements where they have an aura class or aura id in salesforce, it sometimes fails to get the element by the classname or id. I was told that the elements are alls built from salesforce so not sure how much custom stuff the development team has done. The reason my team uses xpath is we are able to get these elements that have these aura classes which cy.get and cy.contains for some reason is not able to handle for Saleforce

[DMaguireKane]

Seems we can add the now removed @cypress/xpath code to our commands.js to get back the cy.xpath(...) functionality, just copy&paste the following into commands.js : @cypress/xpath index.js

• doesn't cover .get and .find but better than no xpath support

[dqiubread]

Seems we can add the now removed @cypress/xpath code to our commands.js to get back the cy.xpath(...) functionality, just copy&paste the following into commands.js : @cypress/xpath index.js

• doesn't cover .get and .find but better than no xpath support

I'm not totally understanding. Wasn't .get and .find never supported? Also Cypress xpath has been removed from the master branch, do you mean we can just make a custom command using what existed of the xpath functionality from before?

[DMaguireKane]

do you mean we can just make a custom command using what existed of the xpath functionality from before?

Yes, copy&paste the full content of the @cypress/xpath index.js linked above (which is linked to the commit before they removed the content) into your command.js and it appears to restore cy.xpath(...) capability (at least in my minimal usage of it so far). It's a crappy workaround for lack of xpath support and potentially will break in the future. I then mentioned that it doesn't cover .get and .find simply as this git issue is specifically about adding xpath support to .get and .find.

[jgonzalezalbisu]

do you mean we can just make a custom command using what existed of the xpath functionality from before?

Yes, copy&paste the full content of the @cypress/xpath index.js linked above (which is linked to the commit before they removed the content) into your command.js and it appears to restore cy.xpath(...) capability (at least in my minimal usage of it so far). It's a crappy workaround for lack of xpath support and potentially will break in the future. I then mentioned that it doesn't cover .get and .find simply as this git issue is specifically about adding xpath support to .get and .find.

@DMaguireKane despite it has been deprecated, is still available for download in the npmjs, why do you suggest we do this to use cy.xpath()command?

[DMaguireKane]

@jgonzalezalbisu Ah, I somehow assumed that it was removed because it no longer worked. Apologies for the wasted notifications. Installing @cypress/xpath and adding require('@cypress/xpath') to command.js still works fine (and add "@cypress/xpath" to tsconfig.json types for intellisense).

[jgonzalezalbisu]

@jgonzalezalbisu Ah, I somehow assumed that it was removed because it no longer worked. Apologies for the wasted notifications. Installing @cypress/xpath and adding require('@cypress/xpath') to command.js still works fine (and add "@cypress/xpath" to tsconfig.json types for intellisense).

@DMaguireKane nothing to apologize for!