MyDataProvider » Blog » How to write right Xpath for Web scrapers

How to write right Xpath for Web scrapers

  • by

Xpath is a way for selection required nodes from html or xml document.
If you do not know what is xpath find basic info at wikipedia or try to google.
When I started to learn about web scraping I knew nothing about xpath.
And when I got understanding how this tool allows me to work with xml or html document it became the #1 tool for me.

Any programming language like Java, c#, Python, NodeJs etc. support Xpath.
There is no built-in modules or libraries for these languages except Javascript.
But you can include or import any popular module into your project or script.
And today I want to show you how to write Xpath queries the easiest way.

I found this extension for google chrome “SelectorsHub”
https://chrome.google.com/webstore/detail/selectorshub/ndgimibanhlabgdgjcpbbndiehljcpfh?hl=en

it is amazing! Just install it, restart and you can copy required xpath for any element.

The main benefit of SelectorsHub is that it allows you to copy relative query for element when chrome built-in or the other tools give you full xpath from the document root – that is not suitable for development and not good in practice.