AI Web Scraping Tool FAQ

Answers to common questions about AIScript, the MyDataProvider AI web scraping tool: what it can scrape, how results are delivered, pricing, security and what to do when a site is hard to scrape. For step-by-step instructions see the user guide.

Getting Started

Do I need programming skills?

No. You describe the data in plain English: the start URL, the records, the fields and the output format. The AI agent writes, runs and tests the scraper. To change it later, you also write the change in words.

What does a good task look like?

"Collect all products from https://example.com/catalog including subcategories: SKU, name, price, stock, image URL, product URL. Save as result.xlsx." The user guide has a checklist and more examples.

How long does it take to get data?

Building a scraper takes from under a minute for a single feed or page to about 5 to 10 minutes for a catalog with detail pages. In our test, a full 1,000-book catalog scraper with three output files was built in about 6 minutes. Then a run takes as long as the site needs: seconds for one page or API call, minutes for a few thousand products, up to a few hours for very large catalogs.

What happened to Runner?

Runner was our previous tool, where you picked a ready source from a list, configured campaigns, category filters and margin templates. AIScript replaces it. Everything Runner did, such as category selection, price formulas, schedules, CSV, Excel, JSON and XML exports and API access, is now described in the task and built by the AI for any site, not only the sources we had prepared.

What Can Be Scraped

Which websites can I scrape?

Most public websites: online stores, catalogs, directories, classifieds, job boards, real estate and news sites. Also REST APIs, JSON, XML and CSV feeds, and sites behind a login with your own account.

Do you extract e-commerce data?

Yes, this is the most common use: category trees, product names, SKUs, EAN/UPC, brands, prices and sale prices, stock, variants with their own prices, attributes, descriptions, images and reviews. For whole stores built on WooCommerce, Shopify, Magento, PrestaShop or OpenCart you can also use the ready store exporters.

Can I scrape only some categories, brands or SKUs?

Yes. List the category URLs, paste a URL with the site's own filters applied, or upload a file with product URLs, SKUs or EANs. See specific categories and products.

Does it handle JavaScript, infinite scroll and pagination?

Yes. The agent first looks for the data API behind the page. When there is none, the sandbox has a headless browser that renders JavaScript, scrolls and clicks.

What if the site blocks requests?

The agent uses polite request rates, real browser sessions and the site's own data endpoints. Marketplaces such as Amazon or Shein and sites with strong anti-bot systems may need our infrastructure and proxy pool; for those we offer custom web scraping and ready scrapers by site.

How many fields can I extract?

As many as the page or feed contains. Name them in the task in the order you want the columns.

Results and Delivery

Which formats are supported?

Excel (.xlsx), CSV, JSON, XML, TXT and custom formats such as WooCommerce or Shopify import files or your own template. One run can produce several files.

Can I access the data via API?

Yes. Each task has an export API: a permanent URL with a key returns the latest file, from any language (Python, PHP, JavaScript, C#, Go) or from Excel, Google Sheets and Power BI.

Can scraping run automatically?

Yes, with the scheduler: every N hours, daily or weekly.

Can the data go straight into my store?

The scraper can write a ready import file for your platform, or call your store's API with a token you save as a secret. For managed imports with variants and images see products import.

Can it change prices with my margin?

Yes. Describe the rule: a margin percent, a margin by category or price range, currency conversion, rounding to .99. See price rules.

Pricing, Limits and Security

How much does it cost?

$20 per month with 20 AI builds or edits a month, unlimited runs of your saved scrapers, schedules and full result files. The 30-day free trial needs no card. See pricing.

What do I get for free?

On the free tier and during the trial you can build scrapers and see and download the first 50 rows of every result, also through the API.

Are my passwords and API keys safe?

Secrets are stored encrypted, given to the script as environment variables only during a run and masked in logs. Each run is an isolated container separated from other users.

Is web scraping legal?

Collecting publicly available data is generally legal, but you must respect the site's terms, copyright and personal data laws such as GDPR and CCPA. Do not collect personal data without a legal basis and do not overload websites.

Can you build the scraper for me?

Yes. If you prefer not to do it yourself, or need guaranteed delivery and maintenance, our team builds and runs the scraper: see the web scraping service or contact us.