Web Scraping API: Get Scraped Data as CSV, Excel or JSON by URL
Every scraper you build in AIScript gets an HTTP API. One permanent URL returns the file of the latest successful run, so Excel, Google Sheets, Power BI, your store or your own code can always load fresh data without logging in or downloading anything by hand. The same URL can also start a new run and return its result.
How It Works
- Build a scraper and run it once by hand or by the scheduler.
- Open the task, press API and create a key. You can create several keys for different systems and revoke any of them.
- Use the export URL from the dialog. Each call returns the files of the latest successful run of this task.
Typical setup: the schedule refreshes the data every morning, and the API serves the newest file to whoever asks. You do not need to know when the run finished.
Endpoints
| Request | What it returns |
|---|---|
GET /api/v1/export/list | JSON with the run id, its time and the list of files (name, size) |
GET /api/v1/export | Without file: the same JSON list. With file: the file itself |
GET /api/v1/export?file=result.csv | The file result.csv of the latest successful run |
GET /api/v1/export?file=result.csv&run_and_wait_timeout=30 | Starts a new run now, waits up to 30 seconds and returns the file of that run |
Base URL: https://script.mydataprovider.com
Parameters
| Parameter | Description |
|---|---|
key | Your API key. Instead of the query parameter you can send the header X-Api-Key: KEY or Authorization: Bearer KEY, which keeps the key out of server logs. |
file | Exact file name from the list, for example result.xlsx. Without it you get the list. |
download=1 | Send the file as an attachment (browser download). By default it is sent inline. |
run_and_wait_timeout | Run the scraper now and wait up to N seconds (maximum 30). For fast scripts such as a single API call or a few pages. |
| anything else | Ignored. Add &_=timestamp if a tool caches URLs; responses are already sent with no-cache headers. |
Response example
GET /api/v1/export/list
X-Api-Key: YOUR_KEY
{
"ok": true,
"run_id": 1842,
"created_at": "2026-11-10 06:00:04",
"files": [
{"name": "result.csv", "size": 284113},
{"name": "result.xlsx", "size": 196552},
{"name": "result.json", "size": 512870}
],
"truncated": false
}
Errors
| Status | Meaning |
|---|---|
| 401 | Missing, wrong or revoked key |
| 404 | No successful run yet, or no file with this name (the response lists available files) |
| 409 | With run_and_wait_timeout: the task is already running, retry later |
| 500 | With run_and_wait_timeout: the run failed (the response includes the error output) |
| 504 | With run_and_wait_timeout: the run did not finish within the timeout and was stopped; the plain export URL still returns the previous successful file |
Code Examples
curl
curl -H "X-Api-Key: YOUR_KEY" \
"https://script.mydataprovider.com/api/v1/export?file=result.csv" -o result.csv
Python
import io, requests, pandas as pd
URL = "https://script.mydataprovider.com/api/v1/export"
r = requests.get(URL, params={"file": "result.csv"},
headers={"X-Api-Key": "YOUR_KEY"}, timeout=60)
r.raise_for_status()
df = pd.read_csv(io.StringIO(r.content.decode("utf-8-sig")))
print(len(df), "rows")
PHP
$ch = curl_init("https://script.mydataprovider.com/api/v1/export?file=result.json");
curl_setopt_array($ch, [
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HTTPHEADER => ["X-Api-Key: YOUR_KEY"],
]);
$products = json_decode(curl_exec($ch), true);
curl_close($ch);
JavaScript (Node.js)
const res = await fetch(
"https://script.mydataprovider.com/api/v1/export?file=result.json",
{ headers: { "X-Api-Key": process.env.MDP_KEY } }
);
const items = await res.json();
Excel, Google Sheets and Power BI
For spreadsheet tools use the URL with the key as a parameter, because they cannot set headers easily:
https://script.mydataprovider.com/api/v1/export?key=YOUR_KEY&file=result.csv
- Excel: Data > From Web, paste the URL, then Load. In Query properties turn on Refresh every N minutes or Refresh data when opening the file. For a CSV choose UTF-8 (65001) and comma as delimiter.
- Google Sheets:
=IMPORTDATA("https://script.mydataprovider.com/api/v1/export?key=YOUR_KEY&file=result.csv"). Sheets refreshes it about every hour. - Power BI: Get data > Web, paste the URL, and set a scheduled refresh in the Power BI service.
Anyone with this URL can read the file, so share the spreadsheet carefully and revoke the key if it leaks. Need help converting nested JSON for Excel? See JSON to Excel.
Real-Time Scraping vs Latest Export
Most integrations should read the latest export: it answers instantly, does not load the source site and works for scrapers that run for hours. Use run_and_wait_timeout only when you need data from this very moment and the script finishes in seconds, for example checking the price of one product or calling one supplier API. Only one run of a task can be active at a time; parallel calls get status 409.
For long jobs, let the scheduler run the scraper and read the latest export from your system after the scheduled time.
Security and Usage Log
- A key gives read access to the exports of one task only. It cannot change the script, the secrets or the schedule.
- Each key has a usage log: time, IP address, user agent, requested file and response status. Use it to see which systems still read the data.
- The schedule can switch itself off when nobody calls the API for a set number of days, so a forgotten integration does not keep scraping. See scheduler.
- On the free tier and during the trial the API returns the first 50 rows; the response then has
"truncated": trueand the headersX-Total-Rows,X-Returned-RowsandX-Truncated. The paid plan returns full files.
Custom API for a Specific Site
Need a real-time API like "give me the price and stock of this ASIN or SKU" for a marketplace with heavy protection, or guaranteed delivery with monitoring? We build and host such APIs as part of our custom web scraping service. Send us the site and the expected request volume.
Have a scraping or integration task? Let our AI solve it.
Describe what data you need or which systems to connect — our AI builds and runs the scraper or integration for you. No coding required.
Free for 14 days, no card needed · already have an account?
Frequently Asked Questions
Is there a free web scraping API?
Every AIScript task gets an export API, and it works on the free tier and during the 30-day trial with the first 50 rows of each file. The paid plan ($20 a month) returns full files.
Can I get scraped data in real time through the API?
Yes, with run_and_wait_timeout the API starts the scraper and waits up to 30 seconds for the result. For bigger jobs, schedule the scraper and read the latest export, which answers instantly.
Which formats can the API return?
Any file the scraper writes: CSV, Excel (.xlsx), JSON, XML, TXT or store import files. One run can produce several files and you pick one with the file parameter.
How do I connect scraped data to Google Sheets?
Create an API key for the task and use =IMPORTDATA() with the export URL that includes key and file=result.csv. Sheets reloads it automatically.
Is the API key safe in a URL?
It is fine for spreadsheets, but everyone who sees the URL can read the export. In code, send the key in the X-Api-Key or Authorization header, and revoke keys you no longer use.