Skip to content
DealAI.lt Documentation

Scrapyd API

Scrapyd API

Section titled “Scrapyd API”

HTTP REST API for managing web scraping jobs.

Base URL: http://78.56.0.236:6800

Endpoints

Section titled “Endpoints”

GET /daemonstatus.json

Section titled “GET /daemonstatus.json”

Get Scrapyd server status.

Response:

{
    "status": "ok",
    "running": 2,
    "pending": 5,
    "finished": 150
}

GET /listjobs.json

Section titled “GET /listjobs.json”

List all jobs by status.

Parameters:

  • project - Project name

POST /schedule.json

Section titled “POST /schedule.json”

Schedule a new spider job.

Parameters:

  • project - Project name
  • spider - Spider name
  • Additional spider arguments

POST /cancel.json

Section titled “POST /cancel.json”

Cancel a running job.

Parameters:

  • project - Project name
  • job - Job ID

PHP Integration

Section titled “PHP Integration”

File: functions.php

function scrapyd_api_request($endpoint, $params = []) {
    // Implementation
}

Next Steps

Section titled “Next Steps”