Scrapyd Control Panel
Overview
Section titled “Overview”The Scrapyd Control Panel provides real-time monitoring and management of web scraping operations.
Features
Section titled “Features”Job Monitoring
Section titled “Job Monitoring”- Real-time job statistics (pending, running, finished)
- Job performance metrics
- Error tracking and logging
- Spider management
Visualizations
Section titled “Visualizations”- Job status distribution (doughnut chart)
- Runtime analysis (bar chart)
- Success rate trends (line chart)
- Performance metrics
Controls
Section titled “Controls”- Schedule new jobs
- Cancel running jobs
- View job logs
- Spider configuration
Implementation
Section titled “Implementation”Page Template: front-page.php
JavaScript: /js/dashboard.js
Access
Section titled “Access”Navigate to your WordPress site homepage (admin access required).
API Integration
Section titled “API Integration”Uses Scrapyd HTTP API for real-time data:
/daemonstatus.json- Server status/listjobs.json- Job queue/schedule.json- Schedule jobs/cancel.json- Cancel jobs
Next Steps
Section titled “Next Steps”- Web Scraping - Scraping system details
- Job Monitoring - Detailed job tracking