Quick Start Guide

Quick Start Overview

This guide will help you get DealAI.lt up and running in your development environment. Follow these steps to set up the platform and verify that all components are working correctly.

Prerequisites Checklist

Before starting, ensure you have:

• Linux server (Ubuntu 20.04+ or Debian 10+)
• PHP 8.0 or higher installed
• PostgreSQL 13+ installed and running
• Elasticsearch 7.x installed and running
• Apache/Nginx web server configured
• WordPress 6.x installed
• Scrapyd server accessible
• Git for version control
• SSH access to your server
• Database credentials
• Administrative privileges

Installation Steps

Step 1: Clone the Theme

cd /var/www/html/wp-content/themes/
git clone <repository-url> products
cd products

Step 2: Configure Database Connection

Create or edit the environment configuration file:

cp .env.example .env
nano .env

Add your database credentials:

DB_HOST=162.55.174.116
DB_PORT=5432
DB_NAME=dealai_products
DB_USER=your_username
DB_PASSWORD=your_password

ELASTICSEARCH_HOST=91.99.113.45
ELASTICSEARCH_PORT=9200

SCRAPYD_HOST=78.56.0.236
SCRAPYD_PORT=6800

Step 3: Verify Database Connection

Run the database verification script:

cd scripts
php check-db.php

Expected output:

✓ Database connection successful
✓ All required tables exist
✓ Product count: 60,000+

Step 4: Verify Elasticsearch Connection

Test the Elasticsearch connection:

php scripts/verify-setup.php

Expected output:

✓ Elasticsearch is accessible
✓ Index 'products' exists
✓ Lithuanian analyzer configured

Step 5: Activate WordPress Theme

1. Log into WordPress Admin Dashboard
2. Navigate to Appearance → Themes
3. Activate the Products theme
4. Verify theme activation

Step 6: Set Up Cron Jobs

Add the following cron jobs for automation:

crontab -e

Add these entries:

# Elasticsearch Auto-Sync (every 5 minutes)
*/5 * * * * /usr/bin/php /var/www/html/wp-content/themes/products/scripts/elasticsearch-auto-sync.php >> /var/log/elasticsearch-sync.log 2>&1

# Product Crawler Manager (every 15 minutes)
*/15 * * * * /usr/bin/php /var/www/html/wp-content/themes/products/scripts/product-crawler-manager.php >> /var/log/crawler-manager.log 2>&1

# Screenshot Manager (daily at 2 AM)
0 2 * * * /usr/bin/php /var/www/html/wp-content/themes/products/scripts/product-screenshot-manager.php >> /var/log/screenshot-manager.log 2>&1

# Category Update (daily at 3 AM)
0 3 * * * /usr/bin/php /var/www/html/wp-content/themes/products/scripts/cron/update-core-categories.php >> /var/log/category-update.log 2>&1

Step 7: Initial Data Sync

Initialize the Elasticsearch index:

cd scripts
php elasticsearch-auto-sync.php

This will start the three-phase synchronization:

1. Initialize - Set up the index
2. Bulk Sync - Process all products in batches
3. Incremental - Keep data up-to-date

Monitor progress in the Elasticsearch Console dashboard.

Step 8: Verify Installation

Check that everything is working:

1. Dashboard Access: Navigate to your WordPress site homepage
2. Product Search: Go to /search/ page
3. Elasticsearch Console: Access /elasticsearch-console/ (admin only)
4. Product Management: Visit /products/ page

Quick Verification Tests

Test 1: Search Functionality

1. Navigate to the search page
2. Enter “laptop” in the search box
3. Verify results are displayed with highlighting
4. Apply filters (category, price range)
5. Check pagination works

Test 2: Scrapyd Dashboard

1. Navigate to the homepage (Scrapyd Control)
2. Verify job statistics are displayed
3. Check that charts render correctly
4. Verify spider list is populated

Test 3: Product Details

1. Go to Product Management page
2. Click on any product
3. Verify all product details display:
   • Images
   • Pricing information
   • Specifications
   • External links

Test 4: Price History

1. Select a product with history data
2. Check that the price chart renders
3. Verify historical data points
4. Test date range filtering

Common First-Time Issues

Database Connection Fails

Problem: “Could not connect to database”

Solution:

# Check PostgreSQL is running
sudo systemctl status postgresql

# Test connection manually
psql -h 162.55.174.116 -U username -d dealai_products

# Verify credentials in .env file

Elasticsearch Not Responding

Problem: “Elasticsearch connection failed”

Solution:

# Check Elasticsearch status
curl http://91.99.113.45:9200/_cluster/health

# Verify network connectivity
ping 91.99.113.45

# Check firewall rules

WordPress Theme Not Activating

Problem: Theme activation fails

Solution:

# Check file permissions
sudo chown -R www-data:www-data /var/www/html/wp-content/themes/products

# Verify PHP version
php -v  # Should be 8.0+

# Check for PHP errors
tail -f /var/log/apache2/error.log

Scrapyd Not Accessible

Problem: “Cannot connect to Scrapyd”

Solution:

# Test Scrapyd endpoint
curl http://78.56.0.236:6800/daemonstatus.json

# Verify Scrapyd is running
ssh [email protected] "sudo systemctl status scrapyd"

Next Steps

Now that you have DealAI.lt running, explore these areas:

Learn the Architecture

Read the Architecture Overview to understand how components interact.

Explore Features

• Product Search - Advanced search implementation
• Price History - Price tracking system
• Category Management - Hierarchical categories

Administrative Tasks

• Scrapyd Control - Managing web scraping
• Elasticsearch Console - Search monitoring
• Product Management - Catalog administration

Development

• File Structure - Navigate the codebase
• API Reference - Integration endpoints
• Development Setup - Development environment

Development Mode

For development, enable WordPress debug mode:

// In wp-config.php
define('WP_DEBUG', true);
define('WP_DEBUG_LOG', true);
define('WP_DEBUG_DISPLAY', false);

Monitor logs:

tail -f /var/www/html/wp-content/debug.log

Performance Tips

For Development

• Use a local Elasticsearch instance
• Enable WordPress query monitor plugin
• Use browser dev tools for frontend debugging

For Production

• Enable OPcache for PHP
• Configure PostgreSQL connection pooling
• Set up WordPress object caching (Redis/Memcached)
• Use a CDN for static assets

Getting Help

If you encounter issues:

1. Check logs: Review error logs in /var/log/ and WordPress debug log
2. Verify setup: Run scripts/verify-setup.php
3. Troubleshooting: See Troubleshooting Guide
4. API docs: Check API Documentation for integration issues

Summary

You should now have:

• ✅ DealAI.lt theme installed and activated
• ✅ Database and Elasticsearch connections verified
• ✅ Cron jobs configured for automation
• ✅ Initial data synchronized
• ✅ All dashboards accessible

Continue with the Architecture Overview to understand the system in depth.