Deployment

Deployment Overview

This guide covers the complete deployment of the Elasticsearch integration system, including prerequisites, installation steps, configuration, and verification.

Prerequisites

System Requirements

Server Requirements

• OS: Linux (Ubuntu 20.04+ / Debian 11+)
• PHP: 8.0+ with required extensions
• Memory: 4GB RAM minimum, 8GB recommended
• Storage: 50GB minimum (grows with product data)
• Network: Stable internet connection

Required PHP Extensions

# Check PHP extensions
php -m | grep -E "(pgsql|curl|json|mbstring|xml|zip|gd)"

# Install missing extensions
sudo apt install php8.1-pgsql php8.1-curl php8.1-json php8.1-mbstring php8.1-xml php8.1-zip php8.1-gd

Database Requirements

• PostgreSQL: 13.0+ with read/write access
• Database Storage: 20GB minimum for 60K products
• Connection: Network access from web server

Elasticsearch Requirements

• Elasticsearch: 7.x (7.10+ recommended)
• Java: OpenJDK 11 or 17
• Memory: 4GB+ for Elasticsearch heap
• Storage: 10GB+ for index data

Network Requirements

Service Connectivity

• Elasticsearch Server: 91.99.113.45:9200
• Database Server: 162.55.174.116:5432
• Web Server: HTTP/HTTPS access for WordPress

Firewall Configuration

# Allow required ports
sudo ufw allow 22/tcp    # SSH
sudo ufw allow 80/tcp    # HTTP
sudo ufw allow 443/tcp   # HTTPS

# For Elasticsearch (if remote)
sudo ufw allow from <trusted-ip> to any port 9200

# For PostgreSQL (if remote)
sudo ufw allow from <trusted-ip> to any port 5432

Installation Steps

Step 1: File Deployment

Deploy Core Files

# Navigate to theme directory
cd /var/www/html/wp-content/themes/products/

# Deploy Elasticsearch files
cp inc/elasticsearch.php inc/
cp inc/elasticsearch-console-manager.php inc/
cp scripts/elasticsearch-auto-sync.php scripts/
cp page-search.php .
cp page-elasticsearch-console.php .

# Set permissions
sudo chown -R www-data:www-data inc/ scripts/ page-*.php
sudo chmod -R 755 inc/ scripts/ page-*.php

File Structure Verification

wp-content/themes/products/
├── inc/
│   ├── elasticsearch.php                 ✅ Core ES client
│   └── elasticsearch-console-manager.php ✅ Console manager
├── scripts/
│   └── elasticsearch-auto-sync.php       ✅ Sync engine
├── page-search.php                       ✅ Search template
├── page-elasticsearch-console.php        ✅ Console template
└── functions.php                         ✅ WordPress integration

Step 2: Database Setup

Add Required Column

-- Connect to database
psql -h 162.55.174.116 -U website -d categories_db

-- Add sync tracking column
ALTER TABLE product ADD COLUMN added_search BOOLEAN DEFAULT FALSE;

-- Create performance indexes
CREATE INDEX idx_product_added_search ON product(added_search);
CREATE INDEX idx_product_composite ON product(added_search, id) WHERE title IS NOT NULL;
CREATE INDEX idx_product_updated ON product(updated_at DESC) WHERE added_search = FALSE;

-- Verify column exists
\d product

Database Connection Test

# Test database connection
php -r "
\$conn = pg_connect('host=162.55.174.116 dbname=categories_db user=website password=*** port=5432');
if (\$conn) {
    echo 'Database connection successful\n';
    \$result = pg_query(\$conn, 'SELECT COUNT(*) FROM product WHERE title IS NOT NULL');
    \$row = pg_fetch_row(\$result);
    echo 'Total products: ' . \$row[0] . '\n';
} else {
    echo 'Database connection failed\n';
}
"

Step 3: Elasticsearch Configuration

Test Elasticsearch Connection

# Test Elasticsearch connectivity
curl -X GET "http://91.99.113.45:9200/"

# Expected response:
# {
#   "name" : "node-1",
#   "cluster_name" : "balerina",
#   "version" : {
#     "number" : "7.17.0"
#   }
# }

Create Index with Lithuanian Mapping

# Create index with proper mapping
curl -X PUT "http://91.99.113.45:9200/products" -H 'Content-Type: application/json' -d'
{
  "settings": {
    "number_of_shards": 3,
    "number_of_replicas": 1,
    "analysis": {
      "analyzer": {
        "lithuanian_analyzer": {
          "tokenizer": "standard",
          "filter": [
            "lowercase",
            "asciifolding",
            "lithuanian_snowball"
          ]
        }
      },
      "filter": {
        "lithuanian_snowball": {
          "type": "snowball",
          "language": "Lithuanian"
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "id": {"type": "integer"},
      "title": {
        "type": "text",
        "analyzer": "lithuanian_analyzer"
      },
      "brand": {"type": "keyword"},
      "price": {"type": "float"},
      "category": {"type": "keyword"},
      "availability": {"type": "keyword"}
    }
  }
}'

Step 4: WordPress Configuration

Create WordPress Pages

# Create search page
wp post create --post_type=page --post_title="Product Search" --post_name="search" --post_status=publish --meta_input='{"_wp_page_template":"page-search.php"}'

# Create console page
wp post create --post_type=page --post_title="Elasticsearch Console" --post_name="elasticsearch-console" --post_status=publish --meta_input='{"_wp_page_template":"page-elasticsearch-console.php"}'

Verify Template Assignment

# Check page templates
wp post list --post_type=page --fields=ID,post_title,post_name,meta_value --meta_key=_wp_page_template

Step 5: Cron Job Setup

Add Cron Jobs

# Edit crontab
sudo crontab -e

# Add Elasticsearch sync job
*/15 * * * * /usr/bin/php /var/www/html/wp-content/themes/products/scripts/elasticsearch-auto-sync.php >> /var/log/dealai/elasticsearch-sync.log 2>&1

Create Log Directory

# Create log directory
sudo mkdir -p /var/log/dealai
sudo chown www-data:www-data /var/log/dealai
sudo chmod 755 /var/log/dealai

Verify Cron Service

# Check cron service status
sudo systemctl status cron

# Test cron job manually
sudo -u www-data php /var/www/html/wp-content/themes/products/scripts/elasticsearch-auto-sync.php

Step 6: Initial Data Sync

Run Initial Sync

# Run initial synchronization
cd /var/www/html/wp-content/themes/products/scripts
php elasticsearch-auto-sync.php

# Monitor progress
tail -f elasticsearch-auto-sync.log

Verify Sync Progress

# Check sync state
cat elasticsearch-sync-state.json

# Check Elasticsearch document count
curl -X GET "http://91.99.113.45:9200/products/_count"

# Check database sync status
psql -h 162.55.174.116 -U website -d categories_db -c "
SELECT
  COUNT(*) as total,
  COUNT(CASE WHEN added_search = true THEN 1 END) as indexed,
  ROUND(COUNT(CASE WHEN added_search = true THEN 1 END)::numeric / COUNT(*) * 100, 1) as progress_pct
FROM product WHERE title IS NOT NULL;"

Configuration

Environment Configuration

Create Environment File

# Create .env file
cat > /var/www/html/wp-content/themes/products/.env << EOF
# Elasticsearch Configuration
ELASTICSEARCH_HOST=91.99.113.45
ELASTICSEARCH_PORT=9200
ELASTICSEARCH_INDEX=products

# Database Configuration
DB_HOST=162.55.174.116
DB_PORT=5432
DB_NAME=categories_db
DB_USER=website
DB_PASSWORD=your_secure_password

# Sync Configuration
SYNC_BATCH_SIZE=500
SYNC_MAX_TIME=240
SYNC_MEMORY_LIMIT=256M
EOF

Set File Permissions

# Secure environment file
sudo chown www-data:www-data /var/www/html/wp-content/themes/products/.env
sudo chmod 600 /var/www/html/wp-content/themes/products/.env

WordPress Configuration

Update functions.php

// Add to functions.php
require_once get_template_directory() . '/inc/elasticsearch.php';
require_once get_template_directory() . '/inc/elasticsearch-console-manager.php';

// Add AJAX handlers
add_action('wp_ajax_get_elasticsearch_stats', 'ajax_get_elasticsearch_stats');
add_action('wp_ajax_search_products', 'ajax_search_products');
// ... other AJAX handlers

Enqueue Scripts and Styles

// Add to functions.php
add_action('wp_enqueue_scripts', 'enqueue_elasticsearch_assets');

function enqueue_elasticsearch_assets() {
    if (is_page_template('page-search.php') || is_page_template('page-elasticsearch-console.php')) {
        wp_enqueue_script('jquery');
        wp_localize_script('jquery', 'ajax_object', [
            'ajax_url' => admin_url('admin-ajax.php'),
            'ajax_nonce' => wp_create_nonce('elasticsearch_nonce'),
            'search_nonce' => wp_create_nonce('search_nonce')
        ]);
    }
}

Verification

System Health Checks

1. Database Connection

# Test database connection
php -r "
require_once '/var/www/html/wp-content/themes/products/inc/elasticsearch.php';
\$elasticsearch = new ProductElasticsearch();
echo 'Database connection: ' . (\$elasticsearch->testDatabaseConnection() ? 'OK' : 'FAILED') . '\n';
"

2. Elasticsearch Connection

# Test Elasticsearch connection
php -r "
require_once '/var/www/html/wp-content/themes/products/inc/elasticsearch.php';
\$elasticsearch = new ProductElasticsearch();
echo 'Elasticsearch connection: ' . (\$elasticsearch->isAvailable() ? 'OK' : 'FAILED') . '\n';
echo 'Document count: ' . \$elasticsearch->getDocumentCount() . '\n';
"

3. WordPress Integration

# Test WordPress pages
curl -I http://your-domain.com/search/
curl -I http://your-domain.com/elasticsearch-console/

Functional Testing

1. Search Functionality

# Test search via AJAX
curl -X POST "http://your-domain.com/wp-admin/admin-ajax.php" \
  -d "action=search_products&nonce=test_nonce&query=test" \
  -H "Content-Type: application/x-www-form-urlencoded"

2. Console Functionality

# Test console stats
curl -X POST "http://your-domain.com/wp-admin/admin-ajax.php" \
  -d "action=get_elasticsearch_stats&nonce=test_nonce" \
  -H "Content-Type: application/x-www-form-urlencoded"

3. Sync Process

# Test sync process
cd /var/www/html/wp-content/themes/products/scripts
php elasticsearch-auto-sync.php stats

Performance Optimization

PHP Configuration

Optimize PHP Settings

# Edit php.ini
sudo nano /etc/php/8.1/fpm/php.ini

# Update settings
memory_limit = 512M
max_execution_time = 300
upload_max_filesize = 64M
post_max_size = 64M

Enable OPcache

# Install OPcache
sudo apt install php8.1-opcache

# Configure OPcache
sudo nano /etc/php/8.1/mods-available/opcache.ini

# Add configuration
opcache.enable=1
opcache.memory_consumption=256
opcache.max_accelerated_files=10000

Database Optimization

PostgreSQL Configuration

# Edit postgresql.conf
sudo nano /etc/postgresql/13/main/postgresql.conf

# Update settings
max_connections = 200
shared_buffers = 4GB
work_mem = 50MB
maintenance_work_mem = 1GB

Database Maintenance

-- Run maintenance
VACUUM ANALYZE product;
VACUUM ANALYZE core_category;

-- Update statistics
ANALYZE product;
ANALYZE core_category;

Elasticsearch Optimization

Index Optimization

# Optimize index
curl -X POST "http://91.99.113.45:9200/products/_optimize?max_num_segments=1"

# Check index health
curl -X GET "http://91.99.113.45:9200/_cluster/health?pretty"

Monitoring Setup

Log Monitoring

Set Up Log Rotation

# Create logrotate configuration
sudo nano /etc/logrotate.d/dealai

# Add configuration
/var/log/dealai/*.log {
    daily
    missingok
    rotate 14
    compress
    delaycompress
    notifempty
    create 0640 www-data www-data
}

Monitor Logs

# Monitor sync logs
tail -f /var/log/dealai/elasticsearch-sync.log

# Monitor error logs
tail -f /var/log/apache2/error.log | grep elasticsearch

Health Monitoring

Set Up Health Checks

# Create health check script
cat > /usr/local/bin/elasticsearch-health-check.sh << 'EOF'
#!/bin/bash

# Check Elasticsearch
curl -s http://91.99.113.45:9200/_cluster/health | grep -q '"status":"green"'
if [ $? -ne 0 ]; then
    echo "Elasticsearch health check failed"
    exit 1
fi

# Check database
psql -h 162.55.174.116 -U website -d categories_db -c "SELECT 1" > /dev/null
if [ $? -ne 0 ]; then
    echo "Database health check failed"
    exit 1
fi

echo "All health checks passed"
EOF

chmod +x /usr/local/bin/elasticsearch-health-check.sh

Troubleshooting

Common Issues

1. Database Connection Failed

# Check PostgreSQL status
sudo systemctl status postgresql

# Test connection
psql -h 162.55.174.116 -U website -d categories_db

# Check firewall
sudo ufw status

2. Elasticsearch Connection Failed

# Check Elasticsearch status
curl -X GET "http://91.99.113.45:9200/"

# Check network connectivity
ping 91.99.113.45
telnet 91.99.113.45 9200

3. Sync Process Not Running

# Check cron service
sudo systemctl status cron

# Check cron jobs
sudo crontab -l

# Test manual sync
php /var/www/html/wp-content/themes/products/scripts/elasticsearch-auto-sync.php

4. WordPress Pages Not Loading

# Check file permissions
ls -la /var/www/html/wp-content/themes/products/page-*.php

# Check WordPress error logs
tail -f /var/www/html/wp-content/debug.log

# Check Apache error logs
tail -f /var/log/apache2/error.log

Recovery Procedures

Reset Sync Process

# Reset sync state
rm /var/www/html/wp-content/themes/products/scripts/elasticsearch-sync-state.json

# Reset database flags
psql -h 162.55.174.116 -U website -d categories_db -c "UPDATE product SET added_search = FALSE;"

# Delete Elasticsearch index
curl -X DELETE "http://91.99.113.45:9200/products"

# Restart sync
php /var/www/html/wp-content/themes/products/scripts/elasticsearch-auto-sync.php

Next Steps

• Troubleshooting - Common issues and solutions
• Performance Optimization - Performance tuning
• Monitoring & Management - Console and monitoring
• API Endpoints - API reference