Installation & Setup

Installation Overview

This comprehensive guide walks you through installing and configuring all components of the DealAI.lt platform from scratch. Follow each section carefully to ensure a successful deployment.

System Requirements

Hardware Requirements

Minimum:

• CPU: 4 cores
• RAM: 8GB
• Storage: 100GB SSD
• Network: 100 Mbps

Recommended:

• CPU: 8+ cores
• RAM: 16GB+
• Storage: 250GB NVMe SSD
• Network: 1 Gbps

Software Requirements

• Ubuntu 20.04 LTS or Debian 11
• Root or sudo access
• Internet connectivity

Pre-Installation Checklist

• Fresh Ubuntu/Debian installation
• Static IP address configured
• Domain name pointing to server (optional)
• SSH access configured
• Firewall rules planned
• Backup strategy defined

Step 1: System Preparation

Update System Packages

sudo apt update && sudo apt upgrade -y
sudo apt install -y software-properties-common apt-transport-https ca-certificates curl wget git unzip

Configure Timezone

sudo timedatectl set-timezone Europe/Vilnius

Install Essential Build Tools

sudo apt install -y build-essential libssl-dev libffi-dev python3-dev

Step 2: Install PHP 8.x

Add PHP Repository

sudo add-apt-repository ppa:ondrej/php -y
sudo apt update

Install PHP and Extensions

sudo apt install -y php8.1 php8.1-fpm php8.1-cli php8.1-common \
  php8.1-pgsql php8.1-curl php8.1-json php8.1-mbstring \
  php8.1-xml php8.1-zip php8.1-gd php8.1-intl php8.1-bcmath

Configure PHP

sudo nano /etc/php/8.1/fpm/php.ini

Update these settings:

memory_limit = 512M
upload_max_filesize = 64M
post_max_size = 64M
max_execution_time = 300
max_input_time = 300
date.timezone = Europe/Vilnius

Restart PHP-FPM:

sudo systemctl restart php8.1-fpm

Step 3: Install PostgreSQL

Install PostgreSQL 13+

sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
sudo apt update
sudo apt install -y postgresql-13

Configure PostgreSQL

Edit configuration:

sudo nano /etc/postgresql/13/main/postgresql.conf

Recommended settings:

max_connections = 200
shared_buffers = 4GB
effective_cache_size = 12GB
maintenance_work_mem = 1GB
work_mem = 50MB

Configure access:

sudo nano /etc/postgresql/13/main/pg_hba.conf

Add:

host    dealai_products    dealai_user    0.0.0.0/0    md5

Restart PostgreSQL:

sudo systemctl restart postgresql

Create Database and User

sudo -u postgres psql

CREATE DATABASE dealai_products;
CREATE USER dealai_user WITH ENCRYPTED PASSWORD 'your_secure_password';
GRANT ALL PRIVILEGES ON DATABASE dealai_products TO dealai_user;
\q

Import Database Schema

psql -h localhost -U dealai_user -d dealai_products -f /path/to/schema.sql

Step 4: Install Elasticsearch

Import GPG Key

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

Add Repository

echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-7.x.list

Install Elasticsearch

sudo apt update
sudo apt install -y elasticsearch

Configure Elasticsearch

sudo nano /etc/elasticsearch/elasticsearch.yml

Configuration:

cluster.name: dealai-cluster
node.name: node-1
network.host: 0.0.0.0
http.port: 9200
discovery.type: single-node

# Memory settings
bootstrap.memory_lock: true

# Indices
index.number_of_shards: 3
index.number_of_replicas: 1

Set JVM heap:

sudo nano /etc/elasticsearch/jvm.options

-Xms4g
-Xmx4g

Start Elasticsearch

sudo systemctl enable elasticsearch
sudo systemctl start elasticsearch

Verify Installation

curl -X GET "localhost:9200/"

Step 5: Install Web Server

Option A: Apache

sudo apt install -y apache2 libapache2-mod-php8.1

# Enable modules
sudo a2enmod rewrite ssl headers proxy_fcgi setenvif
sudo a2enconf php8.1-fpm

# Configure virtual host
sudo nano /etc/apache2/sites-available/dealai.conf

<VirtualHost *:80>
    ServerName dealai.lt
    DocumentRoot /var/www/html

    <Directory /var/www/html>
        AllowOverride All
        Require all granted
    </Directory>

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

sudo a2ensite dealai.conf
sudo systemctl restart apache2

Option B: Nginx

sudo apt install -y nginx

sudo nano /etc/nginx/sites-available/dealai

server {
    listen 80;
    server_name dealai.lt;
    root /var/www/html;
    index index.php index.html;

    location / {
        try_files $uri $uri/ /index.php?$args;
    }

    location ~ \.php$ {
        include snippets/fastcgi-php.conf;
        fastcgi_pass unix:/run/php/php8.1-fpm.sock;
    }

    location ~ /\.ht {
        deny all;
    }
}

sudo ln -s /etc/nginx/sites-available/dealai /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl restart nginx

Step 6: Install WordPress

Download WordPress

cd /tmp
wget https://wordpress.org/latest.tar.gz
tar -xzf latest.tar.gz
sudo mv wordpress/* /var/www/html/
sudo chown -R www-data:www-data /var/www/html
sudo chmod -R 755 /var/www/html

Configure WordPress

cd /var/www/html
cp wp-config-sample.php wp-config.php
nano wp-config.php

Update database settings:

define('DB_NAME', 'wordpress_db');
define('DB_USER', 'wp_user');
define('DB_PASSWORD', 'your_password');
define('DB_HOST', 'localhost');

// For separate PostgreSQL (requires plugin)
// Configure in theme functions.php

Complete WordPress Installation

Visit http://your-domain/wp-admin/install.php and follow the setup wizard.

Step 7: Install DealAI Theme

Clone Theme Repository

cd /var/www/html/wp-content/themes/
git clone <your-repository-url> products
cd products

Install Composer Dependencies

curl -sS https://getcomposer.org/installer | php
php composer.phar install

Configure Environment

cp .env.example .env
nano .env

# PostgreSQL Configuration
DB_HOST=162.55.174.116
DB_PORT=5432
DB_NAME=dealai_products
DB_USER=dealai_user
DB_PASSWORD=your_password

# Elasticsearch Configuration
ELASTICSEARCH_HOST=91.99.113.45
ELASTICSEARCH_PORT=9200
ELASTICSEARCH_INDEX=products

# Scrapyd Configuration
SCRAPYD_HOST=78.56.0.236
SCRAPYD_PORT=6800
SCRAPYD_PROJECT=dealai_scrapers

Set Permissions

sudo chown -R www-data:www-data /var/www/html/wp-content/themes/products
sudo chmod -R 755 /var/www/html/wp-content/themes/products

Activate Theme

1. Log into WordPress admin
2. Go to Appearance → Themes
3. Activate “Products” theme

Step 8: Configure Elasticsearch Index

Create Index with Lithuanian Analyzer

cd /var/www/html/wp-content/themes/products/scripts
php -r "require 'elasticsearch-setup.php';"

Or manually:

curl -X PUT "localhost:9200/products" -H 'Content-Type: application/json' -d'
{
  "settings": {
    "analysis": {
      "analyzer": {
        "lithuanian": {
          "type": "snowball",
          "language": "Lithuanian"
        }
      }
    },
    "number_of_shards": 3,
    "number_of_replicas": 1
  },
  "mappings": {
    "properties": {
      "title": {
        "type": "text",
        "analyzer": "lithuanian"
      },
      "brand": {
        "type": "keyword"
      },
      "price": {
        "type": "float"
      },
      "category": {
        "type": "keyword"
      }
    }
  }
}'

Step 9: Initial Data Sync

Run Initial Sync

cd /var/www/html/wp-content/themes/products/scripts
php elasticsearch-auto-sync.php

Monitor progress:

tail -f elasticsearch-auto-sync.log

Step 10: Configure Cron Jobs

Edit Crontab

sudo crontab -e

Add Scheduled Tasks

# Elasticsearch Auto-Sync (every 5 minutes)
*/5 * * * * /usr/bin/php /var/www/html/wp-content/themes/products/scripts/elasticsearch-auto-sync.php >> /var/log/dealai/elasticsearch-sync.log 2>&1

# Product Crawler Manager (every 15 minutes)
*/15 * * * * /usr/bin/php /var/www/html/wp-content/themes/products/scripts/product-crawler-manager.php >> /var/log/dealai/crawler-manager.log 2>&1

# Screenshot Manager (daily at 2 AM)
0 2 * * * /usr/bin/php /var/www/html/wp-content/themes/products/scripts/product-screenshot-manager.php >> /var/log/dealai/screenshot-manager.log 2>&1

# Category Update (daily at 3 AM)
0 3 * * * /usr/bin/php /var/www/html/wp-content/themes/products/scripts/cron/update-core-categories.php >> /var/log/dealai/category-update.log 2>&1

Create Log Directory

sudo mkdir -p /var/log/dealai
sudo chown www-data:www-data /var/log/dealai

Step 11: Configure Firewall

UFW Configuration

sudo ufw allow 22/tcp    # SSH
sudo ufw allow 80/tcp    # HTTP
sudo ufw allow 443/tcp   # HTTPS
sudo ufw enable

For Elasticsearch (if remote):

sudo ufw allow from <trusted-ip> to any port 9200

Step 12: SSL/TLS Configuration (Optional)

Using Let’s Encrypt

sudo apt install -y certbot python3-certbot-apache  # For Apache
# OR
sudo apt install -y certbot python3-certbot-nginx   # For Nginx

sudo certbot --apache -d dealai.lt -d www.dealai.lt  # Apache
# OR
sudo certbot --nginx -d dealai.lt -d www.dealai.lt   # Nginx

Auto-renewal:

sudo certbot renew --dry-run

Step 13: Verification

Test Database Connection

cd /var/www/html/wp-content/themes/products/scripts
php check-db.php

Test Elasticsearch

curl -X GET "localhost:9200/_cat/indices?v"
curl -X GET "localhost:9200/products/_count"

Test WordPress

Visit: http://your-domain/

Test Search

Visit: http://your-domain/search/

Test Dashboards

• Scrapyd Control: http://your-domain/
• Elasticsearch Console: http://your-domain/elasticsearch-console/
• Product Management: http://your-domain/products/

Post-Installation

Security Hardening

# Remove default WordPress files
rm /var/www/html/readme.html
rm /var/www/html/license.txt

# Restrict wp-config.php
chmod 600 /var/www/html/wp-config.php

Performance Optimization

Install caching plugin:

• W3 Total Cache
• WP Super Cache
• Redis Object Cache

Monitoring Setup

Install monitoring tools:

• New Relic / Datadog for APM
• Prometheus + Grafana for metrics
• ELK Stack for log aggregation

Troubleshooting

See the Troubleshooting Guide for common issues and solutions.

Next Steps

• Architecture Overview - Understand the system
• Core Systems - Explore components
• Development Setup - Configure development environment