Technology Stack

Technology Overview

DealAI.lt is built on a carefully selected stack of proven, enterprise-grade technologies that work together to provide a robust, scalable, and maintainable platform.

Backend Technologies

WordPress 6.x

Role: Application framework and content management

Why WordPress?

Mature, battle-tested platform
Rich plugin ecosystem
Built-in user management and authentication
Powerful templating system
Strong community support
Easy deployment and maintenance

Key Features Used:

Custom theme development
WordPress hooks and filters
Admin AJAX API
Custom post types (if needed)
WordPress database abstraction
Transients for caching

Version: 6.x (latest stable)

PHP 8.x

Role: Server-side programming language

Why PHP 8?

Native WordPress compatibility
JIT compiler for performance
Named arguments for clarity
Match expressions for cleaner code
Union types for better type safety
Null-safe operator for cleaner code

Key Features Used:

PDO for database connections
cURL for HTTP requests
JSON handling for data exchange
Session management
File system operations
Error handling with exceptions

Requirements: PHP 8.0 or higher

Extensions Required:

- php-pgsql      (PostgreSQL support)
- php-curl       (HTTP requests)
- php-json       (JSON processing)
- php-mbstring   (Multi-byte strings)
- php-xml        (XML parsing)
- php-gd         (Image processing)
- php-zip        (Archive handling)

PostgreSQL 13+

Role: Primary relational database

Why PostgreSQL?

Superior performance for complex queries
JSONB support for flexible data
Full ACID compliance
Advanced indexing options (GiST, GIN, BRIN)
Excellent analytics capabilities
Strong data integrity features
Better performance than MySQL for our use case

Key Features Used:

Full-text search capabilities
JSONB columns for flexible data
Array data types
Time-series data optimization
Advanced indexing strategies
Window functions for analytics
Foreign key constraints

Server Configuration:

Host: 162.55.174.116
Port: 5432
Database: dealai_products

Performance Tuning:

shared_buffers = 25% of RAM
effective_cache_size = 75% of RAM
work_mem = 50MB
maintenance_work_mem = 1GB

Search Technology

Elasticsearch 7.x

Role: Full-text search and analytics engine

Why Elasticsearch?

Lightning-fast full-text search
Real-time indexing and search
Highly scalable and distributed
Rich query DSL
Aggregation framework for analytics
Multi-language support including Lithuanian

Key Features Used:

Custom Lithuanian analyzer with snowball stemming
Multi-field text analysis
Fuzzy queries for typo tolerance
Boolean queries for complex filters
Aggregations for faceted search
Highlighting for search results
Bulk API for efficient indexing

Server Configuration:

Host: 91.99.113.45
Port: 9200
Index: products

Lithuanian Analyzer Configuration:

{
  "analysis": {
    "analyzer": {
      "lithuanian": {
        "type": "snowball",
        "language": "Lithuanian"
      }
    }
  }
}

Index Mapping:

title - text with Lithuanian analyzer
brand - keyword + text
description - text with Lithuanian analyzer
price - float with range indexing
category - keyword for filtering
sku - keyword for exact matching

Data Collection

Scrapyd

Role: Distributed web scraping framework

Why Scrapyd?

Built for Scrapy spiders
RESTful HTTP API
Job scheduling and management
Distributed architecture
Process isolation
Web-based monitoring

Key Features Used:

Spider deployment via API
Job scheduling with priorities
Real-time job monitoring
Log access and retrieval
Multiple concurrent jobs
Resource management

Server Configuration:

Host: 78.56.0.236
Port: 6800

API Endpoints:

/daemonstatus.json - Server status
/listjobs.json - Job listing
/schedule.json - Schedule spider
/cancel.json - Cancel job
/listspiders.json - Available spiders

Python 3.8+

Role: Web scraping scripts

Why Python?

Excellent for web scraping (Scrapy, Beautiful Soup)
Rich data processing libraries
Easy integration with Scrapyd
Strong text processing capabilities

Key Libraries:

Scrapy - Web scraping framework
Requests - HTTP library
lxml - HTML/XML parsing
psycopg2 - PostgreSQL adapter
elasticsearch-py - Elasticsearch client

Frontend Technologies

Tailwind CSS 3.x

Role: Utility-first CSS framework

Why Tailwind?

Rapid UI development
Consistent design system
No CSS bloat with PurgeCSS
Responsive design utilities
Dark mode support built-in
Highly customizable

Key Features Used:

Responsive breakpoints
Flexbox and Grid utilities
Color palette system
Typography utilities
Form styling
Component classes

Flowbite

Role: Tailwind CSS component library

Why Flowbite?

Pre-built UI components
Tailwind-native design
Accessible components
JavaScript interactions included
Modern, clean aesthetics

Components Used:

Navigation menus
Modal dialogs
Dropdown menus
Alert boxes
Card layouts
Tables

jQuery 3.x

Role: JavaScript utility library

Why jQuery?

WordPress compatibility
Easy DOM manipulation
Cross-browser compatibility
AJAX helper functions
Event handling
Wide plugin ecosystem

Key Features Used:

AJAX requests to WordPress
DOM manipulation
Event listeners
Animation effects
Plugin integration

Chart.js 3.x

Role: Data visualization library

Why Chart.js?

Beautiful, responsive charts
Wide variety of chart types
Canvas-based rendering
Animation support
Plugin architecture
Easy to configure

Chart Types Used:

Line charts - Price history over time
Bar charts - Product counts by category
Doughnut charts - Status distribution
Scatter plots - Price vs. discount analysis

lightGallery

Role: Image gallery and lightbox

Why lightGallery?

Touch-friendly interface
Responsive image handling
Thumbnail navigation
Zoom capabilities
Video support
Modular architecture

Features Used:

Product image galleries
Screenshot viewing
Thumbnail navigation
Keyboard controls
Full-screen mode

Development Tools

Git

Role: Version control

Why Git?

Industry standard
Distributed workflow
Branch management
Collaboration features
Integration with hosting platforms

Composer

Role: PHP dependency management

Why Composer?

Standard for PHP projects
Autoloading classes
Version management
Package discovery

Dependencies:

{
  "require": {
    "vlucas/phpdotenv": "^5.0",
    "elasticsearch/elasticsearch": "^7.0"
  }
}

npm

Role: Frontend package management

Why npm?

JavaScript ecosystem standard
Build tool integration
Script running
Wide package availability

Dependencies:

{
  "dependencies": {
    "tailwindcss": "^3.0",
    "chart.js": "^3.0",
    "lightgallery": "^2.0"
  }
}

Infrastructure

Linux (Ubuntu/Debian)

Role: Server operating system

Why Linux?

Stable and secure
Excellent server performance
Strong community support
Wide software compatibility
Cost-effective

Services Required:

Apache/Nginx web server
PostgreSQL database server
Elasticsearch service
Cron daemon
PHP-FPM

Apache HTTP Server / Nginx

Role: Web server

Why Apache/Nginx?

Apache: WordPress compatibility, .htaccess support
Nginx: Better performance, reverse proxy capabilities

Configuration:

PHP-FPM integration
HTTPS/SSL support
URL rewriting
Static file serving
Gzip compression

Cron

Role: Task scheduling

Why Cron?

Native to Linux
Reliable scheduling
Simple configuration
Log output capture
Low resource usage

Scheduled Tasks:

Elasticsearch synchronization
Crawler job management
Screenshot capture
Category updates
Maintenance tasks

Optional Technologies

Redis / Memcached

Role: Object caching

Benefits:

Faster page loads
Reduced database queries
Session storage
Rate limiting

CDN (Cloudflare, AWS CloudFront)

Role: Content delivery

Benefits:

Faster asset delivery
DDoS protection
SSL/TLS termination
Geographic distribution

Supervisor

Role: Process management

Benefits:

Keep services running
Automatic restarts
Log management
Resource monitoring

Version Requirements Summary

Technology	Minimum Version	Recommended
PHP	8.0	8.1+
PostgreSQL	13.0	13.x latest
Elasticsearch	7.0	7.10+
WordPress	6.0	6.x latest
Python	3.8	3.10+
Node.js	14.0	16.x LTS
Ubuntu/Debian	20.04/10	Latest LTS

Technology Selection Rationale

Why Not MySQL?

PostgreSQL was chosen over MySQL for:

Better JSON support
Superior full-text search
Advanced analytics capabilities
Better performance for complex queries
More reliable data integrity

Why Not React/Vue?

Server-side rendering with WordPress was chosen for:

Better SEO out of the box
Faster initial page loads
Simpler deployment
WordPress ecosystem compatibility
Lower complexity for the use case

Why Not MongoDB?

PostgreSQL with JSONB was chosen over MongoDB for:

ACID compliance critical for e-commerce
Better query optimization
Proven reliability
SQL query capabilities
Simpler operations

Next Steps

Installation Guide - Set up the stack
Database Configuration - Configure PostgreSQL
Elasticsearch Setup - Configure search
Development Environment - Local development setup