Benchmark

The MVP developed at the moment is Magento 2. In final version the size of HTML and indices will be bigger and results will deviate. At the same time performance will be tuned therefore levelling is expected. The benchmark results shared as they are, not the final.

We will benchmark Lightna product page and category page in two cases:

• Minimal network latency (2ms) - Local machine i5-1145G7 @ 2.60GHz × 8
• Normal network latency (35ms) - Hosting VPS 4 cores

Lightna setup:

1. Configuration storage: Opcache
2. Content, Product, Route storage: Redis
3. Session storage: Files

Server Software:        Apache/2.4.52
Server Port:            80

Document Path:          /olivia-1-4-zip-light-jacket.html
Document Length:        Variable

Concurrency Level:      15
Time taken for tests:   9.134 seconds
Complete requests:      50000
Failed requests:        0
Total transferred:      720694411 bytes
HTML transferred:       712194411 bytes
Requests per second:    5474.06 [#/sec] (mean)
Time per request:       2.740 [ms] (mean)
Time per request:       0.183 [ms] (mean, across all concurrent requests)
Transfer rate:          77053.18 [Kbytes/sec] received
-------------------------------------------------------
CPU Load:               95%
Normal response time    2.7 ms (mean, when no server load)

(Includes realtime interaction with Elasticsearch)

Server Software:        Apache/2.4.52
Server Port:            80

Document Path:          /gear/bags.html
Document Length:        Variable

Concurrency Level:      15
Time taken for tests:   16.829 seconds
Complete requests:      30000
Failed requests:        0
Total transferred:      1051680993 bytes
HTML transferred:       1046580993 bytes
Requests per second:    1782.68 [#/sec] (mean)
Time per request:       8.414 [ms] (mean)
Time per request:       0.561 [ms] (mean, across all concurrent requests)
Transfer rate:          61029.07 [Kbytes/sec] received
-------------------------------------------------------
CPU Load:               98%
Normal response time    3.2 ms (mean, when no server load)

Server Software:        nginx/1.18.0
Server Port:            443
SSL/TLS Protocol:       TLSv1.3,TLS_AES_256_GCM_SHA384,2048,256
Server Temp Key:        X25519 253 bits

Document Path:          /olivia-1-4-zip-light-jacket.html
Document Length:        Variable

Concurrency Level:      50
Time taken for tests:   45.239 seconds
Complete requests:      50000
Failed requests:        0
Total transferred:      720300000 bytes
HTML transferred:       711850000 bytes
Requests per second:    1105.25 [#/sec] (mean)
Time per request:       45.239 [ms] (mean)
Time per request:       0.905 [ms] (mean, across all concurrent requests)
Transfer rate:          15548.99 [Kbytes/sec] received
-------------------------------------------------------
CPU Load:               85%
Normal response time    31.7 ms (mean, when no server load)

(Includes realtime interaction with Elasticsearch)

Server Software:        nginx/1.18.0
Server Port:            443
SSL/TLS Protocol:       TLSv1.3,TLS_AES_256_GCM_SHA384,2048,256
Server Temp Key:        X25519 253 bits

Document Path:          /gear/bags.html
Document Length:        Variable

Concurrency Level:      50
Time taken for tests:   18.872 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      350480000 bytes
HTML transferred:       348790000 bytes
Requests per second:    529.88 [#/sec] (mean)
Time per request:       94.361 [ms] (mean)
Time per request:       1.887 [ms] (mean, across all concurrent requests)
Transfer rate:          18135.98 [Kbytes/sec] received
-------------------------------------------------------
CPU Load:               95%
Normal response time    46.6 ms (mean, when no server load)

(Indexation of Magento sample data: Config, URLs, CMS pages, Categories and 2046 Products)

index opcache config.......... ok
index redis url_rewrite....... ok
index redis cms............... ok
index redis product........... ok
index redis content_page...... ok
index redis content_product... ok
index redis category.......... ok
index redis content_category.. ok

real	0m0.556s
user	0m0.205s
sys	0m0.046s