Entity Index

The indexer is designed for fast data assembly for representation based on existing database data, ensuring high performance. It is not intended for heavy or slow operations, especially API requests or complex calculations. Such operations should be handled as a separate process that prepares and stores data in the database for the indexer to use.

If index data changes only on deployment, consider moving it to the compiler instead of the indexer.

Register Entity

Entities and their indexes are tightly connected. To create an index, the entity must be declared first. An entity can exist without an index, but an index cannot exist without an entity.
config/entity.yaml:

entity:
  my_entity:
    # Required
    entity: MyProject\MyModule\App\Entity\MyEntity
    # Optional
    index: MyProject\MyModule\App\Index\MyEntity
    # Optional
    storage: redis
    # Optional
    versioned: true
    # Optional
    layout: my-entity
    # Optional
    visibility: public
    # Optional
    data: MyProject\MyModule\App\Data\MyEntity

Where:

entity	Entity class
index	Index class
storage	The storage name used for storing the index, can be omitted to use the default storage
versioned	Determines whether the index is global or versioned, true by default
layout	Required only for entities with their own pages on the website, specifies the page layout
visibility	Required only for entities with their own pages on the website. Specifies whether the entity page is public or private. If not specified, the page will be treated as public. This value affects rendering behavior only when Lightna is configured in page cache compatible mode. See the Page Cache Compatibility section on the Configuration page.
data	Required only for entities with their own pages, specifies the entity's Data Object

Declare Entity

<?php
 
declare(strict_types=1);
 
namespace MyProject\MyModule\App\Entity;
 
use Lightna\Engine\App\Entity\EntityA;
 
class MyEntity extends EntityA
{
    public const NAME = 'my_entity';
    public const STORAGE_PREFIX = 'MY.ENTITY_';
    // Optional, true by default
    public const SCOPED = true;
 
    /** @AppConfig(entity/my_entity/storage) */
    protected string $storageName;
}

Where:

EntityA	Base Lightna entity class
NAME	Entity code that matches the code in YAML
STORAGE_PREFIX	Prefix used in storage keys, allowing only one underscore at the end
SCOPED	Determines if the index is shared between scopes or created per scope, can be omitted as it defaults to true in the parent class
$storageName	Specifies the path to the storage config using @AppConfig(entity/my_entity/storage)

Implement Entity Index

<?php
 
declare(strict_types=1);
 
namespace MyProject\MyModule\App\Index;
 
use Lightna\Engine\App\Index\IndexAbstract;
use MyProject\MyModule\App\Entity\MyEntity as MyEntityEntity;
 
class MyEntity extends IndexAbstract
{
    protected MyEntityEntity $entity;
    protected bool $hasRoutes = true;
 
    public function getDataBatch(array $ids): array
    {
        return [];
    }
 
    public function getRoutesBatch(array $ids): array
    {
        return [];
    }
 
    public function scan(string|int $lastId = null): array
    {
        return [];
    }
 
    public function gcCheck(array $ids): array
    {
        return [];
    }
}

Where:

IndexAbstract	Abstract index class
$entity	Entity that will be stored in the index
$hasRoutes	Required only for entities with their own pages on the website, specifies if the entity has routes, false by default
getDataBatch()	Receives a batch of $ids for reindexing and should return data for the specified IDs
getRoutesBatch()	Required only for entities with their own pages on the website. Receives a batch of $ids for reindexing and should return routes for the specified IDs
scan()	Receives $lastId and should return the next batch of IDs
gcCheck()	Receives a batch of existing $ids in the index and should return an array of unavailable IDs to clean

The index class should not implement all the logic itself. It serves as a dispatcher that calls other models, such as a DataProvider for retrieving data in batches and Query model for scanning and cleaning.

When implementing the index, you do not need to clean up unavailable items, as this is handled by IndexAbstract. Simply omitting an unavailable item from the data batch will allow the indexer to remove it automatically. However, you can return more item IDs than those provided in the argument if updating related items of the same entity is necessary.

Declare Entity Data Object

To access index data through a Data Object, you need to declare it and define its data structure. Read more in Data Objects.

Watch Entity Changelog

To refresh the index when database changes occur, register the tables you need to watch.
config/backend/indexer.yaml:

indexer:
  changelog:
    tables:
      my_entity_table: [ ]
      my_entity_related_table: [ additional_field ]

Where:

my_entity_table, my_entity_related_table	Tables to be watched. Changes must be applied using make schema.update
additional_field	By default, the changelog records all row keys and changed fields. If a field must always be included in the changelog, list it here.

Process Entity Changelog

By default, any unprocessed changelog entries are automatically cleaned without triggering an index refresh. To process your changelog records, you need a changelog collector.
config/backend/indexer.yaml:

indexer:
  changelog:
    tables:
      my_entity_table: [ ]
      my_entity_related_table: [ additional_field ]
    batch: 
      collectors: 
        my_entity: MyProject\MyModule\App\Index\Changelog\Collector\MyEntity 

Implement the collector:

<?php
 
declare(strict_types=1);
 
namespace MyProject\MyModule\App\Index\Changelog\Collector;
 
use Lightna\Engine\App\Index\Changelog\Collect;
use Lightna\Engine\App\Index\Changelog\CollectorInterface;
use Lightna\Engine\App\ObjectA;
 
class MyEntity extends ObjectA implements CollectorInterface
{
    protected Collect $collect;
 
    public function collect(string $table, array $changelog): array
    {
        if ($table === 'my_entity_table') {
            return [
                'my_entity' => $this->collect->ids($changelog, 'my_entity_id'),
            ];
        }
 
        return [];
    }
}

Key Aspects:

• The collector's collect method is called for each changelog batch per table. This allows you to apply conditions and trigger entity reindexing based on changes in other application tables.
• The collector returns pairs of entity => IDs for reindexing, enabling it to trigger reindexing for other entities.
• The changelog contains data about changed fields, including their old_value and new_value, allowing the collector to filter out unnecessary changes.
• The changelog aggregates data since the last changelog consumption, ensuring that only one record per table row is stored. In case of multiple changes to the same row, new_value is updated for the affected fields.

Planned

• More composable architecture for entity and index classes with optional class declarations
• More precise control over dependencies and relationships between different items in the indices