RAG in Laravel: embeddings, pgvector, and a personality worth recognizing

🇷🇺 Русская версия

The idea is simple: you have a Telegram channel, a shelf of PDF books, and a couple hundred posts — and you want the model to write as if it were their author. Not "in the style of", but actually: same metaphors, same rhythm, same irony. That takes RAG (Retrieval-Augmented Generation) — fetching the most relevant fragments by vector similarity and feeding them to the model as context.

This article covers building that pipeline in Laravel: embeddings via laravel/ai, storage in PostgreSQL with pgvector, chunking, ingestion, search — and how to turn it all into a recognizable personality.

Stack

• Laravel 12, PHP 8.3
• laravel/ai — the official package: providers, Embeddings::, Agent
• PostgreSQL + pgvector — vectors and cosine search
• The pgvector/pgvector:pg15 image in docker-compose.yml

No third-party "RAG frameworks": everything is plain services, migrations, and artisan commands.

1. Migration: a vector column and an HNSW index

First the extension and the tables:

DB::statement('CREATE EXTENSION IF NOT EXISTS vector');

Schema::create('rag_documents', function (Blueprint $table) {
    $table->id();
    $table->string('source', 32);          // telegram, book, telegraph
    $table->text('source_ref');            // unique source key
    $table->string('title')->nullable();
    $table->string('content_hash', 64);    // sha256 of the content
    $table->unsignedInteger('chunk_count')->default(0);
    // ...
    $table->unique('source_ref');
});

Schema::create('rag_chunks', function (Blueprint $table) {
    $table->id();
    $table->foreignId('document_id')->constrained('rag_documents')->cascadeOnDelete();
    $table->unsignedInteger('seq');
    $table->text('content');
    $table->unique(['document_id', 'seq']);
});

The vector itself goes through a separate DB::statement, because the dimension count only lives in config:

$dimensions = (int) config('rag.embedding.dimensions');

DB::statement("ALTER TABLE rag_chunks ADD COLUMN embedding vector({$dimensions})");
DB::statement('CREATE INDEX rag_chunks_embedding_idx
    ON rag_chunks USING hnsw (embedding vector_cosine_ops)');

HNSW instead of plain ivfflat: the index builds online and needs no rebuild on insert. For a pipeline that receives content in batches, that is exactly what you want.

2. Configuration and DI

Everything configurable lives in config/rag.php:

'embedding' => [
    'provider'   => env('RAG_EMBEDDING_PROVIDER', 'openai'),
    'model'      => env('RAG_EMBEDDING_MODEL', 'text-embedding-3-small'),
    'dimensions' => (int) env('RAG_EMBEDDING_DIMENSIONS', 1536),
    'batch_size' => (int) env('RAG_EMBEDDING_BATCH_SIZE', 100),
],
'chunking' => [
    'target_chars'  => (int) env('RAG_CHUNK_TARGET_CHARS', 1200),
    'overlap_chars' => (int) env('RAG_CHUNK_OVERLAP_CHARS', 200),
],
'retrieval' => [
    'top_k'          => (int) env('RAG_RETRIEVAL_TOP_K', 12),
    'min_similarity' => (float) env('RAG_RETRIEVAL_MIN_SIMILARITY', 0.30),
],

Services that need config are registered as singletons in AppServiceProvider:

$this->app->singleton(Embedder::class, fn () => new Embedder(
    config('rag.embedding.provider'),
    config('rag.embedding.model'),
    (int) config('rag.embedding.dimensions'),
    (int) config('rag.embedding.batch_size'),
    (int) config('rag.embedding.timeout'),
));

$this->app->singleton(Chunker::class, fn (): Chunker => new Chunker(
    (int) config('rag.chunking.target_chars'),
    (int) config('rag.chunking.overlap_chars'),
));

3. Embeddings: a thin wrapper over laravel/ai

The package takes text — and returns a vector. The wrapper exists for exactly one reason: batching. Providers cap request size, and a single document easily splits into a hundred chunks:

public function embedMany(array $texts): array
{
    $vectors = [];

    foreach (array_chunk($texts, $this->batchSize) as $batch) {
        $response = Embeddings::for(array_values($batch))
            ->dimensions($this->dimensions)
            ->timeout($this->timeout)
            ->generate($this->provider, $this->model);

        foreach ($response->embeddings as $embedding) {
            $vectors[] = $embedding;
        }
    }

    return $vectors;
}

Embeddings::fake() ships out of the box and returns random vectors — useful in tests (see the testing section below).

4. Chunking: paragraph boundaries, not character counts

Naive str_split($text, 1200) cuts mid-sentence and destroys meaning. Chunker works differently:

1. normalize line endings and collapse blank runs;
2. split into paragraphs on \n\n;
3. accumulate paragraphs up to target_chars;
4. on overflow — carry an overlap from the tail of the current chunk (overlap_chars chars) so a query landing on the boundary sees both sides;
5. oversized paragraphs get split by sentences (preg_split('/(?<=[.!?…])\s+/u')).

if ($current !== '' && mb_strlen($current) + 2 + mb_strlen($piece) > $this->targetChars) {
    $chunks[] = trim($current);
    $current = $this->overlapChars > 0 ? mb_substr($current, -$this->overlapChars) : '';
}

1200 chars with a 200-char overlap is a working balance: each chunk is self-contained enough to be useful alone, and small enough that cosine search does not drown in noise.

5. Ingestion: deduplication by hash

RagIngestor is the single entry point for content into the database. The key idea is sha256 over the text:

$hash = hash('sha256', $article->content);
$existing = RagDocument::query()->where('source_ref', $article->sourceRef)->first();

if ($existing !== null && ! $force && $existing->content_hash === $hash) {
    $skipped++;
    continue;   // content unchanged — no need to re-embed
}

$document = RagDocument::query()->updateOrCreate(
    ['source_ref' => $article->sourceRef],
    ['source' => $article->source, 'title' => $article->title, 'content_hash' => $hash, /* ... */],
);

$vectors = $this->embedder->embedMany($chunks = $this->chunker->chunk($article->content));
$this->store->replaceChunks($document, $chunks, $vectors);

Hashing saves money: re-running ai:ingest over an already-processed channel costs almost nothing. replaceChunks deletes the document's old chunks and inserts new ones in batches of 50 inside a transaction — all or nothing.

6. Vector search in plain SQL

pgvector needs no ORM magic — the <=> (cosine distance) operator works in a raw query:

return DB::table('rag_chunks as c')
    ->join('rag_documents as d', 'd.id', '=', 'c.document_id')
    ->selectRaw('1 - (c.embedding <=> ?::vector) as similarity', [$literal])
    ->whereRaw('1 - (c.embedding <=> ?::vector) >= ?', [$literal, $minSimilarity])
    ->orderByRaw('c.embedding <=> ?::vector', [$literal])
    ->limit($limit)
    ->get();

Three details people usually trip over:

• ?::vector — the [0.1,-0.2,...] literal is bound as text and cast to vector. No JSON wrappers.
• 1 - distance — pgvector returns distance (0 = identical, 2 = opposite). Similarity comes from subtraction.
• min_similarity — the threshold cuts off accidental matches. Without it, top_k=12 always returns 12 chunks even when the topic is absent from the database, and the model starts hallucinating on top of noise.

7. Retrieval and context assembly

RagRetriever has two methods: retrieve() for debugging and context() for the prompt. The second assembles attributed source blocks within a character budget:

### Source: Title (telegram, https://t.me/s/.../42)

<chunk content>

---

### Source: ...

Attribution is not decoration: the model sees where a fact came from, and the max_context_chars limit (15000) keeps the context from eating the whole window and evicting the instructions.

8. Personality: an agent with a voice

Embeddings without a personality produce a dry bot. What makes texts recognizable is a second component — an agent with a thought-through prompt. In laravel/ai that is the Agent contract:

class KasperskyAgent implements Agent, HasProviderOptions
{
    use Promptable;

    public function __construct(private readonly string $instructions) {}

    public function instructions(): string
    {
        return $this->instructions;
    }

    public function temperature(): float
    {
        return 0.85;
    }
}

Instructions come from the outside — the retriever injects its context, the agent knows nothing by heart. Temperature sits above the default 0.7: personality lives at the edge where variability remains, while factuality is held by the context.

What turns a prompt into a personality rather than an instruction set:

1. The negative list outweighs the positive one. Banning "In this article we will explore", "let's take a look", "in conclusion" is worth more than any "be creative" — AI clichés give the model away instantly.
2. Signature details. crc = 0xEDB88320, the smell of rosin, an old car — specifics that cannot be generated from a template.
3. Rhythm as a requirement. "Alternate choppy short sentences with long, flowing ones" is an instruction that physically breaks the even "model" tone.
4. Few-shot via context. Original texts are passed into the prompt not as examples to copy, but as calibration material: "first silently absorb the rhythm… do not copy verbatim".
5. No self-introduction. The model must not write its own name — the publication context does that.

The assembly lives in ArticleWriter:

public function write(string $topic, ?int $topK = null): string
{
    $context = $this->retriever->context($topic, $topK, config('rag.writer.max_context_chars'));

    $response = (new KasperskyAgent($this->instructions($context)))->prompt(
        "Напиши статью на тему: {$topic}",
        provider: config('rag.writer.provider'),
        model: config('rag.writer.model'),
    );

    return trim($response->text);
}

Context and instructions merge into a single instructions() call, while the user prompt stays short. That makes it easier to test and easier to swap providers: changing RAG_WRITER_MODEL in .env breaks nothing.

9. Commands

The pipeline lives in artisan:

# Telegram channel + books into Postgres
php artisan ai:ingest all

# follow cross-links out of telegra.ph
php artisan ai:ingest-links --depth=2

# inspect which chunks actually match a query
php artisan ai:ask "what about struct alignment" --top-k=5

# generate an article and save it to a file
php artisan ai:write "life under the PE header" --save=storage/app/public/blog/pe/index.md

ai:ask is the main debugging tool: if a query returns garbage, the problem is in chunking or the min_similarity threshold — not in the model.

10. Sources: Kris Kaspersky

The agent's voice is not invented in a prompt — it is assembled from real texts by Kris Kaspersky (1976–2017), a Russian hacker, reverse engineer, and author of books on low-level programming and disassembling. His texts set the rhythm, irony, and way of breaking down topics that ArticleWriter later reproduces.

Telegram channel — https://t.me/KasperskyKris. It is read through the t.me/s/... web preview, so no API token is required — the channel name is set in RAG_TELEGRAM_CHANNEL.

Books (PDF, in Russian) — placed in the RAG_BOOKS_PATH directory; available for download right here:

Both sources are ingested with a single command: php artisan ai:ingest all.

11. Tests without API keys

laravel/ai can fake both embeddings and the agent:

it('generates an article using the retrieved context', function () {
    Embeddings::fake();
    KasperskyAgent::fake(['READY ARTICLE']);

    app(RagIngestor::class)->ingest([/* ... */]);

    $text = app(ArticleWriter::class)->write('about silicon', 5);

    expect($text)->toBe('READY ARTICLE');

    KasperskyAgent::assertPrompted(
        fn ($prompt) => $prompt->contains('about silicon')
            && str_contains($prompt->agent->instructions(), 'silicon')
    );
});

The whole pipeline — ingestion, chunks, retrieval, prompt — is covered by Postgres tests without a single paid request. The only requirements: RefreshDatabase and a running pgvector.

Summary

The full pipeline is five classes and one migration:

Nothing magical: cosine distance in SQL, batches going out, and a prompt that forbids more than it allows. The rest is detail visible only on your own data: tune target_chars, raise min_similarity, give the agent a couple more paragraphs of context. And yes — embeddings power the search, but the personality is what readers come looking for.