Lorebook Package Documentation

Package: @character-foundry/lorebook Environment: Node.js and Browser

The @character-foundry/lorebook package provides primitives for parsing, extracting, and managing lorebooks (also known as world info or memory books) across different AI character card formats.

Core Concepts

Architecture

Data Structures

Workflows

Parsing Standalone Lorebooks

Extracting Lorebooks from Cards

Managing Linked Lorebooks

Entry Manipulation

Format Conversion

API Reference

Design Philosophy

Core Concepts

What is a Lorebook?

A lorebook is a collection of entries that provide contextual information to AI models during roleplay. Each entry has:

Character cards can contain lorebooks in two ways:

A key design principle: cards can have multiple separate lorebooks. We never "smoosh" them into one giant lorebook. This preserves:

┌─────────────────────────────────────────────────────────────────┐ │ Card (CCv3Data) │ ├─────────────────────────────────────────────────────────────────┤ │ data.character_book → Primary embedded lorebook │ │ data.extensions.additional_lorebooks → Array of extra books │ │ data.extensions.chub.linked_lorebooks → Linked refs (URLs) │ │ data.extensions.world_infos → More linked refs │ └─────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ LorebookCollection │ ├─────────────────────────────────────────────────────────────────┤ │ embedded: CCv3CharacterBook[] ← Multiple embedded books │ │ linked: LinkedLorebook[] ← Fetched external books │ └─────────────────────────────────────────────────────────────────┘

Module Structure

Module Purpose

parser.ts Detect format and normalize to CCv3

extractor.ts Extract refs and separate entries by source

inserter.ts Stamp entries with source metadata, manage card lorebooks

handler.ts Format conversion, serialization, entry utilities

Data Structures

LorebookFormat

type LorebookFormat = | 'ccv3' // Standard CCv3 character_book | 'sillytavern' // SillyTavern world_info JSON | 'agnai' // Agnai lorebook format | 'risu' // RisuAI .risulorebook | 'wyvern' // Wyvern format | 'unknown';

ParsedLorebook

Result of parsing a standalone lorebook file:

interface ParsedLorebook { book: CCv3CharacterBook; // Normalized to CCv3 originalFormat: LorebookFormat; originalShape: unknown; // Raw data for round-trip }

LorebookRef

Reference to a linked lorebook found in card extensions:

interface LorebookRef { url: string; // Source URL platform: string; // 'chub' | 'risu' | 'janitor' | etc. id?: string; // Platform-specific ID name?: string; // Display name if available }

LinkedLorebook

A fetched/downloaded linked lorebook with source tracking:

interface LinkedLorebook { source: string; // Source URL platform: string; // Platform identifier sourceId?: string; // ID on source platform fetchedAt: string; // ISO timestamp name?: string; // Display name book: CCv3CharacterBook; // The actual content }

EntrySourceMeta

Metadata stamped on entries from linked lorebooks:

interface EntrySourceMeta { linkedFrom: string; // Source URL platform: string; // Platform identifier fetchedAt: string; // When fetched originalEntryId?: string; // Original entry name/ID lorebookName?: string; // Lorebook name for display }

Stored in entry.extensions.lorebookSource.

Complete collection of lorebooks from a card:

interface LorebookCollection { embedded: CCv3CharacterBook[]; // Directly in card linked: LinkedLorebook[]; // From external sources }

Workflows

Parsing Standalone Lorebooks

Parse a standalone lorebook file (JSON) and normalize to CCv3:

import { parseLorebook, detectLorebookFormat } from '@character-foundry/lorebook';

// Auto-detect format and normalize const jsonBuffer = await fs.readFile('lorebook.json'); const { book, originalFormat, originalShape } = parseLorebook(jsonBuffer);

console.log(Detected format: ${originalFormat}); console.log(Entries: ${book.entries.length});

// Or detect format without parsing const format = detectLorebookFormat(JSON.parse(jsonBuffer.toString()));

Supported formats:

import { getLorebookCollection, extractLorebookRefs, extractLinkedEntries } from '@character-foundry/lorebook';

// Get the full collection (embedded + linked) const collection = getLorebookCollection(card);

console.log(Embedded lorebooks: ${collection.embedded.length}); console.log(Linked lorebooks: ${collection.linked.length});

// List embedded lorebook names for (const book of collection.embedded) { console.log(- ${book.name}: ${book.entries.length} entries); }

// Find linked lorebook references (URLs to fetch) const refs = extractLorebookRefs(card); for (const ref of refs) { console.log(Linked: ${ref.url} (${ref.platform})); }

Where linked refs are found:

Complete workflow for adding a linked lorebook to a card:

import { extractLorebookRefs, parseLorebook, createLinkedLorebook, addLinkedLorebookToCard, removeLinkedEntriesBySource, } from '@character-foundry/lorebook';

// 1. Find linked refs in card const refs = extractLorebookRefs(card);

// 2. App fetches the lorebook (not our job!) const lorebookJson = await yourFetchFunction(refs[0].url);

// 3. Parse and stamp with source metadata const { book } = parseLorebook(lorebookJson); const linked = createLinkedLorebook( book, refs[0].url, // sourceUrl refs[0].platform, refs[0].id );

// 4. Add to card (stored in extensions.additional_lorebooks) let updatedCard = addLinkedLorebookToCard(card, linked);

// Later: Remove linked entries when unlinking updatedCard = removeLinkedEntriesBySource(updatedCard, refs[0].url);

Entry Manipulation

Work with individual entries:

import { findEntriesByKeys, findEntryByNameOrId, updateEntry, addEntry, removeEntry, reorderEntries, } from '@character-foundry/lorebook';

// Search entries by trigger keys const matches = findEntriesByKeys(book, ['dragon', 'fire'], { caseSensitive: false, matchAll: false, // Any key matches (OR) });

// Find specific entry const entry = findEntryByNameOrId(book, 'Dragons'); const entryById = findEntryByNameOrId(book, 42);

// Update an entry const updated = updateEntry(book, 'Dragons', { content: 'Updated dragon lore...', enabled: true, });

// Add new entry const withNew = addEntry(book, { keys: ['phoenix', 'fire bird'], content: 'Phoenix lore here...', enabled: true, name: 'Phoenix', priority: 10, });

// Remove entry const removed = removeEntry(book, 'Outdated Entry');

// Reorder entries const reordered = reorderEntries(book, [5, 2, 3, 1, 4]);

Format Conversion

Convert between lorebook formats:

import { convertLorebook, serializeLorebook, serializeParsedLorebook, } from '@character-foundry/lorebook';

// Convert CCv3 to SillyTavern format const stFormat = convertLorebook(book, 'sillytavern');

// Serialize to JSON string const json = serializeLorebook(book, 'sillytavern', undefined, true);

// Round-trip: preserve original format const { book, originalFormat, originalShape } = parseLorebook(input); // ... modify book ... const output = serializeParsedLorebook({ book, originalFormat, originalShape });

Format-specific field preservation:

When converting, format-specific fields are preserved in entry.extensions:

// SillyTavern fields preserved in entry.extensions.sillytavern { selectiveLogic: 0, excludeRecursion: false, probability: 100, depth: 4, // ... etc }

// Agnai fields preserved in entry.extensions.agnai { weight: 1, }

API Reference

Parser Module

parseLorebook(data: BinaryData): ParsedLorebook

Parse a standalone lorebook file. Auto-detects format and normalizes to CCv3.

Detect the format of a lorebook JSON object without parsing.

Normalize any lorebook format to CCv3 character_book structure.

extractLorebookRefs(card: CCv3Data): LorebookRef[]

Extract linked lorebook references (URLs) from card extensions.

Separate embedded entries from linked entries using extensions.lorebookSource.

Get all lorebooks from a card as a collection with embedded and linked separated.

stampEntriesWithSource(book, source): CCv3CharacterBook

Stamp all entries with source metadata for tracking.

Create a LinkedLorebook from a fetched book with source info.

Add a linked lorebook to card's extensions.additional_lorebooks.

Add an embedded lorebook. Sets as character_book if none exists, otherwise adds to additional_lorebooks.

Remove all entries that came from a specific source URL.

Replace a lorebook in card (matches by name).

Rebuild card's lorebooks from a LorebookCollection.

convertLorebook(book, targetFormat, originalShape?): unknown

Convert CCv3 to another format. Preserves original shape for round-trip.

Round-trip a parsed lorebook back to its original format.

Merge two lorebooks into one. Use with caution - design preference is to keep separate.

Reorder entries by specifying ID/name order.

Why Multiple Separate Lorebooks?

Authorship: Each lorebook may have a different creator

Updates: Linked lorebooks can be refreshed from source

Modularity: Users can enable/disable individual lorebooks

Clarity: Clear separation between character lore and world lore

Why Primitives Don't Fetch?

The package provides parsing, extraction, and insertion - not downloading. Reasons:

Diffing and merge conflict UI is complex and app-specific. Primitives provide: