docs: add invention refinements implementation plan
8 tasks covering: load-time re-registration fix, reasoning field parsing, revised LLM prompt, full stat passthrough, token usage tracking, race condition guard, and integration tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,871 @@
|
||||
# Invention System Refinements Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Fix duplicate inventions across restarts, add stat-driven invention selection via revised LLM prompt, and add token usage tracking.
|
||||
|
||||
**Architecture:** Six independent changes: (1) re-register inventions on load, (2) parse `reasoning` field, (3) revised LLM prompt with full stats, (4) pass all 10 stats from inventionSystem, (5) token usage capture from OpenRouter, (6) global pendingItemIds race guard. All changes are additive — existing tests should continue passing with minor updates.
|
||||
|
||||
**Tech Stack:** TypeScript, vitest, OpenRouter API (fetch-based)
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Re-register inventions into item/recipe registries on load
|
||||
|
||||
This is the root cause of duplicates. On server restart, invented items exist in the timeline but not in the registries, so the duplicate check passes.
|
||||
|
||||
**Files:**
|
||||
- Modify: `server/src/game/GameLoop.ts:104-139` (loadFromSave method)
|
||||
- Test: `server/src/game/__tests__/inventionReload.test.ts` (new)
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
Create `server/src/game/__tests__/inventionReload.test.ts`:
|
||||
|
||||
```typescript
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { ItemRegistry } from '../../industry/itemRegistry.js';
|
||||
import { RecipeRegistry } from '../../industry/recipeRegistry.js';
|
||||
import type { InventionEntry } from '../../industry/inventionTimeline.js';
|
||||
|
||||
// Extracted helper we'll create in Step 3
|
||||
import { reregisterInventions } from '../inventionLoader.js';
|
||||
|
||||
describe('reregisterInventions', () => {
|
||||
it('registers loaded inventions into item and recipe registries', () => {
|
||||
const itemRegistry = ItemRegistry.createDefault();
|
||||
const recipeRegistry = RecipeRegistry.createDefault();
|
||||
|
||||
const inventions: InventionEntry[] = [{
|
||||
itemId: 'rope',
|
||||
inventorEntityId: 'e1' as any,
|
||||
inventorName: 'Gwendolyn',
|
||||
tick: 100,
|
||||
day: 1,
|
||||
name: 'Rope',
|
||||
category: 'material',
|
||||
inputs: [{ itemId: 'log', quantity: 2 }],
|
||||
workshopType: null,
|
||||
toolRequired: null,
|
||||
}];
|
||||
|
||||
reregisterInventions(inventions, itemRegistry, recipeRegistry);
|
||||
|
||||
expect(itemRegistry.get('rope')).toBeDefined();
|
||||
expect(itemRegistry.get('rope')!.source).toBe('invented');
|
||||
expect(itemRegistry.get('rope')!.name).toBe('Rope');
|
||||
expect(recipeRegistry.get('craft_rope')).toBeDefined();
|
||||
expect(recipeRegistry.get('craft_rope')!.source).toBe('invented');
|
||||
});
|
||||
|
||||
it('registers structure inventions with build_ prefix', () => {
|
||||
const itemRegistry = ItemRegistry.createDefault();
|
||||
const recipeRegistry = RecipeRegistry.createDefault();
|
||||
|
||||
const inventions: InventionEntry[] = [{
|
||||
itemId: 'kiln',
|
||||
inventorEntityId: 'e1' as any,
|
||||
inventorName: 'Smithy',
|
||||
tick: 200,
|
||||
day: 2,
|
||||
name: 'Kiln',
|
||||
category: 'structure',
|
||||
inputs: [{ itemId: 'stone', quantity: 5 }],
|
||||
workshopType: 'kiln',
|
||||
toolRequired: null,
|
||||
}];
|
||||
|
||||
reregisterInventions(inventions, itemRegistry, recipeRegistry);
|
||||
|
||||
expect(recipeRegistry.get('build_kiln')).toBeDefined();
|
||||
expect(recipeRegistry.get('build_kiln')!.workshopType).toBe('kiln');
|
||||
});
|
||||
|
||||
it('skips inventions that already exist in registry (idempotent)', () => {
|
||||
const itemRegistry = ItemRegistry.createDefault();
|
||||
const recipeRegistry = RecipeRegistry.createDefault();
|
||||
|
||||
const inventions: InventionEntry[] = [{
|
||||
itemId: 'rope',
|
||||
inventorEntityId: 'e1' as any,
|
||||
inventorName: 'Gwendolyn',
|
||||
tick: 100,
|
||||
day: 1,
|
||||
name: 'Rope',
|
||||
category: 'material',
|
||||
inputs: [{ itemId: 'log', quantity: 2 }],
|
||||
workshopType: null,
|
||||
toolRequired: null,
|
||||
}];
|
||||
|
||||
// Register twice — should not throw
|
||||
reregisterInventions(inventions, itemRegistry, recipeRegistry);
|
||||
reregisterInventions(inventions, itemRegistry, recipeRegistry);
|
||||
|
||||
expect(itemRegistry.get('rope')).toBeDefined();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npm -w server run test -- --run server/src/game/__tests__/inventionReload.test.ts`
|
||||
Expected: FAIL — `inventionLoader.js` does not exist
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
Create `server/src/game/inventionLoader.ts`:
|
||||
|
||||
```typescript
|
||||
import type { ItemRegistry } from '../industry/itemRegistry.js';
|
||||
import type { RecipeRegistry } from '../industry/recipeRegistry.js';
|
||||
import type { InventionEntry } from '../industry/inventionTimeline.js';
|
||||
|
||||
export function reregisterInventions(
|
||||
inventions: InventionEntry[],
|
||||
itemRegistry: ItemRegistry,
|
||||
recipeRegistry: RecipeRegistry,
|
||||
): void {
|
||||
for (const entry of inventions) {
|
||||
// Skip if already registered (idempotent)
|
||||
if (itemRegistry.get(entry.itemId)) continue;
|
||||
|
||||
itemRegistry.register({
|
||||
id: entry.itemId,
|
||||
name: entry.name,
|
||||
description: '', // not stored in invention DB — empty is fine
|
||||
category: entry.category,
|
||||
source: 'invented',
|
||||
inventedBy: {
|
||||
entityId: entry.inventorEntityId,
|
||||
name: entry.inventorName,
|
||||
tick: entry.tick,
|
||||
day: entry.day,
|
||||
},
|
||||
});
|
||||
|
||||
const recipePrefix = entry.category === 'structure' ? 'build_' : 'craft_';
|
||||
const recipeId = `${recipePrefix}${entry.itemId}`;
|
||||
|
||||
if (!recipeRegistry.get(recipeId)) {
|
||||
recipeRegistry.register({
|
||||
id: recipeId,
|
||||
outputItemId: entry.itemId,
|
||||
outputQuantity: 1,
|
||||
inputs: entry.inputs,
|
||||
workshopType: entry.workshopType ?? undefined,
|
||||
toolRequired: entry.toolRequired ?? undefined,
|
||||
source: 'invented',
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `npm -w server run test -- --run server/src/game/__tests__/inventionReload.test.ts`
|
||||
Expected: PASS
|
||||
|
||||
**Step 5: Wire into GameLoop.loadFromSave**
|
||||
|
||||
In `server/src/game/GameLoop.ts`, add import:
|
||||
```typescript
|
||||
import { reregisterInventions } from './inventionLoader.js';
|
||||
```
|
||||
|
||||
After line 120 (`inventionTimeline.loadFromPersistence(inventions);`), add:
|
||||
```typescript
|
||||
// Re-register invented items/recipes so duplicate checks work
|
||||
const itemRegistry = this.world.getSingleton<ItemRegistry>('itemRegistry')!;
|
||||
const recipeRegistry = this.world.getSingleton<RecipeRegistry>('recipeRegistry')!;
|
||||
reregisterInventions(inventions, itemRegistry, recipeRegistry);
|
||||
```
|
||||
|
||||
**Step 6: Run all tests**
|
||||
|
||||
Run: `npm -w server run test`
|
||||
Expected: All pass
|
||||
|
||||
**Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add server/src/game/inventionLoader.ts server/src/game/__tests__/inventionReload.test.ts server/src/game/GameLoop.ts
|
||||
git commit -m "fix: re-register inventions into registries on save load
|
||||
|
||||
Inventions loaded from DB were only added to the timeline, not
|
||||
the item/recipe registries. This allowed duplicate inventions
|
||||
after server restarts since the validation check was empty."
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Add `reasoning` field to invention parser
|
||||
|
||||
**Files:**
|
||||
- Modify: `server/src/industry/inventionParser.ts:1-34`
|
||||
- Modify: `server/src/industry/inventionValidator.ts:1-10` (RawInvention type)
|
||||
- Test: `server/src/industry/__tests__/inventionParser.test.ts`
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
Add to `server/src/industry/__tests__/inventionParser.test.ts`:
|
||||
|
||||
```typescript
|
||||
it('parses reasoning field when present', () => {
|
||||
const json = '{"name": "Rope", "description": "Twisted fibers", "reasoning": "High dexterity suits crafting", "category": "material", "inputs": [{"itemId": "log", "quantity": 2}]}';
|
||||
const result = parseInventionResponse(json);
|
||||
expect(result).not.toBeNull();
|
||||
expect(result!.reasoning).toBe('High dexterity suits crafting');
|
||||
});
|
||||
|
||||
it('defaults reasoning to undefined when absent', () => {
|
||||
const json = '{"name": "Rope", "description": "Twisted fibers", "category": "material", "inputs": [{"itemId": "log", "quantity": 2}]}';
|
||||
const result = parseInventionResponse(json);
|
||||
expect(result).not.toBeNull();
|
||||
expect(result!.reasoning).toBeUndefined();
|
||||
});
|
||||
```
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npm -w server run test -- --run server/src/industry/__tests__/inventionParser.test.ts`
|
||||
Expected: FAIL — `reasoning` property not on type
|
||||
|
||||
**Step 3: Update RawInvention type and parser**
|
||||
|
||||
In `server/src/industry/inventionValidator.ts`, add `reasoning` to `RawInvention`:
|
||||
```typescript
|
||||
export interface RawInvention {
|
||||
name: string;
|
||||
description: string;
|
||||
reasoning?: string;
|
||||
category: string;
|
||||
inputs: { itemId: string; quantity: number }[];
|
||||
workshopType?: string | null;
|
||||
toolRequired?: string | null;
|
||||
}
|
||||
```
|
||||
|
||||
In `server/src/industry/inventionParser.ts`, add to the return object (after `toolRequired` line):
|
||||
```typescript
|
||||
reasoning: typeof parsed.reasoning === 'string' ? parsed.reasoning : undefined,
|
||||
```
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `npm -w server run test -- --run server/src/industry/__tests__/inventionParser.test.ts`
|
||||
Expected: PASS
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add server/src/industry/inventionParser.ts server/src/industry/inventionValidator.ts server/src/industry/__tests__/inventionParser.test.ts
|
||||
git commit -m "feat: parse reasoning field from invention LLM response"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Revise invention LLM prompt
|
||||
|
||||
**Files:**
|
||||
- Modify: `server/src/llm/templates.ts:45-61` (invention template)
|
||||
|
||||
**Step 1: Update the invention template**
|
||||
|
||||
Replace the `invention` entry in `server/src/llm/templates.ts`:
|
||||
|
||||
```typescript
|
||||
invention: {
|
||||
name: 'invention',
|
||||
systemPrompt:
|
||||
'You are an inventor in a medieval fantasy village simulation. ' +
|
||||
'Given available materials and an NPC\'s stats, consider 3 possible inventions, ' +
|
||||
'then select the one that best fits the NPC\'s personality. ' +
|
||||
'Respond ONLY with a valid JSON object. No explanation, no markdown, just JSON.',
|
||||
userPrompt:
|
||||
'{{npcName}} is having a creative moment.\n\n' +
|
||||
'Stats (each ranges 3-18, 10 is average):\n' +
|
||||
'Strength:{{strength}} Dexterity:{{dexterity}} Constitution:{{constitution}} Intelligence:{{intelligence}}\n' +
|
||||
'Perception:{{perception}} Sociability:{{sociability}} Courage:{{courage}} Curiosity:{{curiosity}}\n' +
|
||||
'Empathy:{{empathy}} Temperament:{{temperament}}\n\n' +
|
||||
'Available materials:\n{{materials}}\n\n' +
|
||||
'Known items (do not reinvent):\n{{knownItems}}\n\n' +
|
||||
'Consider 3 possible inventions using 2-3 existing materials, then pick\n' +
|
||||
'the one that best matches this NPC\'s personality and stats.\n' +
|
||||
'Use Title Case for the item name.\n\n' +
|
||||
'Respond with JSON:\n' +
|
||||
'{"name": "Item Name", "description": "brief flavor text", ' +
|
||||
'"reasoning": "why this suits the NPC", ' +
|
||||
'"category": "resource|tool|material|structure", ' +
|
||||
'"inputs": [{"itemId": "existing_item_id", "quantity": N}], ' +
|
||||
'"workshopType": null, "toolRequired": null}',
|
||||
},
|
||||
```
|
||||
|
||||
**Step 2: Run all tests**
|
||||
|
||||
Run: `npm -w server run test`
|
||||
Expected: Some invention tests may fail because the template variables changed (e.g. `existingInventions` → `knownItems`, added stat variables). These will be fixed in Task 4.
|
||||
|
||||
**Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add server/src/llm/templates.ts
|
||||
git commit -m "feat: revise invention prompt for stat-driven selection
|
||||
|
||||
Prompt now includes full 10-stat block with readable names and
|
||||
scale context. Instructs LLM to consider 3 ideas and pick the
|
||||
best match for the NPC's personality. Uses Title Case instruction."
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Pass full stats from inventionSystem to LLM
|
||||
|
||||
**Files:**
|
||||
- Modify: `server/src/systems/inventionSystem.ts:60-79` (the generate call and variable building)
|
||||
- Modify: `server/src/systems/__tests__/inventionSystem.test.ts`
|
||||
- Modify: `server/src/systems/__tests__/inventionIntegration.test.ts`
|
||||
|
||||
**Step 1: Update inventionSystem.ts**
|
||||
|
||||
Replace the variable-building and generate call (lines 61-79) with:
|
||||
|
||||
```typescript
|
||||
const name = world.getComponent<string>(entityId, 'name') ?? 'Unknown';
|
||||
const allItems = itemRegistry.getAll();
|
||||
const materialsList = allItems
|
||||
.map(i => `- ${i.id} (${i.category}): ${i.name}`)
|
||||
.join('\n');
|
||||
|
||||
const knownItems = allItems
|
||||
.map(i => i.name)
|
||||
.join(', ');
|
||||
|
||||
// Gather all 10 stats for personality-driven invention selection
|
||||
const str = getEffectiveStat(world, entityId, 'strength');
|
||||
const dex = getEffectiveStat(world, entityId, 'dexterity');
|
||||
const con = getEffectiveStat(world, entityId, 'constitution');
|
||||
const per = getEffectiveStat(world, entityId, 'perception');
|
||||
const soc = getEffectiveStat(world, entityId, 'sociability');
|
||||
const cou = getEffectiveStat(world, entityId, 'courage');
|
||||
const cur = getEffectiveStat(world, entityId, 'curiosity');
|
||||
const emp = getEffectiveStat(world, entityId, 'empathy');
|
||||
const tmp = getEffectiveStat(world, entityId, 'temperament');
|
||||
|
||||
pendingEntities.add(entityId);
|
||||
|
||||
llmService.generate('invention', {
|
||||
npcName: name,
|
||||
strength: String(str),
|
||||
dexterity: String(dex),
|
||||
constitution: String(con),
|
||||
intelligence: String(intelligence),
|
||||
perception: String(per),
|
||||
sociability: String(soc),
|
||||
courage: String(cou),
|
||||
curiosity: String(curiosity),
|
||||
empathy: String(emp),
|
||||
temperament: String(tmp),
|
||||
materials: materialsList,
|
||||
knownItems,
|
||||
}).then(response => {
|
||||
```
|
||||
|
||||
Note: `intelligence` and `curiosity` are already computed above for the probability check, so reuse them.
|
||||
|
||||
**Step 2: Update inventionSystem.test.ts**
|
||||
|
||||
In the test `'triggers LLM call when random check passes and NPC has items'`, update the expect to check for new variables:
|
||||
|
||||
```typescript
|
||||
expect(llm.generate).toHaveBeenCalledWith('invention', expect.objectContaining({
|
||||
npcName: 'TestNPC',
|
||||
strength: '10',
|
||||
intelligence: '18',
|
||||
curiosity: '18',
|
||||
empathy: '10',
|
||||
knownItems: expect.any(String),
|
||||
}));
|
||||
```
|
||||
|
||||
**Step 3: Update inventionIntegration.test.ts**
|
||||
|
||||
Change the assertion on line 106 from:
|
||||
```typescript
|
||||
expect(secondCallArgs.materials).toContain('rope');
|
||||
```
|
||||
to:
|
||||
```typescript
|
||||
expect(secondCallArgs.materials).toContain('rope');
|
||||
expect(secondCallArgs.knownItems).toContain('Rope');
|
||||
```
|
||||
|
||||
Also verify that `existingInventions` is no longer passed and `knownItems` is used instead.
|
||||
|
||||
**Step 4: Run all tests**
|
||||
|
||||
Run: `npm -w server run test`
|
||||
Expected: All pass
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add server/src/systems/inventionSystem.ts server/src/systems/__tests__/inventionSystem.test.ts server/src/systems/__tests__/inventionIntegration.test.ts
|
||||
git commit -m "feat: pass full NPC stat block to invention LLM prompt
|
||||
|
||||
All 10 stats with readable names sent to LLM for personality-
|
||||
driven invention selection. Known items list includes both seed
|
||||
and invented items for better duplicate prevention."
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Token usage tracking from OpenRouter
|
||||
|
||||
**Files:**
|
||||
- Modify: `server/src/llm/openRouterClient.ts:17,64-66` (return type, extract usage)
|
||||
- Modify: `server/src/llm/llmService.ts:9,78-93` (capture and log usage)
|
||||
- Modify: `server/src/llm/generationQueue.ts:1-6` (pass through new return type)
|
||||
- Test: `server/src/llm/__tests__/tokenTracking.test.ts` (new)
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
Create `server/src/llm/__tests__/tokenTracking.test.ts`:
|
||||
|
||||
```typescript
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { createTokenTracker, type TokenTracker } from '../tokenTracker.js';
|
||||
|
||||
describe('TokenTracker', () => {
|
||||
it('records and retrieves token usage', () => {
|
||||
const tracker = createTokenTracker(10);
|
||||
tracker.record('invention', 340, 120);
|
||||
|
||||
const recent = tracker.getRecent();
|
||||
expect(recent).toHaveLength(1);
|
||||
expect(recent[0].templateName).toBe('invention');
|
||||
expect(recent[0].promptTokens).toBe(340);
|
||||
expect(recent[0].completionTokens).toBe(120);
|
||||
expect(recent[0].totalTokens).toBe(460);
|
||||
});
|
||||
|
||||
it('respects ring buffer capacity', () => {
|
||||
const tracker = createTokenTracker(3);
|
||||
tracker.record('a', 10, 5);
|
||||
tracker.record('b', 20, 10);
|
||||
tracker.record('c', 30, 15);
|
||||
tracker.record('d', 40, 20);
|
||||
|
||||
const recent = tracker.getRecent();
|
||||
expect(recent).toHaveLength(3);
|
||||
expect(recent[0].templateName).toBe('b');
|
||||
expect(recent[2].templateName).toBe('d');
|
||||
});
|
||||
|
||||
it('formats summary line', () => {
|
||||
const tracker = createTokenTracker(10);
|
||||
tracker.record('invention', 340, 120);
|
||||
|
||||
const summary = tracker.lastSummary();
|
||||
expect(summary).toBe('[LLM] invention: 340 in / 120 out tokens');
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npm -w server run test -- --run server/src/llm/__tests__/tokenTracking.test.ts`
|
||||
Expected: FAIL — `tokenTracker.js` does not exist
|
||||
|
||||
**Step 3: Create tokenTracker**
|
||||
|
||||
Create `server/src/llm/tokenTracker.ts`:
|
||||
|
||||
```typescript
|
||||
export interface TokenUsageEntry {
|
||||
timestamp: number;
|
||||
templateName: string;
|
||||
promptTokens: number;
|
||||
completionTokens: number;
|
||||
totalTokens: number;
|
||||
}
|
||||
|
||||
export interface TokenTracker {
|
||||
record(templateName: string, promptTokens: number, completionTokens: number): void;
|
||||
getRecent(): TokenUsageEntry[];
|
||||
lastSummary(): string;
|
||||
}
|
||||
|
||||
export function createTokenTracker(capacity: number = 100): TokenTracker {
|
||||
const entries: TokenUsageEntry[] = [];
|
||||
|
||||
let lastEntry: TokenUsageEntry | null = null;
|
||||
|
||||
return {
|
||||
record(templateName: string, promptTokens: number, completionTokens: number): void {
|
||||
const entry: TokenUsageEntry = {
|
||||
timestamp: Date.now(),
|
||||
templateName,
|
||||
promptTokens,
|
||||
completionTokens,
|
||||
totalTokens: promptTokens + completionTokens,
|
||||
};
|
||||
|
||||
if (entries.length >= capacity) {
|
||||
entries.shift();
|
||||
}
|
||||
entries.push(entry);
|
||||
lastEntry = entry;
|
||||
},
|
||||
|
||||
getRecent(): TokenUsageEntry[] {
|
||||
return [...entries];
|
||||
},
|
||||
|
||||
lastSummary(): string {
|
||||
if (!lastEntry) return '';
|
||||
return `[LLM] ${lastEntry.templateName}: ${lastEntry.promptTokens} in / ${lastEntry.completionTokens} out tokens`;
|
||||
},
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `npm -w server run test -- --run server/src/llm/__tests__/tokenTracking.test.ts`
|
||||
Expected: PASS
|
||||
|
||||
**Step 5: Update OpenRouter client to return usage data**
|
||||
|
||||
In `server/src/llm/openRouterClient.ts`:
|
||||
|
||||
Add a new interface and update the return type:
|
||||
|
||||
```typescript
|
||||
export interface TokenUsage {
|
||||
promptTokens: number;
|
||||
completionTokens: number;
|
||||
}
|
||||
|
||||
export interface CompletionSuccess {
|
||||
content: string;
|
||||
usage: TokenUsage | null;
|
||||
}
|
||||
```
|
||||
|
||||
Change `CompletionResult` type to:
|
||||
```typescript
|
||||
export type CompletionResult = CompletionSuccess | null | RateLimitInfo;
|
||||
```
|
||||
|
||||
Add a type guard:
|
||||
```typescript
|
||||
export function isSuccess(result: CompletionResult): result is CompletionSuccess {
|
||||
return result != null && typeof result === 'object' && 'content' in result;
|
||||
}
|
||||
```
|
||||
|
||||
In the `complete` method, change the success return (line 64-66) from:
|
||||
```typescript
|
||||
const content = data?.choices?.[0]?.message?.content;
|
||||
return content ?? null;
|
||||
```
|
||||
to:
|
||||
```typescript
|
||||
const content = data?.choices?.[0]?.message?.content;
|
||||
if (!content) return null;
|
||||
const usage = data?.usage;
|
||||
return {
|
||||
content,
|
||||
usage: usage ? {
|
||||
promptTokens: usage.prompt_tokens ?? 0,
|
||||
completionTokens: usage.completion_tokens ?? 0,
|
||||
} : null,
|
||||
};
|
||||
```
|
||||
|
||||
**Step 6: Update generationQueue to pass through new type**
|
||||
|
||||
The `generationQueue.ts` already uses `CompletionResult` — no changes needed since the type flows through.
|
||||
|
||||
**Step 7: Update llmService to use new return type**
|
||||
|
||||
In `server/src/llm/llmService.ts`:
|
||||
|
||||
Add imports:
|
||||
```typescript
|
||||
import { isSuccess } from './openRouterClient.js';
|
||||
import { createTokenTracker, type TokenTracker } from './tokenTracker.js';
|
||||
```
|
||||
|
||||
After `const counters = createUsageCounters();` add:
|
||||
```typescript
|
||||
const tokenTracker = createTokenTracker(100);
|
||||
```
|
||||
|
||||
Replace the result handling block (lines 80-93) — change from checking `typeof result === 'string'` to using `isSuccess()`:
|
||||
|
||||
```typescript
|
||||
if (isRateLimited(result)) {
|
||||
switchToFallback(result.resetAt);
|
||||
return null;
|
||||
}
|
||||
|
||||
if (isSuccess(result)) {
|
||||
counters.record(currentModel);
|
||||
if (result.usage) {
|
||||
tokenTracker.record(templateName, result.usage.promptTokens, result.usage.completionTokens);
|
||||
console.log(tokenTracker.lastSummary());
|
||||
}
|
||||
const totalRequests = Object.values(counters.getStats()).reduce((sum, s) => sum + s.total, 0);
|
||||
if (totalRequests % 100 === 0) {
|
||||
console.log(`[LLM] ${counters.getSummary()}`);
|
||||
}
|
||||
return result.content;
|
||||
}
|
||||
|
||||
return result;
|
||||
```
|
||||
|
||||
Add `tokenTracker` to the returned interface — add method to the `LlmService` interface:
|
||||
```typescript
|
||||
tokenUsage(): TokenTracker;
|
||||
```
|
||||
|
||||
And in the disabled service stub:
|
||||
```typescript
|
||||
tokenUsage: () => createTokenTracker(0),
|
||||
```
|
||||
|
||||
And in the real service return:
|
||||
```typescript
|
||||
tokenUsage(): TokenTracker {
|
||||
return tokenTracker;
|
||||
},
|
||||
```
|
||||
|
||||
**Step 8: Run all tests**
|
||||
|
||||
Run: `npm -w server run test`
|
||||
Expected: Some tests may need updating where they mock `llm.generate` to return a string — now it goes through the client which returns `CompletionSuccess`. But since `llmService.generate()` still returns `string | null` (it unwraps `result.content`), downstream tests should be unaffected. The mock LLM in tests returns strings directly via the `generate` mock, which bypasses the client layer entirely, so existing test mocks remain valid.
|
||||
|
||||
**Step 9: Commit**
|
||||
|
||||
```bash
|
||||
git add server/src/llm/tokenTracker.ts server/src/llm/__tests__/tokenTracking.test.ts server/src/llm/openRouterClient.ts server/src/llm/llmService.ts
|
||||
git commit -m "feat: track token usage from OpenRouter API responses
|
||||
|
||||
Captures prompt_tokens and completion_tokens from API response.
|
||||
Stores in 100-entry ring buffer, logs per-call summary. Adds
|
||||
tokenUsage() accessor to LlmService interface."
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Global pendingItemIds race guard
|
||||
|
||||
**Files:**
|
||||
- Modify: `server/src/systems/inventionSystem.ts:27,86-97` (add set, check before register)
|
||||
- Modify: `server/src/systems/__tests__/inventionSystem.test.ts`
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
Add to `server/src/systems/__tests__/inventionSystem.test.ts`:
|
||||
|
||||
```typescript
|
||||
it('prevents two NPCs from inventing the same item simultaneously', async () => {
|
||||
const { world, itemRegistry, timeline } = setupWorld();
|
||||
const npc1 = addNPC(world, { intelligence: 18, curiosity: 18, inv: new Map([['log', 5]]) });
|
||||
const npc2 = addNPC(world, { intelligence: 18, curiosity: 18, inv: new Map([['log', 5]]) });
|
||||
|
||||
const llm = createMockLlm();
|
||||
const ropeJson = JSON.stringify({
|
||||
name: 'Rope',
|
||||
description: 'Twisted fibers',
|
||||
category: 'material',
|
||||
inputs: [{ itemId: 'log', quantity: 2 }],
|
||||
});
|
||||
// Both NPCs get the same invention idea
|
||||
(llm.generate as ReturnType<typeof vi.fn>)
|
||||
.mockResolvedValueOnce(ropeJson)
|
||||
.mockResolvedValueOnce(ropeJson);
|
||||
|
||||
const system = createInventionSystem(llm, createMockNarration(), createEventMemoryService());
|
||||
|
||||
vi.spyOn(Math, 'random').mockReturnValue(0);
|
||||
system.update(world, 100);
|
||||
vi.restoreAllMocks();
|
||||
|
||||
// Wait for both to resolve
|
||||
await new Promise(r => setTimeout(r, 50));
|
||||
|
||||
// Only one should have been registered
|
||||
expect(timeline.getAll()).toHaveLength(1);
|
||||
});
|
||||
```
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npm -w server run test -- --run server/src/systems/__tests__/inventionSystem.test.ts`
|
||||
Expected: FAIL — both register, timeline has 2 entries (or second throws)
|
||||
|
||||
**Step 3: Add pendingItemIds set**
|
||||
|
||||
In `server/src/systems/inventionSystem.ts`, after `const pendingEntities = new Set<EntityId>();`:
|
||||
|
||||
```typescript
|
||||
const pendingItemIds = new Set<string>();
|
||||
```
|
||||
|
||||
In the `.then()` handler, after `const validation = validateInvention(parsed, itemRegistry);`, add:
|
||||
|
||||
```typescript
|
||||
if (!validation.valid) return;
|
||||
|
||||
// Race guard: check if another NPC already claimed this itemId
|
||||
if (pendingItemIds.has(validation.itemId!)) return;
|
||||
pendingItemIds.add(validation.itemId!);
|
||||
```
|
||||
|
||||
After `registerInvention(...)` succeeds (after the `onInventionCreated` and event recording), add cleanup:
|
||||
|
||||
```typescript
|
||||
pendingItemIds.delete(validation.itemId!);
|
||||
```
|
||||
|
||||
Also add cleanup in the `.catch()`:
|
||||
No change needed there — if it errors, no itemId was claimed.
|
||||
|
||||
Actually, the `pendingItemIds` should be cleaned up after registration completes, not before. Since `registerInvention` is synchronous, we can delete it right after. But we want to keep it in the set until registration completes so we protect during the async window. Let me restructure:
|
||||
|
||||
The flow should be:
|
||||
1. LLM returns → parse → validate
|
||||
2. Check `pendingItemIds` → if taken, return
|
||||
3. Add to `pendingItemIds`
|
||||
4. `registerInvention()` (synchronous)
|
||||
5. Delete from `pendingItemIds`
|
||||
|
||||
This is already correct since all the `.then()` callbacks run on the microtask queue (not truly concurrent), but the set protects against the case where two promises resolve on the same tick.
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `npm -w server run test -- --run server/src/systems/__tests__/inventionSystem.test.ts`
|
||||
Expected: PASS
|
||||
|
||||
**Step 5: Run all tests**
|
||||
|
||||
Run: `npm -w server run test`
|
||||
Expected: All pass
|
||||
|
||||
**Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add server/src/systems/inventionSystem.ts server/src/systems/__tests__/inventionSystem.test.ts
|
||||
git commit -m "fix: prevent race condition with global pendingItemIds set
|
||||
|
||||
Two NPCs could invent the same item if both LLM requests
|
||||
returned before either registered. The pendingItemIds set
|
||||
blocks the second registration."
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Final integration test and cleanup
|
||||
|
||||
**Files:**
|
||||
- Modify: `server/src/systems/__tests__/inventionIntegration.test.ts`
|
||||
|
||||
**Step 1: Add integration test for stat-driven prompt**
|
||||
|
||||
Add a new test to `inventionIntegration.test.ts`:
|
||||
|
||||
```typescript
|
||||
it('passes full stat block and knownItems to LLM prompt', () => {
|
||||
// Setup world with NPC with distinctive stats
|
||||
const world = new World();
|
||||
const itemRegistry = ItemRegistry.createDefault();
|
||||
const recipeRegistry = RecipeRegistry.createDefault();
|
||||
const timeline = createInventionTimeline();
|
||||
world.setSingleton('itemRegistry', itemRegistry);
|
||||
world.setSingleton('recipeRegistry', recipeRegistry);
|
||||
world.setSingleton('inventionTimeline', timeline);
|
||||
|
||||
const e = world.createEntity();
|
||||
world.addComponent<Needs>(e, 'needs', { hunger: 80, energy: 80, productivity: 20 });
|
||||
world.addComponent<NPCBrain>(e, 'npcBrain', { currentGoal: 'gather', goalQueue: [] });
|
||||
world.addComponent<Stats>(e, 'stats', {
|
||||
strength: 16, dexterity: 8, constitution: 12,
|
||||
intelligence: 14, perception: 11,
|
||||
sociability: 7, courage: 18, curiosity: 15,
|
||||
empathy: 5, temperament: 13,
|
||||
});
|
||||
world.addComponent<StatModifiers>(e, 'statModifiers', { modifiers: [] });
|
||||
world.addComponent<Map<string, number>>(e, 'inventory', new Map([['log', 3]]));
|
||||
world.addComponent<string>(e, 'name', 'Brutus');
|
||||
|
||||
const generateMock = vi.fn().mockResolvedValue(null);
|
||||
const llm: LlmService = {
|
||||
generate: generateMock,
|
||||
queueDepth: () => 0,
|
||||
clear: () => {},
|
||||
activeModel: () => 'test-model',
|
||||
usageStats: () => ({}),
|
||||
usageSummary: () => '',
|
||||
};
|
||||
|
||||
const system = createInventionSystem(llm, createMockNarration(), createEventMemoryService());
|
||||
|
||||
vi.spyOn(Math, 'random').mockReturnValue(0);
|
||||
system.update(world, 100);
|
||||
|
||||
expect(generateMock).toHaveBeenCalledWith('invention', expect.objectContaining({
|
||||
npcName: 'Brutus',
|
||||
strength: '16',
|
||||
dexterity: '8',
|
||||
constitution: '12',
|
||||
intelligence: '14',
|
||||
perception: '11',
|
||||
sociability: '7',
|
||||
courage: '18',
|
||||
curiosity: '15',
|
||||
empathy: '5',
|
||||
temperament: '13',
|
||||
knownItems: expect.stringContaining('Log'),
|
||||
}));
|
||||
});
|
||||
```
|
||||
|
||||
Also need to add `createMockNarration` helper if not already present in this file (it is not — copy from inventionSystem.test.ts or import).
|
||||
|
||||
**Step 2: Run all tests**
|
||||
|
||||
Run: `npm -w server run test`
|
||||
Expected: All pass
|
||||
|
||||
**Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add server/src/systems/__tests__/inventionIntegration.test.ts
|
||||
git commit -m "test: add integration test for stat-driven invention prompt"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 8: Run full test suite and verify
|
||||
|
||||
**Step 1: Run all tests**
|
||||
|
||||
Run: `npm -w server run test`
|
||||
Expected: All 408+ tests pass
|
||||
|
||||
**Step 2: Manual smoke test (optional)**
|
||||
|
||||
Run: `npm -w server run dev`
|
||||
Watch for invention log output. Verify:
|
||||
- On load, inventions are re-registered (check log for "Loaded save")
|
||||
- Token usage logs appear: `[LLM] invention: NNN in / NNN out tokens`
|
||||
- No duplicate inventions
|
||||
|
||||
**Step 3: Final commit if any cleanup needed**
|
||||
Reference in New Issue
Block a user