KBs and Embeddings for both structured & import

This commit is contained in:
talatops 2026-04-11 17:22:59 +05:00
parent 4dd56f3aae
commit e49a0047aa
28 changed files with 2132 additions and 723 deletions

View File

@ -16,7 +16,8 @@
<img src="https://img.shields.io/badge/Next.js-000000?logo=nextdotjs&logoColor=white" alt="Next.js" /> <img src="https://img.shields.io/badge/Next.js-000000?logo=nextdotjs&logoColor=white" alt="Next.js" />
<img src="https://img.shields.io/badge/TypeScript-3178C6?logo=typescript&logoColor=white" alt="TypeScript" /> <img src="https://img.shields.io/badge/TypeScript-3178C6?logo=typescript&logoColor=white" alt="TypeScript" />
<img src="https://img.shields.io/badge/PostgreSQL-4169E1?logo=postgresql&logoColor=white" alt="PostgreSQL" /> <img src="https://img.shields.io/badge/PostgreSQL-4169E1?logo=postgresql&logoColor=white" alt="PostgreSQL" />
<img src="https://img.shields.io/badge/AWS%20Bedrock-232F3E?logo=amazonaws&logoColor=white" alt="AWS Bedrock" /> <img src="https://img.shields.io/badge/OpenAI-412991?logo=openai&logoColor=white" alt="OpenAI" />
<img src="https://img.shields.io/badge/Groq-F55036?logo=groq&logoColor=white" alt="Groq" />
<img src="https://img.shields.io/badge/Gmail%20API-EA4335?logo=gmail&logoColor=white" alt="Gmail API" /> <img src="https://img.shields.io/badge/Gmail%20API-EA4335?logo=gmail&logoColor=white" alt="Gmail API" />
<img src="https://img.shields.io/badge/Docker-2496ED?logo=docker&logoColor=white" alt="Docker" /> <img src="https://img.shields.io/badge/Docker-2496ED?logo=docker&logoColor=white" alt="Docker" />
<img src="https://img.shields.io/badge/NextAuth-000000?logo=nextdotjs&logoColor=white" alt="NextAuth" /> <img src="https://img.shields.io/badge/NextAuth-000000?logo=nextdotjs&logoColor=white" alt="NextAuth" />
@ -51,7 +52,7 @@ Everything runs inside a single Next.js App Router project (frontend + backend +
- [JUNOCHAT & AI](#future-enhancements-for-junochat--ai) - [JUNOCHAT & AI](#future-enhancements-for-junochat--ai)
6. [Local Development](#local-development) 6. [Local Development](#local-development)
7. [Docker & Deployment](#docker--deployment) 7. [Docker & Deployment](#docker--deployment)
8. [AWS Bedrock & AI Setup](#aws-bedrock--ai-setup) 8. [OpenAI, Groq & AI Setup](#openai-groq--ai-setup)
9. [Prisma & Database](#prisma--database) 9. [Prisma & Database](#prisma--database)
10. [License](#license) 10. [License](#license)
@ -74,7 +75,7 @@ JUNO centralizes communication and operations between **vendors** (Shopify store
- Edit synced product price/quantity **per vendor** with inline controls. - Edit synced product price/quantity **per vendor** with inline controls.
- Both sides are supported by: - Both sides are supported by:
- **JUNOCHAT** – an AI assistant powered by AWS Bedrock (Titan + Claude) that can answer questions based on the knowledge base and FAQs. - **JUNOCHAT** – an AI assistant powered by **OpenAI** (embeddings + chat where configured) and **Groq** (fast chat for storefront widget, Gmail flows, and KB fallback) that can answer questions using the knowledge base, FAQs, and store context.
- A **Juno Engine** area for Gmail and ticketing workflows. - A **Juno Engine** area for Gmail and ticketing workflows.
--- ---
@ -167,9 +168,10 @@ JUNO centralizes communication and operations between **vendors** (Shopify store
JUNOCHAT (a.k.a. Juno Engine chat) is the AI layer that: JUNOCHAT (a.k.a. Juno Engine chat) is the AI layer that:
- Uses **AWS Bedrock**: - Uses **OpenAI** and **Groq**:
- **Amazon Titan Text Embeddings V2** for vector embeddings. - **OpenAI** (`text-embedding-3-small`, etc.) for knowledge‑base **embeddings** stored in Postgres/pgvector.
- **Anthropic Claude 3 Haiku** for chat and knowledge‑base answers. - **OpenAI** (`gpt-4o-mini` by default, configurable) for **KB “Ask”** answers when `OPENAI_API_KEY` is set.
- **Groq** for **storefront widget** replies, **Gmail** classification/safety/preview flows, and **KB fallback** if OpenAI chat fails.
- Supports: - Supports:
- FAQ ingestion and retrieval. - FAQ ingestion and retrieval.
- Knowledge document upload + chunking. - Knowledge document upload + chunking.
@ -217,7 +219,7 @@ JUNOCHAT (a.k.a. Juno Engine chat) is the AI layer that:
- **Background / Integrations**: - **Background / Integrations**:
- Gmail APIs (OAuth) - Gmail APIs (OAuth)
- Shopify APIs (orders, products, inventory) - Shopify APIs (orders, products, inventory)
- AWS Bedrock (Titan embeddings + Claude 3 Haiku chat) - OpenAI (embeddings + optional chat) and Groq (chat / classification)
- **Deployment**: Docker + GitHub Actions → AWS EC2 - **Deployment**: Docker + GitHub Actions → AWS EC2
--- ---
@ -277,7 +279,7 @@ JUNOCHAT (a.k.a. Juno Engine chat) is the AI layer that:
**How JUNO helps:** **How JUNO helps:**
- **JUNOCHAT + Juno Engine**: - **JUNOCHAT + Juno Engine**:
- Ingest FAQs and internal docs into a vectorized knowledge base (via Bedrock embeddings). - Ingest FAQs and internal docs into a vectorized knowledge base (via OpenAI embeddings).
- Answer questions using **RAG** against that knowledge. - Answer questions using **RAG** against that knowledge.
- Integrate with Gmail so tickets and AI context live in the same place. - Integrate with Gmail so tickets and AI context live in the same place.
@ -307,9 +309,9 @@ Copy `.env.example` (or the variables listed below) into a local `.env` file and
- `SMTP_SECURE` - `SMTP_SECURE`
- `SMTP_USER` - `SMTP_USER`
- `SMTP_PASSWORD` - `SMTP_PASSWORD`
- `AWS_ACCESS_KEY_ID` (for AWS Bedrock) - `OPENAI_API_KEY` (embeddings + KB chat; optional `OPENAI_CHAT_MODEL`, `OPENAI_EMBEDDING_MODEL`, `OPENAI_EMBEDDING_DIMENSIONS`)
- `AWS_SECRET_ACCESS_KEY` (for AWS Bedrock) - `GROQ_API_KEY` (storefront chat, Gmail AI helpers, KB chat fallback; optional `GROQ_MODEL`)
- `AWS_REGION` (default: `us-east-1`) - `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` / `AWS_REGION` (if you use S3 or other AWS services—not required for AI-only local dev)
- `ADMIN_EMAIL` - `ADMIN_EMAIL`
- `ADMIN_PASSWORD` - `ADMIN_PASSWORD`
@ -376,51 +378,34 @@ This gives you a sharp deployment with an automatic rollback path.
--- ---
## AWS Bedrock & AI Setup ## OpenAI, Groq & AI Setup
The knowledge base and JUNOCHAT features use AWS Bedrock for AI‑powered embeddings and chat completion. ### OpenAI (embeddings + KB answers)
### Prerequisites 1. Create an API key in the [OpenAI dashboard](https://platform.openai.com/).
2. Add to `.env`:
1. **AWS Account**: Create an account at `https://aws.amazon.com/`
2. **Enable Bedrock Access**:
- Go to `https://console.aws.amazon.com/bedrock/`
- Navigate to **Model access**
- Request access to:
- **Amazon Titan Text Embeddings V2** (for embeddings)
- **Anthropic Claude 3 Haiku** (for chat completion)
### IAM Setup
Create an IAM user or role with Bedrock permissions:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["bedrock:InvokeModel"],
"Resource": [
"arn:aws:bedrock:*::foundation-model/amazon.titan-embed-text-v2:0",
"arn:aws:bedrock:*::foundation-model/anthropic.claude-3-haiku-20240307-v1:0"
]
}
]
}
```
Create access keys for that user and add them to `.env`:
```bash ```bash
AWS_ACCESS_KEY_ID="your-access-key-id" OPENAI_API_KEY="sk-..."
AWS_SECRET_ACCESS_KEY="your-secret-access-key" # Optional overrides:
AWS_REGION="us-east-1" # or your preferred region # OPENAI_EMBEDDING_MODEL="text-embedding-3-small"
# OPENAI_EMBEDDING_DIMENSIONS="1024"
# OPENAI_CHAT_MODEL="gpt-4o-mini"
``` ```
### Database Migration & Re‑Embedding Embeddings are used when indexing knowledge documents, FAQs, and the structured store profile. KB **Ask** uses OpenAI chat first, then falls back to Groq.
After setting up AWS Bedrock: ### Groq (storefront chat, Gmail helpers, KB fallback)
1. Create an API key at [Groq Console](https://console.groq.com/).
2. Add to `.env`:
```bash
GROQ_API_KEY="gsk_..."
# Optional: GROQ_MODEL="llama-3.1-8b-instant"
```
### Database migration & re‑embedding
1. Apply Prisma migrations: 1. Apply Prisma migrations:
@ -428,17 +413,13 @@ After setting up AWS Bedrock:
npx prisma migrate deploy npx prisma migrate deploy
``` ```
2. Re‑embed existing knowledge base data: 2. After changing embedding provider or dimensions, re‑embed:
```bash ```bash
npm run re-embed-knowledge-base npm run re-embed-knowledge-base
``` ```
This script: This regenerates vectors and tags chunk metadata with `provider: openai` (required for KB similarity search filters).
- Reads all existing `KnowledgeChunk` records.
- Generates new embeddings using Titan V2.
- Updates the database in batches to avoid rate limiting.
--- ---

433
package-lock.json generated
View File

@ -10,7 +10,6 @@
"dependencies": { "dependencies": {
"@auth/core": "^0.41.0", "@auth/core": "^0.41.0",
"@auth/prisma-adapter": "^2.11.1", "@auth/prisma-adapter": "^2.11.1",
"@aws-sdk/client-bedrock-runtime": "^3.989.0",
"@aws-sdk/client-s3": "^3.989.0", "@aws-sdk/client-s3": "^3.989.0",
"@prisma/adapter-pg": "^7.6.0", "@prisma/adapter-pg": "^7.6.0",
"@prisma/client": "^7.6.0", "@prisma/client": "^7.6.0",
@ -32,6 +31,7 @@
"next": "^16.2.1", "next": "^16.2.1",
"next-auth": "^5.0.0-beta.30", "next-auth": "^5.0.0-beta.30",
"nodemailer": "^7.0.7", "nodemailer": "^7.0.7",
"openai": "^4.104.0",
"otplib": "^13.2.1", "otplib": "^13.2.1",
"pdf-parse": "^2.4.5", "pdf-parse": "^2.4.5",
"pg": "^8.18.0", "pg": "^8.18.0",
@ -297,63 +297,6 @@
"node": ">=14.0.0" "node": ">=14.0.0"
} }
}, },
"node_modules/@aws-sdk/client-bedrock-runtime": {
"version": "3.989.0",
"resolved": "https://registry.npmjs.org/@aws-sdk/client-bedrock-runtime/-/client-bedrock-runtime-3.989.0.tgz",
"integrity": "sha512-qVa5B0wXjIuPRhX1dcZo1sa9Y4ycI9tiqK7B4FLok67gUWckiKmEf1xQDFrTmc2eCK5g0CTaeiRdbeM1eWmW1Q==",
"dependencies": {
"@aws-crypto/sha256-browser": "5.2.0",
"@aws-crypto/sha256-js": "5.2.0",
"@aws-sdk/core": "^3.973.9",
"@aws-sdk/credential-provider-node": "^3.972.8",
"@aws-sdk/eventstream-handler-node": "^3.972.5",
"@aws-sdk/middleware-eventstream": "^3.972.3",
"@aws-sdk/middleware-host-header": "^3.972.3",
"@aws-sdk/middleware-logger": "^3.972.3",
"@aws-sdk/middleware-recursion-detection": "^3.972.3",
"@aws-sdk/middleware-user-agent": "^3.972.9",
"@aws-sdk/middleware-websocket": "^3.972.6",
"@aws-sdk/region-config-resolver": "^3.972.3",
"@aws-sdk/token-providers": "3.989.0",
"@aws-sdk/types": "^3.973.1",
"@aws-sdk/util-endpoints": "3.989.0",
"@aws-sdk/util-user-agent-browser": "^3.972.3",
"@aws-sdk/util-user-agent-node": "^3.972.7",
"@smithy/config-resolver": "^4.4.6",
"@smithy/core": "^3.23.0",
"@smithy/eventstream-serde-browser": "^4.2.8",
"@smithy/eventstream-serde-config-resolver": "^4.3.8",
"@smithy/eventstream-serde-node": "^4.2.8",
"@smithy/fetch-http-handler": "^5.3.9",
"@smithy/hash-node": "^4.2.8",
"@smithy/invalid-dependency": "^4.2.8",
"@smithy/middleware-content-length": "^4.2.8",
"@smithy/middleware-endpoint": "^4.4.14",
"@smithy/middleware-retry": "^4.4.31",
"@smithy/middleware-serde": "^4.2.9",
"@smithy/middleware-stack": "^4.2.8",
"@smithy/node-config-provider": "^4.3.8",
"@smithy/node-http-handler": "^4.4.10",
"@smithy/protocol-http": "^5.3.8",
"@smithy/smithy-client": "^4.11.3",
"@smithy/types": "^4.12.0",
"@smithy/url-parser": "^4.2.8",
"@smithy/util-base64": "^4.3.0",
"@smithy/util-body-length-browser": "^4.2.0",
"@smithy/util-body-length-node": "^4.2.1",
"@smithy/util-defaults-mode-browser": "^4.3.30",
"@smithy/util-defaults-mode-node": "^4.2.33",
"@smithy/util-endpoints": "^3.2.8",
"@smithy/util-middleware": "^4.2.8",
"@smithy/util-retry": "^4.2.8",
"@smithy/util-stream": "^4.5.12",
"@smithy/util-utf8": "^4.2.0",
"tslib": "^2.6.2"
},
"engines": {
"node": ">=20.0.0"
}
},
"node_modules/@aws-sdk/client-s3": { "node_modules/@aws-sdk/client-s3": {
"version": "3.990.0", "version": "3.990.0",
"resolved": "https://registry.npmjs.org/@aws-sdk/client-s3/-/client-s3-3.990.0.tgz", "resolved": "https://registry.npmjs.org/@aws-sdk/client-s3/-/client-s3-3.990.0.tgz",
@ -951,20 +894,6 @@
"node": ">=20.0.0" "node": ">=20.0.0"
} }
}, },
"node_modules/@aws-sdk/eventstream-handler-node": {
"version": "3.972.5",
"resolved": "https://registry.npmjs.org/@aws-sdk/eventstream-handler-node/-/eventstream-handler-node-3.972.5.tgz",
"integrity": "sha512-xEmd3dnyn83K6t4AJxBJA63wpEoCD45ERFG0XMTViD2E/Ohls9TLxjOWPb1PAxR9/46cKy/TImez1GoqP6xVNQ==",
"dependencies": {
"@aws-sdk/types": "^3.973.1",
"@smithy/eventstream-codec": "^4.2.8",
"@smithy/types": "^4.12.0",
"tslib": "^2.6.2"
},
"engines": {
"node": ">=20.0.0"
}
},
"node_modules/@aws-sdk/middleware-bucket-endpoint": { "node_modules/@aws-sdk/middleware-bucket-endpoint": {
"version": "3.972.3", "version": "3.972.3",
"resolved": "https://registry.npmjs.org/@aws-sdk/middleware-bucket-endpoint/-/middleware-bucket-endpoint-3.972.3.tgz", "resolved": "https://registry.npmjs.org/@aws-sdk/middleware-bucket-endpoint/-/middleware-bucket-endpoint-3.972.3.tgz",
@ -982,20 +911,6 @@
"node": ">=20.0.0" "node": ">=20.0.0"
} }
}, },
"node_modules/@aws-sdk/middleware-eventstream": {
"version": "3.972.3",
"resolved": "https://registry.npmjs.org/@aws-sdk/middleware-eventstream/-/middleware-eventstream-3.972.3.tgz",
"integrity": "sha512-pbvZ6Ye/Ks6BAZPa3RhsNjHrvxU9li25PMhSdDpbX0jzdpKpAkIR65gXSNKmA/REnSdEMWSD4vKUW+5eMFzB6w==",
"dependencies": {
"@aws-sdk/types": "^3.973.1",
"@smithy/protocol-http": "^5.3.8",
"@smithy/types": "^4.12.0",
"tslib": "^2.6.2"
},
"engines": {
"node": ">=20.0.0"
}
},
"node_modules/@aws-sdk/middleware-expect-continue": { "node_modules/@aws-sdk/middleware-expect-continue": {
"version": "3.972.3", "version": "3.972.3",
"resolved": "https://registry.npmjs.org/@aws-sdk/middleware-expect-continue/-/middleware-expect-continue-3.972.3.tgz", "resolved": "https://registry.npmjs.org/@aws-sdk/middleware-expect-continue/-/middleware-expect-continue-3.972.3.tgz",
@ -1158,76 +1073,6 @@
"node": ">=20.0.0" "node": ">=20.0.0"
} }
}, },
"node_modules/@aws-sdk/middleware-websocket": {
"version": "3.972.6",
"resolved": "https://registry.npmjs.org/@aws-sdk/middleware-websocket/-/middleware-websocket-3.972.6.tgz",
"integrity": "sha512-1DedO6N3m8zQ/vG6twNiHtsdwBgk773VdavLEbB3NXeKZDlzSK1BTviqWwvJdKx5UnIy4kGGP6WWpCEFEt/bhQ==",
"dependencies": {
"@aws-sdk/types": "^3.973.1",
"@aws-sdk/util-format-url": "^3.972.3",
"@smithy/eventstream-codec": "^4.2.8",
"@smithy/eventstream-serde-browser": "^4.2.8",
"@smithy/fetch-http-handler": "^5.3.9",
"@smithy/protocol-http": "^5.3.8",
"@smithy/signature-v4": "^5.3.8",
"@smithy/types": "^4.12.0",
"@smithy/util-base64": "^4.3.0",
"@smithy/util-hex-encoding": "^4.2.0",
"@smithy/util-utf8": "^4.2.0",
"tslib": "^2.6.2"
},
"engines": {
"node": ">= 14.0.0"
}
},
"node_modules/@aws-sdk/nested-clients": {
"version": "3.989.0",
"resolved": "https://registry.npmjs.org/@aws-sdk/nested-clients/-/nested-clients-3.989.0.tgz",
"integrity": "sha512-Dbk2HMPU3mb6RrSRzgf0WCaWSbgtZG258maCpuN2/ONcAQNpOTw99V5fU5CA1qVK6Vkm4Fwj2cnOnw7wbGVlOw==",
"dependencies": {
"@aws-crypto/sha256-browser": "5.2.0",
"@aws-crypto/sha256-js": "5.2.0",
"@aws-sdk/core": "^3.973.9",
"@aws-sdk/middleware-host-header": "^3.972.3",
"@aws-sdk/middleware-logger": "^3.972.3",
"@aws-sdk/middleware-recursion-detection": "^3.972.3",
"@aws-sdk/middleware-user-agent": "^3.972.9",
"@aws-sdk/region-config-resolver": "^3.972.3",
"@aws-sdk/types": "^3.973.1",
"@aws-sdk/util-endpoints": "3.989.0",
"@aws-sdk/util-user-agent-browser": "^3.972.3",
"@aws-sdk/util-user-agent-node": "^3.972.7",
"@smithy/config-resolver": "^4.4.6",
"@smithy/core": "^3.23.0",
"@smithy/fetch-http-handler": "^5.3.9",
"@smithy/hash-node": "^4.2.8",
"@smithy/invalid-dependency": "^4.2.8",
"@smithy/middleware-content-length": "^4.2.8",
"@smithy/middleware-endpoint": "^4.4.14",
"@smithy/middleware-retry": "^4.4.31",
"@smithy/middleware-serde": "^4.2.9",
"@smithy/middleware-stack": "^4.2.8",
"@smithy/node-config-provider": "^4.3.8",
"@smithy/node-http-handler": "^4.4.10",
"@smithy/protocol-http": "^5.3.8",
"@smithy/smithy-client": "^4.11.3",
"@smithy/types": "^4.12.0",
"@smithy/url-parser": "^4.2.8",
"@smithy/util-base64": "^4.3.0",
"@smithy/util-body-length-browser": "^4.2.0",
"@smithy/util-body-length-node": "^4.2.1",
"@smithy/util-defaults-mode-browser": "^4.3.30",
"@smithy/util-defaults-mode-node": "^4.2.33",
"@smithy/util-endpoints": "^3.2.8",
"@smithy/util-middleware": "^4.2.8",
"@smithy/util-retry": "^4.2.8",
"@smithy/util-utf8": "^4.2.0",
"tslib": "^2.6.2"
},
"engines": {
"node": ">=20.0.0"
}
},
"node_modules/@aws-sdk/region-config-resolver": { "node_modules/@aws-sdk/region-config-resolver": {
"version": "3.972.3", "version": "3.972.3",
"resolved": "https://registry.npmjs.org/@aws-sdk/region-config-resolver/-/region-config-resolver-3.972.3.tgz", "resolved": "https://registry.npmjs.org/@aws-sdk/region-config-resolver/-/region-config-resolver-3.972.3.tgz",
@ -1259,23 +1104,6 @@
"node": ">=20.0.0" "node": ">=20.0.0"
} }
}, },
"node_modules/@aws-sdk/token-providers": {
"version": "3.989.0",
"resolved": "https://registry.npmjs.org/@aws-sdk/token-providers/-/token-providers-3.989.0.tgz",
"integrity": "sha512-OdBByMv+OjOZoekrk4THPFpLuND5aIQbDHCGh3n2rvifAbm31+6e0OLhxSeCF1UMPm+nKq12bXYYEoCIx5SQBg==",
"dependencies": {
"@aws-sdk/core": "^3.973.9",
"@aws-sdk/nested-clients": "3.989.0",
"@aws-sdk/types": "^3.973.1",
"@smithy/property-provider": "^4.2.8",
"@smithy/shared-ini-file-loader": "^4.4.3",
"@smithy/types": "^4.12.0",
"tslib": "^2.6.2"
},
"engines": {
"node": ">=20.0.0"
}
},
"node_modules/@aws-sdk/types": { "node_modules/@aws-sdk/types": {
"version": "3.973.1", "version": "3.973.1",
"resolved": "https://registry.npmjs.org/@aws-sdk/types/-/types-3.973.1.tgz", "resolved": "https://registry.npmjs.org/@aws-sdk/types/-/types-3.973.1.tgz",
@ -1299,35 +1127,6 @@
"node": ">=20.0.0" "node": ">=20.0.0"
} }
}, },
"node_modules/@aws-sdk/util-endpoints": {
"version": "3.989.0",
"resolved": "https://registry.npmjs.org/@aws-sdk/util-endpoints/-/util-endpoints-3.989.0.tgz",
"integrity": "sha512-eKmAOeQM4Qusq0jtcbZPiNWky8XaojByKC/n+THbJ8vJf7t4ys8LlcZ4PrBSHZISe9cC484mQsPVOQh6iySjqw==",
"dependencies": {
"@aws-sdk/types": "^3.973.1",
"@smithy/types": "^4.12.0",
"@smithy/url-parser": "^4.2.8",
"@smithy/util-endpoints": "^3.2.8",
"tslib": "^2.6.2"
},
"engines": {
"node": ">=20.0.0"
}
},
"node_modules/@aws-sdk/util-format-url": {
"version": "3.972.3",
"resolved": "https://registry.npmjs.org/@aws-sdk/util-format-url/-/util-format-url-3.972.3.tgz",
"integrity": "sha512-n7F2ycckcKFXa01vAsT/SJdjFHfKH9s96QHcs5gn8AaaigASICeME8WdUL9uBp8XV/OVwEt8+6gzn6KFUgQa8g==",
"dependencies": {
"@aws-sdk/types": "^3.973.1",
"@smithy/querystring-builder": "^4.2.8",
"@smithy/types": "^4.12.0",
"tslib": "^2.6.2"
},
"engines": {
"node": ">=20.0.0"
}
},
"node_modules/@aws-sdk/util-locate-window": { "node_modules/@aws-sdk/util-locate-window": {
"version": "3.965.4", "version": "3.965.4",
"resolved": "https://registry.npmjs.org/@aws-sdk/util-locate-window/-/util-locate-window-3.965.4.tgz", "resolved": "https://registry.npmjs.org/@aws-sdk/util-locate-window/-/util-locate-window-3.965.4.tgz",
@ -4956,6 +4755,15 @@
"undici-types": "~6.21.0" "undici-types": "~6.21.0"
} }
}, },
"node_modules/@types/node-fetch": {
"version": "2.6.13",
"resolved": "https://registry.npmjs.org/@types/node-fetch/-/node-fetch-2.6.13.tgz",
"integrity": "sha512-QGpRVpzSaUs30JBSGPjOg4Uveu384erbHBoT1zeONvyCfwQxIkUshLAOqN/k9EjGviPRmWTTe6aH2qySWKTVSw==",
"dependencies": {
"@types/node": "*",
"form-data": "^4.0.4"
}
},
"node_modules/@types/nodemailer": { "node_modules/@types/nodemailer": {
"version": "7.0.9", "version": "7.0.9",
"resolved": "https://registry.npmjs.org/@types/nodemailer/-/nodemailer-7.0.9.tgz", "resolved": "https://registry.npmjs.org/@types/nodemailer/-/nodemailer-7.0.9.tgz",
@ -5686,6 +5494,17 @@
"node": ">=10.0.0" "node": ">=10.0.0"
} }
}, },
"node_modules/abort-controller": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/abort-controller/-/abort-controller-3.0.0.tgz",
"integrity": "sha512-h8lQ8tacZYnR3vNQTgibj+tODHI5/+l06Au2Pcriv/Gmet0eaj4TwWH41sO9wnHDiQsEj19q0drzdWdeAHtweg==",
"dependencies": {
"event-target-shim": "^5.0.0"
},
"engines": {
"node": ">=6.5"
}
},
"node_modules/acorn": { "node_modules/acorn": {
"version": "8.15.0", "version": "8.15.0",
"resolved": "https://registry.npmjs.org/acorn/-/acorn-8.15.0.tgz", "resolved": "https://registry.npmjs.org/acorn/-/acorn-8.15.0.tgz",
@ -5715,6 +5534,17 @@
"node": ">= 14" "node": ">= 14"
} }
}, },
"node_modules/agentkeepalive": {
"version": "4.6.0",
"resolved": "https://registry.npmjs.org/agentkeepalive/-/agentkeepalive-4.6.0.tgz",
"integrity": "sha512-kja8j7PjmncONqaTsB8fQ+wE2mSU2DJ9D4XKoJ5PFWIdRMa6SLSN1ff4mOr4jCbfRSsxR4keIiySJU0N9T5hIQ==",
"dependencies": {
"humanize-ms": "^1.2.1"
},
"engines": {
"node": ">= 8.0.0"
}
},
"node_modules/ajv": { "node_modules/ajv": {
"version": "6.14.0", "version": "6.14.0",
"resolved": "https://registry.npmjs.org/ajv/-/ajv-6.14.0.tgz", "resolved": "https://registry.npmjs.org/ajv/-/ajv-6.14.0.tgz",
@ -5955,6 +5785,11 @@
"node": ">= 0.4" "node": ">= 0.4"
} }
}, },
"node_modules/asynckit": {
"version": "0.4.0",
"resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz",
"integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q=="
},
"node_modules/available-typed-arrays": { "node_modules/available-typed-arrays": {
"version": "1.0.7", "version": "1.0.7",
"resolved": "https://registry.npmjs.org/available-typed-arrays/-/available-typed-arrays-1.0.7.tgz", "resolved": "https://registry.npmjs.org/available-typed-arrays/-/available-typed-arrays-1.0.7.tgz",
@ -6454,6 +6289,17 @@
"resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.4.tgz", "resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.4.tgz",
"integrity": "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==" "integrity": "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA=="
}, },
"node_modules/combined-stream": {
"version": "1.0.8",
"resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz",
"integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==",
"dependencies": {
"delayed-stream": "~1.0.0"
},
"engines": {
"node": ">= 0.8"
}
},
"node_modules/concat-map": { "node_modules/concat-map": {
"version": "0.0.1", "version": "0.0.1",
"resolved": "https://registry.npmjs.org/concat-map/-/concat-map-0.0.1.tgz", "resolved": "https://registry.npmjs.org/concat-map/-/concat-map-0.0.1.tgz",
@ -6768,6 +6614,14 @@
"resolved": "https://registry.npmjs.org/defu/-/defu-6.1.4.tgz", "resolved": "https://registry.npmjs.org/defu/-/defu-6.1.4.tgz",
"integrity": "sha512-mEQCMmwJu317oSz8CwdIOdwf3xMif1ttiM8LTufzc3g6kR+9Pe236twL8j3IYT1F7GfRgGcW6MWxzZjLIkuHIg==" "integrity": "sha512-mEQCMmwJu317oSz8CwdIOdwf3xMif1ttiM8LTufzc3g6kR+9Pe236twL8j3IYT1F7GfRgGcW6MWxzZjLIkuHIg=="
}, },
"node_modules/delayed-stream": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz",
"integrity": "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==",
"engines": {
"node": ">=0.4.0"
}
},
"node_modules/denque": { "node_modules/denque": {
"version": "2.1.0", "version": "2.1.0",
"resolved": "https://registry.npmjs.org/denque/-/denque-2.1.0.tgz", "resolved": "https://registry.npmjs.org/denque/-/denque-2.1.0.tgz",
@ -7035,7 +6889,6 @@
"version": "2.1.0", "version": "2.1.0",
"resolved": "https://registry.npmjs.org/es-set-tostringtag/-/es-set-tostringtag-2.1.0.tgz", "resolved": "https://registry.npmjs.org/es-set-tostringtag/-/es-set-tostringtag-2.1.0.tgz",
"integrity": "sha512-j6vWzfrGVfyXxge+O0x5sh6cvxAog0a/4Rdd2K36zCMV5eJ+/+tOAngRO8cODMNWbVRdVlmGZQL2YS3yR8bIUA==", "integrity": "sha512-j6vWzfrGVfyXxge+O0x5sh6cvxAog0a/4Rdd2K36zCMV5eJ+/+tOAngRO8cODMNWbVRdVlmGZQL2YS3yR8bIUA==",
"dev": true,
"dependencies": { "dependencies": {
"es-errors": "^1.3.0", "es-errors": "^1.3.0",
"get-intrinsic": "^1.2.6", "get-intrinsic": "^1.2.6",
@ -7554,6 +7407,14 @@
"node": ">=0.10.0" "node": ">=0.10.0"
} }
}, },
"node_modules/event-target-shim": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/event-target-shim/-/event-target-shim-5.0.1.tgz",
"integrity": "sha512-i/2XbnSz/uxRCU6+NdVJgKWDTM427+MqYbkQzD321DuCQJUqOuJKIA0IM2+W2xtYHdKOmZ4dR6fExsd4SXL+WQ==",
"engines": {
"node": ">=6"
}
},
"node_modules/eventemitter3": { "node_modules/eventemitter3": {
"version": "5.0.4", "version": "5.0.4",
"resolved": "https://registry.npmjs.org/eventemitter3/-/eventemitter3-5.0.4.tgz", "resolved": "https://registry.npmjs.org/eventemitter3/-/eventemitter3-5.0.4.tgz",
@ -7792,6 +7653,38 @@
"url": "https://github.com/sponsors/isaacs" "url": "https://github.com/sponsors/isaacs"
} }
}, },
"node_modules/form-data": {
"version": "4.0.5",
"resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.5.tgz",
"integrity": "sha512-8RipRLol37bNs2bhoV67fiTEvdTrbMUYcFTiy3+wuuOnUog2QBHCZWXDRijWQfAkhBj2Uf5UnVaiWwA5vdd82w==",
"dependencies": {
"asynckit": "^0.4.0",
"combined-stream": "^1.0.8",
"es-set-tostringtag": "^2.1.0",
"hasown": "^2.0.2",
"mime-types": "^2.1.12"
},
"engines": {
"node": ">= 6"
}
},
"node_modules/form-data-encoder": {
"version": "1.7.2",
"resolved": "https://registry.npmjs.org/form-data-encoder/-/form-data-encoder-1.7.2.tgz",
"integrity": "sha512-qfqtYan3rxrnCk1VYaA4H+Ms9xdpPqvLZa6xmMgFvhO32x7/3J/ExcTd6qpxM0vH2GdMI+poehyBZvqfMTto8A=="
},
"node_modules/formdata-node": {
"version": "4.4.1",
"resolved": "https://registry.npmjs.org/formdata-node/-/formdata-node-4.4.1.tgz",
"integrity": "sha512-0iirZp3uVDjVGt9p49aTaqjk84TrglENEDuqfdlZQ1roC9CWlPk6Avf8EEnZNcAqPonwkG35x4n3ww/1THYAeQ==",
"dependencies": {
"node-domexception": "1.0.0",
"web-streams-polyfill": "4.0.0-beta.3"
},
"engines": {
"node": ">= 12.20"
}
},
"node_modules/framer-motion": { "node_modules/framer-motion": {
"version": "12.34.0", "version": "12.34.0",
"resolved": "https://registry.npmjs.org/framer-motion/-/framer-motion-12.34.0.tgz", "resolved": "https://registry.npmjs.org/framer-motion/-/framer-motion-12.34.0.tgz",
@ -8192,6 +8085,14 @@
"node": ">= 14" "node": ">= 14"
} }
}, },
"node_modules/humanize-ms": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/humanize-ms/-/humanize-ms-1.2.1.tgz",
"integrity": "sha512-Fl70vYtsAFb/C06PTS9dZBo7ihau+Tu/DNCk/OyHhea07S+aeMWpFFkUaXRa8fI+ScZbEI8dfSxwY7gxZ9SAVQ==",
"dependencies": {
"ms": "^2.0.0"
}
},
"node_modules/ieee754": { "node_modules/ieee754": {
"version": "1.1.13", "version": "1.1.13",
"resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.1.13.tgz", "resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.1.13.tgz",
@ -9313,6 +9214,25 @@
"node": ">=8.6" "node": ">=8.6"
} }
}, },
"node_modules/mime-db": {
"version": "1.52.0",
"resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz",
"integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==",
"engines": {
"node": ">= 0.6"
}
},
"node_modules/mime-types": {
"version": "2.1.35",
"resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz",
"integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==",
"dependencies": {
"mime-db": "1.52.0"
},
"engines": {
"node": ">= 0.6"
}
},
"node_modules/minimalistic-assert": { "node_modules/minimalistic-assert": {
"version": "1.0.1", "version": "1.0.1",
"resolved": "https://registry.npmjs.org/minimalistic-assert/-/minimalistic-assert-1.0.1.tgz", "resolved": "https://registry.npmjs.org/minimalistic-assert/-/minimalistic-assert-1.0.1.tgz",
@ -9572,6 +9492,44 @@
"node": "^10 || ^12 || >=14" "node": "^10 || ^12 || >=14"
} }
}, },
"node_modules/node-domexception": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/node-domexception/-/node-domexception-1.0.0.tgz",
"integrity": "sha512-/jKZoMpw0F8GRwl4/eLROPA3cfcXtLApP0QzLmUT/HuPCZWyB7IY9ZrMeKw2O/nFIqPQB3PVM9aYm0F312AXDQ==",
"deprecated": "Use your platform's native DOMException instead",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/jimmywarting"
},
{
"type": "github",
"url": "https://paypal.me/jimmywarting"
}
],
"engines": {
"node": ">=10.5.0"
}
},
"node_modules/node-fetch": {
"version": "2.7.0",
"resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-2.7.0.tgz",
"integrity": "sha512-c4FRfUm/dbcWZ7U+1Wq0AwCyFL+3nt2bEw05wfxSz+DWpWsitgmSgYmy2dQdWyKC1694ELPqMs/YzUSNozLt8A==",
"dependencies": {
"whatwg-url": "^5.0.0"
},
"engines": {
"node": "4.x || >=6.0.0"
},
"peerDependencies": {
"encoding": "^0.1.0"
},
"peerDependenciesMeta": {
"encoding": {
"optional": true
}
}
},
"node_modules/node-fetch-native": { "node_modules/node-fetch-native": {
"version": "1.6.7", "version": "1.6.7",
"resolved": "https://registry.npmjs.org/node-fetch-native/-/node-fetch-native-1.6.7.tgz", "resolved": "https://registry.npmjs.org/node-fetch-native/-/node-fetch-native-1.6.7.tgz",
@ -9740,6 +9698,48 @@
"resolved": "https://registry.npmjs.org/ohash/-/ohash-2.0.11.tgz", "resolved": "https://registry.npmjs.org/ohash/-/ohash-2.0.11.tgz",
"integrity": "sha512-RdR9FQrFwNBNXAr4GixM8YaRZRJ5PUWbKYbE5eOsrwAjJW0q2REGcf79oYPsLyskQCZG1PLN+S/K1V00joZAoQ==" "integrity": "sha512-RdR9FQrFwNBNXAr4GixM8YaRZRJ5PUWbKYbE5eOsrwAjJW0q2REGcf79oYPsLyskQCZG1PLN+S/K1V00joZAoQ=="
}, },
"node_modules/openai": {
"version": "4.104.0",
"resolved": "https://registry.npmjs.org/openai/-/openai-4.104.0.tgz",
"integrity": "sha512-p99EFNsA/yX6UhVO93f5kJsDRLAg+CTA2RBqdHK4RtK8u5IJw32Hyb2dTGKbnnFmnuoBv5r7Z2CURI9sGZpSuA==",
"dependencies": {
"@types/node": "^18.11.18",
"@types/node-fetch": "^2.6.4",
"abort-controller": "^3.0.0",
"agentkeepalive": "^4.2.1",
"form-data-encoder": "1.7.2",
"formdata-node": "^4.3.2",
"node-fetch": "^2.6.7"
},
"bin": {
"openai": "bin/cli"
},
"peerDependencies": {
"ws": "^8.18.0",
"zod": "^3.23.8"
},
"peerDependenciesMeta": {
"ws": {
"optional": true
},
"zod": {
"optional": true
}
}
},
"node_modules/openai/node_modules/@types/node": {
"version": "18.19.130",
"resolved": "https://registry.npmjs.org/@types/node/-/node-18.19.130.tgz",
"integrity": "sha512-GRaXQx6jGfL8sKfaIDD6OupbIHBr9jv7Jnaml9tB7l4v068PAOXqfcujMMo5PhbIs6ggR1XODELqahT2R8v0fg==",
"dependencies": {
"undici-types": "~5.26.4"
}
},
"node_modules/openai/node_modules/undici-types": {
"version": "5.26.5",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-5.26.5.tgz",
"integrity": "sha512-JlCMO+ehdEIKqlFxk6IfVoAUVmgz7cU7zD/h9XZ0qzeosSHmUJVOzSQvvYSYWXkFXC+IfLKSIffhv0sVZup6pA=="
},
"node_modules/operetta": { "node_modules/operetta": {
"version": "0.0.5", "version": "0.0.5",
"resolved": "https://registry.npmjs.org/operetta/-/operetta-0.0.5.tgz", "resolved": "https://registry.npmjs.org/operetta/-/operetta-0.0.5.tgz",
@ -11378,6 +11378,11 @@
"node": ">=8.0" "node": ">=8.0"
} }
}, },
"node_modules/tr46": {
"version": "0.0.3",
"resolved": "https://registry.npmjs.org/tr46/-/tr46-0.0.3.tgz",
"integrity": "sha512-N3WMsuqV66lT30CrXNbEjx4GEwlow3v6rr4mCcv6prnfwhS01rkgyFdjPNBYd9br7LpXV1+Emh01fHnq2Gdgrw=="
},
"node_modules/ts-api-utils": { "node_modules/ts-api-utils": {
"version": "2.4.0", "version": "2.4.0",
"resolved": "https://registry.npmjs.org/ts-api-utils/-/ts-api-utils-2.4.0.tgz", "resolved": "https://registry.npmjs.org/ts-api-utils/-/ts-api-utils-2.4.0.tgz",
@ -12422,6 +12427,28 @@
"node": ">= 16" "node": ">= 16"
} }
}, },
"node_modules/web-streams-polyfill": {
"version": "4.0.0-beta.3",
"resolved": "https://registry.npmjs.org/web-streams-polyfill/-/web-streams-polyfill-4.0.0-beta.3.tgz",
"integrity": "sha512-QW95TCTaHmsYfHDybGMwO5IJIM93I/6vTRk+daHTWFPhwh+C8Cg7j7XyKrwrj8Ib6vYXe0ocYNrmzY4xAAN6ug==",
"engines": {
"node": ">= 14"
}
},
"node_modules/webidl-conversions": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-3.0.1.tgz",
"integrity": "sha512-2JAn3z8AR6rjK8Sm8orRC0h/bcl/DqL7tRPdGZ4I1CjdF+EaMLmYxBHyXuKL849eucPFhvBoxMsflfOb8kxaeQ=="
},
"node_modules/whatwg-url": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/whatwg-url/-/whatwg-url-5.0.0.tgz",
"integrity": "sha512-saE57nupxk6v3HY35+jzBwYa0rKSy0XR8JSxZPwgLr7ys0IBzhGviA1/TUGJLmSVqs8pb9AnvICXEuOHLprYTw==",
"dependencies": {
"tr46": "~0.0.3",
"webidl-conversions": "^3.0.0"
}
},
"node_modules/which": { "node_modules/which": {
"version": "2.0.2", "version": "2.0.2",
"resolved": "https://registry.npmjs.org/which/-/which-2.0.2.tgz", "resolved": "https://registry.npmjs.org/which/-/which-2.0.2.tgz",

View File

@ -19,7 +19,6 @@
"dependencies": { "dependencies": {
"@auth/core": "^0.41.0", "@auth/core": "^0.41.0",
"@auth/prisma-adapter": "^2.11.1", "@auth/prisma-adapter": "^2.11.1",
"@aws-sdk/client-bedrock-runtime": "^3.989.0",
"@aws-sdk/client-s3": "^3.989.0", "@aws-sdk/client-s3": "^3.989.0",
"@prisma/adapter-pg": "^7.6.0", "@prisma/adapter-pg": "^7.6.0",
"@prisma/client": "^7.6.0", "@prisma/client": "^7.6.0",
@ -41,6 +40,7 @@
"next": "^16.2.1", "next": "^16.2.1",
"next-auth": "^5.0.0-beta.30", "next-auth": "^5.0.0-beta.30",
"nodemailer": "^7.0.7", "nodemailer": "^7.0.7",
"openai": "^4.104.0",
"otplib": "^13.2.1", "otplib": "^13.2.1",
"pdf-parse": "^2.4.5", "pdf-parse": "^2.4.5",
"pg": "^8.18.0", "pg": "^8.18.0",

View File

@ -0,0 +1,9 @@
-- CreateEnum
CREATE TYPE "KnowledgeBaseMode" AS ENUM ('UNSET', 'STRUCTURED', 'DOCUMENTS');
-- AlterTable
ALTER TABLE "Store" ADD COLUMN "knowledgeBaseMode" "KnowledgeBaseMode" NOT NULL DEFAULT 'UNSET';
ALTER TABLE "Store" ADD COLUMN "knowledgeTemplate" JSONB;
-- AlterEnum: add STRUCTURED to KnowledgeChunkSourceType
ALTER TYPE "KnowledgeChunkSourceType" ADD VALUE 'STRUCTURED';

View File

@ -114,6 +114,11 @@ model Store {
storefrontChatBrandColor String? storefrontChatBrandColor String?
storefrontChatFontFamily String? storefrontChatFontFamily String?
storefrontChatLogoUrl String? storefrontChatLogoUrl String?
/** Primary knowledge source: structured questionnaire vs uploaded documents (mutually exclusive). */
knowledgeBaseMode KnowledgeBaseMode @default(UNSET)
/** Saved structured KB answers (version, per-field text, misc). Embedded as STRUCTURED chunks. */
knowledgeTemplate Json?
// Relations // Relations
connections Connection[] connections Connection[]
@ -485,6 +490,13 @@ enum KnowledgeDocumentStatus {
enum KnowledgeChunkSourceType { enum KnowledgeChunkSourceType {
DOCUMENT DOCUMENT
FAQ FAQ
STRUCTURED
}
enum KnowledgeBaseMode {
UNSET
STRUCTURED
DOCUMENTS
} }
model KnowledgeDocument { model KnowledgeDocument {

View File

@ -1,40 +1,37 @@
#!/usr/bin/env tsx #!/usr/bin/env tsx
/** /**
* Re-embedding script for Knowledge Base * Re-embed all KnowledgeChunk rows with OpenAI (text-embedding-3-small @ 1024 dims)
* * and tag metadata with provider openai (required for KB vector search filters).
* This script re-embeds all existing KnowledgeChunk records using AWS Bedrock *
* Titan Text Embeddings V2 (1024 dimensions) to replace Gemini embeddings (1536 dimensions). * Usage: npm run re-embed-knowledge-base
*
* Usage:
* npm run re-embed-knowledge-base
* or
* tsx scripts/re-embed-knowledge-base.ts
*/ */
import { prisma } from "../src/lib/prisma" import { prisma } from "../src/lib/prisma"
import { embedBatch } from "../src/lib/kb/bedrockEmbeddings" import { embedBatch } from "../src/lib/kb/openaiEmbeddings"
import {
kbEmbeddingMetadata,
KB_EMBEDDING_DIMENSIONS,
} from "../src/lib/kb/embeddingMeta"
import { logger } from "../src/lib/logger" import { logger } from "../src/lib/logger"
// AWS Bedrock Titan V2 limit: 2000 requests/minute const BATCH_SIZE = 32
// Use smaller batches with delays to stay under limit
const BATCH_SIZE = 10 // Process chunks in smaller batches to avoid rate limits
interface KnowledgeChunk { interface KnowledgeChunkRow {
id: string id: string
content: string content: string
storeId: string storeId: string
metadata: unknown
} }
async function reEmbedKnowledgeBase() { async function reEmbedKnowledgeBase() {
try { try {
logger.info("Starting knowledge base re-embedding process...") logger.info("Starting knowledge base re-embedding (OpenAI)...")
// Fetch all knowledge chunks const sql = `SELECT "id", "content", "storeId", "metadata"
const sql = `SELECT "id", "content", "storeId"
FROM "KnowledgeChunk" FROM "KnowledgeChunk"
WHERE "content" IS NOT NULL AND "content" != '' WHERE "content" IS NOT NULL AND "content" != ''
ORDER BY "createdAt" ASC` ORDER BY "createdAt" ASC`
const chunks = await prisma.$queryRawUnsafe<KnowledgeChunk[]>(sql) const chunks = await prisma.$queryRawUnsafe<KnowledgeChunkRow[]>(sql)
if (chunks.length === 0) { if (chunks.length === 0) {
logger.info("No knowledge chunks found to re-embed.") logger.info("No knowledge chunks found to re-embed.")
@ -47,7 +44,6 @@ async function reEmbedKnowledgeBase() {
let failed = 0 let failed = 0
const totalBatches = Math.ceil(chunks.length / BATCH_SIZE) const totalBatches = Math.ceil(chunks.length / BATCH_SIZE)
// Process in batches
for (let i = 0; i < chunks.length; i += BATCH_SIZE) { for (let i = 0; i < chunks.length; i += BATCH_SIZE) {
const batch = chunks.slice(i, i + BATCH_SIZE) const batch = chunks.slice(i, i + BATCH_SIZE)
const batchNumber = Math.floor(i / BATCH_SIZE) + 1 const batchNumber = Math.floor(i / BATCH_SIZE) + 1
@ -57,8 +53,7 @@ async function reEmbedKnowledgeBase() {
) )
try { try {
// Generate embeddings for this batch const texts = batch.map((c) => c.content)
const texts = batch.map((chunk: KnowledgeChunk) => chunk.content)
const embeddings = await embedBatch(texts) const embeddings = await embedBatch(texts)
if (embeddings.length !== batch.length) { if (embeddings.length !== batch.length) {
@ -69,60 +64,55 @@ async function reEmbedKnowledgeBase() {
continue continue
} }
// Update each chunk with its new embedding await Promise.all(
const updatePromises = batch.map(async (chunk: KnowledgeChunk, index: number) => { batch.map(async (chunk, index) => {
const embedding = embeddings[index] const embedding = embeddings[index]
if (!embedding || embedding.length === 0) { if (!embedding || embedding.length === 0) {
logger.warn( logger.warn(`Skipping chunk ${chunk.id}: empty embedding`)
`Skipping chunk ${chunk.id}: empty embedding generated`, failed++
) return
failed++
return
}
if (embedding.length !== 1024) {
logger.warn(
`Chunk ${chunk.id}: embedding dimension is ${embedding.length}, expected 1024`,
)
// Pad or truncate to 1024
if (embedding.length < 1024) {
embedding.push(...new Array(1024 - embedding.length).fill(0))
} else {
embedding.splice(1024)
} }
}
try { if (embedding.length !== KB_EMBEDDING_DIMENSIONS) {
await prisma.$executeRawUnsafe( logger.error(
` `Chunk ${chunk.id}: expected ${KB_EMBEDDING_DIMENSIONS} dims, got ${embedding.length}`,
UPDATE "KnowledgeChunk" )
SET "embedding" = $1::vector(1024), failed++
"updatedAt" = NOW() return
WHERE "id" = $2 }
`,
`[${embedding.join(",")}]`,
chunk.id,
)
processed++
} catch (error) {
logger.error(`Failed to update chunk ${chunk.id}:`, error)
failed++
}
})
await Promise.all(updatePromises) const prevMeta =
chunk.metadata &&
typeof chunk.metadata === "object" &&
!Array.isArray(chunk.metadata)
? (chunk.metadata as Record<string, unknown>)
: {}
const nextMeta = {
...prevMeta,
...kbEmbeddingMetadata(),
}
logger.info( try {
`Batch ${batchNumber} completed. Processed: ${processed}, Failed: ${failed}`, await prisma.$executeRawUnsafe(
`
UPDATE "KnowledgeChunk"
SET "embedding" = $1::vector,
"metadata" = $2::jsonb,
"updatedAt" = NOW()
WHERE "id" = $3
`,
`[${embedding.join(",")}]`,
JSON.stringify(nextMeta),
chunk.id,
)
processed++
} catch (error) {
logger.error(`Failed to update chunk ${chunk.id}:`, error)
failed++
}
}),
) )
// Add delay between batches to avoid rate limiting
// With 2s delay per request + batch processing, we need extra time between batches
if (i + BATCH_SIZE < chunks.length) {
// Wait 5 seconds between batches to ensure we stay under 2000 req/min limit
await new Promise((resolve) => setTimeout(resolve, 5000))
}
} catch (error) { } catch (error) {
logger.error(`Error processing batch ${batchNumber}:`, error) logger.error(`Error processing batch ${batchNumber}:`, error)
failed += batch.length failed += batch.length
@ -140,7 +130,6 @@ async function reEmbedKnowledgeBase() {
} }
} }
// Run the script
if (require.main === module) { if (require.main === module) {
reEmbedKnowledgeBase() reEmbedKnowledgeBase()
.then(() => { .then(() => {

View File

@ -40,6 +40,23 @@ export async function POST(
return NextResponse.json({ message: "Store not found" }, { status: 404 }) return NextResponse.json({ message: "Store not found" }, { status: 404 })
} }
if (store.knowledgeBaseMode === "STRUCTURED") {
return NextResponse.json(
{
message:
"Document upload is disabled while using structured knowledge. Switch to document-based knowledge in Settings → Knowledge base.",
},
{ status: 409 },
)
}
if (store.knowledgeBaseMode === "UNSET") {
await prisma.store.update({
where: { id: store.id },
data: { knowledgeBaseMode: "DOCUMENTS" },
})
}
const formData = await request.formData() const formData = await request.formData()
const file = formData.get("file") const file = formData.get("file")

View File

@ -2,7 +2,8 @@ import { NextResponse } from "next/server"
import { auth } from "@/auth" import { auth } from "@/auth"
import { prisma } from "@/lib/prisma" import { prisma } from "@/lib/prisma"
import { logger } from "@/lib/logger" import { logger } from "@/lib/logger"
import { embedBatch } from "@/lib/kb/bedrockEmbeddings" import { embedBatch } from "@/lib/kb/openaiEmbeddings"
import { kbEmbeddingMetadata } from "@/lib/kb/embeddingMeta"
import type { FaqItem } from "@prisma/client" import type { FaqItem } from "@prisma/client"
type FaqImportItem = { type FaqImportItem = {
@ -35,6 +36,16 @@ export async function POST(
return NextResponse.json({ message: "Store not found" }, { status: 404 }) return NextResponse.json({ message: "Store not found" }, { status: 404 })
} }
if (store.knowledgeBaseMode === "STRUCTURED") {
return NextResponse.json(
{
message:
"FAQ import is disabled while using structured knowledge. Switch to document-based knowledge in Settings.",
},
{ status: 409 },
)
}
let items: FaqImportItem[] = [] let items: FaqImportItem[] = []
const contentType = request.headers.get("content-type") || "" const contentType = request.headers.get("content-type") || ""
@ -135,6 +146,7 @@ export async function POST(
{ {
question: faq.question, question: faq.question,
answer: faq.answer, answer: faq.answer,
...kbEmbeddingMetadata(),
} as unknown as object, } as unknown as object,
) )
}), }),

View File

@ -2,7 +2,8 @@ import { NextResponse } from "next/server"
import { auth } from "@/auth" import { auth } from "@/auth"
import { prisma } from "@/lib/prisma" import { prisma } from "@/lib/prisma"
import { logger } from "@/lib/logger" import { logger } from "@/lib/logger"
import { embedBatch } from "@/lib/kb/bedrockEmbeddings" import { embedBatch } from "@/lib/kb/openaiEmbeddings"
import { kbEmbeddingMetadata } from "@/lib/kb/embeddingMeta"
export async function GET( export async function GET(
_request: Request, _request: Request,
@ -67,6 +68,16 @@ export async function POST(
return NextResponse.json({ message: "Store not found" }, { status: 404 }) return NextResponse.json({ message: "Store not found" }, { status: 404 })
} }
if (store.knowledgeBaseMode === "STRUCTURED") {
return NextResponse.json(
{
message:
"Manual FAQs are disabled while using structured knowledge. Switch to document-based knowledge in Settings to add FAQs.",
},
{ status: 409 },
)
}
const body = await request.json() const body = await request.json()
const { question, answer, tags } = body as { const { question, answer, tags } = body as {
question?: string question?: string
@ -115,6 +126,7 @@ export async function POST(
{ {
question, question,
answer, answer,
...kbEmbeddingMetadata(),
} as unknown as object, } as unknown as object,
) )
} }

View File

@ -2,20 +2,22 @@ import { NextResponse } from "next/server"
import { auth } from "@/auth" import { auth } from "@/auth"
import { prisma } from "@/lib/prisma" import { prisma } from "@/lib/prisma"
import { logger } from "@/lib/logger" import { logger } from "@/lib/logger"
import { embedBatch } from "@/lib/kb/bedrockEmbeddings" import { embedBatch } from "@/lib/kb/openaiEmbeddings"
import { chatCompletion } from "@/lib/kb/bedrockChat" import { KB_EMBEDDING_PROVIDER } from "@/lib/kb/embeddingMeta"
import { generateKbRagAnswer } from "@/lib/kb/kbAnswer"
type QueryBody = { type QueryBody = {
query?: string query?: string
topK?: number topK?: number
includeFaq?: boolean includeFaq?: boolean
includeDocuments?: boolean includeDocuments?: boolean
includeStructured?: boolean
} }
type KnowledgeChunkRow = { type KnowledgeChunkRow = {
id: string id: string
content: string content: string
sourceType: "DOCUMENT" | "FAQ" sourceType: "DOCUMENT" | "FAQ" | "STRUCTURED"
documentId: string | null documentId: string | null
faqItemId: string | null faqItemId: string | null
metadata: unknown metadata: unknown
@ -58,10 +60,14 @@ export async function POST(
const topK = Math.min(Math.max(body.topK ?? 8, 1), 20) const topK = Math.min(Math.max(body.topK ?? 8, 1), 20)
const includeFaq = body.includeFaq ?? true const includeFaq = body.includeFaq ?? true
const includeDocuments = body.includeDocuments ?? true const includeDocuments = body.includeDocuments ?? true
const includeStructured = body.includeStructured ?? true
if (!includeFaq && !includeDocuments) { if (!includeFaq && !includeDocuments && !includeStructured) {
return NextResponse.json( return NextResponse.json(
{ message: "At least one of FAQs or Documents must be included" }, {
message:
"At least one of FAQs, documents, or structured knowledge must be included",
},
{ status: 400 }, { status: 400 },
) )
} }
@ -74,14 +80,18 @@ export async function POST(
) )
} }
const filters: string[] = ['"storeId" = $1'] const sourceTypes: string[] = []
if (includeFaq && !includeDocuments) { if (includeFaq) sourceTypes.push("'FAQ'")
filters.push(`"sourceType" = 'FAQ'`) if (includeDocuments) sourceTypes.push("'DOCUMENT'")
} else if (!includeFaq && includeDocuments) { if (includeStructured) sourceTypes.push("'STRUCTURED'")
filters.push(`"sourceType" = 'DOCUMENT'`)
}
const whereClause = filters.length ? `WHERE ${filters.join(" AND ")}` : "" const filters: string[] = [
'"storeId" = $1',
`"sourceType" IN (${sourceTypes.join(", ")})`,
`COALESCE(("metadata"::jsonb->>'provider'), '') = '${KB_EMBEDDING_PROVIDER}'`,
]
const whereClause = `WHERE ${filters.join(" AND ")}`
const chunks = await prisma.$queryRawUnsafe<KnowledgeChunkRow[]>( const chunks = await prisma.$queryRawUnsafe<KnowledgeChunkRow[]>(
` `
@ -113,7 +123,9 @@ export async function POST(
const label = const label =
chunk.sourceType === "FAQ" chunk.sourceType === "FAQ"
? `FAQ #${index + 1}` ? `FAQ #${index + 1}`
: `Document chunk #${index + 1}` : chunk.sourceType === "STRUCTURED"
? `Store profile #${index + 1}`
: `Document chunk #${index + 1}`
return `${label} (similarity: ${chunk.similarity.toFixed(2)}):\n${ return `${label} (similarity: ${chunk.similarity.toFixed(2)}):\n${
chunk.content chunk.content
}` }`
@ -123,23 +135,21 @@ export async function POST(
const systemPrompt = const systemPrompt =
"You are an AI assistant helping answer customer and vendor questions for a specific store.\n" + "You are an AI assistant helping answer customer and vendor questions for a specific store.\n" +
"Use ONLY the provided context from the store's documents and FAQs.\n" + "Use ONLY the provided context from the store's documents, FAQs, and structured store profile.\n" +
"If the answer is not clearly present in the context, say that you do not know.\n" + "If the answer is not clearly present in the context, say that you do not know.\n" +
"Be concise and helpful.\n" "Be concise and helpful.\n"
const userMessage = `Context:\n${context}\n\nQuestion:\n${question}` const userMessage = `Context:\n${context}\n\nQuestion:\n${question}`
let answer: string const { text: answer, source } = await generateKbRagAnswer(userMessage, {
try { systemPrompt,
answer = await chatCompletion(userMessage, { maxTokens: 512,
systemPrompt, })
maxTokens: 512,
}) if (source === "none") {
} catch (error) {
logger.error("Bedrock RAG answer error", error)
return NextResponse.json( return NextResponse.json(
{ message: "Failed to generate answer from Bedrock" }, { message: "Failed to generate an answer" },
{ status: 500 }, { status: 502 },
) )
} }
@ -161,4 +171,3 @@ export async function POST(
) )
} }
} }

View File

@ -0,0 +1,214 @@
import { NextResponse } from "next/server"
import { auth } from "@/auth"
import { prisma } from "@/lib/prisma"
import { logger } from "@/lib/logger"
import {
knowledgeTemplatePutSchema,
normalizeTemplateAnswers,
type KnowledgeTemplateStored,
} from "@/lib/kb/knowledgeTemplate"
import { replaceStructuredKnowledgeChunks } from "@/lib/kb/structuredTemplateChunks"
import {
deleteDocumentKbForStore,
deleteFaqKbForStore,
deleteStructuredKbForStore,
} from "@/lib/kb/storeKbCleanup"
import { Prisma, type KnowledgeBaseMode } from "@prisma/client"
export async function GET(
_request: Request,
{ params }: { params: Promise<{ storeId: string }> },
) {
try {
const session = await auth()
if (!session?.user?.id) {
return NextResponse.json({ message: "Unauthorized" }, { status: 401 })
}
const { storeId } = await params
const store = await prisma.store.findFirst({
where: { id: storeId, userId: session.user.id },
select: {
id: true,
knowledgeBaseMode: true,
knowledgeTemplate: true,
},
})
if (!store) {
return NextResponse.json({ message: "Store not found" }, { status: 404 })
}
const [documentCount, faqCount, structuredChunkCount] = await Promise.all([
prisma.knowledgeDocument.count({ where: { storeId: store.id } }),
prisma.faqItem.count({ where: { storeId: store.id } }),
prisma.knowledgeChunk.count({
where: { storeId: store.id, sourceType: "STRUCTURED" },
}),
])
return NextResponse.json({
knowledgeBaseMode: store.knowledgeBaseMode,
knowledgeTemplate: store.knowledgeTemplate,
counts: {
documents: documentCount,
faqs: faqCount,
structuredChunks: structuredChunkCount,
},
})
} catch (error) {
logger.error("GET knowledge-template error", error)
return NextResponse.json(
{ message: "Failed to load knowledge template" },
{ status: 500 },
)
}
}
export async function PUT(
request: Request,
{ params }: { params: Promise<{ storeId: string }> },
) {
try {
const session = await auth()
if (!session?.user?.id) {
return NextResponse.json({ message: "Unauthorized" }, { status: 401 })
}
const { storeId } = await params
let store = await prisma.store.findFirst({
where: { id: storeId, userId: session.user.id },
})
if (!store) {
return NextResponse.json({ message: "Store not found" }, { status: 404 })
}
const json = await request.json().catch(() => null)
const parsed = knowledgeTemplatePutSchema.safeParse(json)
if (!parsed.success) {
return NextResponse.json(
{ message: "Invalid body", issues: parsed.error.flatten() },
{ status: 400 },
)
}
const body = parsed.data
const confirm = body.confirmDestructiveSwitch === true
const docCount = await prisma.knowledgeDocument.count({
where: { storeId: store.id },
})
const faqCount = await prisma.faqItem.count({ where: { storeId: store.id } })
const structuredCount = await prisma.knowledgeChunk.count({
where: { storeId: store.id, sourceType: "STRUCTURED" },
})
const hasTemplateJson = store.knowledgeTemplate != null
if (body.knowledgeBaseMode !== undefined) {
const target = body.knowledgeBaseMode as KnowledgeBaseMode
if (target !== store.knowledgeBaseMode) {
if (target === "STRUCTURED") {
const destructive = docCount > 0 || faqCount > 0
if (destructive && !confirm) {
return NextResponse.json(
{
message:
"Switching to structured knowledge will remove uploaded documents and FAQs. Send confirmDestructiveSwitch: true to proceed.",
code: "CONFIRM_REQUIRED",
},
{ status: 400 },
)
}
if (destructive) {
await deleteDocumentKbForStore(prisma, store.id)
await deleteFaqKbForStore(prisma, store.id)
}
await prisma.store.update({
where: { id: store.id },
data: { knowledgeBaseMode: "STRUCTURED" },
})
} else if (target === "DOCUMENTS") {
const destructive = structuredCount > 0 || hasTemplateJson
if (destructive && !confirm) {
return NextResponse.json(
{
message:
"Switching to document-based knowledge will remove your structured profile and its embeddings. Send confirmDestructiveSwitch: true to proceed.",
code: "CONFIRM_REQUIRED",
},
{ status: 400 },
)
}
await deleteStructuredKbForStore(prisma, store.id)
await prisma.store.update({
where: { id: store.id },
data: {
knowledgeBaseMode: "DOCUMENTS",
knowledgeTemplate: Prisma.JsonNull,
},
})
}
}
}
store = await prisma.store.findFirstOrThrow({
where: { id: store.id, userId: session.user.id },
})
if (body.template !== undefined) {
if (store.knowledgeBaseMode === "DOCUMENTS") {
return NextResponse.json(
{
message:
"This store uses document-based knowledge. Switch to structured knowledge before editing the template.",
},
{ status: 409 },
)
}
const answers = normalizeTemplateAnswers(body.template.answers)
const misc = body.template.misc?.trim() || undefined
const stored: KnowledgeTemplateStored = {
version: 1,
answers,
misc,
updatedAt: new Date().toISOString(),
}
const nextMode: KnowledgeBaseMode =
store.knowledgeBaseMode === "UNSET" ? "STRUCTURED" : store.knowledgeBaseMode
await prisma.store.update({
where: { id: store.id },
data: {
knowledgeBaseMode: nextMode,
knowledgeTemplate: stored as object,
},
})
await replaceStructuredKnowledgeChunks(store.id, stored)
}
const refreshed = await prisma.store.findFirst({
where: { id: store.id },
select: {
knowledgeBaseMode: true,
knowledgeTemplate: true,
},
})
return NextResponse.json({
success: true,
knowledgeBaseMode: refreshed?.knowledgeBaseMode,
knowledgeTemplate: refreshed?.knowledgeTemplate,
})
} catch (error) {
logger.error("PUT knowledge-template error", error)
return NextResponse.json(
{ message: "Failed to save knowledge template" },
{ status: 500 },
)
}
}

View File

@ -1,138 +1,394 @@
"use client" "use client"
import { useEffect, useState } from "react" import { useCallback, useEffect, useState } from "react"
import { DocumentsSection } from "@/components/kb/DocumentsSection" import { DocumentsFaqKnowledgeModal } from "@/components/kb/DocumentsFaqKnowledgeModal"
import { FaqSection } from "@/components/kb/FaqSection" import { StructuredKnowledgeModal } from "@/components/kb/StructuredKnowledgeModal"
import { AskSection } from "@/components/kb/AskSection"
import { useTheme } from "@/components/ThemeProvider" import { useTheme } from "@/components/ThemeProvider"
import { cn } from "@/lib/utils" import { cn } from "@/lib/utils"
import type { KnowledgeTemplateStored } from "@/lib/kb/knowledgeTemplate"
import { BookOpen, ChevronRight, FileStack, Loader2 } from "lucide-react"
type KbCounts = {
documents: number
faqs: number
structuredChunks: number
}
export default function KnowledgeBasePage() { export default function KnowledgeBasePage() {
const { theme } = useTheme() const { theme } = useTheme()
const isLight = theme === "light" const isLight = theme === "light"
const [activeStoreId, setActiveStoreId] = useState<string | null>(null) const [activeStoreId, setActiveStoreId] = useState<string | null>(null)
const [kbLoading, setKbLoading] = useState(true)
const [knowledgeBaseMode, setKnowledgeBaseMode] = useState<
"UNSET" | "STRUCTURED" | "DOCUMENTS"
>("UNSET")
const [structuredOpen, setStructuredOpen] = useState(false)
const [documentsModalOpen, setDocumentsModalOpen] = useState(false)
const [modeBusy, setModeBusy] = useState<"documents" | null>(null)
const [toast, setToast] = useState<string | null>(null)
const loadKb = useCallback(async (sid: string) => {
const res = await fetch(`/api/stores/${encodeURIComponent(sid)}/knowledge-template`)
if (!res.ok) return
const j = (await res.json()) as {
knowledgeBaseMode: "UNSET" | "STRUCTURED" | "DOCUMENTS"
knowledgeTemplate: KnowledgeTemplateStored | null
counts: KbCounts
}
setKnowledgeBaseMode(j.knowledgeBaseMode)
}, [])
useEffect(() => { useEffect(() => {
let cancelled = false let cancelled = false
; (async () => { ; (async () => {
try { try {
const res = await fetch("/api/stores") const res = await fetch("/api/stores")
if (!res.ok) return if (!res.ok) return
const data = (await res.json()) as { const data = (await res.json()) as {
stores?: { id: string; isActive: boolean }[] stores?: { id: string; isActive: boolean }[]
} }
const active = (data.stores || []).find((s) => s.isActive) const active = (data.stores || []).find((s) => s.isActive)
if (!cancelled) setActiveStoreId(active?.id ?? null)
if (!cancelled) {
setActiveStoreId(active?.id ?? null)
}
} catch { } catch {
if (!cancelled) { if (!cancelled) setActiveStoreId(null)
setActiveStoreId(null)
}
} }
})() })()
return () => { return () => {
cancelled = true cancelled = true
} }
}, []) }, [])
const [activeTab, setActiveTab] = useState<"documents" | "faqs" | "ask">( useEffect(() => {
"documents", if (!activeStoreId) {
) setKbLoading(false)
setKnowledgeBaseMode("UNSET")
return
}
let cancelled = false
; (async () => {
try {
setKbLoading(true)
await loadKb(activeStoreId)
} finally {
if (!cancelled) setKbLoading(false)
}
})()
return () => {
cancelled = true
}
}, [activeStoreId, loadKb])
async function ensureDocumentsMode(
confirmDestructiveSwitch?: boolean,
): Promise<boolean> {
if (!activeStoreId) return false
if (knowledgeBaseMode === "DOCUMENTS") return true
setModeBusy("documents")
setToast(null)
try {
const res = await fetch(
`/api/stores/${encodeURIComponent(activeStoreId)}/knowledge-template`,
{
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
knowledgeBaseMode: "DOCUMENTS",
...(confirmDestructiveSwitch
? { confirmDestructiveSwitch: true }
: {}),
}),
},
)
const j = await res.json().catch(() => ({}))
if (res.status === 400 && j.code === "CONFIRM_REQUIRED") {
const ok = window.confirm(
"This removes your structured profile and its embeddings. Continue?",
)
if (!ok) return false
return await ensureDocumentsMode(true)
}
if (!res.ok) {
throw new Error(j.message || "Failed to update mode")
}
await loadKb(activeStoreId)
setToast("Using documents & FAQs for this store.")
return true
} catch (e) {
setToast(e instanceof Error ? e.message : "Could not switch mode")
return false
} finally {
setModeBusy(null)
}
}
async function openDocumentsKnowledgePanel() {
if (!activeStoreId) return
const ok = await ensureDocumentsMode()
if (ok) setDocumentsModalOpen(true)
}
const hasStore = Boolean(activeStoreId) const hasStore = Boolean(activeStoreId)
const structuredActive = knowledgeBaseMode === "STRUCTURED"
const modeLabel =
knowledgeBaseMode === "STRUCTURED"
? "Structured profile"
: knowledgeBaseMode === "DOCUMENTS"
? "Documents & FAQs"
: "Not chosen yet"
return ( return (
<div className="flex flex-col h-full space-y-6"> <div className="flex flex-col h-full space-y-5">
<div className="flex items-center justify-between"> <div>
<div> <h1
<h1 className={cn("text-2xl font-bold", isLight ? "text-slate-900" : "text-white")}> className={cn(
Knowledge Base "text-2xl font-bold tracking-tight",
</h1> isLight ? "text-slate-900" : "text-white",
<p className={cn("text-sm", isLight ? "text-slate-500" : "text-slate-400")}> )}
Centralize your store&apos;s documents and FAQs so JUNO can >
answer customer and vendor questions. Knowledge base
</p> </h1>
</div> <p
className={cn(
"text-sm mt-1 max-w-2xl",
isLight ? "text-slate-500" : "text-slate-400",
)}
>
One source of truth per store for AI: either a guided profile or
uploaded docs & FAQs—not both.
</p>
</div> </div>
{!hasStore ? ( {!hasStore ? (
<div className="flex-1 flex items-center justify-center"> <div className="flex-1 flex items-center justify-center py-16">
<div className="text-center max-w-md"> <p
<h2 className={cn("text-lg font-semibold mb-2", isLight ? "text-slate-900" : "text-white")}> className={cn(
No active store selected "text-sm text-center max-w-sm",
</h2> isLight ? "text-slate-500" : "text-slate-400",
<p className={cn("text-sm", isLight ? "text-slate-500" : "text-slate-400")}> )}
Select a store from the top bar to manage its knowledge >
base. Select an active store from the top bar to manage its knowledge
</p> base.
</div> </p>
</div>
) : kbLoading ? (
<div className="flex items-center gap-2 text-sm text-slate-400 py-8">
<Loader2 className="w-4 h-4 animate-spin" />
Loading…
</div> </div>
) : ( ) : (
<> <>
<div className={cn("flex items-center space-x-2 border-b", isLight ? "border-slate-100" : "border-white/10")}> <div
<button id="kb-source"
type="button" className={cn(
onClick={() => setActiveTab("documents")} "rounded-xl border p-4 scroll-mt-4",
className={cn( isLight
"px-4 py-2 text-sm font-medium border-b-2 -mb-px transition-colors", ? "border-slate-200 bg-white shadow-sm"
activeTab === "documents" : "border-white/10 bg-white/[0.04]",
? "border-indigo-500 text-indigo-600" )}
: isLight >
? "border-transparent text-slate-500 hover:text-slate-900 hover:border-slate-300" <div className="flex flex-col sm:flex-row sm:items-center sm:justify-between gap-3 mb-3">
: "border-transparent text-slate-400 hover:text-white hover:border-slate-500" <div>
)} <p
> className={cn(
Documents "text-[11px] font-bold uppercase tracking-wider",
</button> isLight ? "text-slate-400" : "text-slate-500",
<button )}
type="button" >
onClick={() => setActiveTab("faqs")} Knowledge source
className={cn( </p>
"px-4 py-2 text-sm font-medium border-b-2 -mb-px transition-colors", <p
activeTab === "faqs" className={cn(
? "border-indigo-500 text-indigo-600" "text-sm font-medium mt-0.5",
: isLight isLight ? "text-slate-800" : "text-slate-200",
? "border-transparent text-slate-500 hover:text-slate-900 hover:border-slate-300" )}
: "border-transparent text-slate-400 hover:text-white hover:border-slate-500" >
)} {modeLabel}
> </p>
FAQs </div>
</button> </div>
<button
type="button" <div className="grid grid-cols-1 sm:grid-cols-2 gap-2 sm:gap-3">
onClick={() => setActiveTab("ask")} <button
className={cn( type="button"
"px-4 py-2 text-sm font-medium border-b-2 -mb-px transition-colors", onClick={() => setStructuredOpen(true)}
activeTab === "ask" className={cn(
? "border-indigo-500 text-indigo-600" "flex items-center gap-3 rounded-lg border px-4 py-3 text-left transition-all",
: isLight knowledgeBaseMode === "STRUCTURED"
? "border-transparent text-slate-500 hover:text-slate-900 hover:border-slate-300" ? isLight
: "border-transparent text-slate-400 hover:text-white hover:border-slate-500" ? "border-indigo-400 bg-indigo-50/80 ring-1 ring-indigo-200"
)} : "border-indigo-500/50 bg-indigo-500/10"
> : isLight
Ask ? "border-slate-200 hover:border-slate-300 hover:bg-slate-50/80"
</button> : "border-white/10 hover:border-white/20 hover:bg-white/[0.06]",
)}
>
<div
className={cn(
"flex h-10 w-10 items-center justify-center rounded-lg shrink-0",
isLight ? "bg-indigo-100" : "bg-indigo-500/20",
)}
>
<BookOpen
className={cn(
"h-5 w-5",
isLight ? "text-indigo-600" : "text-indigo-300",
)}
/>
</div>
<div className="min-w-0 flex-1">
<p
className={cn(
"text-sm font-semibold",
isLight ? "text-slate-900" : "text-white",
)}
>
Structured profile
</p>
<p
className={cn(
"text-xs mt-0.5 line-clamp-2",
isLight ? "text-slate-500" : "text-slate-400",
)}
>
Guided questions · opens in a panel
</p>
</div>
<ChevronRight
className={cn(
"h-4 w-4 shrink-0",
isLight ? "text-slate-400" : "text-slate-500",
)}
/>
</button>
<button
type="button"
disabled={modeBusy !== null}
onClick={() => void openDocumentsKnowledgePanel()}
className={cn(
"flex items-center gap-3 rounded-lg border px-4 py-3 text-left transition-all disabled:opacity-60 disabled:cursor-not-allowed",
knowledgeBaseMode === "DOCUMENTS"
? isLight
? "border-emerald-400 bg-emerald-50/80 ring-1 ring-emerald-200"
: "border-emerald-500/50 bg-emerald-500/10"
: isLight
? "border-slate-200 hover:border-slate-300 hover:bg-slate-50/80"
: "border-white/10 hover:border-white/20 hover:bg-white/[0.06]",
)}
>
<div
className={cn(
"flex h-10 w-10 items-center justify-center rounded-lg shrink-0",
isLight ? "bg-emerald-100" : "bg-emerald-500/20",
)}
>
{modeBusy === "documents" ? (
<Loader2 className="h-5 w-5 animate-spin text-emerald-600" />
) : (
<FileStack
className={cn(
"h-5 w-5",
isLight
? "text-emerald-700"
: "text-emerald-300",
)}
/>
)}
</div>
<div className="min-w-0 flex-1">
<p
className={cn(
"text-sm font-semibold",
isLight ? "text-slate-900" : "text-white",
)}
>
Documents &amp; FAQs
</p>
<p
className={cn(
"text-xs mt-0.5 line-clamp-2",
isLight ? "text-slate-500" : "text-slate-400",
)}
>
PDFs, FAQs &amp; Ask · opens in a panel
</p>
</div>
<ChevronRight
className={cn(
"h-4 w-4 shrink-0",
isLight ? "text-slate-400" : "text-slate-500",
)}
/>
</button>
</div>
{toast && (
<p
className={cn(
"text-xs mt-3 px-1",
toast.includes("Could not") || toast.includes("Failed")
? isLight
? "text-amber-700"
: "text-amber-400"
: isLight
? "text-emerald-700"
: "text-emerald-400/90",
)}
>
{toast}
</p>
)}
</div> </div>
<div className="flex-1 min-h-0 overflow-y-auto pb-4"> {structuredActive && (
{activeTab === "documents" && ( <p
<DocumentsSection storeId={activeStoreId!} /> className={cn(
)} "text-xs",
{activeTab === "faqs" && ( isLight ? "text-slate-500" : "text-slate-500",
<FaqSection storeId={activeStoreId!} /> )}
)} >
{activeTab === "ask" && ( You&apos;re on a structured profile. Open{" "}
<AskSection storeId={activeStoreId!} /> <button
)} type="button"
</div> onClick={() => setStructuredOpen(true)}
className="font-semibold text-indigo-500 hover:underline"
>
Structured profile
</button>{" "}
to edit, or choose{" "}
<button
type="button"
onClick={() => void openDocumentsKnowledgePanel()}
className="font-semibold text-emerald-600 hover:underline dark:text-emerald-400"
>
Documents &amp; FAQs
</button>{" "}
to switch (you&apos;ll confirm replacing structured data).
</p>
)}
<StructuredKnowledgeModal
open={structuredOpen}
onOpenChange={setStructuredOpen}
storeId={activeStoreId!}
knowledgeBaseMode={knowledgeBaseMode}
isLight={isLight}
onSaved={async () => {
await loadKb(activeStoreId!)
setToast("Structured profile saved and embedded.")
}}
/>
<DocumentsFaqKnowledgeModal
open={documentsModalOpen}
onOpenChange={setDocumentsModalOpen}
storeId={activeStoreId!}
isLight={isLight}
/>
</> </>
)} )}
</div> </div>
) )
} }

View File

@ -0,0 +1,6 @@
import { redirect } from "next/navigation"
/** Old location; knowledge base lives under JUNO Engine. */
export default function KnowledgeBaseSettingsRedirectPage() {
redirect("/dashboard/juno-engine/knowledge-base")
}

View File

@ -10,7 +10,7 @@ type AskSectionProps = {
type Source = { type Source = {
id: string id: string
type: "DOCUMENT" | "FAQ" type: "DOCUMENT" | "FAQ" | "STRUCTURED"
documentId: string | null documentId: string | null
faqId: string | null faqId: string | null
similarity: number similarity: number
@ -22,6 +22,7 @@ export function AskSection({ storeId }: AskSectionProps) {
const [query, setQuery] = useState("") const [query, setQuery] = useState("")
const [includeFaq, setIncludeFaq] = useState(true) const [includeFaq, setIncludeFaq] = useState(true)
const [includeDocuments, setIncludeDocuments] = useState(true) const [includeDocuments, setIncludeDocuments] = useState(true)
const [includeStructured, setIncludeStructured] = useState(true)
const [answer, setAnswer] = useState<string | null>(null) const [answer, setAnswer] = useState<string | null>(null)
const [sources, setSources] = useState<Source[]>([]) const [sources, setSources] = useState<Source[]>([])
const [isLoading, setIsLoading] = useState(false) const [isLoading, setIsLoading] = useState(false)
@ -48,6 +49,7 @@ export function AskSection({ storeId }: AskSectionProps) {
query: query.trim(), query: query.trim(),
includeFaq, includeFaq,
includeDocuments, includeDocuments,
includeStructured,
}), }),
}, },
) )
@ -75,8 +77,8 @@ export function AskSection({ storeId }: AskSectionProps) {
<div> <div>
<h3 className={cn("text-lg font-semibold", isLight ? "text-slate-900" : "text-white")}>Ask JUNO</h3> <h3 className={cn("text-lg font-semibold", isLight ? "text-slate-900" : "text-white")}>Ask JUNO</h3>
<p className={cn("text-xs", isLight ? "text-slate-500" : "text-slate-400")}> <p className={cn("text-xs", isLight ? "text-slate-500" : "text-slate-400")}>
Ask questions across your store&apos;s documents and FAQs. The Ask questions across documents, FAQs, and your structured store profile.
AI will answer using only your knowledge base. The AI uses only your knowledge base.
</p> </p>
</div> </div>
@ -105,7 +107,7 @@ export function AskSection({ storeId }: AskSectionProps) {
)} )}
/> />
<div className="flex items-center justify-between gap-4"> <div className="flex items-center justify-between gap-4">
<div className="flex items-center gap-4 text-xs"> <div className="flex flex-wrap items-center gap-4 text-xs">
<label className={cn( <label className={cn(
"inline-flex items-center gap-2 cursor-pointer transition-colors", "inline-flex items-center gap-2 cursor-pointer transition-colors",
isLight ? "text-slate-600 hover:text-slate-900" : "text-slate-400 hover:text-white" isLight ? "text-slate-600 hover:text-slate-900" : "text-slate-400 hover:text-white"
@ -120,6 +122,20 @@ export function AskSection({ storeId }: AskSectionProps) {
/> />
<span className="font-medium">Use documents</span> <span className="font-medium">Use documents</span>
</label> </label>
<label className={cn(
"inline-flex items-center gap-2 cursor-pointer transition-colors",
isLight ? "text-slate-600 hover:text-slate-900" : "text-slate-400 hover:text-white"
)}>
<input
type="checkbox"
className="rounded border-slate-300 bg-white text-indigo-600 focus:ring-indigo-500/20"
checked={includeStructured}
onChange={(e) =>
setIncludeStructured(e.target.checked)
}
/>
<span className="font-medium">Use structured profile</span>
</label>
<label className={cn( <label className={cn(
"inline-flex items-center gap-2 cursor-pointer transition-colors", "inline-flex items-center gap-2 cursor-pointer transition-colors",
isLight ? "text-slate-600 hover:text-slate-900" : "text-slate-400 hover:text-white" isLight ? "text-slate-600 hover:text-slate-900" : "text-slate-400 hover:text-white"
@ -178,7 +194,9 @@ export function AskSection({ storeId }: AskSectionProps) {
)}> )}>
{source.type === "FAQ" {source.type === "FAQ"
? "FAQ" ? "FAQ"
: "Document"} : source.type === "STRUCTURED"
? "Profile"
: "Document"}
</span> </span>
<span className={isLight ? "text-slate-500" : "text-slate-400"}> <span className={isLight ? "text-slate-500" : "text-slate-400"}>
Source #{index + 1} Source #{index + 1}

View File

@ -0,0 +1,171 @@
"use client"
import { useEffect, useState } from "react"
import { createPortal } from "react-dom"
import { DocumentsSection } from "@/components/kb/DocumentsSection"
import { FaqSection } from "@/components/kb/FaqSection"
import { AskSection } from "@/components/kb/AskSection"
import { cn } from "@/lib/utils"
import { X } from "lucide-react"
type TabId = "documents" | "faqs" | "ask"
type DocumentsFaqKnowledgeModalProps = {
open: boolean
onOpenChange: (open: boolean) => void
storeId: string
isLight: boolean
}
export function DocumentsFaqKnowledgeModal({
open,
onOpenChange,
storeId,
isLight,
}: DocumentsFaqKnowledgeModalProps) {
const [mounted, setMounted] = useState(false)
const [tab, setTab] = useState<TabId>("documents")
useEffect(() => {
setMounted(true)
}, [])
useEffect(() => {
if (!open) return
const onKey = (e: KeyboardEvent) => {
if (e.key === "Escape") onOpenChange(false)
}
window.addEventListener("keydown", onKey)
return () => window.removeEventListener("keydown", onKey)
}, [open, onOpenChange])
useEffect(() => {
if (open) {
document.body.style.overflow = "hidden"
} else {
document.body.style.overflow = ""
}
return () => {
document.body.style.overflow = ""
}
}, [open])
if (!mounted || !open) return null
const panelClass = isLight
? "bg-white border-slate-200 text-slate-900 shadow-xl"
: "bg-slate-900/95 border-white/10 text-white shadow-2xl backdrop-blur-xl"
const tabs: { id: TabId; label: string }[] = [
{ id: "documents", label: "Documents" },
{ id: "faqs", label: "FAQs" },
{ id: "ask", label: "Ask" },
]
return createPortal(
<div
className="fixed inset-0 z-[200] flex items-end sm:items-center justify-center p-0 sm:p-6"
role="dialog"
aria-modal="true"
aria-labelledby="docs-kb-modal-title"
>
<button
type="button"
className="absolute inset-0 bg-black/50 backdrop-blur-sm"
aria-label="Close dialog"
onClick={() => onOpenChange(false)}
/>
<div
className={cn(
"relative flex w-full flex-col rounded-t-2xl border sm:rounded-2xl",
"max-h-[92vh] sm:max-h-[88vh] sm:max-w-4xl lg:max-w-5xl",
panelClass,
"sm:mx-4",
)}
onClick={(e) => e.stopPropagation()}
>
<div
className={cn(
"flex shrink-0 items-start justify-between gap-3 border-b px-5 py-4",
isLight ? "border-slate-100" : "border-white/10",
)}
>
<div>
<h2
id="docs-kb-modal-title"
className={cn(
"text-lg font-bold tracking-tight sm:text-xl",
isLight ? "text-slate-900" : "text-white",
)}
>
Documents &amp; FAQs
</h2>
<p
className={cn(
"mt-1 max-w-xl text-xs sm:text-sm",
isLight ? "text-slate-500" : "text-slate-400",
)}
>
Upload files, manage FAQs, and test retrieval with Ask—all for
this store.
</p>
</div>
<button
type="button"
onClick={() => onOpenChange(false)}
className={cn(
"shrink-0 rounded-lg p-2 transition-colors",
isLight
? "text-slate-500 hover:bg-slate-100"
: "text-slate-400 hover:bg-white/10",
)}
aria-label="Close"
>
<X className="h-5 w-5" />
</button>
</div>
<div
className={cn(
"flex shrink-0 gap-1 border-b px-4 pt-2",
isLight ? "border-slate-100" : "border-white/10",
)}
>
{tabs.map((t) => (
<button
key={t.id}
type="button"
onClick={() => setTab(t.id)}
className={cn(
"rounded-t-lg px-4 py-2.5 text-sm font-medium transition-colors",
tab === t.id
? isLight
? "bg-slate-100 text-indigo-600"
: "bg-white/10 text-indigo-300"
: isLight
? "text-slate-500 hover:bg-slate-50 hover:text-slate-800"
: "text-slate-400 hover:bg-white/5 hover:text-white",
)}
>
{t.label}
</button>
))}
</div>
<div
className={cn(
"min-h-0 flex-1 overflow-y-auto px-4 py-4 sm:px-5 sm:py-5",
"max-h-[calc(92vh-11rem)] sm:max-h-[calc(88vh-10rem)]",
)}
>
{tab === "documents" && (
<DocumentsSection storeId={storeId} locked={false} />
)}
{tab === "faqs" && <FaqSection storeId={storeId} locked={false} />}
{tab === "ask" && <AskSection storeId={storeId} />}
</div>
</div>
</div>,
document.body,
)
}

View File

@ -1,6 +1,7 @@
"use client" "use client"
import { useEffect, useState } from "react" import { useEffect, useState } from "react"
import Link from "next/link"
import { useTheme } from "@/components/ThemeProvider" import { useTheme } from "@/components/ThemeProvider"
import { cn } from "@/lib/utils" import { cn } from "@/lib/utils"
@ -17,9 +18,17 @@ type KnowledgeDocument = {
type DocumentsSectionProps = { type DocumentsSectionProps = {
storeId: string storeId: string
locked?: boolean
lockedMessage?: string
settingsHref?: string
} }
export function DocumentsSection({ storeId }: DocumentsSectionProps) { export function DocumentsSection({
storeId,
locked = false,
lockedMessage = "This store uses structured knowledge. Switch to Documents & FAQs above or edit your structured profile.",
settingsHref = "/dashboard/juno-engine/knowledge-base#kb-source",
}: DocumentsSectionProps) {
const { theme } = useTheme() const { theme } = useTheme()
const isLight = theme === "light" const isLight = theme === "light"
const [documents, setDocuments] = useState<KnowledgeDocument[]>([]) const [documents, setDocuments] = useState<KnowledgeDocument[]>([])
@ -118,6 +127,27 @@ export function DocumentsSection({ storeId }: DocumentsSectionProps) {
return ( return (
<div className="space-y-4"> <div className="space-y-4">
{locked && (
<div
className={cn(
"rounded-xl border px-4 py-3 text-sm",
isLight
? "border-amber-200 bg-amber-50 text-amber-900"
: "border-amber-500/30 bg-amber-500/10 text-amber-100",
)}
>
<p>{lockedMessage}</p>
<Link
href={settingsHref}
className={cn(
"mt-2 inline-block text-xs font-bold underline",
isLight ? "text-indigo-600" : "text-indigo-300",
)}
>
Knowledge base
</Link>
</div>
)}
<div className="flex items-center justify-between"> <div className="flex items-center justify-between">
<div> <div>
<h3 className={cn("text-lg font-semibold", isLight ? "text-slate-900" : "text-white")}> <h3 className={cn("text-lg font-semibold", isLight ? "text-slate-900" : "text-white")}>
@ -127,14 +157,21 @@ export function DocumentsSection({ storeId }: DocumentsSectionProps) {
Upload PDFs or DOCX files to power AI answers. Upload PDFs or DOCX files to power AI answers.
</p> </p>
</div> </div>
<label className="inline-flex items-center px-4 py-2 rounded-lg bg-indigo-600 hover:bg-indigo-700 text-sm font-medium text-white cursor-pointer transition-colors shadow-sm"> <label
className={cn(
"inline-flex items-center px-4 py-2 rounded-lg text-sm font-medium text-white transition-colors shadow-sm",
locked
? "bg-slate-500 cursor-not-allowed opacity-60"
: "bg-indigo-600 hover:bg-indigo-700 cursor-pointer",
)}
>
<span>{isUploading ? "Uploading..." : "Upload document"}</span> <span>{isUploading ? "Uploading..." : "Upload document"}</span>
<input <input
type="file" type="file"
accept=".pdf,.doc,.docx,application/pdf,application/msword,application/vnd.openxmlformats-officedocument.wordprocessingml.document" accept=".pdf,.doc,.docx,application/pdf,application/msword,application/vnd.openxmlformats-officedocument.wordprocessingml.document"
className="hidden" className="hidden"
onChange={handleUpload} onChange={handleUpload}
disabled={isUploading} disabled={isUploading || locked}
/> />
</label> </label>
</div> </div>
@ -204,7 +241,8 @@ export function DocumentsSection({ storeId }: DocumentsSectionProps) {
<button <button
type="button" type="button"
onClick={() => void handleDelete(doc.id)} onClick={() => void handleDelete(doc.id)}
className="p-1.5 rounded-lg border border-red-200 text-red-500 hover:bg-red-50 transition-colors" disabled={locked}
className="p-1.5 rounded-lg border border-red-200 text-red-500 hover:bg-red-50 transition-colors disabled:opacity-40 disabled:pointer-events-none"
title="Delete document" title="Delete document"
> >
<svg xmlns="http://www.w3.org/2000/svg" className="h-4 w-4" fill="none" viewBox="0 0 24 24" stroke="currentColor"> <svg xmlns="http://www.w3.org/2000/svg" className="h-4 w-4" fill="none" viewBox="0 0 24 24" stroke="currentColor">

View File

@ -1,6 +1,7 @@
"use client" "use client"
import { useEffect, useState } from "react" import { useEffect, useState } from "react"
import Link from "next/link"
import { useTheme } from "@/components/ThemeProvider" import { useTheme } from "@/components/ThemeProvider"
import { cn } from "@/lib/utils" import { cn } from "@/lib/utils"
@ -14,9 +15,17 @@ type FaqItem = {
type FaqSectionProps = { type FaqSectionProps = {
storeId: string storeId: string
locked?: boolean
lockedMessage?: string
settingsHref?: string
} }
export function FaqSection({ storeId }: FaqSectionProps) { export function FaqSection({
storeId,
locked = false,
lockedMessage = "Manual FAQs are disabled while using structured knowledge. Switch mode using the cards above.",
settingsHref = "/dashboard/juno-engine/knowledge-base#kb-source",
}: FaqSectionProps) {
const { theme } = useTheme() const { theme } = useTheme()
const isLight = theme === "light" const isLight = theme === "light"
const [faqs, setFaqs] = useState<FaqItem[]>([]) const [faqs, setFaqs] = useState<FaqItem[]>([])
@ -167,6 +176,27 @@ export function FaqSection({ storeId }: FaqSectionProps) {
return ( return (
<div className="space-y-4"> <div className="space-y-4">
{locked && (
<div
className={cn(
"rounded-xl border px-4 py-3 text-sm",
isLight
? "border-amber-200 bg-amber-50 text-amber-900"
: "border-amber-500/30 bg-amber-500/10 text-amber-100",
)}
>
<p>{lockedMessage}</p>
<Link
href={settingsHref}
className={cn(
"mt-2 inline-block text-xs font-bold underline",
isLight ? "text-indigo-600" : "text-indigo-300",
)}
>
Knowledge base
</Link>
</div>
)}
<div className="flex items-center justify-between"> <div className="flex items-center justify-between">
<div> <div>
<h3 className={cn("text-lg font-semibold", isLight ? "text-slate-900" : "text-white")}>FAQs</h3> <h3 className={cn("text-lg font-semibold", isLight ? "text-slate-900" : "text-white")}>FAQs</h3>
@ -175,12 +205,18 @@ export function FaqSection({ storeId }: FaqSectionProps) {
trusted answers. trusted answers.
</p> </p>
</div> </div>
<label className={cn( <label
"inline-flex items-center px-3 py-2 rounded-lg text-xs font-medium cursor-pointer transition-colors border shadow-sm", className={cn(
isLight "inline-flex items-center px-3 py-2 rounded-lg text-xs font-medium transition-colors border shadow-sm",
? "bg-white border-slate-200 text-slate-700 hover:bg-slate-50" locked
: "bg-slate-800/80 hover:bg-slate-800 border-white/10 text-slate-100 shadow-xl" ? isLight
)}> ? "opacity-50 cursor-not-allowed bg-white border-slate-200 text-slate-400"
: "opacity-50 cursor-not-allowed bg-slate-800/80 border-white/10 text-slate-500"
: isLight
? "cursor-pointer bg-white border-slate-200 text-slate-700 hover:bg-slate-50"
: "cursor-pointer bg-slate-800/80 hover:bg-slate-800 border-white/10 text-slate-100 shadow-xl",
)}
>
<span> <span>
{isImporting ? "Importing..." : "Import from JSON file"} {isImporting ? "Importing..." : "Import from JSON file"}
</span> </span>
@ -189,7 +225,7 @@ export function FaqSection({ storeId }: FaqSectionProps) {
accept="application/json" accept="application/json"
className="hidden" className="hidden"
onChange={handleImport} onChange={handleImport}
disabled={isImporting} disabled={isImporting || locked}
/> />
</label> </label>
</div> </div>
@ -204,6 +240,7 @@ export function FaqSection({ storeId }: FaqSectionProps) {
onSubmit={handleAddFaq} onSubmit={handleAddFaq}
className={cn( className={cn(
"border rounded-xl p-4 space-y-4 transition-all", "border rounded-xl p-4 space-y-4 transition-all",
locked && "opacity-50 pointer-events-none",
isLight ? "bg-white border-slate-100 shadow-sm" : "bg-slate-900/40 border-white/5 backdrop-blur-sm" isLight ? "bg-white border-slate-100 shadow-sm" : "bg-slate-900/40 border-white/5 backdrop-blur-sm"
)} )}
> >

View File

@ -0,0 +1,373 @@
"use client"
import { useCallback, useEffect, useState } from "react"
import { createPortal } from "react-dom"
import {
KNOWLEDGE_TEMPLATE_FIELDS,
type KnowledgeTemplateFieldKey,
type KnowledgeTemplateStored,
} from "@/lib/kb/knowledgeTemplate"
import { cn } from "@/lib/utils"
import { Loader2, Save, X } from "lucide-react"
type KbCounts = {
documents: number
faqs: number
structuredChunks: number
}
type StructuredKnowledgeModalProps = {
open: boolean
onOpenChange: (open: boolean) => void
storeId: string
knowledgeBaseMode: "UNSET" | "STRUCTURED" | "DOCUMENTS"
onSaved: () => void | Promise<void>
isLight: boolean
}
export function StructuredKnowledgeModal({
open,
onOpenChange,
storeId,
knowledgeBaseMode,
onSaved,
isLight,
}: StructuredKnowledgeModalProps) {
const [mounted, setMounted] = useState(false)
const [loading, setLoading] = useState(false)
const [saving, setSaving] = useState(false)
const [error, setError] = useState<string | null>(null)
const [answers, setAnswers] = useState<Record<KnowledgeTemplateFieldKey, string>>(
() =>
KNOWLEDGE_TEMPLATE_FIELDS.reduce(
(acc, f) => {
acc[f.key] = ""
return acc
},
{} as Record<KnowledgeTemplateFieldKey, string>,
),
)
const [misc, setMisc] = useState("")
const loadForm = useCallback(async () => {
const res = await fetch(
`/api/stores/${encodeURIComponent(storeId)}/knowledge-template`,
)
if (!res.ok) return
const data = (await res.json()) as {
knowledgeTemplate: KnowledgeTemplateStored | null
}
const tmpl = data.knowledgeTemplate
setAnswers((prev) => {
const next = { ...prev }
for (const f of KNOWLEDGE_TEMPLATE_FIELDS) {
const v = tmpl?.answers?.[f.key]
next[f.key] = typeof v === "string" ? v : ""
}
return next
})
setMisc(typeof tmpl?.misc === "string" ? tmpl.misc : "")
}, [storeId])
useEffect(() => {
setMounted(true)
}, [])
useEffect(() => {
if (!open || !storeId) return
let cancelled = false
; (async () => {
try {
setLoading(true)
setError(null)
await loadForm()
} finally {
if (!cancelled) setLoading(false)
}
})()
return () => {
cancelled = true
}
}, [open, storeId, loadForm])
useEffect(() => {
if (!open) return
const onKey = (e: KeyboardEvent) => {
if (e.key === "Escape") onOpenChange(false)
}
window.addEventListener("keydown", onKey)
return () => window.removeEventListener("keydown", onKey)
}, [open, onOpenChange])
useEffect(() => {
if (open) {
document.body.style.overflow = "hidden"
} else {
document.body.style.overflow = ""
}
return () => {
document.body.style.overflow = ""
}
}, [open])
async function handleSave(e: React.FormEvent) {
e.preventDefault()
setSaving(true)
setError(null)
const payload: Record<string, unknown> = {
template: {
answers,
misc: misc.trim() || undefined,
},
}
if (knowledgeBaseMode === "DOCUMENTS") {
payload.knowledgeBaseMode = "STRUCTURED"
}
const trySave = async (confirmDestructiveSwitch?: boolean) => {
const body = {
...payload,
...(confirmDestructiveSwitch ? { confirmDestructiveSwitch: true } : {}),
}
const res = await fetch(
`/api/stores/${encodeURIComponent(storeId)}/knowledge-template`,
{
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
},
)
const j = await res.json().catch(() => ({}))
if (res.status === 400 && j.code === "CONFIRM_REQUIRED") {
const ok = window.confirm(
"Switching to structured knowledge will remove all uploaded documents and FAQs for this store. Continue?",
)
if (!ok) return false
return await trySave(true)
}
if (!res.ok) {
throw new Error(j.message || "Failed to save")
}
return true
}
try {
const ok = await trySave()
if (ok) {
await onSaved()
onOpenChange(false)
}
} catch (err) {
setError(err instanceof Error ? err.message : "Failed to save")
} finally {
setSaving(false)
}
}
if (!mounted || !open) return null
const panelClass = isLight
? "bg-white border-slate-200 text-slate-900 shadow-xl"
: "bg-slate-900/95 border-white/10 text-white shadow-2xl backdrop-blur-xl"
return createPortal(
<div
className="fixed inset-0 z-[200] flex items-end sm:items-center justify-center p-0 sm:p-4"
role="dialog"
aria-modal="true"
aria-labelledby="structured-kb-title"
>
<button
type="button"
className="absolute inset-0 bg-black/50 backdrop-blur-sm"
aria-label="Close dialog"
onClick={() => onOpenChange(false)}
/>
<div
className={cn(
"relative flex min-h-0 w-full max-h-[92vh] flex-col rounded-t-2xl border sm:mx-4 sm:max-h-[90vh] sm:rounded-2xl",
"sm:max-w-3xl lg:max-w-4xl",
panelClass,
)}
onClick={(e) => e.stopPropagation()}
>
<div
className={cn(
"flex items-start justify-between gap-3 px-5 py-4 border-b shrink-0",
isLight ? "border-slate-100" : "border-white/10",
)}
>
<div>
<h2
id="structured-kb-title"
className={cn(
"text-lg font-bold tracking-tight",
isLight ? "text-slate-900" : "text-white",
)}
>
Structured store profile
</h2>
<p
className={cn(
"text-xs mt-1",
isLight ? "text-slate-500" : "text-slate-400",
)}
>
Answers are embedded for AI retrieval. Skip any field you
don&apos;t need.
</p>
</div>
<button
type="button"
onClick={() => onOpenChange(false)}
className={cn(
"p-2 rounded-lg transition-colors shrink-0",
isLight
? "hover:bg-slate-100 text-slate-500"
: "hover:bg-white/10 text-slate-400",
)}
aria-label="Close"
>
<X className="w-5 h-5" />
</button>
</div>
{loading ? (
<div className="flex items-center justify-center gap-2 py-16 text-sm text-slate-400">
<Loader2 className="w-5 h-5 animate-spin" />
Loading…
</div>
) : (
<form
onSubmit={(e) => void handleSave(e)}
className="flex min-h-0 flex-1 flex-col"
>
<div
className={cn(
"custom-scrollbar min-h-0 flex-1 space-y-5 overflow-y-auto px-5 py-4",
)}
>
{error && (
<div
className={cn(
"text-xs rounded-lg px-3 py-2 border",
isLight
? "text-red-600 bg-red-50 border-red-100"
: "text-red-400 bg-red-500/10 border-red-500/20",
)}
>
{error}
</div>
)}
{KNOWLEDGE_TEMPLATE_FIELDS.map((field) => (
<div key={field.key}>
<label
className={cn(
"block text-sm font-semibold",
isLight ? "text-slate-900" : "text-white",
)}
>
{field.label}
</label>
<p
className={cn(
"text-[11px] leading-snug mt-0.5 mb-1.5",
isLight ? "text-slate-500" : "text-slate-500",
)}
>
{field.description}
</p>
<textarea
value={answers[field.key]}
onChange={(e) =>
setAnswers((a) => ({
...a,
[field.key]: e.target.value,
}))
}
placeholder={field.placeholder}
rows={3}
className={cn(
"w-full px-3 py-2 rounded-lg border text-sm resize-y min-h-[72px] focus:outline-none focus:ring-2 focus:ring-indigo-500/30",
isLight
? "bg-slate-50 border-slate-200 text-slate-900 placeholder:text-slate-400"
: "bg-slate-950/50 border-white/10 text-white placeholder:text-slate-500",
)}
/>
</div>
))}
<div>
<label
className={cn(
"block text-sm font-semibold",
isLight ? "text-slate-900" : "text-white",
)}
>
Anything else?
</label>
<p
className={cn(
"text-[11px] mt-0.5 mb-1.5",
isLight ? "text-slate-500" : "text-slate-500",
)}
>
Extra notes customers should know.
</p>
<textarea
value={misc}
onChange={(e) => setMisc(e.target.value)}
placeholder="Optional…"
rows={3}
className={cn(
"w-full px-3 py-2 rounded-lg border text-sm resize-y min-h-[72px] focus:outline-none focus:ring-2 focus:ring-indigo-500/30",
isLight
? "bg-slate-50 border-slate-200 text-slate-900 placeholder:text-slate-400"
: "bg-slate-950/50 border-white/10 text-white placeholder:text-slate-500",
)}
/>
</div>
</div>
<div
className={cn(
"flex items-center justify-end gap-2 px-5 py-4 border-t shrink-0",
isLight ? "border-slate-100 bg-slate-50/80" : "border-white/10 bg-slate-950/50",
)}
>
<button
type="button"
onClick={() => onOpenChange(false)}
className={cn(
"px-4 py-2 rounded-lg text-sm font-medium transition-colors",
isLight
? "text-slate-600 hover:bg-slate-200/80"
: "text-slate-300 hover:bg-white/10",
)}
>
Cancel
</button>
<button
type="submit"
disabled={saving}
className="inline-flex items-center gap-2 px-4 py-2 rounded-lg bg-indigo-600 hover:bg-indigo-700 text-white text-sm font-bold disabled:opacity-50"
>
{saving ? (
<Loader2 className="w-4 h-4 animate-spin" />
) : (
<Save className="w-4 h-4" />
)}
Save &amp; embed
</button>
</div>
</form>
)}
</div>
</div>,
document.body,
)
}

View File

@ -1,87 +0,0 @@
import { BedrockRuntimeClient, InvokeModelCommand } from "@aws-sdk/client-bedrock-runtime"
import { logger } from "@/lib/logger"
const AWS_REGION = process.env.AWS_REGION || "us-east-1"
const BEDROCK_CHAT_MODEL =
process.env.BEDROCK_CHAT_MODEL || "anthropic.claude-3-haiku-20240307-v1:0"
// Initialize Bedrock client
const bedrockClient = new BedrockRuntimeClient({
region: AWS_REGION,
credentials: process.env.AWS_ACCESS_KEY_ID && process.env.AWS_SECRET_ACCESS_KEY
? {
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
}
: undefined, // Will use default credential chain (IAM role, etc.)
})
export interface ChatCompletionOptions {
systemPrompt?: string
maxTokens?: number
temperature?: number
}
export async function chatCompletion(
userMessage: string,
options: ChatCompletionOptions = {},
): Promise<string> {
const {
systemPrompt = "",
maxTokens = 512,
temperature = 0.7,
} = options
try {
// Claude Messages API format
const requestBody = {
anthropic_version: "bedrock-2023-05-31",
max_tokens: maxTokens,
temperature: temperature,
system: systemPrompt,
messages: [
{
role: "user" as const,
content: userMessage,
},
],
}
const command = new InvokeModelCommand({
modelId: BEDROCK_CHAT_MODEL,
contentType: "application/json",
accept: "application/json",
body: JSON.stringify(requestBody),
})
const response = await bedrockClient.send(command)
if (!response.body) {
logger.error("Empty response body from Bedrock chat API")
return "I am unable to generate an answer at this time."
}
// Parse response body
const responseBody = JSON.parse(
new TextDecoder().decode(response.body),
) as {
content?: Array<{
type?: string
text?: string
}>
stop_reason?: string
}
// Extract text from response
const textContent = responseBody.content
?.filter((item) => item.type === "text")
.map((item) => item.text || "")
.join("")
.trim()
return textContent || "I am unable to generate an answer at this time."
} catch (error) {
logger.error("Bedrock chat completion failed", error)
return "I am unable to generate an answer at this time."
}
}

View File

@ -1,168 +0,0 @@
import { BedrockRuntimeClient, InvokeModelCommand } from "@aws-sdk/client-bedrock-runtime"
import { logger } from "@/lib/logger"
const AWS_REGION = process.env.AWS_REGION || "us-east-1"
// Hard-code Titan Text Embeddings v2 so all embeddings go through this model
const BEDROCK_EMBEDDING_MODEL = "amazon.titan-embed-text-v2:0"
/**
* AWS Bedrock Rate Limits:
* - Titan Text Embeddings V2: 2,000 requests/minute (~33 req/sec)
* - We use 2-second delays between requests (30 req/min) to stay well under the limit
* - Throttling errors trigger exponential backoff: 5s, 10s, 20s, 40s, 80s
*/
// Initialize Bedrock client with retry configuration
const bedrockClient = new BedrockRuntimeClient({
region: AWS_REGION,
credentials: process.env.AWS_ACCESS_KEY_ID && process.env.AWS_SECRET_ACCESS_KEY
? {
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
}
: undefined, // Will use default credential chain (IAM role, etc.)
maxAttempts: 5, // AWS SDK default retry attempts
})
if (!process.env.AWS_ACCESS_KEY_ID && !process.env.AWS_SECRET_ACCESS_KEY) {
logger.warn(
"AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are not set. Bedrock will use default credential chain.",
)
}
/**
* Retry helper with exponential backoff for throttling errors
*/
async function retryWithBackoff<T>(
fn: () => Promise<T>,
maxRetries: number = 5,
baseDelay: number = 5000, // Start with 5 seconds for throttling
): Promise<T> {
let lastError: unknown
for (let attempt = 0; attempt <= maxRetries; attempt++) {
try {
return await fn()
} catch (error: any) {
lastError = error
// Check if it's a throttling error
const isThrottling =
error?.name === "ThrottlingException" ||
error?.$metadata?.httpStatusCode === 429 ||
error?.message?.includes("Too many requests")
if (isThrottling && attempt < maxRetries) {
// Exponential backoff with longer delays: 5s, 10s, 20s, 40s, 80s
const delay = baseDelay * Math.pow(2, attempt)
logger.warn(
`Bedrock throttling detected, retrying in ${delay / 1000}s (attempt ${attempt + 1}/${maxRetries + 1})`,
)
await new Promise((resolve) => setTimeout(resolve, delay))
continue
}
// If not throttling or max retries reached, throw
throw error
}
}
throw lastError
}
export async function embedBatch(texts: string[]): Promise<number[][]> {
if (texts.length === 0) return []
const results: number[][] = []
// AWS Bedrock Titan V2 limit: 2000 requests/minute = ~33 req/sec
// Use 2 seconds delay to stay well under limit (30 req/min = safe margin)
const REQUEST_DELAY = 2000 // 2 seconds between requests
for (let i = 0; i < texts.length; i++) {
const text = texts[i]
const trimmed = text.trim()
if (!trimmed) {
results.push([])
continue
}
try {
// Titan Text Embeddings V2 request format
const requestBody = {
inputText: trimmed,
dimensions: 1024, // Use 1024 dimensions to match database schema
}
const command = new InvokeModelCommand({
modelId: BEDROCK_EMBEDDING_MODEL,
contentType: "application/json",
accept: "application/json",
body: JSON.stringify(requestBody),
})
// Use retry logic with exponential backoff
const response = await retryWithBackoff(async () => {
return await bedrockClient.send(command)
})
if (!response.body) {
logger.error("Empty response body from Bedrock embedding API")
results.push([])
continue
}
// Parse response body
const responseBody = JSON.parse(
new TextDecoder().decode(response.body),
) as {
embedding?: number[]
inputTokenCount?: number
}
const vector = responseBody.embedding
if (!vector || !Array.isArray(vector)) {
logger.error("Invalid embedding response from Bedrock", responseBody)
results.push([])
continue
}
// Verify dimension matches expected (1024)
if (vector.length !== 1024) {
logger.warn(
`Expected 1024 dimensions but got ${vector.length}. Truncating or padding as needed.`,
)
// Pad or truncate to 1024
if (vector.length < 1024) {
vector.push(...new Array(1024 - vector.length).fill(0))
} else {
vector.splice(1024)
}
}
results.push(vector)
// Add delay between requests to avoid rate limiting (except for last request)
if (i < texts.length - 1) {
await new Promise((resolve) => setTimeout(resolve, REQUEST_DELAY))
}
} catch (error: any) {
const isThrottling =
error?.name === "ThrottlingException" ||
error?.$metadata?.httpStatusCode === 429 ||
error?.message?.includes("Too many requests")
if (isThrottling) {
logger.error(
`Bedrock throttling error after retries for text ${i + 1}/${texts.length}`,
error,
)
} else {
logger.error(`Bedrock embedding request failed for text ${i + 1}/${texts.length}`, error)
}
// On any error, push an empty vector and continue.
results.push([])
}
}
return results
}

View File

@ -0,0 +1,27 @@
/** OpenAI embedding model used for pgvector rows (must match query-time filters). */
export const KB_EMBEDDING_MODEL =
process.env.OPENAI_EMBEDDING_MODEL?.trim() || "text-embedding-3-small"
const parsedDims = Number(process.env.OPENAI_EMBEDDING_DIMENSIONS || "1024")
export const KB_EMBEDDING_DIMENSIONS =
Number.isFinite(parsedDims) && parsedDims > 0 ? parsedDims : 1024
export const KB_EMBEDDING_PROVIDER = "openai" as const
export type KbChunkEmbeddingMetadata = {
provider: typeof KB_EMBEDDING_PROVIDER
embeddingModel: string
embeddingDims: number
sourceKey?: string
}
export function kbEmbeddingMetadata(
extra?: Partial<Pick<KbChunkEmbeddingMetadata, "sourceKey">>,
): KbChunkEmbeddingMetadata {
return {
provider: KB_EMBEDDING_PROVIDER,
embeddingModel: KB_EMBEDDING_MODEL,
embeddingDims: KB_EMBEDDING_DIMENSIONS,
...extra,
}
}

View File

@ -1,6 +1,7 @@
import { prisma } from "@/lib/prisma" import { prisma } from "@/lib/prisma"
import { logger } from "@/lib/logger" import { logger } from "@/lib/logger"
import { embedBatch } from "@/lib/kb/bedrockEmbeddings" import { embedBatch } from "@/lib/kb/openaiEmbeddings"
import { kbEmbeddingMetadata } from "@/lib/kb/embeddingMeta"
import type { TextChunk } from "@/lib/kb/documentProcessing" import type { TextChunk } from "@/lib/kb/documentProcessing"
export async function indexDocumentChunks(chunks: TextChunk[]) { export async function indexDocumentChunks(chunks: TextChunk[]) {
@ -17,6 +18,10 @@ export async function indexDocumentChunks(chunks: TextChunk[]) {
const values = chunks.map((chunk, idx) => { const values = chunks.map((chunk, idx) => {
const embedding = embeddings[idx] const embedding = embeddings[idx]
const vectorLiteral = embedding.length ? `'[${embedding.join(",")}]'` : "NULL" const vectorLiteral = embedding.length ? `'[${embedding.join(",")}]'` : "NULL"
const metadata = {
...chunk.metadata,
...kbEmbeddingMetadata(),
}
return prisma.$executeRawUnsafe( return prisma.$executeRawUnsafe(
` `
@ -37,7 +42,7 @@ export async function indexDocumentChunks(chunks: TextChunk[]) {
chunk.storeId, chunk.storeId,
chunk.documentId, chunk.documentId,
chunk.content, chunk.content,
chunk.metadata as unknown as object, metadata as unknown as object,
) )
}) })

49
src/lib/kb/kbAnswer.ts Normal file
View File

@ -0,0 +1,49 @@
import { openaiChatCompletion } from "@/lib/kb/openaiChat"
import { groqChatCompletion } from "@/lib/ai/groqClient"
import { logger } from "@/lib/logger"
/**
* Generate an answer for KB RAG: OpenAI chat first, then Groq if OpenAI is unavailable or empty.
*/
export async function generateKbRagAnswer(
userMessage: string,
options: { systemPrompt: string; maxTokens?: number },
): Promise<{ text: string; source: "openai" | "groq" | "none" }> {
const maxTokens = options.maxTokens ?? 512
const openai = await openaiChatCompletion(
[
{ role: "system", content: options.systemPrompt },
{ role: "user", content: userMessage },
],
{ maxTokens, temperature: 0.3 },
)
if (openai.content?.trim()) {
return { text: openai.content, source: "openai" }
}
logger.info("KB answer falling back to Groq", { openaiError: openai.error })
const groq = await groqChatCompletion(
[
{ role: "system", content: options.systemPrompt },
{ role: "user", content: userMessage },
],
{ maxTokens, temperature: 0.2 },
)
if (groq.content?.trim()) {
logger.info("KB answer generated via Groq fallback")
return { text: groq.content, source: "groq" }
}
logger.error("KB answer: both OpenAI and Groq failed", {
openaiError: openai.error,
groqError: groq.error,
})
return {
text: "I could not generate an answer right now. Please try again later.",
source: "none",
}
}

View File

@ -0,0 +1,104 @@
import { z } from "zod"
/** Six fixed prompts + optional misc (open-ended). */
export const KNOWLEDGE_TEMPLATE_FIELD_KEYS = [
"brandVoice",
"shippingReturns",
"sizingFit",
"materialsCare",
"policies",
"differentiators",
] as const
export type KnowledgeTemplateFieldKey = (typeof KNOWLEDGE_TEMPLATE_FIELD_KEYS)[number]
export const KNOWLEDGE_TEMPLATE_FIELDS: {
key: KnowledgeTemplateFieldKey
label: string
description: string
placeholder: string
}[] = [
{
key: "brandVoice",
label: "Brand voice and personality",
description:
"How should your store sound? Tone, formality, phrases to use or avoid, and what makes your brand feel human.",
placeholder:
"e.g. Friendly and upbeat; we say “we” and avoid slang; we’re optimistic but never pushy…",
},
{
key: "shippingReturns",
label: "Shipping, returns, and exchanges",
description:
"Regions you ship to, typical timelines, costs, return windows, and how customers start a return.",
placeholder:
"e.g. US & Canada; 3–5 business days standard; free over $75; 30-day returns via portal…",
},
{
key: "sizingFit",
label: "Sizing and fit",
description:
"Fit notes, size guides, model references, and how customers should choose a size.",
placeholder:
"e.g. True to size; relaxed fit on hoodies; see size chart on each product page…",
},
{
key: "materialsCare",
label: "Materials, care, and quality",
description:
"Fabrics, ingredients, durability, care instructions, and sustainability claims you stand behind.",
placeholder:
"e.g. 100% organic cotton; machine wash cold; hang dry; OEKO-TEX where noted…",
},
{
key: "policies",
label: "Policies customers ask about",
description:
"Warranty, price adjustments, pre-orders, backorders, subscriptions, or limits worth stating clearly.",
placeholder:
"e.g. 1-year defect warranty; no price matching; pre-orders ship when noted on the product…",
},
{
key: "differentiators",
label: "What makes your store special",
description:
"Differentiators, origin story, craftsmanship, community, or anything that sets you apart from competitors.",
placeholder:
"e.g. Small-batch, made in Portland; we donate 1% to ocean cleanup; limited seasonal drops…",
},
]
const answerMax = 12_000
const miscMax = 16_000
export const knowledgeTemplatePutSchema = z.object({
knowledgeBaseMode: z.enum(["STRUCTURED", "DOCUMENTS"]).optional(),
confirmDestructiveSwitch: z.boolean().optional(),
template: z
.object({
answers: z.record(z.string().max(answerMax)).optional(),
misc: z.string().max(miscMax).optional(),
})
.optional(),
})
export type KnowledgeTemplateStored = {
version: number
answers: Partial<Record<KnowledgeTemplateFieldKey, string>>
misc?: string
updatedAt: string
}
export function normalizeTemplateAnswers(
raw: Record<string, string> | undefined,
): Partial<Record<KnowledgeTemplateFieldKey, string>> {
if (!raw) return {}
const out: Partial<Record<KnowledgeTemplateFieldKey, string>> = {}
for (const key of KNOWLEDGE_TEMPLATE_FIELD_KEYS) {
const v = raw[key]
if (typeof v === "string" && v.trim()) {
out[key] = v.trim()
}
}
return out
}

53
src/lib/kb/openaiChat.ts Normal file
View File

@ -0,0 +1,53 @@
import OpenAI from "openai"
import { logger } from "@/lib/logger"
const DEFAULT_MODEL = "gpt-4o-mini"
export type OpenAiChatMessage = {
role: "system" | "user" | "assistant"
content: string
}
export type OpenAiChatOptions = {
model?: string
maxTokens?: number
temperature?: number
}
/**
* Chat completions via OpenAI (used for KB RAG answers; embeddings stay in openaiEmbeddings.ts).
*/
export async function openaiChatCompletion(
messages: OpenAiChatMessage[],
options: OpenAiChatOptions = {},
): Promise<{ content: string | null; error?: string }> {
const apiKey = process.env.OPENAI_API_KEY?.trim()
if (!apiKey) {
logger.warn("OPENAI_API_KEY is not set; skipping OpenAI chat")
return { content: null, error: "missing_api_key" }
}
const model =
options.model?.trim() ||
process.env.OPENAI_CHAT_MODEL?.trim() ||
DEFAULT_MODEL
try {
const client = new OpenAI({ apiKey })
const res = await client.chat.completions.create({
model,
messages,
max_tokens: options.maxTokens ?? 512,
temperature: options.temperature ?? 0.3,
})
const text = res.choices[0]?.message?.content?.trim() ?? null
if (!text) {
return { content: null, error: "empty_response" }
}
return { content: text }
} catch (err) {
const message = err instanceof Error ? err.message : String(err)
logger.warn("OpenAI chat completion failed", { message })
return { content: null, error: message }
}
}

View File

@ -0,0 +1,118 @@
import OpenAI from "openai"
import { logger } from "@/lib/logger"
import {
KB_EMBEDDING_DIMENSIONS,
KB_EMBEDDING_MODEL,
} from "@/lib/kb/embeddingMeta"
const MAX_INPUTS_PER_REQUEST = 64
const MAX_RETRIES = 5
const BASE_DELAY_MS = 800
function sleep(ms: number) {
return new Promise((r) => setTimeout(r, ms))
}
function isRetryableStatus(status: number | undefined) {
return status === 429 || (status !== undefined && status >= 500)
}
/**
* Batch embeddings via OpenAI (text-embedding-3-small + dimensions for pgvector / cost tradeoff).
* Returns one vector per input; empty strings yield empty arrays (callers skip DB insert).
*/
export async function embedBatch(texts: string[]): Promise<number[][]> {
if (texts.length === 0) return []
const apiKey = process.env.OPENAI_API_KEY?.trim()
if (!apiKey) {
logger.error("OPENAI_API_KEY is not set; cannot generate embeddings")
return texts.map(() => [])
}
const client = new OpenAI({ apiKey })
const out: number[][] = new Array(texts.length).fill(null).map(() => [])
for (let start = 0; start < texts.length; start += MAX_INPUTS_PER_REQUEST) {
const end = Math.min(start + MAX_INPUTS_PER_REQUEST, texts.length)
const slice = texts.slice(start, end)
const normalized = slice.map((t) => (t.trim().length ? t.trim() : ""))
const indicesWithText: number[] = []
const inputs: string[] = []
normalized.forEach((t, i) => {
if (t.length) {
indicesWithText.push(start + i)
inputs.push(t)
}
})
if (inputs.length === 0) continue
let attempt = 0
let embeddings: number[][] | null = null
while (attempt <= MAX_RETRIES) {
try {
const started = Date.now()
const res = await client.embeddings.create({
model: KB_EMBEDDING_MODEL,
input: inputs,
dimensions: KB_EMBEDDING_DIMENSIONS,
})
logger.info("OpenAI embeddings batch", {
count: inputs.length,
ms: Date.now() - started,
model: KB_EMBEDDING_MODEL,
})
const byIndex = new Map<number, number[]>()
for (const item of res.data) {
const vec = item.embedding
if (!Array.isArray(vec) || vec.length !== KB_EMBEDDING_DIMENSIONS) {
logger.error("Unexpected embedding dimensions from OpenAI", {
expected: KB_EMBEDDING_DIMENSIONS,
got: vec?.length,
})
throw new Error("Invalid embedding dimensions")
}
byIndex.set(item.index, vec)
}
embeddings = []
for (let i = 0; i < inputs.length; i++) {
const vec = byIndex.get(i)
embeddings.push(vec ?? [])
}
break
} catch (err: unknown) {
const status =
err && typeof err === "object" && "status" in err
? Number((err as { status?: number }).status)
: undefined
if (!isRetryableStatus(status) || attempt >= MAX_RETRIES) {
logger.error("OpenAI embedding request failed", {
attempt,
status,
message: err instanceof Error ? err.message : String(err),
})
embeddings = inputs.map(() => [])
break
}
const delay = BASE_DELAY_MS * Math.pow(2, attempt)
logger.warn(`OpenAI embeddings retry in ${delay}ms`, { attempt, status })
await sleep(delay)
attempt++
}
}
if (!embeddings) continue
embeddings.forEach((vec, j) => {
const globalIdx = indicesWithText[j]!
out[globalIdx] = vec
})
}
return out
}

View File

@ -0,0 +1,50 @@
import fs from "fs/promises"
import path from "path"
import type { PrismaClient } from "@prisma/client"
import { logger } from "@/lib/logger"
export async function deleteDocumentKbForStore(
prisma: PrismaClient,
storeId: string,
) {
const docs = await prisma.$queryRawUnsafe<{ id: string; storagePath: string }[]>(
`SELECT "id", "storagePath" FROM "KnowledgeDocument" WHERE "storeId" = $1`,
storeId,
)
for (const d of docs) {
try {
await fs.unlink(path.resolve(d.storagePath))
} catch (e) {
logger.warn("Failed to delete KB file on disk", {
path: d.storagePath,
message: e instanceof Error ? e.message : String(e),
})
}
}
await prisma.$executeRawUnsafe(
`DELETE FROM "KnowledgeChunk" WHERE "storeId" = $1 AND "sourceType"::text = 'DOCUMENT'`,
storeId,
)
await prisma.$executeRawUnsafe(
`DELETE FROM "KnowledgeDocument" WHERE "storeId" = $1`,
storeId,
)
}
export async function deleteFaqKbForStore(prisma: PrismaClient, storeId: string) {
await prisma.$executeRawUnsafe(
`DELETE FROM "KnowledgeChunk" WHERE "storeId" = $1 AND "sourceType"::text = 'FAQ'`,
storeId,
)
await prisma.faqItem.deleteMany({ where: { storeId } })
}
export async function deleteStructuredKbForStore(
prisma: PrismaClient,
storeId: string,
) {
await prisma.$executeRawUnsafe(
`DELETE FROM "KnowledgeChunk" WHERE "storeId" = $1 AND "sourceType"::text = 'STRUCTURED'`,
storeId,
)
}

View File

@ -0,0 +1,77 @@
import { prisma } from "@/lib/prisma"
import { logger } from "@/lib/logger"
import { embedBatch } from "@/lib/kb/openaiEmbeddings"
import { kbEmbeddingMetadata } from "@/lib/kb/embeddingMeta"
import {
KNOWLEDGE_TEMPLATE_FIELDS,
type KnowledgeTemplateStored,
} from "@/lib/kb/knowledgeTemplate"
export async function replaceStructuredKnowledgeChunks(
storeId: string,
stored: KnowledgeTemplateStored,
) {
await prisma.$executeRawUnsafe(
`DELETE FROM "KnowledgeChunk" WHERE "storeId" = $1 AND "sourceType"::text = 'STRUCTURED'`,
storeId,
)
const rows: { content: string; metadata: ReturnType<typeof kbEmbeddingMetadata> }[] =
[]
for (const field of KNOWLEDGE_TEMPLATE_FIELDS) {
const text = stored.answers[field.key]
if (text?.trim()) {
rows.push({
content: `Q: ${field.label}\nA: ${text.trim()}`,
metadata: kbEmbeddingMetadata({ sourceKey: field.key }),
})
}
}
if (stored.misc?.trim()) {
rows.push({
content: `Q: Anything else customers should know\nA: ${stored.misc.trim()}`,
metadata: kbEmbeddingMetadata({ sourceKey: "misc" }),
})
}
if (!rows.length) {
return
}
const embeddings = await embedBatch(rows.map((r) => r.content))
if (embeddings.length !== rows.length) {
throw new Error("Embedding count mismatch for structured template")
}
for (let i = 0; i < rows.length; i++) {
const embedding = embeddings[i]!
if (!embedding.length) {
logger.error("Empty embedding for structured template row", { storeId, i })
throw new Error("Failed to embed structured knowledge template")
}
const vectorLiteral = `'[${embedding.join(",")}]'`
await prisma.$executeRawUnsafe(
`
INSERT INTO "KnowledgeChunk"
("id", "storeId", "documentId", "faqItemId", "sourceType", "content", "metadata", "embedding", "createdAt", "updatedAt")
VALUES (
gen_random_uuid()::text,
$1,
NULL,
NULL,
'STRUCTURED',
$2,
$3,
${vectorLiteral},
NOW(),
NOW()
)
`,
storeId,
rows[i]!.content,
rows[i]!.metadata as unknown as object,
)
}
}