KBs and Embeddings for both structured & import
This commit is contained in:
talatops
parent
4dd56f3aae
commit
e49a0047aa
91
README.md
91
README.md
@ -16,7 +16,8 @@
|
||||
<img src="https://img.shields.io/badge/Next.js-000000?logo=nextdotjs&logoColor=white" alt="Next.js" />
|
||||
<img src="https://img.shields.io/badge/TypeScript-3178C6?logo=typescript&logoColor=white" alt="TypeScript" />
|
||||
<img src="https://img.shields.io/badge/PostgreSQL-4169E1?logo=postgresql&logoColor=white" alt="PostgreSQL" />
|
||||
<img src="https://img.shields.io/badge/AWS%20Bedrock-232F3E?logo=amazonaws&logoColor=white" alt="AWS Bedrock" />
|
||||
<img src="https://img.shields.io/badge/OpenAI-412991?logo=openai&logoColor=white" alt="OpenAI" />
|
||||
<img src="https://img.shields.io/badge/Groq-F55036?logo=groq&logoColor=white" alt="Groq" />
|
||||
<img src="https://img.shields.io/badge/Gmail%20API-EA4335?logo=gmail&logoColor=white" alt="Gmail API" />
|
||||
<img src="https://img.shields.io/badge/Docker-2496ED?logo=docker&logoColor=white" alt="Docker" />
|
||||
<img src="https://img.shields.io/badge/NextAuth-000000?logo=nextdotjs&logoColor=white" alt="NextAuth" />
|
||||
@ -51,7 +52,7 @@ Everything runs inside a single Next.js App Router project (frontend + backend +
|
||||
- [JUNOCHAT & AI](#future-enhancements-for-junochat--ai)
|
||||
6. [Local Development](#local-development)
|
||||
7. [Docker & Deployment](#docker--deployment)
|
||||
8. [AWS Bedrock & AI Setup](#aws-bedrock--ai-setup)
|
||||
8. [OpenAI, Groq & AI Setup](#openai-groq--ai-setup)
|
||||
9. [Prisma & Database](#prisma--database)
|
||||
10. [License](#license)
|
||||
|
||||
@ -74,7 +75,7 @@ JUNO centralizes communication and operations between **vendors** (Shopify store
|
||||
- Edit synced product price/quantity **per vendor** with inline controls.
|
||||
|
||||
- Both sides are supported by:
|
||||
- **JUNOCHAT** – an AI assistant powered by AWS Bedrock (Titan + Claude) that can answer questions based on the knowledge base and FAQs.
|
||||
- **JUNOCHAT** – an AI assistant powered by **OpenAI** (embeddings + chat where configured) and **Groq** (fast chat for storefront widget, Gmail flows, and KB fallback) that can answer questions using the knowledge base, FAQs, and store context.
|
||||
- A **Juno Engine** area for Gmail and ticketing workflows.
|
||||
|
||||
---
|
||||
@ -167,9 +168,10 @@ JUNO centralizes communication and operations between **vendors** (Shopify store
|
||||
|
||||
JUNOCHAT (a.k.a. Juno Engine chat) is the AI layer that:
|
||||
|
||||
- Uses **AWS Bedrock**:
|
||||
- **Amazon Titan Text Embeddings V2** for vector embeddings.
|
||||
- **Anthropic Claude 3 Haiku** for chat and knowledge‑base answers.
|
||||
- Uses **OpenAI** and **Groq**:
|
||||
- **OpenAI** (`text-embedding-3-small`, etc.) for knowledge‑base **embeddings** stored in Postgres/pgvector.
|
||||
- **OpenAI** (`gpt-4o-mini` by default, configurable) for **KB “Ask”** answers when `OPENAI_API_KEY` is set.
|
||||
- **Groq** for **storefront widget** replies, **Gmail** classification/safety/preview flows, and **KB fallback** if OpenAI chat fails.
|
||||
- Supports:
|
||||
- FAQ ingestion and retrieval.
|
||||
- Knowledge document upload + chunking.
|
||||
@ -217,7 +219,7 @@ JUNOCHAT (a.k.a. Juno Engine chat) is the AI layer that:
|
||||
- **Background / Integrations**:
|
||||
- Gmail APIs (OAuth)
|
||||
- Shopify APIs (orders, products, inventory)
|
||||
- AWS Bedrock (Titan embeddings + Claude 3 Haiku chat)
|
||||
- OpenAI (embeddings + optional chat) and Groq (chat / classification)
|
||||
- **Deployment**: Docker + GitHub Actions → AWS EC2
|
||||
|
||||
---
|
||||
@ -277,7 +279,7 @@ JUNOCHAT (a.k.a. Juno Engine chat) is the AI layer that:
|
||||
**How JUNO helps:**
|
||||
|
||||
- **JUNOCHAT + Juno Engine**:
|
||||
- Ingest FAQs and internal docs into a vectorized knowledge base (via Bedrock embeddings).
|
||||
- Ingest FAQs and internal docs into a vectorized knowledge base (via OpenAI embeddings).
|
||||
- Answer questions using **RAG** against that knowledge.
|
||||
- Integrate with Gmail so tickets and AI context live in the same place.
|
||||
|
||||
@ -307,9 +309,9 @@ Copy `.env.example` (or the variables listed below) into a local `.env` file and
|
||||
- `SMTP_SECURE`
|
||||
- `SMTP_USER`
|
||||
- `SMTP_PASSWORD`
|
||||
- `AWS_ACCESS_KEY_ID` (for AWS Bedrock)
|
||||
- `AWS_SECRET_ACCESS_KEY` (for AWS Bedrock)
|
||||
- `AWS_REGION` (default: `us-east-1`)
|
||||
- `OPENAI_API_KEY` (embeddings + KB chat; optional `OPENAI_CHAT_MODEL`, `OPENAI_EMBEDDING_MODEL`, `OPENAI_EMBEDDING_DIMENSIONS`)
|
||||
- `GROQ_API_KEY` (storefront chat, Gmail AI helpers, KB chat fallback; optional `GROQ_MODEL`)
|
||||
- `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` / `AWS_REGION` (if you use S3 or other AWS services—not required for AI-only local dev)
|
||||
- `ADMIN_EMAIL`
|
||||
- `ADMIN_PASSWORD`
|
||||
|
||||
@ -376,51 +378,34 @@ This gives you a sharp deployment with an automatic rollback path.
|
||||
|
||||
---
|
||||
|
||||
## AWS Bedrock & AI Setup
|
||||
## OpenAI, Groq & AI Setup
|
||||
|
||||
The knowledge base and JUNOCHAT features use AWS Bedrock for AI‑powered embeddings and chat completion.
|
||||
### OpenAI (embeddings + KB answers)
|
||||
|
||||
### Prerequisites
|
||||
|
||||
1. **AWS Account**: Create an account at `https://aws.amazon.com/`
|
||||
2. **Enable Bedrock Access**:
|
||||
- Go to `https://console.aws.amazon.com/bedrock/`
|
||||
- Navigate to **Model access**
|
||||
- Request access to:
|
||||
- **Amazon Titan Text Embeddings V2** (for embeddings)
|
||||
- **Anthropic Claude 3 Haiku** (for chat completion)
|
||||
|
||||
### IAM Setup
|
||||
|
||||
Create an IAM user or role with Bedrock permissions:
|
||||
|
||||
```json
|
||||
{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Action": ["bedrock:InvokeModel"],
|
||||
"Resource": [
|
||||
"arn:aws:bedrock:*::foundation-model/amazon.titan-embed-text-v2:0",
|
||||
"arn:aws:bedrock:*::foundation-model/anthropic.claude-3-haiku-20240307-v1:0"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Create access keys for that user and add them to `.env`:
|
||||
1. Create an API key in the [OpenAI dashboard](https://platform.openai.com/).
|
||||
2. Add to `.env`:
|
||||
|
||||
```bash
|
||||
AWS_ACCESS_KEY_ID="your-access-key-id"
|
||||
AWS_SECRET_ACCESS_KEY="your-secret-access-key"
|
||||
AWS_REGION="us-east-1" # or your preferred region
|
||||
OPENAI_API_KEY="sk-..."
|
||||
# Optional overrides:
|
||||
# OPENAI_EMBEDDING_MODEL="text-embedding-3-small"
|
||||
# OPENAI_EMBEDDING_DIMENSIONS="1024"
|
||||
# OPENAI_CHAT_MODEL="gpt-4o-mini"
|
||||
```
|
||||
|
||||
### Database Migration & Re‑Embedding
|
||||
Embeddings are used when indexing knowledge documents, FAQs, and the structured store profile. KB **Ask** uses OpenAI chat first, then falls back to Groq.
|
||||
|
||||
After setting up AWS Bedrock:
|
||||
### Groq (storefront chat, Gmail helpers, KB fallback)
|
||||
|
||||
1. Create an API key at [Groq Console](https://console.groq.com/).
|
||||
2. Add to `.env`:
|
||||
|
||||
```bash
|
||||
GROQ_API_KEY="gsk_..."
|
||||
# Optional: GROQ_MODEL="llama-3.1-8b-instant"
|
||||
```
|
||||
|
||||
### Database migration & re‑embedding
|
||||
|
||||
1. Apply Prisma migrations:
|
||||
|
||||
@ -428,17 +413,13 @@ After setting up AWS Bedrock:
|
||||
npx prisma migrate deploy
|
||||
```
|
||||
|
||||
2. Re‑embed existing knowledge base data:
|
||||
2. After changing embedding provider or dimensions, re‑embed:
|
||||
|
||||
```bash
|
||||
npm run re-embed-knowledge-base
|
||||
```
|
||||
|
||||
This script:
|
||||
|
||||
- Reads all existing `KnowledgeChunk` records.
|
||||
- Generates new embeddings using Titan V2.
|
||||
- Updates the database in batches to avoid rate limiting.
|
||||
This regenerates vectors and tags chunk metadata with `provider: openai` (required for KB similarity search filters).
|
||||
|
||||
---
|
||||
|
||||
|
||||
433
package-lock.json
generated
433
package-lock.json
generated
@ -10,7 +10,6 @@
|
||||
"dependencies": {
|
||||
"@auth/core": "^0.41.0",
|
||||
"@auth/prisma-adapter": "^2.11.1",
|
||||
"@aws-sdk/client-bedrock-runtime": "^3.989.0",
|
||||
"@aws-sdk/client-s3": "^3.989.0",
|
||||
"@prisma/adapter-pg": "^7.6.0",
|
||||
"@prisma/client": "^7.6.0",
|
||||
@ -32,6 +31,7 @@
|
||||
"next": "^16.2.1",
|
||||
"next-auth": "^5.0.0-beta.30",
|
||||
"nodemailer": "^7.0.7",
|
||||
"openai": "^4.104.0",
|
||||
"otplib": "^13.2.1",
|
||||
"pdf-parse": "^2.4.5",
|
||||
"pg": "^8.18.0",
|
||||
@ -297,63 +297,6 @@
|
||||
"node": ">=14.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@aws-sdk/client-bedrock-runtime": {
|
||||
"version": "3.989.0",
|
||||
"resolved": "https://registry.npmjs.org/@aws-sdk/client-bedrock-runtime/-/client-bedrock-runtime-3.989.0.tgz",
|
||||
"integrity": "sha512-qVa5B0wXjIuPRhX1dcZo1sa9Y4ycI9tiqK7B4FLok67gUWckiKmEf1xQDFrTmc2eCK5g0CTaeiRdbeM1eWmW1Q==",
|
||||
"dependencies": {
|
||||
"@aws-crypto/sha256-browser": "5.2.0",
|
||||
"@aws-crypto/sha256-js": "5.2.0",
|
||||
"@aws-sdk/core": "^3.973.9",
|
||||
"@aws-sdk/credential-provider-node": "^3.972.8",
|
||||
"@aws-sdk/eventstream-handler-node": "^3.972.5",
|
||||
"@aws-sdk/middleware-eventstream": "^3.972.3",
|
||||
"@aws-sdk/middleware-host-header": "^3.972.3",
|
||||
"@aws-sdk/middleware-logger": "^3.972.3",
|
||||
"@aws-sdk/middleware-recursion-detection": "^3.972.3",
|
||||
"@aws-sdk/middleware-user-agent": "^3.972.9",
|
||||
"@aws-sdk/middleware-websocket": "^3.972.6",
|
||||
"@aws-sdk/region-config-resolver": "^3.972.3",
|
||||
"@aws-sdk/token-providers": "3.989.0",
|
||||
"@aws-sdk/types": "^3.973.1",
|
||||
"@aws-sdk/util-endpoints": "3.989.0",
|
||||
"@aws-sdk/util-user-agent-browser": "^3.972.3",
|
||||
"@aws-sdk/util-user-agent-node": "^3.972.7",
|
||||
"@smithy/config-resolver": "^4.4.6",
|
||||
"@smithy/core": "^3.23.0",
|
||||
"@smithy/eventstream-serde-browser": "^4.2.8",
|
||||
"@smithy/eventstream-serde-config-resolver": "^4.3.8",
|
||||
"@smithy/eventstream-serde-node": "^4.2.8",
|
||||
"@smithy/fetch-http-handler": "^5.3.9",
|
||||
"@smithy/hash-node": "^4.2.8",
|
||||
"@smithy/invalid-dependency": "^4.2.8",
|
||||
"@smithy/middleware-content-length": "^4.2.8",
|
||||
"@smithy/middleware-endpoint": "^4.4.14",
|
||||
"@smithy/middleware-retry": "^4.4.31",
|
||||
"@smithy/middleware-serde": "^4.2.9",
|
||||
"@smithy/middleware-stack": "^4.2.8",
|
||||
"@smithy/node-config-provider": "^4.3.8",
|
||||
"@smithy/node-http-handler": "^4.4.10",
|
||||
"@smithy/protocol-http": "^5.3.8",
|
||||
"@smithy/smithy-client": "^4.11.3",
|
||||
"@smithy/types": "^4.12.0",
|
||||
"@smithy/url-parser": "^4.2.8",
|
||||
"@smithy/util-base64": "^4.3.0",
|
||||
"@smithy/util-body-length-browser": "^4.2.0",
|
||||
"@smithy/util-body-length-node": "^4.2.1",
|
||||
"@smithy/util-defaults-mode-browser": "^4.3.30",
|
||||
"@smithy/util-defaults-mode-node": "^4.2.33",
|
||||
"@smithy/util-endpoints": "^3.2.8",
|
||||
"@smithy/util-middleware": "^4.2.8",
|
||||
"@smithy/util-retry": "^4.2.8",
|
||||
"@smithy/util-stream": "^4.5.12",
|
||||
"@smithy/util-utf8": "^4.2.0",
|
||||
"tslib": "^2.6.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@aws-sdk/client-s3": {
|
||||
"version": "3.990.0",
|
||||
"resolved": "https://registry.npmjs.org/@aws-sdk/client-s3/-/client-s3-3.990.0.tgz",
|
||||
@ -951,20 +894,6 @@
|
||||
"node": ">=20.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@aws-sdk/eventstream-handler-node": {
|
||||
"version": "3.972.5",
|
||||
"resolved": "https://registry.npmjs.org/@aws-sdk/eventstream-handler-node/-/eventstream-handler-node-3.972.5.tgz",
|
||||
"integrity": "sha512-xEmd3dnyn83K6t4AJxBJA63wpEoCD45ERFG0XMTViD2E/Ohls9TLxjOWPb1PAxR9/46cKy/TImez1GoqP6xVNQ==",
|
||||
"dependencies": {
|
||||
"@aws-sdk/types": "^3.973.1",
|
||||
"@smithy/eventstream-codec": "^4.2.8",
|
||||
"@smithy/types": "^4.12.0",
|
||||
"tslib": "^2.6.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@aws-sdk/middleware-bucket-endpoint": {
|
||||
"version": "3.972.3",
|
||||
"resolved": "https://registry.npmjs.org/@aws-sdk/middleware-bucket-endpoint/-/middleware-bucket-endpoint-3.972.3.tgz",
|
||||
@ -982,20 +911,6 @@
|
||||
"node": ">=20.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@aws-sdk/middleware-eventstream": {
|
||||
"version": "3.972.3",
|
||||
"resolved": "https://registry.npmjs.org/@aws-sdk/middleware-eventstream/-/middleware-eventstream-3.972.3.tgz",
|
||||
"integrity": "sha512-pbvZ6Ye/Ks6BAZPa3RhsNjHrvxU9li25PMhSdDpbX0jzdpKpAkIR65gXSNKmA/REnSdEMWSD4vKUW+5eMFzB6w==",
|
||||
"dependencies": {
|
||||
"@aws-sdk/types": "^3.973.1",
|
||||
"@smithy/protocol-http": "^5.3.8",
|
||||
"@smithy/types": "^4.12.0",
|
||||
"tslib": "^2.6.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@aws-sdk/middleware-expect-continue": {
|
||||
"version": "3.972.3",
|
||||
"resolved": "https://registry.npmjs.org/@aws-sdk/middleware-expect-continue/-/middleware-expect-continue-3.972.3.tgz",
|
||||
@ -1158,76 +1073,6 @@
|
||||
"node": ">=20.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@aws-sdk/middleware-websocket": {
|
||||
"version": "3.972.6",
|
||||
"resolved": "https://registry.npmjs.org/@aws-sdk/middleware-websocket/-/middleware-websocket-3.972.6.tgz",
|
||||
"integrity": "sha512-1DedO6N3m8zQ/vG6twNiHtsdwBgk773VdavLEbB3NXeKZDlzSK1BTviqWwvJdKx5UnIy4kGGP6WWpCEFEt/bhQ==",
|
||||
"dependencies": {
|
||||
"@aws-sdk/types": "^3.973.1",
|
||||
"@aws-sdk/util-format-url": "^3.972.3",
|
||||
"@smithy/eventstream-codec": "^4.2.8",
|
||||
"@smithy/eventstream-serde-browser": "^4.2.8",
|
||||
"@smithy/fetch-http-handler": "^5.3.9",
|
||||
"@smithy/protocol-http": "^5.3.8",
|
||||
"@smithy/signature-v4": "^5.3.8",
|
||||
"@smithy/types": "^4.12.0",
|
||||
"@smithy/util-base64": "^4.3.0",
|
||||
"@smithy/util-hex-encoding": "^4.2.0",
|
||||
"@smithy/util-utf8": "^4.2.0",
|
||||
"tslib": "^2.6.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 14.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@aws-sdk/nested-clients": {
|
||||
"version": "3.989.0",
|
||||
"resolved": "https://registry.npmjs.org/@aws-sdk/nested-clients/-/nested-clients-3.989.0.tgz",
|
||||
"integrity": "sha512-Dbk2HMPU3mb6RrSRzgf0WCaWSbgtZG258maCpuN2/ONcAQNpOTw99V5fU5CA1qVK6Vkm4Fwj2cnOnw7wbGVlOw==",
|
||||
"dependencies": {
|
||||
"@aws-crypto/sha256-browser": "5.2.0",
|
||||
"@aws-crypto/sha256-js": "5.2.0",
|
||||
"@aws-sdk/core": "^3.973.9",
|
||||
"@aws-sdk/middleware-host-header": "^3.972.3",
|
||||
"@aws-sdk/middleware-logger": "^3.972.3",
|
||||
"@aws-sdk/middleware-recursion-detection": "^3.972.3",
|
||||
"@aws-sdk/middleware-user-agent": "^3.972.9",
|
||||
"@aws-sdk/region-config-resolver": "^3.972.3",
|
||||
"@aws-sdk/types": "^3.973.1",
|
||||
"@aws-sdk/util-endpoints": "3.989.0",
|
||||
"@aws-sdk/util-user-agent-browser": "^3.972.3",
|
||||
"@aws-sdk/util-user-agent-node": "^3.972.7",
|
||||
"@smithy/config-resolver": "^4.4.6",
|
||||
"@smithy/core": "^3.23.0",
|
||||
"@smithy/fetch-http-handler": "^5.3.9",
|
||||
"@smithy/hash-node": "^4.2.8",
|
||||
"@smithy/invalid-dependency": "^4.2.8",
|
||||
"@smithy/middleware-content-length": "^4.2.8",
|
||||
"@smithy/middleware-endpoint": "^4.4.14",
|
||||
"@smithy/middleware-retry": "^4.4.31",
|
||||
"@smithy/middleware-serde": "^4.2.9",
|
||||
"@smithy/middleware-stack": "^4.2.8",
|
||||
"@smithy/node-config-provider": "^4.3.8",
|
||||
"@smithy/node-http-handler": "^4.4.10",
|
||||
"@smithy/protocol-http": "^5.3.8",
|
||||
"@smithy/smithy-client": "^4.11.3",
|
||||
"@smithy/types": "^4.12.0",
|
||||
"@smithy/url-parser": "^4.2.8",
|
||||
"@smithy/util-base64": "^4.3.0",
|
||||
"@smithy/util-body-length-browser": "^4.2.0",
|
||||
"@smithy/util-body-length-node": "^4.2.1",
|
||||
"@smithy/util-defaults-mode-browser": "^4.3.30",
|
||||
"@smithy/util-defaults-mode-node": "^4.2.33",
|
||||
"@smithy/util-endpoints": "^3.2.8",
|
||||
"@smithy/util-middleware": "^4.2.8",
|
||||
"@smithy/util-retry": "^4.2.8",
|
||||
"@smithy/util-utf8": "^4.2.0",
|
||||
"tslib": "^2.6.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@aws-sdk/region-config-resolver": {
|
||||
"version": "3.972.3",
|
||||
"resolved": "https://registry.npmjs.org/@aws-sdk/region-config-resolver/-/region-config-resolver-3.972.3.tgz",
|
||||
@ -1259,23 +1104,6 @@
|
||||
"node": ">=20.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@aws-sdk/token-providers": {
|
||||
"version": "3.989.0",
|
||||
"resolved": "https://registry.npmjs.org/@aws-sdk/token-providers/-/token-providers-3.989.0.tgz",
|
||||
"integrity": "sha512-OdBByMv+OjOZoekrk4THPFpLuND5aIQbDHCGh3n2rvifAbm31+6e0OLhxSeCF1UMPm+nKq12bXYYEoCIx5SQBg==",
|
||||
"dependencies": {
|
||||
"@aws-sdk/core": "^3.973.9",
|
||||
"@aws-sdk/nested-clients": "3.989.0",
|
||||
"@aws-sdk/types": "^3.973.1",
|
||||
"@smithy/property-provider": "^4.2.8",
|
||||
"@smithy/shared-ini-file-loader": "^4.4.3",
|
||||
"@smithy/types": "^4.12.0",
|
||||
"tslib": "^2.6.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@aws-sdk/types": {
|
||||
"version": "3.973.1",
|
||||
"resolved": "https://registry.npmjs.org/@aws-sdk/types/-/types-3.973.1.tgz",
|
||||
@ -1299,35 +1127,6 @@
|
||||
"node": ">=20.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@aws-sdk/util-endpoints": {
|
||||
"version": "3.989.0",
|
||||
"resolved": "https://registry.npmjs.org/@aws-sdk/util-endpoints/-/util-endpoints-3.989.0.tgz",
|
||||
"integrity": "sha512-eKmAOeQM4Qusq0jtcbZPiNWky8XaojByKC/n+THbJ8vJf7t4ys8LlcZ4PrBSHZISe9cC484mQsPVOQh6iySjqw==",
|
||||
"dependencies": {
|
||||
"@aws-sdk/types": "^3.973.1",
|
||||
"@smithy/types": "^4.12.0",
|
||||
"@smithy/url-parser": "^4.2.8",
|
||||
"@smithy/util-endpoints": "^3.2.8",
|
||||
"tslib": "^2.6.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@aws-sdk/util-format-url": {
|
||||
"version": "3.972.3",
|
||||
"resolved": "https://registry.npmjs.org/@aws-sdk/util-format-url/-/util-format-url-3.972.3.tgz",
|
||||
"integrity": "sha512-n7F2ycckcKFXa01vAsT/SJdjFHfKH9s96QHcs5gn8AaaigASICeME8WdUL9uBp8XV/OVwEt8+6gzn6KFUgQa8g==",
|
||||
"dependencies": {
|
||||
"@aws-sdk/types": "^3.973.1",
|
||||
"@smithy/querystring-builder": "^4.2.8",
|
||||
"@smithy/types": "^4.12.0",
|
||||
"tslib": "^2.6.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@aws-sdk/util-locate-window": {
|
||||
"version": "3.965.4",
|
||||
"resolved": "https://registry.npmjs.org/@aws-sdk/util-locate-window/-/util-locate-window-3.965.4.tgz",
|
||||
@ -4956,6 +4755,15 @@
|
||||
"undici-types": "~6.21.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/node-fetch": {
|
||||
"version": "2.6.13",
|
||||
"resolved": "https://registry.npmjs.org/@types/node-fetch/-/node-fetch-2.6.13.tgz",
|
||||
"integrity": "sha512-QGpRVpzSaUs30JBSGPjOg4Uveu384erbHBoT1zeONvyCfwQxIkUshLAOqN/k9EjGviPRmWTTe6aH2qySWKTVSw==",
|
||||
"dependencies": {
|
||||
"@types/node": "*",
|
||||
"form-data": "^4.0.4"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/nodemailer": {
|
||||
"version": "7.0.9",
|
||||
"resolved": "https://registry.npmjs.org/@types/nodemailer/-/nodemailer-7.0.9.tgz",
|
||||
@ -5686,6 +5494,17 @@
|
||||
"node": ">=10.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/abort-controller": {
|
||||
"version": "3.0.0",
|
||||
"resolved": "https://registry.npmjs.org/abort-controller/-/abort-controller-3.0.0.tgz",
|
||||
"integrity": "sha512-h8lQ8tacZYnR3vNQTgibj+tODHI5/+l06Au2Pcriv/Gmet0eaj4TwWH41sO9wnHDiQsEj19q0drzdWdeAHtweg==",
|
||||
"dependencies": {
|
||||
"event-target-shim": "^5.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=6.5"
|
||||
}
|
||||
},
|
||||
"node_modules/acorn": {
|
||||
"version": "8.15.0",
|
||||
"resolved": "https://registry.npmjs.org/acorn/-/acorn-8.15.0.tgz",
|
||||
@ -5715,6 +5534,17 @@
|
||||
"node": ">= 14"
|
||||
}
|
||||
},
|
||||
"node_modules/agentkeepalive": {
|
||||
"version": "4.6.0",
|
||||
"resolved": "https://registry.npmjs.org/agentkeepalive/-/agentkeepalive-4.6.0.tgz",
|
||||
"integrity": "sha512-kja8j7PjmncONqaTsB8fQ+wE2mSU2DJ9D4XKoJ5PFWIdRMa6SLSN1ff4mOr4jCbfRSsxR4keIiySJU0N9T5hIQ==",
|
||||
"dependencies": {
|
||||
"humanize-ms": "^1.2.1"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 8.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/ajv": {
|
||||
"version": "6.14.0",
|
||||
"resolved": "https://registry.npmjs.org/ajv/-/ajv-6.14.0.tgz",
|
||||
@ -5955,6 +5785,11 @@
|
||||
"node": ">= 0.4"
|
||||
}
|
||||
},
|
||||
"node_modules/asynckit": {
|
||||
"version": "0.4.0",
|
||||
"resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz",
|
||||
"integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q=="
|
||||
},
|
||||
"node_modules/available-typed-arrays": {
|
||||
"version": "1.0.7",
|
||||
"resolved": "https://registry.npmjs.org/available-typed-arrays/-/available-typed-arrays-1.0.7.tgz",
|
||||
@ -6454,6 +6289,17 @@
|
||||
"resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.4.tgz",
|
||||
"integrity": "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA=="
|
||||
},
|
||||
"node_modules/combined-stream": {
|
||||
"version": "1.0.8",
|
||||
"resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz",
|
||||
"integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==",
|
||||
"dependencies": {
|
||||
"delayed-stream": "~1.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.8"
|
||||
}
|
||||
},
|
||||
"node_modules/concat-map": {
|
||||
"version": "0.0.1",
|
||||
"resolved": "https://registry.npmjs.org/concat-map/-/concat-map-0.0.1.tgz",
|
||||
@ -6768,6 +6614,14 @@
|
||||
"resolved": "https://registry.npmjs.org/defu/-/defu-6.1.4.tgz",
|
||||
"integrity": "sha512-mEQCMmwJu317oSz8CwdIOdwf3xMif1ttiM8LTufzc3g6kR+9Pe236twL8j3IYT1F7GfRgGcW6MWxzZjLIkuHIg=="
|
||||
},
|
||||
"node_modules/delayed-stream": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz",
|
||||
"integrity": "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==",
|
||||
"engines": {
|
||||
"node": ">=0.4.0"
|
||||
}
|
||||
},
|
||||
"node_modules/denque": {
|
||||
"version": "2.1.0",
|
||||
"resolved": "https://registry.npmjs.org/denque/-/denque-2.1.0.tgz",
|
||||
@ -7035,7 +6889,6 @@
|
||||
"version": "2.1.0",
|
||||
"resolved": "https://registry.npmjs.org/es-set-tostringtag/-/es-set-tostringtag-2.1.0.tgz",
|
||||
"integrity": "sha512-j6vWzfrGVfyXxge+O0x5sh6cvxAog0a/4Rdd2K36zCMV5eJ+/+tOAngRO8cODMNWbVRdVlmGZQL2YS3yR8bIUA==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"es-errors": "^1.3.0",
|
||||
"get-intrinsic": "^1.2.6",
|
||||
@ -7554,6 +7407,14 @@
|
||||
"node": ">=0.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/event-target-shim": {
|
||||
"version": "5.0.1",
|
||||
"resolved": "https://registry.npmjs.org/event-target-shim/-/event-target-shim-5.0.1.tgz",
|
||||
"integrity": "sha512-i/2XbnSz/uxRCU6+NdVJgKWDTM427+MqYbkQzD321DuCQJUqOuJKIA0IM2+W2xtYHdKOmZ4dR6fExsd4SXL+WQ==",
|
||||
"engines": {
|
||||
"node": ">=6"
|
||||
}
|
||||
},
|
||||
"node_modules/eventemitter3": {
|
||||
"version": "5.0.4",
|
||||
"resolved": "https://registry.npmjs.org/eventemitter3/-/eventemitter3-5.0.4.tgz",
|
||||
@ -7792,6 +7653,38 @@
|
||||
"url": "https://github.com/sponsors/isaacs"
|
||||
}
|
||||
},
|
||||
"node_modules/form-data": {
|
||||
"version": "4.0.5",
|
||||
"resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.5.tgz",
|
||||
"integrity": "sha512-8RipRLol37bNs2bhoV67fiTEvdTrbMUYcFTiy3+wuuOnUog2QBHCZWXDRijWQfAkhBj2Uf5UnVaiWwA5vdd82w==",
|
||||
"dependencies": {
|
||||
"asynckit": "^0.4.0",
|
||||
"combined-stream": "^1.0.8",
|
||||
"es-set-tostringtag": "^2.1.0",
|
||||
"hasown": "^2.0.2",
|
||||
"mime-types": "^2.1.12"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 6"
|
||||
}
|
||||
},
|
||||
"node_modules/form-data-encoder": {
|
||||
"version": "1.7.2",
|
||||
"resolved": "https://registry.npmjs.org/form-data-encoder/-/form-data-encoder-1.7.2.tgz",
|
||||
"integrity": "sha512-qfqtYan3rxrnCk1VYaA4H+Ms9xdpPqvLZa6xmMgFvhO32x7/3J/ExcTd6qpxM0vH2GdMI+poehyBZvqfMTto8A=="
|
||||
},
|
||||
"node_modules/formdata-node": {
|
||||
"version": "4.4.1",
|
||||
"resolved": "https://registry.npmjs.org/formdata-node/-/formdata-node-4.4.1.tgz",
|
||||
"integrity": "sha512-0iirZp3uVDjVGt9p49aTaqjk84TrglENEDuqfdlZQ1roC9CWlPk6Avf8EEnZNcAqPonwkG35x4n3ww/1THYAeQ==",
|
||||
"dependencies": {
|
||||
"node-domexception": "1.0.0",
|
||||
"web-streams-polyfill": "4.0.0-beta.3"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 12.20"
|
||||
}
|
||||
},
|
||||
"node_modules/framer-motion": {
|
||||
"version": "12.34.0",
|
||||
"resolved": "https://registry.npmjs.org/framer-motion/-/framer-motion-12.34.0.tgz",
|
||||
@ -8192,6 +8085,14 @@
|
||||
"node": ">= 14"
|
||||
}
|
||||
},
|
||||
"node_modules/humanize-ms": {
|
||||
"version": "1.2.1",
|
||||
"resolved": "https://registry.npmjs.org/humanize-ms/-/humanize-ms-1.2.1.tgz",
|
||||
"integrity": "sha512-Fl70vYtsAFb/C06PTS9dZBo7ihau+Tu/DNCk/OyHhea07S+aeMWpFFkUaXRa8fI+ScZbEI8dfSxwY7gxZ9SAVQ==",
|
||||
"dependencies": {
|
||||
"ms": "^2.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/ieee754": {
|
||||
"version": "1.1.13",
|
||||
"resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.1.13.tgz",
|
||||
@ -9313,6 +9214,25 @@
|
||||
"node": ">=8.6"
|
||||
}
|
||||
},
|
||||
"node_modules/mime-db": {
|
||||
"version": "1.52.0",
|
||||
"resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz",
|
||||
"integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==",
|
||||
"engines": {
|
||||
"node": ">= 0.6"
|
||||
}
|
||||
},
|
||||
"node_modules/mime-types": {
|
||||
"version": "2.1.35",
|
||||
"resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz",
|
||||
"integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==",
|
||||
"dependencies": {
|
||||
"mime-db": "1.52.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.6"
|
||||
}
|
||||
},
|
||||
"node_modules/minimalistic-assert": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/minimalistic-assert/-/minimalistic-assert-1.0.1.tgz",
|
||||
@ -9572,6 +9492,44 @@
|
||||
"node": "^10 || ^12 || >=14"
|
||||
}
|
||||
},
|
||||
"node_modules/node-domexception": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/node-domexception/-/node-domexception-1.0.0.tgz",
|
||||
"integrity": "sha512-/jKZoMpw0F8GRwl4/eLROPA3cfcXtLApP0QzLmUT/HuPCZWyB7IY9ZrMeKw2O/nFIqPQB3PVM9aYm0F312AXDQ==",
|
||||
"deprecated": "Use your platform's native DOMException instead",
|
||||
"funding": [
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/jimmywarting"
|
||||
},
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://paypal.me/jimmywarting"
|
||||
}
|
||||
],
|
||||
"engines": {
|
||||
"node": ">=10.5.0"
|
||||
}
|
||||
},
|
||||
"node_modules/node-fetch": {
|
||||
"version": "2.7.0",
|
||||
"resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-2.7.0.tgz",
|
||||
"integrity": "sha512-c4FRfUm/dbcWZ7U+1Wq0AwCyFL+3nt2bEw05wfxSz+DWpWsitgmSgYmy2dQdWyKC1694ELPqMs/YzUSNozLt8A==",
|
||||
"dependencies": {
|
||||
"whatwg-url": "^5.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": "4.x || >=6.0.0"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"encoding": "^0.1.0"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"encoding": {
|
||||
"optional": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"node_modules/node-fetch-native": {
|
||||
"version": "1.6.7",
|
||||
"resolved": "https://registry.npmjs.org/node-fetch-native/-/node-fetch-native-1.6.7.tgz",
|
||||
@ -9740,6 +9698,48 @@
|
||||
"resolved": "https://registry.npmjs.org/ohash/-/ohash-2.0.11.tgz",
|
||||
"integrity": "sha512-RdR9FQrFwNBNXAr4GixM8YaRZRJ5PUWbKYbE5eOsrwAjJW0q2REGcf79oYPsLyskQCZG1PLN+S/K1V00joZAoQ=="
|
||||
},
|
||||
"node_modules/openai": {
|
||||
"version": "4.104.0",
|
||||
"resolved": "https://registry.npmjs.org/openai/-/openai-4.104.0.tgz",
|
||||
"integrity": "sha512-p99EFNsA/yX6UhVO93f5kJsDRLAg+CTA2RBqdHK4RtK8u5IJw32Hyb2dTGKbnnFmnuoBv5r7Z2CURI9sGZpSuA==",
|
||||
"dependencies": {
|
||||
"@types/node": "^18.11.18",
|
||||
"@types/node-fetch": "^2.6.4",
|
||||
"abort-controller": "^3.0.0",
|
||||
"agentkeepalive": "^4.2.1",
|
||||
"form-data-encoder": "1.7.2",
|
||||
"formdata-node": "^4.3.2",
|
||||
"node-fetch": "^2.6.7"
|
||||
},
|
||||
"bin": {
|
||||
"openai": "bin/cli"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"ws": "^8.18.0",
|
||||
"zod": "^3.23.8"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"ws": {
|
||||
"optional": true
|
||||
},
|
||||
"zod": {
|
||||
"optional": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"node_modules/openai/node_modules/@types/node": {
|
||||
"version": "18.19.130",
|
||||
"resolved": "https://registry.npmjs.org/@types/node/-/node-18.19.130.tgz",
|
||||
"integrity": "sha512-GRaXQx6jGfL8sKfaIDD6OupbIHBr9jv7Jnaml9tB7l4v068PAOXqfcujMMo5PhbIs6ggR1XODELqahT2R8v0fg==",
|
||||
"dependencies": {
|
||||
"undici-types": "~5.26.4"
|
||||
}
|
||||
},
|
||||
"node_modules/openai/node_modules/undici-types": {
|
||||
"version": "5.26.5",
|
||||
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-5.26.5.tgz",
|
||||
"integrity": "sha512-JlCMO+ehdEIKqlFxk6IfVoAUVmgz7cU7zD/h9XZ0qzeosSHmUJVOzSQvvYSYWXkFXC+IfLKSIffhv0sVZup6pA=="
|
||||
},
|
||||
"node_modules/operetta": {
|
||||
"version": "0.0.5",
|
||||
"resolved": "https://registry.npmjs.org/operetta/-/operetta-0.0.5.tgz",
|
||||
@ -11378,6 +11378,11 @@
|
||||
"node": ">=8.0"
|
||||
}
|
||||
},
|
||||
"node_modules/tr46": {
|
||||
"version": "0.0.3",
|
||||
"resolved": "https://registry.npmjs.org/tr46/-/tr46-0.0.3.tgz",
|
||||
"integrity": "sha512-N3WMsuqV66lT30CrXNbEjx4GEwlow3v6rr4mCcv6prnfwhS01rkgyFdjPNBYd9br7LpXV1+Emh01fHnq2Gdgrw=="
|
||||
},
|
||||
"node_modules/ts-api-utils": {
|
||||
"version": "2.4.0",
|
||||
"resolved": "https://registry.npmjs.org/ts-api-utils/-/ts-api-utils-2.4.0.tgz",
|
||||
@ -12422,6 +12427,28 @@
|
||||
"node": ">= 16"
|
||||
}
|
||||
},
|
||||
"node_modules/web-streams-polyfill": {
|
||||
"version": "4.0.0-beta.3",
|
||||
"resolved": "https://registry.npmjs.org/web-streams-polyfill/-/web-streams-polyfill-4.0.0-beta.3.tgz",
|
||||
"integrity": "sha512-QW95TCTaHmsYfHDybGMwO5IJIM93I/6vTRk+daHTWFPhwh+C8Cg7j7XyKrwrj8Ib6vYXe0ocYNrmzY4xAAN6ug==",
|
||||
"engines": {
|
||||
"node": ">= 14"
|
||||
}
|
||||
},
|
||||
"node_modules/webidl-conversions": {
|
||||
"version": "3.0.1",
|
||||
"resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-3.0.1.tgz",
|
||||
"integrity": "sha512-2JAn3z8AR6rjK8Sm8orRC0h/bcl/DqL7tRPdGZ4I1CjdF+EaMLmYxBHyXuKL849eucPFhvBoxMsflfOb8kxaeQ=="
|
||||
},
|
||||
"node_modules/whatwg-url": {
|
||||
"version": "5.0.0",
|
||||
"resolved": "https://registry.npmjs.org/whatwg-url/-/whatwg-url-5.0.0.tgz",
|
||||
"integrity": "sha512-saE57nupxk6v3HY35+jzBwYa0rKSy0XR8JSxZPwgLr7ys0IBzhGviA1/TUGJLmSVqs8pb9AnvICXEuOHLprYTw==",
|
||||
"dependencies": {
|
||||
"tr46": "~0.0.3",
|
||||
"webidl-conversions": "^3.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/which": {
|
||||
"version": "2.0.2",
|
||||
"resolved": "https://registry.npmjs.org/which/-/which-2.0.2.tgz",
|
||||
|
||||
@ -19,7 +19,6 @@
|
||||
"dependencies": {
|
||||
"@auth/core": "^0.41.0",
|
||||
"@auth/prisma-adapter": "^2.11.1",
|
||||
"@aws-sdk/client-bedrock-runtime": "^3.989.0",
|
||||
"@aws-sdk/client-s3": "^3.989.0",
|
||||
"@prisma/adapter-pg": "^7.6.0",
|
||||
"@prisma/client": "^7.6.0",
|
||||
@ -41,6 +40,7 @@
|
||||
"next": "^16.2.1",
|
||||
"next-auth": "^5.0.0-beta.30",
|
||||
"nodemailer": "^7.0.7",
|
||||
"openai": "^4.104.0",
|
||||
"otplib": "^13.2.1",
|
||||
"pdf-parse": "^2.4.5",
|
||||
"pg": "^8.18.0",
|
||||
|
||||
@ -0,0 +1,9 @@
|
||||
-- CreateEnum
|
||||
CREATE TYPE "KnowledgeBaseMode" AS ENUM ('UNSET', 'STRUCTURED', 'DOCUMENTS');
|
||||
|
||||
-- AlterTable
|
||||
ALTER TABLE "Store" ADD COLUMN "knowledgeBaseMode" "KnowledgeBaseMode" NOT NULL DEFAULT 'UNSET';
|
||||
ALTER TABLE "Store" ADD COLUMN "knowledgeTemplate" JSONB;
|
||||
|
||||
-- AlterEnum: add STRUCTURED to KnowledgeChunkSourceType
|
||||
ALTER TYPE "KnowledgeChunkSourceType" ADD VALUE 'STRUCTURED';
|
||||
@ -115,6 +115,11 @@ model Store {
|
||||
storefrontChatFontFamily String?
|
||||
storefrontChatLogoUrl String?
|
||||
|
||||
/** Primary knowledge source: structured questionnaire vs uploaded documents (mutually exclusive). */
|
||||
knowledgeBaseMode KnowledgeBaseMode @default(UNSET)
|
||||
/** Saved structured KB answers (version, per-field text, misc). Embedded as STRUCTURED chunks. */
|
||||
knowledgeTemplate Json?
|
||||
|
||||
// Relations
|
||||
connections Connection[]
|
||||
cachedProducts CachedProduct[]
|
||||
@ -485,6 +490,13 @@ enum KnowledgeDocumentStatus {
|
||||
enum KnowledgeChunkSourceType {
|
||||
DOCUMENT
|
||||
FAQ
|
||||
STRUCTURED
|
||||
}
|
||||
|
||||
enum KnowledgeBaseMode {
|
||||
UNSET
|
||||
STRUCTURED
|
||||
DOCUMENTS
|
||||
}
|
||||
|
||||
model KnowledgeDocument {
|
||||
|
||||
@ -1,40 +1,37 @@
|
||||
#!/usr/bin/env tsx
|
||||
/**
|
||||
* Re-embedding script for Knowledge Base
|
||||
* Re-embed all KnowledgeChunk rows with OpenAI (text-embedding-3-small @ 1024 dims)
|
||||
* and tag metadata with provider openai (required for KB vector search filters).
|
||||
*
|
||||
* This script re-embeds all existing KnowledgeChunk records using AWS Bedrock
|
||||
* Titan Text Embeddings V2 (1024 dimensions) to replace Gemini embeddings (1536 dimensions).
|
||||
*
|
||||
* Usage:
|
||||
* npm run re-embed-knowledge-base
|
||||
* or
|
||||
* tsx scripts/re-embed-knowledge-base.ts
|
||||
* Usage: npm run re-embed-knowledge-base
|
||||
*/
|
||||
|
||||
import { prisma } from "../src/lib/prisma"
|
||||
import { embedBatch } from "../src/lib/kb/bedrockEmbeddings"
|
||||
import { embedBatch } from "../src/lib/kb/openaiEmbeddings"
|
||||
import {
|
||||
kbEmbeddingMetadata,
|
||||
KB_EMBEDDING_DIMENSIONS,
|
||||
} from "../src/lib/kb/embeddingMeta"
|
||||
import { logger } from "../src/lib/logger"
|
||||
|
||||
// AWS Bedrock Titan V2 limit: 2000 requests/minute
|
||||
// Use smaller batches with delays to stay under limit
|
||||
const BATCH_SIZE = 10 // Process chunks in smaller batches to avoid rate limits
|
||||
const BATCH_SIZE = 32
|
||||
|
||||
interface KnowledgeChunk {
|
||||
interface KnowledgeChunkRow {
|
||||
id: string
|
||||
content: string
|
||||
storeId: string
|
||||
metadata: unknown
|
||||
}
|
||||
|
||||
async function reEmbedKnowledgeBase() {
|
||||
try {
|
||||
logger.info("Starting knowledge base re-embedding process...")
|
||||
logger.info("Starting knowledge base re-embedding (OpenAI)...")
|
||||
|
||||
// Fetch all knowledge chunks
|
||||
const sql = `SELECT "id", "content", "storeId"
|
||||
const sql = `SELECT "id", "content", "storeId", "metadata"
|
||||
FROM "KnowledgeChunk"
|
||||
WHERE "content" IS NOT NULL AND "content" != ''
|
||||
ORDER BY "createdAt" ASC`
|
||||
const chunks = await prisma.$queryRawUnsafe<KnowledgeChunk[]>(sql)
|
||||
const chunks = await prisma.$queryRawUnsafe<KnowledgeChunkRow[]>(sql)
|
||||
|
||||
if (chunks.length === 0) {
|
||||
logger.info("No knowledge chunks found to re-embed.")
|
||||
@ -47,7 +44,6 @@ async function reEmbedKnowledgeBase() {
|
||||
let failed = 0
|
||||
const totalBatches = Math.ceil(chunks.length / BATCH_SIZE)
|
||||
|
||||
// Process in batches
|
||||
for (let i = 0; i < chunks.length; i += BATCH_SIZE) {
|
||||
const batch = chunks.slice(i, i + BATCH_SIZE)
|
||||
const batchNumber = Math.floor(i / BATCH_SIZE) + 1
|
||||
@ -57,8 +53,7 @@ async function reEmbedKnowledgeBase() {
|
||||
)
|
||||
|
||||
try {
|
||||
// Generate embeddings for this batch
|
||||
const texts = batch.map((chunk: KnowledgeChunk) => chunk.content)
|
||||
const texts = batch.map((c) => c.content)
|
||||
const embeddings = await embedBatch(texts)
|
||||
|
||||
if (embeddings.length !== batch.length) {
|
||||
@ -69,39 +64,46 @@ async function reEmbedKnowledgeBase() {
|
||||
continue
|
||||
}
|
||||
|
||||
// Update each chunk with its new embedding
|
||||
const updatePromises = batch.map(async (chunk: KnowledgeChunk, index: number) => {
|
||||
await Promise.all(
|
||||
batch.map(async (chunk, index) => {
|
||||
const embedding = embeddings[index]
|
||||
|
||||
if (!embedding || embedding.length === 0) {
|
||||
logger.warn(
|
||||
`Skipping chunk ${chunk.id}: empty embedding generated`,
|
||||
logger.warn(`Skipping chunk ${chunk.id}: empty embedding`)
|
||||
failed++
|
||||
return
|
||||
}
|
||||
|
||||
if (embedding.length !== KB_EMBEDDING_DIMENSIONS) {
|
||||
logger.error(
|
||||
`Chunk ${chunk.id}: expected ${KB_EMBEDDING_DIMENSIONS} dims, got ${embedding.length}`,
|
||||
)
|
||||
failed++
|
||||
return
|
||||
}
|
||||
|
||||
if (embedding.length !== 1024) {
|
||||
logger.warn(
|
||||
`Chunk ${chunk.id}: embedding dimension is ${embedding.length}, expected 1024`,
|
||||
)
|
||||
// Pad or truncate to 1024
|
||||
if (embedding.length < 1024) {
|
||||
embedding.push(...new Array(1024 - embedding.length).fill(0))
|
||||
} else {
|
||||
embedding.splice(1024)
|
||||
}
|
||||
const prevMeta =
|
||||
chunk.metadata &&
|
||||
typeof chunk.metadata === "object" &&
|
||||
!Array.isArray(chunk.metadata)
|
||||
? (chunk.metadata as Record<string, unknown>)
|
||||
: {}
|
||||
const nextMeta = {
|
||||
...prevMeta,
|
||||
...kbEmbeddingMetadata(),
|
||||
}
|
||||
|
||||
try {
|
||||
await prisma.$executeRawUnsafe(
|
||||
`
|
||||
UPDATE "KnowledgeChunk"
|
||||
SET "embedding" = $1::vector(1024),
|
||||
SET "embedding" = $1::vector,
|
||||
"metadata" = $2::jsonb,
|
||||
"updatedAt" = NOW()
|
||||
WHERE "id" = $2
|
||||
WHERE "id" = $3
|
||||
`,
|
||||
`[${embedding.join(",")}]`,
|
||||
JSON.stringify(nextMeta),
|
||||
chunk.id,
|
||||
)
|
||||
processed++
|
||||
@ -109,20 +111,8 @@ async function reEmbedKnowledgeBase() {
|
||||
logger.error(`Failed to update chunk ${chunk.id}:`, error)
|
||||
failed++
|
||||
}
|
||||
})
|
||||
|
||||
await Promise.all(updatePromises)
|
||||
|
||||
logger.info(
|
||||
`Batch ${batchNumber} completed. Processed: ${processed}, Failed: ${failed}`,
|
||||
}),
|
||||
)
|
||||
|
||||
// Add delay between batches to avoid rate limiting
|
||||
// With 2s delay per request + batch processing, we need extra time between batches
|
||||
if (i + BATCH_SIZE < chunks.length) {
|
||||
// Wait 5 seconds between batches to ensure we stay under 2000 req/min limit
|
||||
await new Promise((resolve) => setTimeout(resolve, 5000))
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error(`Error processing batch ${batchNumber}:`, error)
|
||||
failed += batch.length
|
||||
@ -140,7 +130,6 @@ async function reEmbedKnowledgeBase() {
|
||||
}
|
||||
}
|
||||
|
||||
// Run the script
|
||||
if (require.main === module) {
|
||||
reEmbedKnowledgeBase()
|
||||
.then(() => {
|
||||
|
||||
@ -40,6 +40,23 @@ export async function POST(
|
||||
return NextResponse.json({ message: "Store not found" }, { status: 404 })
|
||||
}
|
||||
|
||||
if (store.knowledgeBaseMode === "STRUCTURED") {
|
||||
return NextResponse.json(
|
||||
{
|
||||
message:
|
||||
"Document upload is disabled while using structured knowledge. Switch to document-based knowledge in Settings → Knowledge base.",
|
||||
},
|
||||
{ status: 409 },
|
||||
)
|
||||
}
|
||||
|
||||
if (store.knowledgeBaseMode === "UNSET") {
|
||||
await prisma.store.update({
|
||||
where: { id: store.id },
|
||||
data: { knowledgeBaseMode: "DOCUMENTS" },
|
||||
})
|
||||
}
|
||||
|
||||
const formData = await request.formData()
|
||||
const file = formData.get("file")
|
||||
|
||||
|
||||
@ -2,7 +2,8 @@ import { NextResponse } from "next/server"
|
||||
import { auth } from "@/auth"
|
||||
import { prisma } from "@/lib/prisma"
|
||||
import { logger } from "@/lib/logger"
|
||||
import { embedBatch } from "@/lib/kb/bedrockEmbeddings"
|
||||
import { embedBatch } from "@/lib/kb/openaiEmbeddings"
|
||||
import { kbEmbeddingMetadata } from "@/lib/kb/embeddingMeta"
|
||||
import type { FaqItem } from "@prisma/client"
|
||||
|
||||
type FaqImportItem = {
|
||||
@ -35,6 +36,16 @@ export async function POST(
|
||||
return NextResponse.json({ message: "Store not found" }, { status: 404 })
|
||||
}
|
||||
|
||||
if (store.knowledgeBaseMode === "STRUCTURED") {
|
||||
return NextResponse.json(
|
||||
{
|
||||
message:
|
||||
"FAQ import is disabled while using structured knowledge. Switch to document-based knowledge in Settings.",
|
||||
},
|
||||
{ status: 409 },
|
||||
)
|
||||
}
|
||||
|
||||
let items: FaqImportItem[] = []
|
||||
|
||||
const contentType = request.headers.get("content-type") || ""
|
||||
@ -135,6 +146,7 @@ export async function POST(
|
||||
{
|
||||
question: faq.question,
|
||||
answer: faq.answer,
|
||||
...kbEmbeddingMetadata(),
|
||||
} as unknown as object,
|
||||
)
|
||||
}),
|
||||
|
||||
@ -2,7 +2,8 @@ import { NextResponse } from "next/server"
|
||||
import { auth } from "@/auth"
|
||||
import { prisma } from "@/lib/prisma"
|
||||
import { logger } from "@/lib/logger"
|
||||
import { embedBatch } from "@/lib/kb/bedrockEmbeddings"
|
||||
import { embedBatch } from "@/lib/kb/openaiEmbeddings"
|
||||
import { kbEmbeddingMetadata } from "@/lib/kb/embeddingMeta"
|
||||
|
||||
export async function GET(
|
||||
_request: Request,
|
||||
@ -67,6 +68,16 @@ export async function POST(
|
||||
return NextResponse.json({ message: "Store not found" }, { status: 404 })
|
||||
}
|
||||
|
||||
if (store.knowledgeBaseMode === "STRUCTURED") {
|
||||
return NextResponse.json(
|
||||
{
|
||||
message:
|
||||
"Manual FAQs are disabled while using structured knowledge. Switch to document-based knowledge in Settings to add FAQs.",
|
||||
},
|
||||
{ status: 409 },
|
||||
)
|
||||
}
|
||||
|
||||
const body = await request.json()
|
||||
const { question, answer, tags } = body as {
|
||||
question?: string
|
||||
@ -115,6 +126,7 @@ export async function POST(
|
||||
{
|
||||
question,
|
||||
answer,
|
||||
...kbEmbeddingMetadata(),
|
||||
} as unknown as object,
|
||||
)
|
||||
}
|
||||
|
||||
@ -2,20 +2,22 @@ import { NextResponse } from "next/server"
|
||||
import { auth } from "@/auth"
|
||||
import { prisma } from "@/lib/prisma"
|
||||
import { logger } from "@/lib/logger"
|
||||
import { embedBatch } from "@/lib/kb/bedrockEmbeddings"
|
||||
import { chatCompletion } from "@/lib/kb/bedrockChat"
|
||||
import { embedBatch } from "@/lib/kb/openaiEmbeddings"
|
||||
import { KB_EMBEDDING_PROVIDER } from "@/lib/kb/embeddingMeta"
|
||||
import { generateKbRagAnswer } from "@/lib/kb/kbAnswer"
|
||||
|
||||
type QueryBody = {
|
||||
query?: string
|
||||
topK?: number
|
||||
includeFaq?: boolean
|
||||
includeDocuments?: boolean
|
||||
includeStructured?: boolean
|
||||
}
|
||||
|
||||
type KnowledgeChunkRow = {
|
||||
id: string
|
||||
content: string
|
||||
sourceType: "DOCUMENT" | "FAQ"
|
||||
sourceType: "DOCUMENT" | "FAQ" | "STRUCTURED"
|
||||
documentId: string | null
|
||||
faqItemId: string | null
|
||||
metadata: unknown
|
||||
@ -58,10 +60,14 @@ export async function POST(
|
||||
const topK = Math.min(Math.max(body.topK ?? 8, 1), 20)
|
||||
const includeFaq = body.includeFaq ?? true
|
||||
const includeDocuments = body.includeDocuments ?? true
|
||||
const includeStructured = body.includeStructured ?? true
|
||||
|
||||
if (!includeFaq && !includeDocuments) {
|
||||
if (!includeFaq && !includeDocuments && !includeStructured) {
|
||||
return NextResponse.json(
|
||||
{ message: "At least one of FAQs or Documents must be included" },
|
||||
{
|
||||
message:
|
||||
"At least one of FAQs, documents, or structured knowledge must be included",
|
||||
},
|
||||
{ status: 400 },
|
||||
)
|
||||
}
|
||||
@ -74,14 +80,18 @@ export async function POST(
|
||||
)
|
||||
}
|
||||
|
||||
const filters: string[] = ['"storeId" = $1']
|
||||
if (includeFaq && !includeDocuments) {
|
||||
filters.push(`"sourceType" = 'FAQ'`)
|
||||
} else if (!includeFaq && includeDocuments) {
|
||||
filters.push(`"sourceType" = 'DOCUMENT'`)
|
||||
}
|
||||
const sourceTypes: string[] = []
|
||||
if (includeFaq) sourceTypes.push("'FAQ'")
|
||||
if (includeDocuments) sourceTypes.push("'DOCUMENT'")
|
||||
if (includeStructured) sourceTypes.push("'STRUCTURED'")
|
||||
|
||||
const whereClause = filters.length ? `WHERE ${filters.join(" AND ")}` : ""
|
||||
const filters: string[] = [
|
||||
'"storeId" = $1',
|
||||
`"sourceType" IN (${sourceTypes.join(", ")})`,
|
||||
`COALESCE(("metadata"::jsonb->>'provider'), '') = '${KB_EMBEDDING_PROVIDER}'`,
|
||||
]
|
||||
|
||||
const whereClause = `WHERE ${filters.join(" AND ")}`
|
||||
|
||||
const chunks = await prisma.$queryRawUnsafe<KnowledgeChunkRow[]>(
|
||||
`
|
||||
@ -113,6 +123,8 @@ export async function POST(
|
||||
const label =
|
||||
chunk.sourceType === "FAQ"
|
||||
? `FAQ #${index + 1}`
|
||||
: chunk.sourceType === "STRUCTURED"
|
||||
? `Store profile #${index + 1}`
|
||||
: `Document chunk #${index + 1}`
|
||||
return `${label} (similarity: ${chunk.similarity.toFixed(2)}):\n${
|
||||
chunk.content
|
||||
@ -123,23 +135,21 @@ export async function POST(
|
||||
|
||||
const systemPrompt =
|
||||
"You are an AI assistant helping answer customer and vendor questions for a specific store.\n" +
|
||||
"Use ONLY the provided context from the store's documents and FAQs.\n" +
|
||||
"Use ONLY the provided context from the store's documents, FAQs, and structured store profile.\n" +
|
||||
"If the answer is not clearly present in the context, say that you do not know.\n" +
|
||||
"Be concise and helpful.\n"
|
||||
|
||||
const userMessage = `Context:\n${context}\n\nQuestion:\n${question}`
|
||||
|
||||
let answer: string
|
||||
try {
|
||||
answer = await chatCompletion(userMessage, {
|
||||
const { text: answer, source } = await generateKbRagAnswer(userMessage, {
|
||||
systemPrompt,
|
||||
maxTokens: 512,
|
||||
})
|
||||
} catch (error) {
|
||||
logger.error("Bedrock RAG answer error", error)
|
||||
|
||||
if (source === "none") {
|
||||
return NextResponse.json(
|
||||
{ message: "Failed to generate answer from Bedrock" },
|
||||
{ status: 500 },
|
||||
{ message: "Failed to generate an answer" },
|
||||
{ status: 502 },
|
||||
)
|
||||
}
|
||||
|
||||
@ -161,4 +171,3 @@ export async function POST(
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
214
src/app/api/stores/[storeId]/knowledge-template/route.ts
Normal file
214
src/app/api/stores/[storeId]/knowledge-template/route.ts
Normal file
@ -0,0 +1,214 @@
|
||||
import { NextResponse } from "next/server"
|
||||
import { auth } from "@/auth"
|
||||
import { prisma } from "@/lib/prisma"
|
||||
import { logger } from "@/lib/logger"
|
||||
import {
|
||||
knowledgeTemplatePutSchema,
|
||||
normalizeTemplateAnswers,
|
||||
type KnowledgeTemplateStored,
|
||||
} from "@/lib/kb/knowledgeTemplate"
|
||||
import { replaceStructuredKnowledgeChunks } from "@/lib/kb/structuredTemplateChunks"
|
||||
import {
|
||||
deleteDocumentKbForStore,
|
||||
deleteFaqKbForStore,
|
||||
deleteStructuredKbForStore,
|
||||
} from "@/lib/kb/storeKbCleanup"
|
||||
import { Prisma, type KnowledgeBaseMode } from "@prisma/client"
|
||||
|
||||
export async function GET(
|
||||
_request: Request,
|
||||
{ params }: { params: Promise<{ storeId: string }> },
|
||||
) {
|
||||
try {
|
||||
const session = await auth()
|
||||
if (!session?.user?.id) {
|
||||
return NextResponse.json({ message: "Unauthorized" }, { status: 401 })
|
||||
}
|
||||
|
||||
const { storeId } = await params
|
||||
const store = await prisma.store.findFirst({
|
||||
where: { id: storeId, userId: session.user.id },
|
||||
select: {
|
||||
id: true,
|
||||
knowledgeBaseMode: true,
|
||||
knowledgeTemplate: true,
|
||||
},
|
||||
})
|
||||
|
||||
if (!store) {
|
||||
return NextResponse.json({ message: "Store not found" }, { status: 404 })
|
||||
}
|
||||
|
||||
const [documentCount, faqCount, structuredChunkCount] = await Promise.all([
|
||||
prisma.knowledgeDocument.count({ where: { storeId: store.id } }),
|
||||
prisma.faqItem.count({ where: { storeId: store.id } }),
|
||||
prisma.knowledgeChunk.count({
|
||||
where: { storeId: store.id, sourceType: "STRUCTURED" },
|
||||
}),
|
||||
])
|
||||
|
||||
return NextResponse.json({
|
||||
knowledgeBaseMode: store.knowledgeBaseMode,
|
||||
knowledgeTemplate: store.knowledgeTemplate,
|
||||
counts: {
|
||||
documents: documentCount,
|
||||
faqs: faqCount,
|
||||
structuredChunks: structuredChunkCount,
|
||||
},
|
||||
})
|
||||
} catch (error) {
|
||||
logger.error("GET knowledge-template error", error)
|
||||
return NextResponse.json(
|
||||
{ message: "Failed to load knowledge template" },
|
||||
{ status: 500 },
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
export async function PUT(
|
||||
request: Request,
|
||||
{ params }: { params: Promise<{ storeId: string }> },
|
||||
) {
|
||||
try {
|
||||
const session = await auth()
|
||||
if (!session?.user?.id) {
|
||||
return NextResponse.json({ message: "Unauthorized" }, { status: 401 })
|
||||
}
|
||||
|
||||
const { storeId } = await params
|
||||
let store = await prisma.store.findFirst({
|
||||
where: { id: storeId, userId: session.user.id },
|
||||
})
|
||||
|
||||
if (!store) {
|
||||
return NextResponse.json({ message: "Store not found" }, { status: 404 })
|
||||
}
|
||||
|
||||
const json = await request.json().catch(() => null)
|
||||
const parsed = knowledgeTemplatePutSchema.safeParse(json)
|
||||
if (!parsed.success) {
|
||||
return NextResponse.json(
|
||||
{ message: "Invalid body", issues: parsed.error.flatten() },
|
||||
{ status: 400 },
|
||||
)
|
||||
}
|
||||
|
||||
const body = parsed.data
|
||||
const confirm = body.confirmDestructiveSwitch === true
|
||||
|
||||
const docCount = await prisma.knowledgeDocument.count({
|
||||
where: { storeId: store.id },
|
||||
})
|
||||
const faqCount = await prisma.faqItem.count({ where: { storeId: store.id } })
|
||||
const structuredCount = await prisma.knowledgeChunk.count({
|
||||
where: { storeId: store.id, sourceType: "STRUCTURED" },
|
||||
})
|
||||
const hasTemplateJson = store.knowledgeTemplate != null
|
||||
|
||||
if (body.knowledgeBaseMode !== undefined) {
|
||||
const target = body.knowledgeBaseMode as KnowledgeBaseMode
|
||||
if (target !== store.knowledgeBaseMode) {
|
||||
if (target === "STRUCTURED") {
|
||||
const destructive = docCount > 0 || faqCount > 0
|
||||
if (destructive && !confirm) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
message:
|
||||
"Switching to structured knowledge will remove uploaded documents and FAQs. Send confirmDestructiveSwitch: true to proceed.",
|
||||
code: "CONFIRM_REQUIRED",
|
||||
},
|
||||
{ status: 400 },
|
||||
)
|
||||
}
|
||||
if (destructive) {
|
||||
await deleteDocumentKbForStore(prisma, store.id)
|
||||
await deleteFaqKbForStore(prisma, store.id)
|
||||
}
|
||||
await prisma.store.update({
|
||||
where: { id: store.id },
|
||||
data: { knowledgeBaseMode: "STRUCTURED" },
|
||||
})
|
||||
} else if (target === "DOCUMENTS") {
|
||||
const destructive = structuredCount > 0 || hasTemplateJson
|
||||
if (destructive && !confirm) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
message:
|
||||
"Switching to document-based knowledge will remove your structured profile and its embeddings. Send confirmDestructiveSwitch: true to proceed.",
|
||||
code: "CONFIRM_REQUIRED",
|
||||
},
|
||||
{ status: 400 },
|
||||
)
|
||||
}
|
||||
await deleteStructuredKbForStore(prisma, store.id)
|
||||
await prisma.store.update({
|
||||
where: { id: store.id },
|
||||
data: {
|
||||
knowledgeBaseMode: "DOCUMENTS",
|
||||
knowledgeTemplate: Prisma.JsonNull,
|
||||
},
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
store = await prisma.store.findFirstOrThrow({
|
||||
where: { id: store.id, userId: session.user.id },
|
||||
})
|
||||
|
||||
if (body.template !== undefined) {
|
||||
if (store.knowledgeBaseMode === "DOCUMENTS") {
|
||||
return NextResponse.json(
|
||||
{
|
||||
message:
|
||||
"This store uses document-based knowledge. Switch to structured knowledge before editing the template.",
|
||||
},
|
||||
{ status: 409 },
|
||||
)
|
||||
}
|
||||
|
||||
const answers = normalizeTemplateAnswers(body.template.answers)
|
||||
const misc = body.template.misc?.trim() || undefined
|
||||
|
||||
const stored: KnowledgeTemplateStored = {
|
||||
version: 1,
|
||||
answers,
|
||||
misc,
|
||||
updatedAt: new Date().toISOString(),
|
||||
}
|
||||
|
||||
const nextMode: KnowledgeBaseMode =
|
||||
store.knowledgeBaseMode === "UNSET" ? "STRUCTURED" : store.knowledgeBaseMode
|
||||
|
||||
await prisma.store.update({
|
||||
where: { id: store.id },
|
||||
data: {
|
||||
knowledgeBaseMode: nextMode,
|
||||
knowledgeTemplate: stored as object,
|
||||
},
|
||||
})
|
||||
|
||||
await replaceStructuredKnowledgeChunks(store.id, stored)
|
||||
}
|
||||
|
||||
const refreshed = await prisma.store.findFirst({
|
||||
where: { id: store.id },
|
||||
select: {
|
||||
knowledgeBaseMode: true,
|
||||
knowledgeTemplate: true,
|
||||
},
|
||||
})
|
||||
|
||||
return NextResponse.json({
|
||||
success: true,
|
||||
knowledgeBaseMode: refreshed?.knowledgeBaseMode,
|
||||
knowledgeTemplate: refreshed?.knowledgeTemplate,
|
||||
})
|
||||
} catch (error) {
|
||||
logger.error("PUT knowledge-template error", error)
|
||||
return NextResponse.json(
|
||||
{ message: "Failed to save knowledge template" },
|
||||
{ status: 500 },
|
||||
)
|
||||
}
|
||||
}
|
||||
@ -1,138 +1,394 @@
|
||||
"use client"
|
||||
|
||||
import { useEffect, useState } from "react"
|
||||
import { DocumentsSection } from "@/components/kb/DocumentsSection"
|
||||
import { FaqSection } from "@/components/kb/FaqSection"
|
||||
import { AskSection } from "@/components/kb/AskSection"
|
||||
import { useCallback, useEffect, useState } from "react"
|
||||
import { DocumentsFaqKnowledgeModal } from "@/components/kb/DocumentsFaqKnowledgeModal"
|
||||
import { StructuredKnowledgeModal } from "@/components/kb/StructuredKnowledgeModal"
|
||||
import { useTheme } from "@/components/ThemeProvider"
|
||||
import { cn } from "@/lib/utils"
|
||||
import type { KnowledgeTemplateStored } from "@/lib/kb/knowledgeTemplate"
|
||||
import { BookOpen, ChevronRight, FileStack, Loader2 } from "lucide-react"
|
||||
|
||||
type KbCounts = {
|
||||
documents: number
|
||||
faqs: number
|
||||
structuredChunks: number
|
||||
}
|
||||
|
||||
export default function KnowledgeBasePage() {
|
||||
const { theme } = useTheme()
|
||||
const isLight = theme === "light"
|
||||
const [activeStoreId, setActiveStoreId] = useState<string | null>(null)
|
||||
const [kbLoading, setKbLoading] = useState(true)
|
||||
const [knowledgeBaseMode, setKnowledgeBaseMode] = useState<
|
||||
"UNSET" | "STRUCTURED" | "DOCUMENTS"
|
||||
>("UNSET")
|
||||
const [structuredOpen, setStructuredOpen] = useState(false)
|
||||
const [documentsModalOpen, setDocumentsModalOpen] = useState(false)
|
||||
const [modeBusy, setModeBusy] = useState<"documents" | null>(null)
|
||||
const [toast, setToast] = useState<string | null>(null)
|
||||
|
||||
const loadKb = useCallback(async (sid: string) => {
|
||||
const res = await fetch(`/api/stores/${encodeURIComponent(sid)}/knowledge-template`)
|
||||
if (!res.ok) return
|
||||
const j = (await res.json()) as {
|
||||
knowledgeBaseMode: "UNSET" | "STRUCTURED" | "DOCUMENTS"
|
||||
knowledgeTemplate: KnowledgeTemplateStored | null
|
||||
counts: KbCounts
|
||||
}
|
||||
setKnowledgeBaseMode(j.knowledgeBaseMode)
|
||||
}, [])
|
||||
|
||||
useEffect(() => {
|
||||
let cancelled = false
|
||||
|
||||
; (async () => {
|
||||
try {
|
||||
const res = await fetch("/api/stores")
|
||||
if (!res.ok) return
|
||||
|
||||
const data = (await res.json()) as {
|
||||
stores?: { id: string; isActive: boolean }[]
|
||||
}
|
||||
const active = (data.stores || []).find((s) => s.isActive)
|
||||
|
||||
if (!cancelled) {
|
||||
setActiveStoreId(active?.id ?? null)
|
||||
}
|
||||
if (!cancelled) setActiveStoreId(active?.id ?? null)
|
||||
} catch {
|
||||
if (!cancelled) {
|
||||
setActiveStoreId(null)
|
||||
}
|
||||
if (!cancelled) setActiveStoreId(null)
|
||||
}
|
||||
})()
|
||||
|
||||
return () => {
|
||||
cancelled = true
|
||||
}
|
||||
}, [])
|
||||
|
||||
const [activeTab, setActiveTab] = useState<"documents" | "faqs" | "ask">(
|
||||
"documents",
|
||||
useEffect(() => {
|
||||
if (!activeStoreId) {
|
||||
setKbLoading(false)
|
||||
setKnowledgeBaseMode("UNSET")
|
||||
return
|
||||
}
|
||||
let cancelled = false
|
||||
; (async () => {
|
||||
try {
|
||||
setKbLoading(true)
|
||||
await loadKb(activeStoreId)
|
||||
} finally {
|
||||
if (!cancelled) setKbLoading(false)
|
||||
}
|
||||
})()
|
||||
return () => {
|
||||
cancelled = true
|
||||
}
|
||||
}, [activeStoreId, loadKb])
|
||||
|
||||
async function ensureDocumentsMode(
|
||||
confirmDestructiveSwitch?: boolean,
|
||||
): Promise<boolean> {
|
||||
if (!activeStoreId) return false
|
||||
if (knowledgeBaseMode === "DOCUMENTS") return true
|
||||
|
||||
setModeBusy("documents")
|
||||
setToast(null)
|
||||
try {
|
||||
const res = await fetch(
|
||||
`/api/stores/${encodeURIComponent(activeStoreId)}/knowledge-template`,
|
||||
{
|
||||
method: "PUT",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
knowledgeBaseMode: "DOCUMENTS",
|
||||
...(confirmDestructiveSwitch
|
||||
? { confirmDestructiveSwitch: true }
|
||||
: {}),
|
||||
}),
|
||||
},
|
||||
)
|
||||
const j = await res.json().catch(() => ({}))
|
||||
if (res.status === 400 && j.code === "CONFIRM_REQUIRED") {
|
||||
const ok = window.confirm(
|
||||
"This removes your structured profile and its embeddings. Continue?",
|
||||
)
|
||||
if (!ok) return false
|
||||
return await ensureDocumentsMode(true)
|
||||
}
|
||||
if (!res.ok) {
|
||||
throw new Error(j.message || "Failed to update mode")
|
||||
}
|
||||
await loadKb(activeStoreId)
|
||||
setToast("Using documents & FAQs for this store.")
|
||||
return true
|
||||
} catch (e) {
|
||||
setToast(e instanceof Error ? e.message : "Could not switch mode")
|
||||
return false
|
||||
} finally {
|
||||
setModeBusy(null)
|
||||
}
|
||||
}
|
||||
|
||||
async function openDocumentsKnowledgePanel() {
|
||||
if (!activeStoreId) return
|
||||
const ok = await ensureDocumentsMode()
|
||||
if (ok) setDocumentsModalOpen(true)
|
||||
}
|
||||
|
||||
const hasStore = Boolean(activeStoreId)
|
||||
const structuredActive = knowledgeBaseMode === "STRUCTURED"
|
||||
|
||||
const modeLabel =
|
||||
knowledgeBaseMode === "STRUCTURED"
|
||||
? "Structured profile"
|
||||
: knowledgeBaseMode === "DOCUMENTS"
|
||||
? "Documents & FAQs"
|
||||
: "Not chosen yet"
|
||||
|
||||
return (
|
||||
<div className="flex flex-col h-full space-y-6">
|
||||
<div className="flex items-center justify-between">
|
||||
<div className="flex flex-col h-full space-y-5">
|
||||
<div>
|
||||
<h1 className={cn("text-2xl font-bold", isLight ? "text-slate-900" : "text-white")}>
|
||||
Knowledge Base
|
||||
<h1
|
||||
className={cn(
|
||||
"text-2xl font-bold tracking-tight",
|
||||
isLight ? "text-slate-900" : "text-white",
|
||||
)}
|
||||
>
|
||||
Knowledge base
|
||||
</h1>
|
||||
<p className={cn("text-sm", isLight ? "text-slate-500" : "text-slate-400")}>
|
||||
Centralize your store's documents and FAQs so JUNO can
|
||||
answer customer and vendor questions.
|
||||
<p
|
||||
className={cn(
|
||||
"text-sm mt-1 max-w-2xl",
|
||||
isLight ? "text-slate-500" : "text-slate-400",
|
||||
)}
|
||||
>
|
||||
One source of truth per store for AI: either a guided profile or
|
||||
uploaded docs & FAQs—not both.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{!hasStore ? (
|
||||
<div className="flex-1 flex items-center justify-center">
|
||||
<div className="text-center max-w-md">
|
||||
<h2 className={cn("text-lg font-semibold mb-2", isLight ? "text-slate-900" : "text-white")}>
|
||||
No active store selected
|
||||
</h2>
|
||||
<p className={cn("text-sm", isLight ? "text-slate-500" : "text-slate-400")}>
|
||||
Select a store from the top bar to manage its knowledge
|
||||
<div className="flex-1 flex items-center justify-center py-16">
|
||||
<p
|
||||
className={cn(
|
||||
"text-sm text-center max-w-sm",
|
||||
isLight ? "text-slate-500" : "text-slate-400",
|
||||
)}
|
||||
>
|
||||
Select an active store from the top bar to manage its knowledge
|
||||
base.
|
||||
</p>
|
||||
</div>
|
||||
) : kbLoading ? (
|
||||
<div className="flex items-center gap-2 text-sm text-slate-400 py-8">
|
||||
<Loader2 className="w-4 h-4 animate-spin" />
|
||||
Loading…
|
||||
</div>
|
||||
) : (
|
||||
<>
|
||||
<div className={cn("flex items-center space-x-2 border-b", isLight ? "border-slate-100" : "border-white/10")}>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setActiveTab("documents")}
|
||||
<div
|
||||
id="kb-source"
|
||||
className={cn(
|
||||
"px-4 py-2 text-sm font-medium border-b-2 -mb-px transition-colors",
|
||||
activeTab === "documents"
|
||||
? "border-indigo-500 text-indigo-600"
|
||||
: isLight
|
||||
? "border-transparent text-slate-500 hover:text-slate-900 hover:border-slate-300"
|
||||
: "border-transparent text-slate-400 hover:text-white hover:border-slate-500"
|
||||
"rounded-xl border p-4 scroll-mt-4",
|
||||
isLight
|
||||
? "border-slate-200 bg-white shadow-sm"
|
||||
: "border-white/10 bg-white/[0.04]",
|
||||
)}
|
||||
>
|
||||
Documents
|
||||
<div className="flex flex-col sm:flex-row sm:items-center sm:justify-between gap-3 mb-3">
|
||||
<div>
|
||||
<p
|
||||
className={cn(
|
||||
"text-[11px] font-bold uppercase tracking-wider",
|
||||
isLight ? "text-slate-400" : "text-slate-500",
|
||||
)}
|
||||
>
|
||||
Knowledge source
|
||||
</p>
|
||||
<p
|
||||
className={cn(
|
||||
"text-sm font-medium mt-0.5",
|
||||
isLight ? "text-slate-800" : "text-slate-200",
|
||||
)}
|
||||
>
|
||||
{modeLabel}
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="grid grid-cols-1 sm:grid-cols-2 gap-2 sm:gap-3">
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setStructuredOpen(true)}
|
||||
className={cn(
|
||||
"flex items-center gap-3 rounded-lg border px-4 py-3 text-left transition-all",
|
||||
knowledgeBaseMode === "STRUCTURED"
|
||||
? isLight
|
||||
? "border-indigo-400 bg-indigo-50/80 ring-1 ring-indigo-200"
|
||||
: "border-indigo-500/50 bg-indigo-500/10"
|
||||
: isLight
|
||||
? "border-slate-200 hover:border-slate-300 hover:bg-slate-50/80"
|
||||
: "border-white/10 hover:border-white/20 hover:bg-white/[0.06]",
|
||||
)}
|
||||
>
|
||||
<div
|
||||
className={cn(
|
||||
"flex h-10 w-10 items-center justify-center rounded-lg shrink-0",
|
||||
isLight ? "bg-indigo-100" : "bg-indigo-500/20",
|
||||
)}
|
||||
>
|
||||
<BookOpen
|
||||
className={cn(
|
||||
"h-5 w-5",
|
||||
isLight ? "text-indigo-600" : "text-indigo-300",
|
||||
)}
|
||||
/>
|
||||
</div>
|
||||
<div className="min-w-0 flex-1">
|
||||
<p
|
||||
className={cn(
|
||||
"text-sm font-semibold",
|
||||
isLight ? "text-slate-900" : "text-white",
|
||||
)}
|
||||
>
|
||||
Structured profile
|
||||
</p>
|
||||
<p
|
||||
className={cn(
|
||||
"text-xs mt-0.5 line-clamp-2",
|
||||
isLight ? "text-slate-500" : "text-slate-400",
|
||||
)}
|
||||
>
|
||||
Guided questions · opens in a panel
|
||||
</p>
|
||||
</div>
|
||||
<ChevronRight
|
||||
className={cn(
|
||||
"h-4 w-4 shrink-0",
|
||||
isLight ? "text-slate-400" : "text-slate-500",
|
||||
)}
|
||||
/>
|
||||
</button>
|
||||
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setActiveTab("faqs")}
|
||||
disabled={modeBusy !== null}
|
||||
onClick={() => void openDocumentsKnowledgePanel()}
|
||||
className={cn(
|
||||
"px-4 py-2 text-sm font-medium border-b-2 -mb-px transition-colors",
|
||||
activeTab === "faqs"
|
||||
? "border-indigo-500 text-indigo-600"
|
||||
"flex items-center gap-3 rounded-lg border px-4 py-3 text-left transition-all disabled:opacity-60 disabled:cursor-not-allowed",
|
||||
knowledgeBaseMode === "DOCUMENTS"
|
||||
? isLight
|
||||
? "border-emerald-400 bg-emerald-50/80 ring-1 ring-emerald-200"
|
||||
: "border-emerald-500/50 bg-emerald-500/10"
|
||||
: isLight
|
||||
? "border-transparent text-slate-500 hover:text-slate-900 hover:border-slate-300"
|
||||
: "border-transparent text-slate-400 hover:text-white hover:border-slate-500"
|
||||
? "border-slate-200 hover:border-slate-300 hover:bg-slate-50/80"
|
||||
: "border-white/10 hover:border-white/20 hover:bg-white/[0.06]",
|
||||
)}
|
||||
>
|
||||
FAQs
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setActiveTab("ask")}
|
||||
<div
|
||||
className={cn(
|
||||
"px-4 py-2 text-sm font-medium border-b-2 -mb-px transition-colors",
|
||||
activeTab === "ask"
|
||||
? "border-indigo-500 text-indigo-600"
|
||||
: isLight
|
||||
? "border-transparent text-slate-500 hover:text-slate-900 hover:border-slate-300"
|
||||
: "border-transparent text-slate-400 hover:text-white hover:border-slate-500"
|
||||
"flex h-10 w-10 items-center justify-center rounded-lg shrink-0",
|
||||
isLight ? "bg-emerald-100" : "bg-emerald-500/20",
|
||||
)}
|
||||
>
|
||||
Ask
|
||||
{modeBusy === "documents" ? (
|
||||
<Loader2 className="h-5 w-5 animate-spin text-emerald-600" />
|
||||
) : (
|
||||
<FileStack
|
||||
className={cn(
|
||||
"h-5 w-5",
|
||||
isLight
|
||||
? "text-emerald-700"
|
||||
: "text-emerald-300",
|
||||
)}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
<div className="min-w-0 flex-1">
|
||||
<p
|
||||
className={cn(
|
||||
"text-sm font-semibold",
|
||||
isLight ? "text-slate-900" : "text-white",
|
||||
)}
|
||||
>
|
||||
Documents & FAQs
|
||||
</p>
|
||||
<p
|
||||
className={cn(
|
||||
"text-xs mt-0.5 line-clamp-2",
|
||||
isLight ? "text-slate-500" : "text-slate-400",
|
||||
)}
|
||||
>
|
||||
PDFs, FAQs & Ask · opens in a panel
|
||||
</p>
|
||||
</div>
|
||||
<ChevronRight
|
||||
className={cn(
|
||||
"h-4 w-4 shrink-0",
|
||||
isLight ? "text-slate-400" : "text-slate-500",
|
||||
)}
|
||||
/>
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div className="flex-1 min-h-0 overflow-y-auto pb-4">
|
||||
{activeTab === "documents" && (
|
||||
<DocumentsSection storeId={activeStoreId!} />
|
||||
{toast && (
|
||||
<p
|
||||
className={cn(
|
||||
"text-xs mt-3 px-1",
|
||||
toast.includes("Could not") || toast.includes("Failed")
|
||||
? isLight
|
||||
? "text-amber-700"
|
||||
: "text-amber-400"
|
||||
: isLight
|
||||
? "text-emerald-700"
|
||||
: "text-emerald-400/90",
|
||||
)}
|
||||
{activeTab === "faqs" && (
|
||||
<FaqSection storeId={activeStoreId!} />
|
||||
)}
|
||||
{activeTab === "ask" && (
|
||||
<AskSection storeId={activeStoreId!} />
|
||||
>
|
||||
{toast}
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{structuredActive && (
|
||||
<p
|
||||
className={cn(
|
||||
"text-xs",
|
||||
isLight ? "text-slate-500" : "text-slate-500",
|
||||
)}
|
||||
>
|
||||
You're on a structured profile. Open{" "}
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setStructuredOpen(true)}
|
||||
className="font-semibold text-indigo-500 hover:underline"
|
||||
>
|
||||
Structured profile
|
||||
</button>{" "}
|
||||
to edit, or choose{" "}
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => void openDocumentsKnowledgePanel()}
|
||||
className="font-semibold text-emerald-600 hover:underline dark:text-emerald-400"
|
||||
>
|
||||
Documents & FAQs
|
||||
</button>{" "}
|
||||
to switch (you'll confirm replacing structured data).
|
||||
</p>
|
||||
)}
|
||||
|
||||
<StructuredKnowledgeModal
|
||||
open={structuredOpen}
|
||||
onOpenChange={setStructuredOpen}
|
||||
storeId={activeStoreId!}
|
||||
knowledgeBaseMode={knowledgeBaseMode}
|
||||
isLight={isLight}
|
||||
onSaved={async () => {
|
||||
await loadKb(activeStoreId!)
|
||||
setToast("Structured profile saved and embedded.")
|
||||
}}
|
||||
/>
|
||||
|
||||
<DocumentsFaqKnowledgeModal
|
||||
open={documentsModalOpen}
|
||||
onOpenChange={setDocumentsModalOpen}
|
||||
storeId={activeStoreId!}
|
||||
isLight={isLight}
|
||||
/>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
6
src/app/dashboard/settings/knowledge-base/page.tsx
Normal file
6
src/app/dashboard/settings/knowledge-base/page.tsx
Normal file
@ -0,0 +1,6 @@
|
||||
import { redirect } from "next/navigation"
|
||||
|
||||
/** Old location; knowledge base lives under JUNO Engine. */
|
||||
export default function KnowledgeBaseSettingsRedirectPage() {
|
||||
redirect("/dashboard/juno-engine/knowledge-base")
|
||||
}
|
||||
@ -10,7 +10,7 @@ type AskSectionProps = {
|
||||
|
||||
type Source = {
|
||||
id: string
|
||||
type: "DOCUMENT" | "FAQ"
|
||||
type: "DOCUMENT" | "FAQ" | "STRUCTURED"
|
||||
documentId: string | null
|
||||
faqId: string | null
|
||||
similarity: number
|
||||
@ -22,6 +22,7 @@ export function AskSection({ storeId }: AskSectionProps) {
|
||||
const [query, setQuery] = useState("")
|
||||
const [includeFaq, setIncludeFaq] = useState(true)
|
||||
const [includeDocuments, setIncludeDocuments] = useState(true)
|
||||
const [includeStructured, setIncludeStructured] = useState(true)
|
||||
const [answer, setAnswer] = useState<string | null>(null)
|
||||
const [sources, setSources] = useState<Source[]>([])
|
||||
const [isLoading, setIsLoading] = useState(false)
|
||||
@ -48,6 +49,7 @@ export function AskSection({ storeId }: AskSectionProps) {
|
||||
query: query.trim(),
|
||||
includeFaq,
|
||||
includeDocuments,
|
||||
includeStructured,
|
||||
}),
|
||||
},
|
||||
)
|
||||
@ -75,8 +77,8 @@ export function AskSection({ storeId }: AskSectionProps) {
|
||||
<div>
|
||||
<h3 className={cn("text-lg font-semibold", isLight ? "text-slate-900" : "text-white")}>Ask JUNO</h3>
|
||||
<p className={cn("text-xs", isLight ? "text-slate-500" : "text-slate-400")}>
|
||||
Ask questions across your store's documents and FAQs. The
|
||||
AI will answer using only your knowledge base.
|
||||
Ask questions across documents, FAQs, and your structured store profile.
|
||||
The AI uses only your knowledge base.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
@ -105,7 +107,7 @@ export function AskSection({ storeId }: AskSectionProps) {
|
||||
)}
|
||||
/>
|
||||
<div className="flex items-center justify-between gap-4">
|
||||
<div className="flex items-center gap-4 text-xs">
|
||||
<div className="flex flex-wrap items-center gap-4 text-xs">
|
||||
<label className={cn(
|
||||
"inline-flex items-center gap-2 cursor-pointer transition-colors",
|
||||
isLight ? "text-slate-600 hover:text-slate-900" : "text-slate-400 hover:text-white"
|
||||
@ -120,6 +122,20 @@ export function AskSection({ storeId }: AskSectionProps) {
|
||||
/>
|
||||
<span className="font-medium">Use documents</span>
|
||||
</label>
|
||||
<label className={cn(
|
||||
"inline-flex items-center gap-2 cursor-pointer transition-colors",
|
||||
isLight ? "text-slate-600 hover:text-slate-900" : "text-slate-400 hover:text-white"
|
||||
)}>
|
||||
<input
|
||||
type="checkbox"
|
||||
className="rounded border-slate-300 bg-white text-indigo-600 focus:ring-indigo-500/20"
|
||||
checked={includeStructured}
|
||||
onChange={(e) =>
|
||||
setIncludeStructured(e.target.checked)
|
||||
}
|
||||
/>
|
||||
<span className="font-medium">Use structured profile</span>
|
||||
</label>
|
||||
<label className={cn(
|
||||
"inline-flex items-center gap-2 cursor-pointer transition-colors",
|
||||
isLight ? "text-slate-600 hover:text-slate-900" : "text-slate-400 hover:text-white"
|
||||
@ -178,6 +194,8 @@ export function AskSection({ storeId }: AskSectionProps) {
|
||||
)}>
|
||||
{source.type === "FAQ"
|
||||
? "FAQ"
|
||||
: source.type === "STRUCTURED"
|
||||
? "Profile"
|
||||
: "Document"}
|
||||
</span>
|
||||
<span className={isLight ? "text-slate-500" : "text-slate-400"}>
|
||||
|
||||
171
src/components/kb/DocumentsFaqKnowledgeModal.tsx
Normal file
171
src/components/kb/DocumentsFaqKnowledgeModal.tsx
Normal file
@ -0,0 +1,171 @@
|
||||
"use client"
|
||||
|
||||
import { useEffect, useState } from "react"
|
||||
import { createPortal } from "react-dom"
|
||||
import { DocumentsSection } from "@/components/kb/DocumentsSection"
|
||||
import { FaqSection } from "@/components/kb/FaqSection"
|
||||
import { AskSection } from "@/components/kb/AskSection"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { X } from "lucide-react"
|
||||
|
||||
type TabId = "documents" | "faqs" | "ask"
|
||||
|
||||
type DocumentsFaqKnowledgeModalProps = {
|
||||
open: boolean
|
||||
onOpenChange: (open: boolean) => void
|
||||
storeId: string
|
||||
isLight: boolean
|
||||
}
|
||||
|
||||
export function DocumentsFaqKnowledgeModal({
|
||||
open,
|
||||
onOpenChange,
|
||||
storeId,
|
||||
isLight,
|
||||
}: DocumentsFaqKnowledgeModalProps) {
|
||||
const [mounted, setMounted] = useState(false)
|
||||
const [tab, setTab] = useState<TabId>("documents")
|
||||
|
||||
useEffect(() => {
|
||||
setMounted(true)
|
||||
}, [])
|
||||
|
||||
useEffect(() => {
|
||||
if (!open) return
|
||||
const onKey = (e: KeyboardEvent) => {
|
||||
if (e.key === "Escape") onOpenChange(false)
|
||||
}
|
||||
window.addEventListener("keydown", onKey)
|
||||
return () => window.removeEventListener("keydown", onKey)
|
||||
}, [open, onOpenChange])
|
||||
|
||||
useEffect(() => {
|
||||
if (open) {
|
||||
document.body.style.overflow = "hidden"
|
||||
} else {
|
||||
document.body.style.overflow = ""
|
||||
}
|
||||
return () => {
|
||||
document.body.style.overflow = ""
|
||||
}
|
||||
}, [open])
|
||||
|
||||
if (!mounted || !open) return null
|
||||
|
||||
const panelClass = isLight
|
||||
? "bg-white border-slate-200 text-slate-900 shadow-xl"
|
||||
: "bg-slate-900/95 border-white/10 text-white shadow-2xl backdrop-blur-xl"
|
||||
|
||||
const tabs: { id: TabId; label: string }[] = [
|
||||
{ id: "documents", label: "Documents" },
|
||||
{ id: "faqs", label: "FAQs" },
|
||||
{ id: "ask", label: "Ask" },
|
||||
]
|
||||
|
||||
return createPortal(
|
||||
<div
|
||||
className="fixed inset-0 z-[200] flex items-end sm:items-center justify-center p-0 sm:p-6"
|
||||
role="dialog"
|
||||
aria-modal="true"
|
||||
aria-labelledby="docs-kb-modal-title"
|
||||
>
|
||||
<button
|
||||
type="button"
|
||||
className="absolute inset-0 bg-black/50 backdrop-blur-sm"
|
||||
aria-label="Close dialog"
|
||||
onClick={() => onOpenChange(false)}
|
||||
/>
|
||||
<div
|
||||
className={cn(
|
||||
"relative flex w-full flex-col rounded-t-2xl border sm:rounded-2xl",
|
||||
"max-h-[92vh] sm:max-h-[88vh] sm:max-w-4xl lg:max-w-5xl",
|
||||
panelClass,
|
||||
"sm:mx-4",
|
||||
)}
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
<div
|
||||
className={cn(
|
||||
"flex shrink-0 items-start justify-between gap-3 border-b px-5 py-4",
|
||||
isLight ? "border-slate-100" : "border-white/10",
|
||||
)}
|
||||
>
|
||||
<div>
|
||||
<h2
|
||||
id="docs-kb-modal-title"
|
||||
className={cn(
|
||||
"text-lg font-bold tracking-tight sm:text-xl",
|
||||
isLight ? "text-slate-900" : "text-white",
|
||||
)}
|
||||
>
|
||||
Documents & FAQs
|
||||
</h2>
|
||||
<p
|
||||
className={cn(
|
||||
"mt-1 max-w-xl text-xs sm:text-sm",
|
||||
isLight ? "text-slate-500" : "text-slate-400",
|
||||
)}
|
||||
>
|
||||
Upload files, manage FAQs, and test retrieval with Ask—all for
|
||||
this store.
|
||||
</p>
|
||||
</div>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => onOpenChange(false)}
|
||||
className={cn(
|
||||
"shrink-0 rounded-lg p-2 transition-colors",
|
||||
isLight
|
||||
? "text-slate-500 hover:bg-slate-100"
|
||||
: "text-slate-400 hover:bg-white/10",
|
||||
)}
|
||||
aria-label="Close"
|
||||
>
|
||||
<X className="h-5 w-5" />
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div
|
||||
className={cn(
|
||||
"flex shrink-0 gap-1 border-b px-4 pt-2",
|
||||
isLight ? "border-slate-100" : "border-white/10",
|
||||
)}
|
||||
>
|
||||
{tabs.map((t) => (
|
||||
<button
|
||||
key={t.id}
|
||||
type="button"
|
||||
onClick={() => setTab(t.id)}
|
||||
className={cn(
|
||||
"rounded-t-lg px-4 py-2.5 text-sm font-medium transition-colors",
|
||||
tab === t.id
|
||||
? isLight
|
||||
? "bg-slate-100 text-indigo-600"
|
||||
: "bg-white/10 text-indigo-300"
|
||||
: isLight
|
||||
? "text-slate-500 hover:bg-slate-50 hover:text-slate-800"
|
||||
: "text-slate-400 hover:bg-white/5 hover:text-white",
|
||||
)}
|
||||
>
|
||||
{t.label}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
|
||||
<div
|
||||
className={cn(
|
||||
"min-h-0 flex-1 overflow-y-auto px-4 py-4 sm:px-5 sm:py-5",
|
||||
"max-h-[calc(92vh-11rem)] sm:max-h-[calc(88vh-10rem)]",
|
||||
)}
|
||||
>
|
||||
{tab === "documents" && (
|
||||
<DocumentsSection storeId={storeId} locked={false} />
|
||||
)}
|
||||
{tab === "faqs" && <FaqSection storeId={storeId} locked={false} />}
|
||||
{tab === "ask" && <AskSection storeId={storeId} />}
|
||||
</div>
|
||||
</div>
|
||||
</div>,
|
||||
document.body,
|
||||
)
|
||||
}
|
||||
@ -1,6 +1,7 @@
|
||||
"use client"
|
||||
|
||||
import { useEffect, useState } from "react"
|
||||
import Link from "next/link"
|
||||
import { useTheme } from "@/components/ThemeProvider"
|
||||
import { cn } from "@/lib/utils"
|
||||
|
||||
@ -17,9 +18,17 @@ type KnowledgeDocument = {
|
||||
|
||||
type DocumentsSectionProps = {
|
||||
storeId: string
|
||||
locked?: boolean
|
||||
lockedMessage?: string
|
||||
settingsHref?: string
|
||||
}
|
||||
|
||||
export function DocumentsSection({ storeId }: DocumentsSectionProps) {
|
||||
export function DocumentsSection({
|
||||
storeId,
|
||||
locked = false,
|
||||
lockedMessage = "This store uses structured knowledge. Switch to Documents & FAQs above or edit your structured profile.",
|
||||
settingsHref = "/dashboard/juno-engine/knowledge-base#kb-source",
|
||||
}: DocumentsSectionProps) {
|
||||
const { theme } = useTheme()
|
||||
const isLight = theme === "light"
|
||||
const [documents, setDocuments] = useState<KnowledgeDocument[]>([])
|
||||
@ -118,6 +127,27 @@ export function DocumentsSection({ storeId }: DocumentsSectionProps) {
|
||||
|
||||
return (
|
||||
<div className="space-y-4">
|
||||
{locked && (
|
||||
<div
|
||||
className={cn(
|
||||
"rounded-xl border px-4 py-3 text-sm",
|
||||
isLight
|
||||
? "border-amber-200 bg-amber-50 text-amber-900"
|
||||
: "border-amber-500/30 bg-amber-500/10 text-amber-100",
|
||||
)}
|
||||
>
|
||||
<p>{lockedMessage}</p>
|
||||
<Link
|
||||
href={settingsHref}
|
||||
className={cn(
|
||||
"mt-2 inline-block text-xs font-bold underline",
|
||||
isLight ? "text-indigo-600" : "text-indigo-300",
|
||||
)}
|
||||
>
|
||||
Knowledge base
|
||||
</Link>
|
||||
</div>
|
||||
)}
|
||||
<div className="flex items-center justify-between">
|
||||
<div>
|
||||
<h3 className={cn("text-lg font-semibold", isLight ? "text-slate-900" : "text-white")}>
|
||||
@ -127,14 +157,21 @@ export function DocumentsSection({ storeId }: DocumentsSectionProps) {
|
||||
Upload PDFs or DOCX files to power AI answers.
|
||||
</p>
|
||||
</div>
|
||||
<label className="inline-flex items-center px-4 py-2 rounded-lg bg-indigo-600 hover:bg-indigo-700 text-sm font-medium text-white cursor-pointer transition-colors shadow-sm">
|
||||
<label
|
||||
className={cn(
|
||||
"inline-flex items-center px-4 py-2 rounded-lg text-sm font-medium text-white transition-colors shadow-sm",
|
||||
locked
|
||||
? "bg-slate-500 cursor-not-allowed opacity-60"
|
||||
: "bg-indigo-600 hover:bg-indigo-700 cursor-pointer",
|
||||
)}
|
||||
>
|
||||
<span>{isUploading ? "Uploading..." : "Upload document"}</span>
|
||||
<input
|
||||
type="file"
|
||||
accept=".pdf,.doc,.docx,application/pdf,application/msword,application/vnd.openxmlformats-officedocument.wordprocessingml.document"
|
||||
className="hidden"
|
||||
onChange={handleUpload}
|
||||
disabled={isUploading}
|
||||
disabled={isUploading || locked}
|
||||
/>
|
||||
</label>
|
||||
</div>
|
||||
@ -204,7 +241,8 @@ export function DocumentsSection({ storeId }: DocumentsSectionProps) {
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => void handleDelete(doc.id)}
|
||||
className="p-1.5 rounded-lg border border-red-200 text-red-500 hover:bg-red-50 transition-colors"
|
||||
disabled={locked}
|
||||
className="p-1.5 rounded-lg border border-red-200 text-red-500 hover:bg-red-50 transition-colors disabled:opacity-40 disabled:pointer-events-none"
|
||||
title="Delete document"
|
||||
>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" className="h-4 w-4" fill="none" viewBox="0 0 24 24" stroke="currentColor">
|
||||
|
||||
@ -1,6 +1,7 @@
|
||||
"use client"
|
||||
|
||||
import { useEffect, useState } from "react"
|
||||
import Link from "next/link"
|
||||
import { useTheme } from "@/components/ThemeProvider"
|
||||
import { cn } from "@/lib/utils"
|
||||
|
||||
@ -14,9 +15,17 @@ type FaqItem = {
|
||||
|
||||
type FaqSectionProps = {
|
||||
storeId: string
|
||||
locked?: boolean
|
||||
lockedMessage?: string
|
||||
settingsHref?: string
|
||||
}
|
||||
|
||||
export function FaqSection({ storeId }: FaqSectionProps) {
|
||||
export function FaqSection({
|
||||
storeId,
|
||||
locked = false,
|
||||
lockedMessage = "Manual FAQs are disabled while using structured knowledge. Switch mode using the cards above.",
|
||||
settingsHref = "/dashboard/juno-engine/knowledge-base#kb-source",
|
||||
}: FaqSectionProps) {
|
||||
const { theme } = useTheme()
|
||||
const isLight = theme === "light"
|
||||
const [faqs, setFaqs] = useState<FaqItem[]>([])
|
||||
@ -167,6 +176,27 @@ export function FaqSection({ storeId }: FaqSectionProps) {
|
||||
|
||||
return (
|
||||
<div className="space-y-4">
|
||||
{locked && (
|
||||
<div
|
||||
className={cn(
|
||||
"rounded-xl border px-4 py-3 text-sm",
|
||||
isLight
|
||||
? "border-amber-200 bg-amber-50 text-amber-900"
|
||||
: "border-amber-500/30 bg-amber-500/10 text-amber-100",
|
||||
)}
|
||||
>
|
||||
<p>{lockedMessage}</p>
|
||||
<Link
|
||||
href={settingsHref}
|
||||
className={cn(
|
||||
"mt-2 inline-block text-xs font-bold underline",
|
||||
isLight ? "text-indigo-600" : "text-indigo-300",
|
||||
)}
|
||||
>
|
||||
Knowledge base
|
||||
</Link>
|
||||
</div>
|
||||
)}
|
||||
<div className="flex items-center justify-between">
|
||||
<div>
|
||||
<h3 className={cn("text-lg font-semibold", isLight ? "text-slate-900" : "text-white")}>FAQs</h3>
|
||||
@ -175,12 +205,18 @@ export function FaqSection({ storeId }: FaqSectionProps) {
|
||||
trusted answers.
|
||||
</p>
|
||||
</div>
|
||||
<label className={cn(
|
||||
"inline-flex items-center px-3 py-2 rounded-lg text-xs font-medium cursor-pointer transition-colors border shadow-sm",
|
||||
isLight
|
||||
? "bg-white border-slate-200 text-slate-700 hover:bg-slate-50"
|
||||
: "bg-slate-800/80 hover:bg-slate-800 border-white/10 text-slate-100 shadow-xl"
|
||||
)}>
|
||||
<label
|
||||
className={cn(
|
||||
"inline-flex items-center px-3 py-2 rounded-lg text-xs font-medium transition-colors border shadow-sm",
|
||||
locked
|
||||
? isLight
|
||||
? "opacity-50 cursor-not-allowed bg-white border-slate-200 text-slate-400"
|
||||
: "opacity-50 cursor-not-allowed bg-slate-800/80 border-white/10 text-slate-500"
|
||||
: isLight
|
||||
? "cursor-pointer bg-white border-slate-200 text-slate-700 hover:bg-slate-50"
|
||||
: "cursor-pointer bg-slate-800/80 hover:bg-slate-800 border-white/10 text-slate-100 shadow-xl",
|
||||
)}
|
||||
>
|
||||
<span>
|
||||
{isImporting ? "Importing..." : "Import from JSON file"}
|
||||
</span>
|
||||
@ -189,7 +225,7 @@ export function FaqSection({ storeId }: FaqSectionProps) {
|
||||
accept="application/json"
|
||||
className="hidden"
|
||||
onChange={handleImport}
|
||||
disabled={isImporting}
|
||||
disabled={isImporting || locked}
|
||||
/>
|
||||
</label>
|
||||
</div>
|
||||
@ -204,6 +240,7 @@ export function FaqSection({ storeId }: FaqSectionProps) {
|
||||
onSubmit={handleAddFaq}
|
||||
className={cn(
|
||||
"border rounded-xl p-4 space-y-4 transition-all",
|
||||
locked && "opacity-50 pointer-events-none",
|
||||
isLight ? "bg-white border-slate-100 shadow-sm" : "bg-slate-900/40 border-white/5 backdrop-blur-sm"
|
||||
)}
|
||||
>
|
||||
|
||||
373
src/components/kb/StructuredKnowledgeModal.tsx
Normal file
373
src/components/kb/StructuredKnowledgeModal.tsx
Normal file
@ -0,0 +1,373 @@
|
||||
"use client"
|
||||
|
||||
import { useCallback, useEffect, useState } from "react"
|
||||
import { createPortal } from "react-dom"
|
||||
import {
|
||||
KNOWLEDGE_TEMPLATE_FIELDS,
|
||||
type KnowledgeTemplateFieldKey,
|
||||
type KnowledgeTemplateStored,
|
||||
} from "@/lib/kb/knowledgeTemplate"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { Loader2, Save, X } from "lucide-react"
|
||||
|
||||
type KbCounts = {
|
||||
documents: number
|
||||
faqs: number
|
||||
structuredChunks: number
|
||||
}
|
||||
|
||||
type StructuredKnowledgeModalProps = {
|
||||
open: boolean
|
||||
onOpenChange: (open: boolean) => void
|
||||
storeId: string
|
||||
knowledgeBaseMode: "UNSET" | "STRUCTURED" | "DOCUMENTS"
|
||||
onSaved: () => void | Promise<void>
|
||||
isLight: boolean
|
||||
}
|
||||
|
||||
export function StructuredKnowledgeModal({
|
||||
open,
|
||||
onOpenChange,
|
||||
storeId,
|
||||
knowledgeBaseMode,
|
||||
onSaved,
|
||||
isLight,
|
||||
}: StructuredKnowledgeModalProps) {
|
||||
const [mounted, setMounted] = useState(false)
|
||||
const [loading, setLoading] = useState(false)
|
||||
const [saving, setSaving] = useState(false)
|
||||
const [error, setError] = useState<string | null>(null)
|
||||
|
||||
const [answers, setAnswers] = useState<Record<KnowledgeTemplateFieldKey, string>>(
|
||||
() =>
|
||||
KNOWLEDGE_TEMPLATE_FIELDS.reduce(
|
||||
(acc, f) => {
|
||||
acc[f.key] = ""
|
||||
return acc
|
||||
},
|
||||
{} as Record<KnowledgeTemplateFieldKey, string>,
|
||||
),
|
||||
)
|
||||
const [misc, setMisc] = useState("")
|
||||
|
||||
const loadForm = useCallback(async () => {
|
||||
const res = await fetch(
|
||||
`/api/stores/${encodeURIComponent(storeId)}/knowledge-template`,
|
||||
)
|
||||
if (!res.ok) return
|
||||
const data = (await res.json()) as {
|
||||
knowledgeTemplate: KnowledgeTemplateStored | null
|
||||
}
|
||||
const tmpl = data.knowledgeTemplate
|
||||
setAnswers((prev) => {
|
||||
const next = { ...prev }
|
||||
for (const f of KNOWLEDGE_TEMPLATE_FIELDS) {
|
||||
const v = tmpl?.answers?.[f.key]
|
||||
next[f.key] = typeof v === "string" ? v : ""
|
||||
}
|
||||
return next
|
||||
})
|
||||
setMisc(typeof tmpl?.misc === "string" ? tmpl.misc : "")
|
||||
}, [storeId])
|
||||
|
||||
useEffect(() => {
|
||||
setMounted(true)
|
||||
}, [])
|
||||
|
||||
useEffect(() => {
|
||||
if (!open || !storeId) return
|
||||
let cancelled = false
|
||||
; (async () => {
|
||||
try {
|
||||
setLoading(true)
|
||||
setError(null)
|
||||
await loadForm()
|
||||
} finally {
|
||||
if (!cancelled) setLoading(false)
|
||||
}
|
||||
})()
|
||||
return () => {
|
||||
cancelled = true
|
||||
}
|
||||
}, [open, storeId, loadForm])
|
||||
|
||||
useEffect(() => {
|
||||
if (!open) return
|
||||
const onKey = (e: KeyboardEvent) => {
|
||||
if (e.key === "Escape") onOpenChange(false)
|
||||
}
|
||||
window.addEventListener("keydown", onKey)
|
||||
return () => window.removeEventListener("keydown", onKey)
|
||||
}, [open, onOpenChange])
|
||||
|
||||
useEffect(() => {
|
||||
if (open) {
|
||||
document.body.style.overflow = "hidden"
|
||||
} else {
|
||||
document.body.style.overflow = ""
|
||||
}
|
||||
return () => {
|
||||
document.body.style.overflow = ""
|
||||
}
|
||||
}, [open])
|
||||
|
||||
async function handleSave(e: React.FormEvent) {
|
||||
e.preventDefault()
|
||||
setSaving(true)
|
||||
setError(null)
|
||||
|
||||
const payload: Record<string, unknown> = {
|
||||
template: {
|
||||
answers,
|
||||
misc: misc.trim() || undefined,
|
||||
},
|
||||
}
|
||||
|
||||
if (knowledgeBaseMode === "DOCUMENTS") {
|
||||
payload.knowledgeBaseMode = "STRUCTURED"
|
||||
}
|
||||
|
||||
const trySave = async (confirmDestructiveSwitch?: boolean) => {
|
||||
const body = {
|
||||
...payload,
|
||||
...(confirmDestructiveSwitch ? { confirmDestructiveSwitch: true } : {}),
|
||||
}
|
||||
const res = await fetch(
|
||||
`/api/stores/${encodeURIComponent(storeId)}/knowledge-template`,
|
||||
{
|
||||
method: "PUT",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(body),
|
||||
},
|
||||
)
|
||||
const j = await res.json().catch(() => ({}))
|
||||
if (res.status === 400 && j.code === "CONFIRM_REQUIRED") {
|
||||
const ok = window.confirm(
|
||||
"Switching to structured knowledge will remove all uploaded documents and FAQs for this store. Continue?",
|
||||
)
|
||||
if (!ok) return false
|
||||
return await trySave(true)
|
||||
}
|
||||
if (!res.ok) {
|
||||
throw new Error(j.message || "Failed to save")
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
try {
|
||||
const ok = await trySave()
|
||||
if (ok) {
|
||||
await onSaved()
|
||||
onOpenChange(false)
|
||||
}
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : "Failed to save")
|
||||
} finally {
|
||||
setSaving(false)
|
||||
}
|
||||
}
|
||||
|
||||
if (!mounted || !open) return null
|
||||
|
||||
const panelClass = isLight
|
||||
? "bg-white border-slate-200 text-slate-900 shadow-xl"
|
||||
: "bg-slate-900/95 border-white/10 text-white shadow-2xl backdrop-blur-xl"
|
||||
|
||||
return createPortal(
|
||||
<div
|
||||
className="fixed inset-0 z-[200] flex items-end sm:items-center justify-center p-0 sm:p-4"
|
||||
role="dialog"
|
||||
aria-modal="true"
|
||||
aria-labelledby="structured-kb-title"
|
||||
>
|
||||
<button
|
||||
type="button"
|
||||
className="absolute inset-0 bg-black/50 backdrop-blur-sm"
|
||||
aria-label="Close dialog"
|
||||
onClick={() => onOpenChange(false)}
|
||||
/>
|
||||
<div
|
||||
className={cn(
|
||||
"relative flex min-h-0 w-full max-h-[92vh] flex-col rounded-t-2xl border sm:mx-4 sm:max-h-[90vh] sm:rounded-2xl",
|
||||
"sm:max-w-3xl lg:max-w-4xl",
|
||||
panelClass,
|
||||
)}
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
<div
|
||||
className={cn(
|
||||
"flex items-start justify-between gap-3 px-5 py-4 border-b shrink-0",
|
||||
isLight ? "border-slate-100" : "border-white/10",
|
||||
)}
|
||||
>
|
||||
<div>
|
||||
<h2
|
||||
id="structured-kb-title"
|
||||
className={cn(
|
||||
"text-lg font-bold tracking-tight",
|
||||
isLight ? "text-slate-900" : "text-white",
|
||||
)}
|
||||
>
|
||||
Structured store profile
|
||||
</h2>
|
||||
<p
|
||||
className={cn(
|
||||
"text-xs mt-1",
|
||||
isLight ? "text-slate-500" : "text-slate-400",
|
||||
)}
|
||||
>
|
||||
Answers are embedded for AI retrieval. Skip any field you
|
||||
don't need.
|
||||
</p>
|
||||
</div>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => onOpenChange(false)}
|
||||
className={cn(
|
||||
"p-2 rounded-lg transition-colors shrink-0",
|
||||
isLight
|
||||
? "hover:bg-slate-100 text-slate-500"
|
||||
: "hover:bg-white/10 text-slate-400",
|
||||
)}
|
||||
aria-label="Close"
|
||||
>
|
||||
<X className="w-5 h-5" />
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{loading ? (
|
||||
<div className="flex items-center justify-center gap-2 py-16 text-sm text-slate-400">
|
||||
<Loader2 className="w-5 h-5 animate-spin" />
|
||||
Loading…
|
||||
</div>
|
||||
) : (
|
||||
<form
|
||||
onSubmit={(e) => void handleSave(e)}
|
||||
className="flex min-h-0 flex-1 flex-col"
|
||||
>
|
||||
<div
|
||||
className={cn(
|
||||
"custom-scrollbar min-h-0 flex-1 space-y-5 overflow-y-auto px-5 py-4",
|
||||
)}
|
||||
>
|
||||
{error && (
|
||||
<div
|
||||
className={cn(
|
||||
"text-xs rounded-lg px-3 py-2 border",
|
||||
isLight
|
||||
? "text-red-600 bg-red-50 border-red-100"
|
||||
: "text-red-400 bg-red-500/10 border-red-500/20",
|
||||
)}
|
||||
>
|
||||
{error}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{KNOWLEDGE_TEMPLATE_FIELDS.map((field) => (
|
||||
<div key={field.key}>
|
||||
<label
|
||||
className={cn(
|
||||
"block text-sm font-semibold",
|
||||
isLight ? "text-slate-900" : "text-white",
|
||||
)}
|
||||
>
|
||||
{field.label}
|
||||
</label>
|
||||
<p
|
||||
className={cn(
|
||||
"text-[11px] leading-snug mt-0.5 mb-1.5",
|
||||
isLight ? "text-slate-500" : "text-slate-500",
|
||||
)}
|
||||
>
|
||||
{field.description}
|
||||
</p>
|
||||
<textarea
|
||||
value={answers[field.key]}
|
||||
onChange={(e) =>
|
||||
setAnswers((a) => ({
|
||||
...a,
|
||||
[field.key]: e.target.value,
|
||||
}))
|
||||
}
|
||||
placeholder={field.placeholder}
|
||||
rows={3}
|
||||
className={cn(
|
||||
"w-full px-3 py-2 rounded-lg border text-sm resize-y min-h-[72px] focus:outline-none focus:ring-2 focus:ring-indigo-500/30",
|
||||
isLight
|
||||
? "bg-slate-50 border-slate-200 text-slate-900 placeholder:text-slate-400"
|
||||
: "bg-slate-950/50 border-white/10 text-white placeholder:text-slate-500",
|
||||
)}
|
||||
/>
|
||||
</div>
|
||||
))}
|
||||
|
||||
<div>
|
||||
<label
|
||||
className={cn(
|
||||
"block text-sm font-semibold",
|
||||
isLight ? "text-slate-900" : "text-white",
|
||||
)}
|
||||
>
|
||||
Anything else?
|
||||
</label>
|
||||
<p
|
||||
className={cn(
|
||||
"text-[11px] mt-0.5 mb-1.5",
|
||||
isLight ? "text-slate-500" : "text-slate-500",
|
||||
)}
|
||||
>
|
||||
Extra notes customers should know.
|
||||
</p>
|
||||
<textarea
|
||||
value={misc}
|
||||
onChange={(e) => setMisc(e.target.value)}
|
||||
placeholder="Optional…"
|
||||
rows={3}
|
||||
className={cn(
|
||||
"w-full px-3 py-2 rounded-lg border text-sm resize-y min-h-[72px] focus:outline-none focus:ring-2 focus:ring-indigo-500/30",
|
||||
isLight
|
||||
? "bg-slate-50 border-slate-200 text-slate-900 placeholder:text-slate-400"
|
||||
: "bg-slate-950/50 border-white/10 text-white placeholder:text-slate-500",
|
||||
)}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div
|
||||
className={cn(
|
||||
"flex items-center justify-end gap-2 px-5 py-4 border-t shrink-0",
|
||||
isLight ? "border-slate-100 bg-slate-50/80" : "border-white/10 bg-slate-950/50",
|
||||
)}
|
||||
>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => onOpenChange(false)}
|
||||
className={cn(
|
||||
"px-4 py-2 rounded-lg text-sm font-medium transition-colors",
|
||||
isLight
|
||||
? "text-slate-600 hover:bg-slate-200/80"
|
||||
: "text-slate-300 hover:bg-white/10",
|
||||
)}
|
||||
>
|
||||
Cancel
|
||||
</button>
|
||||
<button
|
||||
type="submit"
|
||||
disabled={saving}
|
||||
className="inline-flex items-center gap-2 px-4 py-2 rounded-lg bg-indigo-600 hover:bg-indigo-700 text-white text-sm font-bold disabled:opacity-50"
|
||||
>
|
||||
{saving ? (
|
||||
<Loader2 className="w-4 h-4 animate-spin" />
|
||||
) : (
|
||||
<Save className="w-4 h-4" />
|
||||
)}
|
||||
Save & embed
|
||||
</button>
|
||||
</div>
|
||||
</form>
|
||||
)}
|
||||
</div>
|
||||
</div>,
|
||||
document.body,
|
||||
)
|
||||
}
|
||||
@ -1,87 +0,0 @@
|
||||
import { BedrockRuntimeClient, InvokeModelCommand } from "@aws-sdk/client-bedrock-runtime"
|
||||
import { logger } from "@/lib/logger"
|
||||
|
||||
const AWS_REGION = process.env.AWS_REGION || "us-east-1"
|
||||
const BEDROCK_CHAT_MODEL =
|
||||
process.env.BEDROCK_CHAT_MODEL || "anthropic.claude-3-haiku-20240307-v1:0"
|
||||
|
||||
// Initialize Bedrock client
|
||||
const bedrockClient = new BedrockRuntimeClient({
|
||||
region: AWS_REGION,
|
||||
credentials: process.env.AWS_ACCESS_KEY_ID && process.env.AWS_SECRET_ACCESS_KEY
|
||||
? {
|
||||
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
|
||||
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
|
||||
}
|
||||
: undefined, // Will use default credential chain (IAM role, etc.)
|
||||
})
|
||||
|
||||
export interface ChatCompletionOptions {
|
||||
systemPrompt?: string
|
||||
maxTokens?: number
|
||||
temperature?: number
|
||||
}
|
||||
|
||||
export async function chatCompletion(
|
||||
userMessage: string,
|
||||
options: ChatCompletionOptions = {},
|
||||
): Promise<string> {
|
||||
const {
|
||||
systemPrompt = "",
|
||||
maxTokens = 512,
|
||||
temperature = 0.7,
|
||||
} = options
|
||||
|
||||
try {
|
||||
// Claude Messages API format
|
||||
const requestBody = {
|
||||
anthropic_version: "bedrock-2023-05-31",
|
||||
max_tokens: maxTokens,
|
||||
temperature: temperature,
|
||||
system: systemPrompt,
|
||||
messages: [
|
||||
{
|
||||
role: "user" as const,
|
||||
content: userMessage,
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
const command = new InvokeModelCommand({
|
||||
modelId: BEDROCK_CHAT_MODEL,
|
||||
contentType: "application/json",
|
||||
accept: "application/json",
|
||||
body: JSON.stringify(requestBody),
|
||||
})
|
||||
|
||||
const response = await bedrockClient.send(command)
|
||||
|
||||
if (!response.body) {
|
||||
logger.error("Empty response body from Bedrock chat API")
|
||||
return "I am unable to generate an answer at this time."
|
||||
}
|
||||
|
||||
// Parse response body
|
||||
const responseBody = JSON.parse(
|
||||
new TextDecoder().decode(response.body),
|
||||
) as {
|
||||
content?: Array<{
|
||||
type?: string
|
||||
text?: string
|
||||
}>
|
||||
stop_reason?: string
|
||||
}
|
||||
|
||||
// Extract text from response
|
||||
const textContent = responseBody.content
|
||||
?.filter((item) => item.type === "text")
|
||||
.map((item) => item.text || "")
|
||||
.join("")
|
||||
.trim()
|
||||
|
||||
return textContent || "I am unable to generate an answer at this time."
|
||||
} catch (error) {
|
||||
logger.error("Bedrock chat completion failed", error)
|
||||
return "I am unable to generate an answer at this time."
|
||||
}
|
||||
}
|
||||
@ -1,168 +0,0 @@
|
||||
import { BedrockRuntimeClient, InvokeModelCommand } from "@aws-sdk/client-bedrock-runtime"
|
||||
import { logger } from "@/lib/logger"
|
||||
|
||||
const AWS_REGION = process.env.AWS_REGION || "us-east-1"
|
||||
// Hard-code Titan Text Embeddings v2 so all embeddings go through this model
|
||||
const BEDROCK_EMBEDDING_MODEL = "amazon.titan-embed-text-v2:0"
|
||||
|
||||
/**
|
||||
* AWS Bedrock Rate Limits:
|
||||
* - Titan Text Embeddings V2: 2,000 requests/minute (~33 req/sec)
|
||||
* - We use 2-second delays between requests (30 req/min) to stay well under the limit
|
||||
* - Throttling errors trigger exponential backoff: 5s, 10s, 20s, 40s, 80s
|
||||
*/
|
||||
|
||||
// Initialize Bedrock client with retry configuration
|
||||
const bedrockClient = new BedrockRuntimeClient({
|
||||
region: AWS_REGION,
|
||||
credentials: process.env.AWS_ACCESS_KEY_ID && process.env.AWS_SECRET_ACCESS_KEY
|
||||
? {
|
||||
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
|
||||
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
|
||||
}
|
||||
: undefined, // Will use default credential chain (IAM role, etc.)
|
||||
maxAttempts: 5, // AWS SDK default retry attempts
|
||||
})
|
||||
|
||||
if (!process.env.AWS_ACCESS_KEY_ID && !process.env.AWS_SECRET_ACCESS_KEY) {
|
||||
logger.warn(
|
||||
"AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are not set. Bedrock will use default credential chain.",
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Retry helper with exponential backoff for throttling errors
|
||||
*/
|
||||
async function retryWithBackoff<T>(
|
||||
fn: () => Promise<T>,
|
||||
maxRetries: number = 5,
|
||||
baseDelay: number = 5000, // Start with 5 seconds for throttling
|
||||
): Promise<T> {
|
||||
let lastError: unknown
|
||||
|
||||
for (let attempt = 0; attempt <= maxRetries; attempt++) {
|
||||
try {
|
||||
return await fn()
|
||||
} catch (error: any) {
|
||||
lastError = error
|
||||
|
||||
// Check if it's a throttling error
|
||||
const isThrottling =
|
||||
error?.name === "ThrottlingException" ||
|
||||
error?.$metadata?.httpStatusCode === 429 ||
|
||||
error?.message?.includes("Too many requests")
|
||||
|
||||
if (isThrottling && attempt < maxRetries) {
|
||||
// Exponential backoff with longer delays: 5s, 10s, 20s, 40s, 80s
|
||||
const delay = baseDelay * Math.pow(2, attempt)
|
||||
logger.warn(
|
||||
`Bedrock throttling detected, retrying in ${delay / 1000}s (attempt ${attempt + 1}/${maxRetries + 1})`,
|
||||
)
|
||||
await new Promise((resolve) => setTimeout(resolve, delay))
|
||||
continue
|
||||
}
|
||||
|
||||
// If not throttling or max retries reached, throw
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
throw lastError
|
||||
}
|
||||
|
||||
export async function embedBatch(texts: string[]): Promise<number[][]> {
|
||||
if (texts.length === 0) return []
|
||||
|
||||
const results: number[][] = []
|
||||
// AWS Bedrock Titan V2 limit: 2000 requests/minute = ~33 req/sec
|
||||
// Use 2 seconds delay to stay well under limit (30 req/min = safe margin)
|
||||
const REQUEST_DELAY = 2000 // 2 seconds between requests
|
||||
|
||||
for (let i = 0; i < texts.length; i++) {
|
||||
const text = texts[i]
|
||||
const trimmed = text.trim()
|
||||
if (!trimmed) {
|
||||
results.push([])
|
||||
continue
|
||||
}
|
||||
|
||||
try {
|
||||
// Titan Text Embeddings V2 request format
|
||||
const requestBody = {
|
||||
inputText: trimmed,
|
||||
dimensions: 1024, // Use 1024 dimensions to match database schema
|
||||
}
|
||||
|
||||
const command = new InvokeModelCommand({
|
||||
modelId: BEDROCK_EMBEDDING_MODEL,
|
||||
contentType: "application/json",
|
||||
accept: "application/json",
|
||||
body: JSON.stringify(requestBody),
|
||||
})
|
||||
|
||||
// Use retry logic with exponential backoff
|
||||
const response = await retryWithBackoff(async () => {
|
||||
return await bedrockClient.send(command)
|
||||
})
|
||||
|
||||
if (!response.body) {
|
||||
logger.error("Empty response body from Bedrock embedding API")
|
||||
results.push([])
|
||||
continue
|
||||
}
|
||||
|
||||
// Parse response body
|
||||
const responseBody = JSON.parse(
|
||||
new TextDecoder().decode(response.body),
|
||||
) as {
|
||||
embedding?: number[]
|
||||
inputTokenCount?: number
|
||||
}
|
||||
|
||||
const vector = responseBody.embedding
|
||||
if (!vector || !Array.isArray(vector)) {
|
||||
logger.error("Invalid embedding response from Bedrock", responseBody)
|
||||
results.push([])
|
||||
continue
|
||||
}
|
||||
|
||||
// Verify dimension matches expected (1024)
|
||||
if (vector.length !== 1024) {
|
||||
logger.warn(
|
||||
`Expected 1024 dimensions but got ${vector.length}. Truncating or padding as needed.`,
|
||||
)
|
||||
// Pad or truncate to 1024
|
||||
if (vector.length < 1024) {
|
||||
vector.push(...new Array(1024 - vector.length).fill(0))
|
||||
} else {
|
||||
vector.splice(1024)
|
||||
}
|
||||
}
|
||||
|
||||
results.push(vector)
|
||||
|
||||
// Add delay between requests to avoid rate limiting (except for last request)
|
||||
if (i < texts.length - 1) {
|
||||
await new Promise((resolve) => setTimeout(resolve, REQUEST_DELAY))
|
||||
}
|
||||
} catch (error: any) {
|
||||
const isThrottling =
|
||||
error?.name === "ThrottlingException" ||
|
||||
error?.$metadata?.httpStatusCode === 429 ||
|
||||
error?.message?.includes("Too many requests")
|
||||
|
||||
if (isThrottling) {
|
||||
logger.error(
|
||||
`Bedrock throttling error after retries for text ${i + 1}/${texts.length}`,
|
||||
error,
|
||||
)
|
||||
} else {
|
||||
logger.error(`Bedrock embedding request failed for text ${i + 1}/${texts.length}`, error)
|
||||
}
|
||||
// On any error, push an empty vector and continue.
|
||||
results.push([])
|
||||
}
|
||||
}
|
||||
|
||||
return results
|
||||
}
|
||||
27
src/lib/kb/embeddingMeta.ts
Normal file
27
src/lib/kb/embeddingMeta.ts
Normal file
@ -0,0 +1,27 @@
|
||||
/** OpenAI embedding model used for pgvector rows (must match query-time filters). */
|
||||
export const KB_EMBEDDING_MODEL =
|
||||
process.env.OPENAI_EMBEDDING_MODEL?.trim() || "text-embedding-3-small"
|
||||
|
||||
const parsedDims = Number(process.env.OPENAI_EMBEDDING_DIMENSIONS || "1024")
|
||||
export const KB_EMBEDDING_DIMENSIONS =
|
||||
Number.isFinite(parsedDims) && parsedDims > 0 ? parsedDims : 1024
|
||||
|
||||
export const KB_EMBEDDING_PROVIDER = "openai" as const
|
||||
|
||||
export type KbChunkEmbeddingMetadata = {
|
||||
provider: typeof KB_EMBEDDING_PROVIDER
|
||||
embeddingModel: string
|
||||
embeddingDims: number
|
||||
sourceKey?: string
|
||||
}
|
||||
|
||||
export function kbEmbeddingMetadata(
|
||||
extra?: Partial<Pick<KbChunkEmbeddingMetadata, "sourceKey">>,
|
||||
): KbChunkEmbeddingMetadata {
|
||||
return {
|
||||
provider: KB_EMBEDDING_PROVIDER,
|
||||
embeddingModel: KB_EMBEDDING_MODEL,
|
||||
embeddingDims: KB_EMBEDDING_DIMENSIONS,
|
||||
...extra,
|
||||
}
|
||||
}
|
||||
@ -1,6 +1,7 @@
|
||||
import { prisma } from "@/lib/prisma"
|
||||
import { logger } from "@/lib/logger"
|
||||
import { embedBatch } from "@/lib/kb/bedrockEmbeddings"
|
||||
import { embedBatch } from "@/lib/kb/openaiEmbeddings"
|
||||
import { kbEmbeddingMetadata } from "@/lib/kb/embeddingMeta"
|
||||
import type { TextChunk } from "@/lib/kb/documentProcessing"
|
||||
|
||||
export async function indexDocumentChunks(chunks: TextChunk[]) {
|
||||
@ -17,6 +18,10 @@ export async function indexDocumentChunks(chunks: TextChunk[]) {
|
||||
const values = chunks.map((chunk, idx) => {
|
||||
const embedding = embeddings[idx]
|
||||
const vectorLiteral = embedding.length ? `'[${embedding.join(",")}]'` : "NULL"
|
||||
const metadata = {
|
||||
...chunk.metadata,
|
||||
...kbEmbeddingMetadata(),
|
||||
}
|
||||
|
||||
return prisma.$executeRawUnsafe(
|
||||
`
|
||||
@ -37,7 +42,7 @@ export async function indexDocumentChunks(chunks: TextChunk[]) {
|
||||
chunk.storeId,
|
||||
chunk.documentId,
|
||||
chunk.content,
|
||||
chunk.metadata as unknown as object,
|
||||
metadata as unknown as object,
|
||||
)
|
||||
})
|
||||
|
||||
|
||||
49
src/lib/kb/kbAnswer.ts
Normal file
49
src/lib/kb/kbAnswer.ts
Normal file
@ -0,0 +1,49 @@
|
||||
import { openaiChatCompletion } from "@/lib/kb/openaiChat"
|
||||
import { groqChatCompletion } from "@/lib/ai/groqClient"
|
||||
import { logger } from "@/lib/logger"
|
||||
|
||||
/**
|
||||
* Generate an answer for KB RAG: OpenAI chat first, then Groq if OpenAI is unavailable or empty.
|
||||
*/
|
||||
export async function generateKbRagAnswer(
|
||||
userMessage: string,
|
||||
options: { systemPrompt: string; maxTokens?: number },
|
||||
): Promise<{ text: string; source: "openai" | "groq" | "none" }> {
|
||||
const maxTokens = options.maxTokens ?? 512
|
||||
|
||||
const openai = await openaiChatCompletion(
|
||||
[
|
||||
{ role: "system", content: options.systemPrompt },
|
||||
{ role: "user", content: userMessage },
|
||||
],
|
||||
{ maxTokens, temperature: 0.3 },
|
||||
)
|
||||
|
||||
if (openai.content?.trim()) {
|
||||
return { text: openai.content, source: "openai" }
|
||||
}
|
||||
|
||||
logger.info("KB answer falling back to Groq", { openaiError: openai.error })
|
||||
|
||||
const groq = await groqChatCompletion(
|
||||
[
|
||||
{ role: "system", content: options.systemPrompt },
|
||||
{ role: "user", content: userMessage },
|
||||
],
|
||||
{ maxTokens, temperature: 0.2 },
|
||||
)
|
||||
|
||||
if (groq.content?.trim()) {
|
||||
logger.info("KB answer generated via Groq fallback")
|
||||
return { text: groq.content, source: "groq" }
|
||||
}
|
||||
|
||||
logger.error("KB answer: both OpenAI and Groq failed", {
|
||||
openaiError: openai.error,
|
||||
groqError: groq.error,
|
||||
})
|
||||
return {
|
||||
text: "I could not generate an answer right now. Please try again later.",
|
||||
source: "none",
|
||||
}
|
||||
}
|
||||
104
src/lib/kb/knowledgeTemplate.ts
Normal file
104
src/lib/kb/knowledgeTemplate.ts
Normal file
@ -0,0 +1,104 @@
|
||||
import { z } from "zod"
|
||||
|
||||
/** Six fixed prompts + optional misc (open-ended). */
|
||||
export const KNOWLEDGE_TEMPLATE_FIELD_KEYS = [
|
||||
"brandVoice",
|
||||
"shippingReturns",
|
||||
"sizingFit",
|
||||
"materialsCare",
|
||||
"policies",
|
||||
"differentiators",
|
||||
] as const
|
||||
|
||||
export type KnowledgeTemplateFieldKey = (typeof KNOWLEDGE_TEMPLATE_FIELD_KEYS)[number]
|
||||
|
||||
export const KNOWLEDGE_TEMPLATE_FIELDS: {
|
||||
key: KnowledgeTemplateFieldKey
|
||||
label: string
|
||||
description: string
|
||||
placeholder: string
|
||||
}[] = [
|
||||
{
|
||||
key: "brandVoice",
|
||||
label: "Brand voice and personality",
|
||||
description:
|
||||
"How should your store sound? Tone, formality, phrases to use or avoid, and what makes your brand feel human.",
|
||||
placeholder:
|
||||
"e.g. Friendly and upbeat; we say “we” and avoid slang; we’re optimistic but never pushy…",
|
||||
},
|
||||
{
|
||||
key: "shippingReturns",
|
||||
label: "Shipping, returns, and exchanges",
|
||||
description:
|
||||
"Regions you ship to, typical timelines, costs, return windows, and how customers start a return.",
|
||||
placeholder:
|
||||
"e.g. US & Canada; 3–5 business days standard; free over $75; 30-day returns via portal…",
|
||||
},
|
||||
{
|
||||
key: "sizingFit",
|
||||
label: "Sizing and fit",
|
||||
description:
|
||||
"Fit notes, size guides, model references, and how customers should choose a size.",
|
||||
placeholder:
|
||||
"e.g. True to size; relaxed fit on hoodies; see size chart on each product page…",
|
||||
},
|
||||
{
|
||||
key: "materialsCare",
|
||||
label: "Materials, care, and quality",
|
||||
description:
|
||||
"Fabrics, ingredients, durability, care instructions, and sustainability claims you stand behind.",
|
||||
placeholder:
|
||||
"e.g. 100% organic cotton; machine wash cold; hang dry; OEKO-TEX where noted…",
|
||||
},
|
||||
{
|
||||
key: "policies",
|
||||
label: "Policies customers ask about",
|
||||
description:
|
||||
"Warranty, price adjustments, pre-orders, backorders, subscriptions, or limits worth stating clearly.",
|
||||
placeholder:
|
||||
"e.g. 1-year defect warranty; no price matching; pre-orders ship when noted on the product…",
|
||||
},
|
||||
{
|
||||
key: "differentiators",
|
||||
label: "What makes your store special",
|
||||
description:
|
||||
"Differentiators, origin story, craftsmanship, community, or anything that sets you apart from competitors.",
|
||||
placeholder:
|
||||
"e.g. Small-batch, made in Portland; we donate 1% to ocean cleanup; limited seasonal drops…",
|
||||
},
|
||||
]
|
||||
|
||||
const answerMax = 12_000
|
||||
const miscMax = 16_000
|
||||
|
||||
export const knowledgeTemplatePutSchema = z.object({
|
||||
knowledgeBaseMode: z.enum(["STRUCTURED", "DOCUMENTS"]).optional(),
|
||||
confirmDestructiveSwitch: z.boolean().optional(),
|
||||
template: z
|
||||
.object({
|
||||
answers: z.record(z.string().max(answerMax)).optional(),
|
||||
misc: z.string().max(miscMax).optional(),
|
||||
})
|
||||
.optional(),
|
||||
})
|
||||
|
||||
export type KnowledgeTemplateStored = {
|
||||
version: number
|
||||
answers: Partial<Record<KnowledgeTemplateFieldKey, string>>
|
||||
misc?: string
|
||||
updatedAt: string
|
||||
}
|
||||
|
||||
export function normalizeTemplateAnswers(
|
||||
raw: Record<string, string> | undefined,
|
||||
): Partial<Record<KnowledgeTemplateFieldKey, string>> {
|
||||
if (!raw) return {}
|
||||
const out: Partial<Record<KnowledgeTemplateFieldKey, string>> = {}
|
||||
for (const key of KNOWLEDGE_TEMPLATE_FIELD_KEYS) {
|
||||
const v = raw[key]
|
||||
if (typeof v === "string" && v.trim()) {
|
||||
out[key] = v.trim()
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
53
src/lib/kb/openaiChat.ts
Normal file
53
src/lib/kb/openaiChat.ts
Normal file
@ -0,0 +1,53 @@
|
||||
import OpenAI from "openai"
|
||||
import { logger } from "@/lib/logger"
|
||||
|
||||
const DEFAULT_MODEL = "gpt-4o-mini"
|
||||
|
||||
export type OpenAiChatMessage = {
|
||||
role: "system" | "user" | "assistant"
|
||||
content: string
|
||||
}
|
||||
|
||||
export type OpenAiChatOptions = {
|
||||
model?: string
|
||||
maxTokens?: number
|
||||
temperature?: number
|
||||
}
|
||||
|
||||
/**
|
||||
* Chat completions via OpenAI (used for KB RAG answers; embeddings stay in openaiEmbeddings.ts).
|
||||
*/
|
||||
export async function openaiChatCompletion(
|
||||
messages: OpenAiChatMessage[],
|
||||
options: OpenAiChatOptions = {},
|
||||
): Promise<{ content: string | null; error?: string }> {
|
||||
const apiKey = process.env.OPENAI_API_KEY?.trim()
|
||||
if (!apiKey) {
|
||||
logger.warn("OPENAI_API_KEY is not set; skipping OpenAI chat")
|
||||
return { content: null, error: "missing_api_key" }
|
||||
}
|
||||
|
||||
const model =
|
||||
options.model?.trim() ||
|
||||
process.env.OPENAI_CHAT_MODEL?.trim() ||
|
||||
DEFAULT_MODEL
|
||||
|
||||
try {
|
||||
const client = new OpenAI({ apiKey })
|
||||
const res = await client.chat.completions.create({
|
||||
model,
|
||||
messages,
|
||||
max_tokens: options.maxTokens ?? 512,
|
||||
temperature: options.temperature ?? 0.3,
|
||||
})
|
||||
const text = res.choices[0]?.message?.content?.trim() ?? null
|
||||
if (!text) {
|
||||
return { content: null, error: "empty_response" }
|
||||
}
|
||||
return { content: text }
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err)
|
||||
logger.warn("OpenAI chat completion failed", { message })
|
||||
return { content: null, error: message }
|
||||
}
|
||||
}
|
||||
118
src/lib/kb/openaiEmbeddings.ts
Normal file
118
src/lib/kb/openaiEmbeddings.ts
Normal file
@ -0,0 +1,118 @@
|
||||
import OpenAI from "openai"
|
||||
import { logger } from "@/lib/logger"
|
||||
import {
|
||||
KB_EMBEDDING_DIMENSIONS,
|
||||
KB_EMBEDDING_MODEL,
|
||||
} from "@/lib/kb/embeddingMeta"
|
||||
|
||||
const MAX_INPUTS_PER_REQUEST = 64
|
||||
const MAX_RETRIES = 5
|
||||
const BASE_DELAY_MS = 800
|
||||
|
||||
function sleep(ms: number) {
|
||||
return new Promise((r) => setTimeout(r, ms))
|
||||
}
|
||||
|
||||
function isRetryableStatus(status: number | undefined) {
|
||||
return status === 429 || (status !== undefined && status >= 500)
|
||||
}
|
||||
|
||||
/**
|
||||
* Batch embeddings via OpenAI (text-embedding-3-small + dimensions for pgvector / cost tradeoff).
|
||||
* Returns one vector per input; empty strings yield empty arrays (callers skip DB insert).
|
||||
*/
|
||||
export async function embedBatch(texts: string[]): Promise<number[][]> {
|
||||
if (texts.length === 0) return []
|
||||
|
||||
const apiKey = process.env.OPENAI_API_KEY?.trim()
|
||||
if (!apiKey) {
|
||||
logger.error("OPENAI_API_KEY is not set; cannot generate embeddings")
|
||||
return texts.map(() => [])
|
||||
}
|
||||
|
||||
const client = new OpenAI({ apiKey })
|
||||
const out: number[][] = new Array(texts.length).fill(null).map(() => [])
|
||||
|
||||
for (let start = 0; start < texts.length; start += MAX_INPUTS_PER_REQUEST) {
|
||||
const end = Math.min(start + MAX_INPUTS_PER_REQUEST, texts.length)
|
||||
const slice = texts.slice(start, end)
|
||||
const normalized = slice.map((t) => (t.trim().length ? t.trim() : ""))
|
||||
|
||||
const indicesWithText: number[] = []
|
||||
const inputs: string[] = []
|
||||
normalized.forEach((t, i) => {
|
||||
if (t.length) {
|
||||
indicesWithText.push(start + i)
|
||||
inputs.push(t)
|
||||
}
|
||||
})
|
||||
|
||||
if (inputs.length === 0) continue
|
||||
|
||||
let attempt = 0
|
||||
let embeddings: number[][] | null = null
|
||||
|
||||
while (attempt <= MAX_RETRIES) {
|
||||
try {
|
||||
const started = Date.now()
|
||||
const res = await client.embeddings.create({
|
||||
model: KB_EMBEDDING_MODEL,
|
||||
input: inputs,
|
||||
dimensions: KB_EMBEDDING_DIMENSIONS,
|
||||
})
|
||||
logger.info("OpenAI embeddings batch", {
|
||||
count: inputs.length,
|
||||
ms: Date.now() - started,
|
||||
model: KB_EMBEDDING_MODEL,
|
||||
})
|
||||
|
||||
const byIndex = new Map<number, number[]>()
|
||||
for (const item of res.data) {
|
||||
const vec = item.embedding
|
||||
if (!Array.isArray(vec) || vec.length !== KB_EMBEDDING_DIMENSIONS) {
|
||||
logger.error("Unexpected embedding dimensions from OpenAI", {
|
||||
expected: KB_EMBEDDING_DIMENSIONS,
|
||||
got: vec?.length,
|
||||
})
|
||||
throw new Error("Invalid embedding dimensions")
|
||||
}
|
||||
byIndex.set(item.index, vec)
|
||||
}
|
||||
|
||||
embeddings = []
|
||||
for (let i = 0; i < inputs.length; i++) {
|
||||
const vec = byIndex.get(i)
|
||||
embeddings.push(vec ?? [])
|
||||
}
|
||||
break
|
||||
} catch (err: unknown) {
|
||||
const status =
|
||||
err && typeof err === "object" && "status" in err
|
||||
? Number((err as { status?: number }).status)
|
||||
: undefined
|
||||
if (!isRetryableStatus(status) || attempt >= MAX_RETRIES) {
|
||||
logger.error("OpenAI embedding request failed", {
|
||||
attempt,
|
||||
status,
|
||||
message: err instanceof Error ? err.message : String(err),
|
||||
})
|
||||
embeddings = inputs.map(() => [])
|
||||
break
|
||||
}
|
||||
const delay = BASE_DELAY_MS * Math.pow(2, attempt)
|
||||
logger.warn(`OpenAI embeddings retry in ${delay}ms`, { attempt, status })
|
||||
await sleep(delay)
|
||||
attempt++
|
||||
}
|
||||
}
|
||||
|
||||
if (!embeddings) continue
|
||||
|
||||
embeddings.forEach((vec, j) => {
|
||||
const globalIdx = indicesWithText[j]!
|
||||
out[globalIdx] = vec
|
||||
})
|
||||
}
|
||||
|
||||
return out
|
||||
}
|
||||
50
src/lib/kb/storeKbCleanup.ts
Normal file
50
src/lib/kb/storeKbCleanup.ts
Normal file
@ -0,0 +1,50 @@
|
||||
import fs from "fs/promises"
|
||||
import path from "path"
|
||||
import type { PrismaClient } from "@prisma/client"
|
||||
import { logger } from "@/lib/logger"
|
||||
|
||||
export async function deleteDocumentKbForStore(
|
||||
prisma: PrismaClient,
|
||||
storeId: string,
|
||||
) {
|
||||
const docs = await prisma.$queryRawUnsafe<{ id: string; storagePath: string }[]>(
|
||||
`SELECT "id", "storagePath" FROM "KnowledgeDocument" WHERE "storeId" = $1`,
|
||||
storeId,
|
||||
)
|
||||
for (const d of docs) {
|
||||
try {
|
||||
await fs.unlink(path.resolve(d.storagePath))
|
||||
} catch (e) {
|
||||
logger.warn("Failed to delete KB file on disk", {
|
||||
path: d.storagePath,
|
||||
message: e instanceof Error ? e.message : String(e),
|
||||
})
|
||||
}
|
||||
}
|
||||
await prisma.$executeRawUnsafe(
|
||||
`DELETE FROM "KnowledgeChunk" WHERE "storeId" = $1 AND "sourceType"::text = 'DOCUMENT'`,
|
||||
storeId,
|
||||
)
|
||||
await prisma.$executeRawUnsafe(
|
||||
`DELETE FROM "KnowledgeDocument" WHERE "storeId" = $1`,
|
||||
storeId,
|
||||
)
|
||||
}
|
||||
|
||||
export async function deleteFaqKbForStore(prisma: PrismaClient, storeId: string) {
|
||||
await prisma.$executeRawUnsafe(
|
||||
`DELETE FROM "KnowledgeChunk" WHERE "storeId" = $1 AND "sourceType"::text = 'FAQ'`,
|
||||
storeId,
|
||||
)
|
||||
await prisma.faqItem.deleteMany({ where: { storeId } })
|
||||
}
|
||||
|
||||
export async function deleteStructuredKbForStore(
|
||||
prisma: PrismaClient,
|
||||
storeId: string,
|
||||
) {
|
||||
await prisma.$executeRawUnsafe(
|
||||
`DELETE FROM "KnowledgeChunk" WHERE "storeId" = $1 AND "sourceType"::text = 'STRUCTURED'`,
|
||||
storeId,
|
||||
)
|
||||
}
|
||||
77
src/lib/kb/structuredTemplateChunks.ts
Normal file
77
src/lib/kb/structuredTemplateChunks.ts
Normal file
@ -0,0 +1,77 @@
|
||||
import { prisma } from "@/lib/prisma"
|
||||
import { logger } from "@/lib/logger"
|
||||
import { embedBatch } from "@/lib/kb/openaiEmbeddings"
|
||||
import { kbEmbeddingMetadata } from "@/lib/kb/embeddingMeta"
|
||||
import {
|
||||
KNOWLEDGE_TEMPLATE_FIELDS,
|
||||
type KnowledgeTemplateStored,
|
||||
} from "@/lib/kb/knowledgeTemplate"
|
||||
|
||||
export async function replaceStructuredKnowledgeChunks(
|
||||
storeId: string,
|
||||
stored: KnowledgeTemplateStored,
|
||||
) {
|
||||
await prisma.$executeRawUnsafe(
|
||||
`DELETE FROM "KnowledgeChunk" WHERE "storeId" = $1 AND "sourceType"::text = 'STRUCTURED'`,
|
||||
storeId,
|
||||
)
|
||||
|
||||
const rows: { content: string; metadata: ReturnType<typeof kbEmbeddingMetadata> }[] =
|
||||
[]
|
||||
|
||||
for (const field of KNOWLEDGE_TEMPLATE_FIELDS) {
|
||||
const text = stored.answers[field.key]
|
||||
if (text?.trim()) {
|
||||
rows.push({
|
||||
content: `Q: ${field.label}\nA: ${text.trim()}`,
|
||||
metadata: kbEmbeddingMetadata({ sourceKey: field.key }),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
if (stored.misc?.trim()) {
|
||||
rows.push({
|
||||
content: `Q: Anything else customers should know\nA: ${stored.misc.trim()}`,
|
||||
metadata: kbEmbeddingMetadata({ sourceKey: "misc" }),
|
||||
})
|
||||
}
|
||||
|
||||
if (!rows.length) {
|
||||
return
|
||||
}
|
||||
|
||||
const embeddings = await embedBatch(rows.map((r) => r.content))
|
||||
if (embeddings.length !== rows.length) {
|
||||
throw new Error("Embedding count mismatch for structured template")
|
||||
}
|
||||
|
||||
for (let i = 0; i < rows.length; i++) {
|
||||
const embedding = embeddings[i]!
|
||||
if (!embedding.length) {
|
||||
logger.error("Empty embedding for structured template row", { storeId, i })
|
||||
throw new Error("Failed to embed structured knowledge template")
|
||||
}
|
||||
const vectorLiteral = `'[${embedding.join(",")}]'`
|
||||
await prisma.$executeRawUnsafe(
|
||||
`
|
||||
INSERT INTO "KnowledgeChunk"
|
||||
("id", "storeId", "documentId", "faqItemId", "sourceType", "content", "metadata", "embedding", "createdAt", "updatedAt")
|
||||
VALUES (
|
||||
gen_random_uuid()::text,
|
||||
$1,
|
||||
NULL,
|
||||
NULL,
|
||||
'STRUCTURED',
|
||||
$2,
|
||||
$3,
|
||||
${vectorLiteral},
|
||||
NOW(),
|
||||
NOW()
|
||||
)
|
||||
`,
|
||||
storeId,
|
||||
rows[i]!.content,
|
||||
rows[i]!.metadata as unknown as object,
|
||||
)
|
||||
}
|
||||
}
|
||||
Loading…
Reference in New Issue
Block a user