deepchat-data-import
Help developers build third-party tools that import, inspect, migrate, or analyze DeepChat data. Use when Codex needs to work with DeepChat provider configuration, model configuration, MCP/app settings, sessions, messages, legacy chat data, `agent.db`, `chat.db`, SQLCipher encryp
Install
npx skills add https://github.com/ThinkInAIXYZ/deepchat/tree/dev/.agents/skills/deepchat-data-import
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install thinkinaixyz-deepchat@llmmart
git clone https://github.com/ThinkInAIXYZ/deepchat.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole thinkinaixyz/deepchat collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
DeepChat Data Import
Overview
Use this skill to design or implement importers for DeepChat local data. Treat DeepChat's SQLite schema as an internal but documentable contract: inspect the current schema when precision matters, prefer read-only snapshots, and avoid writing to a live profile.
Workflow
- Identify the source: live DeepChat profile, copied profile, sync backup, exported
agent.db, or legacychat.db. - Read references/data-locations.md to locate
agent.db, sidecar files, encryption metadata, and backup paths. - Read references/sqlite-access.md before opening SQLite. Decide whether the database is unencrypted, can be unlocked through Electron safeStorage, or must ask the user for the SQLite password.
- Read references/schema-reference.md for provider config, settings, session, message, and legacy table relationships.
- Read references/import-recipes.md when writing extractor code, mapping DeepChat data to another app, or creating a compatibility import.
Safety Rules
- Get explicit user consent before reading local DeepChat data. Provider keys, OAuth tokens, MCP env vars, prompt text, message traces, and chat content may be sensitive.
- Never open the active
agent.dbread-write from a third-party tool. Copyagent.db,agent.db-wal, andagent.db-shm, or use SQLite backup APIs through DeepChat itself. - If DeepChat is running, either ask the user to quit it or make a WAL-aware snapshot before import.
- Prefer parameterized key APIs for SQLCipher passwords. Do not interpolate passwords into SQL.
- Redact secrets by default in logs, telemetry, previews, and generated sample output.
- When the schema has changed, inspect
schema_versions,sqlite_master, and the table classes undersrc/main/presenter/sqlitePresenter/tables/before assuming column availability.
Source Files
Use these repository files as the current source of truth when updating the skill or answering version-sensitive questions:
src/main/presenter/sqlitePresenter/index.tssrc/main/presenter/sqlitePresenter/connectionConfig.tssrc/main/presenter/databaseSecurityPresenter/index.tssrc/main/presenter/sqlitePresenter/tables/*.tssrc/main/presenter/agentRuntimePresenter/messageStore.tssrc/main/presenter/agentRuntimePresenter/sessionStore.tssrc/main/presenter/configPresenter/**
Files (deepchat)
-
agents
-
openai.yaml 280 B
interface: display_name: "DeepChat Data Import" short_description: "Import DeepChat config and chat data" default_prompt: "Use $deepchat-data-import to help a third-party tool import DeepChat provider config, sessions, messages, and encrypted or unencrypted agent.db data."
-
-
references
-
data-locations.md 2.5 KB
# Data Locations Use this reference when an importer must locate DeepChat data on disk or inside a backup. ## Primary Files DeepChat stores the current main database at: ```text <electron userData>/app_db/agent.db <electron userData>/app_db/agent.db-wal <electron userData>/app_db/agent.db-shm ``` The legacy database, when present, is: ```text <electron userData>/app_db/chat.db ``` Database encryption metadata is outside SQLite: ```text <electron userData>/database-security.json ``` The file is an ElectronStore JSON file. The relevant shape is: ```json { "metadata": { "version": 1, "enabled": true, "cipher": "sqlcipher", "passwordStorage": "safeStorage", "wrappedPassword": "base64-electron-safeStorage-blob", "safeStorageBackend": "basic_text", "lastMigrationAt": 1770000000000, "lastMigrationDirection": "enable" } } ``` `passwordStorage` can be `safeStorage`, `manual`, or `none`. If `enabled` is false, open `agent.db` as normal SQLite. ## Default UserData Paths Electron derives the profile path from the packaged product name `DeepChat` unless the runtime overrides `app.getPath('userData')`. ```text macOS: ~/Library/Application Support/DeepChat Windows: %APPDATA%\DeepChat Linux: ~/.config/DeepChat ``` Treat these as defaults. Portable builds, development builds, tests, or user overrides can point elsewhere. ## Sync Backup Layout DeepChat sync backups use `database/agent.db` as the primary database payload in current backup versions. Some compatibility backups may also contain `database/chat.db` or old JSON settings. When both `agent.db` and `chat.db` exist, prefer `agent.db`. ## Snapshot Rules - If DeepChat is running, copy `agent.db`, `agent.db-wal`, and `agent.db-shm` together. - If DeepChat is closed, `agent.db` alone is usually enough, but copying sidecars is still harmless. - For high-integrity import, open a read-only source connection and run a SQLite backup into a temp file, then import from the temp file. - Do not delete `*.migration-tmp`, `*.migration-rollback`, `agent.db-wal`, or `agent.db-shm` from a user profile. DeepChat owns those lifecycle decisions. ## Related Files Still Outside Agent.db Most sensitive configuration has moved into SQLite. Some lightweight or compatibility JSON files may still exist in userData, including: - `app-settings.json` - `custom_prompts.json` - `system_prompts.json` - `mcp-settings.json` Prefer SQLite tables for current provider, MCP, app setting, prompt, and knowledge config imports. Use JSON files only as legacy fallback. -
import-recipes.md 4.5 KB
# Import Recipes Use this reference when turning DeepChat data into an importer or migration tool. ## Minimal Read-Only Export 1. Locate and snapshot `agent.db`. 2. Open it with the flow in `sqlite-access.md`. 3. Read provider config from `providers`, `provider_models`, `model_status`, and `model_configs`. 4. Read sessions with: ```sql SELECT ns.*, ds.* FROM new_sessions ns LEFT JOIN deepchat_sessions ds ON ds.id = ns.id WHERE ns.session_kind = 'regular' ORDER BY ns.updated_at DESC, ns.id DESC; ``` 5. Read messages per session: ```sql SELECT * FROM deepchat_messages WHERE session_id = ? ORDER BY order_seq ASC, id ASC; ``` 6. Hydrate user messages from `deepchat_user_messages`, `deepchat_user_message_files`, and `deepchat_user_message_links`. 7. Hydrate assistant messages from `deepchat_assistant_blocks`; fall back to parsing `deepchat_messages.content`. 8. Export to the target app's format, redacting secrets unless the user explicitly chooses to include them. ## Provider Config Import When importing providers into another tool, preserve: - provider id, name, API type, base URL, enabled flag, custom flag. - API key and OAuth token only with explicit user consent. - capability provider id for model capability lookup. - model rows from `provider_models`, split by `source`. - enabled/disabled state from `model_status`. - model config from `model_configs.config_json`. Do not rely only on `provider_json`; DeepChat deliberately stores common scalar fields in columns for queryability and migration. ## Session And Message Import Recommended target shape: ```json { "session": { "id": "session-id", "title": "Session title", "agentId": "deepchat", "projectDir": "/path/to/project", "providerId": "openai", "modelId": "gpt-4.1", "createdAt": 1770000000000, "updatedAt": 1770000000000 }, "messages": [ { "id": "message-id", "orderSeq": 1, "role": "user", "status": "sent", "content": { "text": "hello", "files": [], "links": [], "search": false, "think": false }, "metadata": {}, "createdAt": 1770000000000, "updatedAt": 1770000000000 } ] } ``` For assistant messages, keep the assistant block array when possible instead of flattening to text. Tool calls, tool responses, reasoning blocks, image data, action prompts, and error blocks may all be represented as assistant blocks. ## Handling Partial Or Old Rows - If structured user rows are missing, parse `deepchat_messages.content`. - If structured assistant blocks are missing, parse `deepchat_messages.content`. - If `new_sessions` is missing but `conversations` exists, import through the legacy path. - If `agent.db` is missing and `chat.db` exists, open `chat.db` as legacy data. - If a column is missing, check `schema_versions` and use the nearest fallback rather than failing the whole import. ## Writing Back Into DeepChat Avoid third-party direct writes to a user's live DeepChat database. If a tool must generate data for DeepChat: - Prefer creating an export file or backup package that DeepChat can import through its own code. - If implementing inside DeepChat, use Presenter/table helpers instead of raw SQL. - If writing a copied database for controlled migration tests, use one transaction per session and keep table groups consistent: - `new_sessions` - `deepchat_sessions` - `deepchat_messages` - structured user or assistant tables - optional search, trace, usage, pending input, and tape rows - Keep `deepchat_messages.content` compatible even when structured tables are populated, because it remains the fallback path. - Do not update `database-security.json` manually after rekeying; use DeepChat's migration flow. ## Secret Handling Checklist Redact or require explicit opt-in for: - `providers.api_key` - OAuth tokens in `providers.provider_json` - MCP server `env` and custom headers - `app_settings` rows marked `sensitive = 1` - `deepchat_message_traces.headers_json` and `body_json` - file paths in user message files - chat content, system prompts, summaries, and project paths ## Useful Consistency Checks Run these after import from a copied database: ```sql PRAGMA quick_check; SELECT COUNT(*) FROM new_sessions; SELECT COUNT(*) FROM deepchat_sessions; SELECT COUNT(*) FROM deepchat_messages; SELECT m.session_id FROM deepchat_messages m LEFT JOIN new_sessions s ON s.id = m.session_id WHERE s.id IS NULL LIMIT 20; ``` For encrypted databases, validate the password before any import work: ```sql SELECT name FROM sqlite_master LIMIT 1; ``` -
schema-reference.md 7.6 KB
# Schema Reference Use this reference when extracting provider config, settings, sessions, and messages from `agent.db`. Verify against `sqlite_master` for user databases created by newer DeepChat versions. ## Core Version Tables - `schema_versions`: applied migration versions. Read `MAX(version)` to understand how far the DB has migrated. - `config_migrations`: config storage migrations, including the SQLite config migration marker. ## Provider And Config Tables ### providers Primary provider rows. Important columns: - `id`: provider id. - `name`: display name. - `api_type`: provider API adapter type. - `api_key`: sensitive API key. Redact by default. - `base_url`: configured endpoint. - `enabled`: `1` when provider is enabled. - `custom`: `1` for custom providers. - `capability_provider_id`: catalog provider used for capabilities, nullable. - `sort_order`, `last_used_at`, `created_at`, `updated_at`: ordering and timestamps. - `provider_json`: JSON for the rest of `LLM_PROVIDER`, excluding model arrays and enabled/disabled model lists. To reconstruct a provider object, parse `provider_json`, then overlay scalar columns: ```ts { ...JSON.parse(row.provider_json || '{}'), id: row.id, name: row.name, apiType: row.api_type, apiKey: row.api_key, baseUrl: row.base_url, enable: row.enabled === 1, custom: row.custom === 1, capabilityProviderId: row.capability_provider_id } ``` ### provider_models Provider and custom model catalog rows. - Primary key: `(provider_id, model_id, source)`. - `source`: `provider` or `custom`. - `model_json`: JSON for `MODEL_META`; overlay `model_id`, `provider_id`, `name`, `group_name`, and `isCustom`. ### model_status Per-model enabled state. `status_key` is the primary key; rows also include `provider_id`, `model_id`, `enabled`, and `updated_at`. ### model_configs Per-model generation config. - `cache_key`: primary key used by DeepChat config helpers. - `provider_id`, `model_id`, `source`: denormalized lookup fields. - `config_json`: JSON for model config values such as temperature, context length, reasoning, search, image generation, video generation, or TTS options. ### mcp_servers, mcp_settings, agent_settings, app_settings - `mcp_servers`: MCP server configs by `name`, with `config_json`, `sort_order`, and timestamps. - `mcp_settings`: JSON key/value settings for MCP behavior. - `agent_settings`: JSON key/value settings for agent behavior. - `app_settings`: JSON key/value settings, with `sensitive` flag. Current sensitive config such as prompts, knowledge config, hooks, remote control, and API-like settings may live here. `mcp_servers.config_json`, MCP env values, and `app_settings.value_json` can contain secrets. ## Current Session And Message Tables DeepChat's current mainline session model is split across a thin registry and agent-specific data. ### new_sessions One row per visible session or subagent session. Key columns: - `id`: session id. - `agent_id`: agent implementation id. DeepChat chat sessions normally use the DeepChat agent id; ACP sessions use ACP-oriented ids. - `title`: sidebar title. - `project_dir`: nullable project/workspace path. - `is_pinned`, `is_draft`: booleans as integers. - `active_skills`, `disabled_agent_tools`: JSON arrays retained for compatibility. - `subagent_enabled`: boolean as integer. - `session_kind`: `regular` or `subagent`. - `parent_session_id`, `subagent_meta_json`: subagent relationship data. - `created_at`, `updated_at`: epoch milliseconds. Related tables: - `new_projects`: project path, name, optional icon, last access timestamp. - `new_session_active_skills`: structured active skill rows. - `new_session_disabled_agent_tools`: structured disabled tool rows. ### deepchat_sessions DeepChat-specific session config. `id` matches `new_sessions.id`. Important columns: - `provider_id`, `model_id`: selected model. - `permission_mode`: `default` or `full_access`. - `system_prompt`, `temperature`, `context_length`, `max_tokens`, `timeout_ms`. - `thinking_budget`, `reasoning_effort`, `reasoning_visibility`, `verbosity`. - `force_interleaved_thinking_compat`: nullable boolean as integer. - `image_generation_options_json`, `video_generation_options_json`: nullable JSON. - `summary_text`, `summary_cursor_order_seq`, `summary_updated_at`: compaction summary state. ### deepchat_messages Message timeline for a session. - `id`: message id. - `session_id`: references the session id. - `order_seq`: monotonic ordering within session. Sort ascending for conversation order. - `role`: `user` or `assistant`. - `content`: JSON string fallback/materialized content. - `status`: `pending`, `sent`, or `error`. - `is_context_edge`: boolean as integer. - `metadata`: JSON string. - `created_at`, `updated_at`: epoch milliseconds. Basic query: ```sql SELECT * FROM deepchat_messages WHERE session_id = ? ORDER BY order_seq ASC; ``` ### Structured User Message Tables Use these first for current rows; fall back to `deepchat_messages.content` if missing. - `deepchat_user_messages`: `message_id`, `text`, `search_enabled`, `think_enabled`. - `deepchat_user_message_files`: `message_id`, `ordinal`, `name`, `path`, `mime_type`, `size`, `metadata_json`. - `deepchat_user_message_links`: `message_id`, `ordinal`, `url`. Materialized user content: ```json { "text": "user text", "files": [], "links": [], "search": false, "think": false } ``` ### Structured Assistant Blocks Use `deepchat_assistant_blocks` first for assistant messages. It is especially important for pending or recently streamed messages. Columns: - `message_id`, `block_index`: primary key. - `block_type`, `status`, `text_content`. - `tool_call_id`, `tool_name`, `tool_params`, `tool_response`. - `action_type`. - `image_mime_type`. - `reasoning_start_at`, `reasoning_end_at`. - `extra_json`: includes block id, timestamp, image data, tool call extras, and reasoning time. - `updated_at`. Sort by `(message_id, block_index)`. If no structured blocks exist, parse `deepchat_messages.content` as the fallback assistant block array. ### Event, Search, Trace, And Usage Tables These are useful for richer import but optional for basic chat history. - `deepchat_tape_entries`: append-only reconstruction/event facts per session. - `deepchat_pending_inputs`: queued or steer-mode pending input payloads. - `deepchat_search_documents` and FTS shadow tables: derived search index. - `deepchat_message_search_results`: web/search results associated with messages. - `deepchat_message_traces`: provider request traces. Treat as highly sensitive. - `deepchat_usage_stats`: token/cost usage by message, provider, model, and date. ## Legacy Compatibility Tables Current DeepChat keeps legacy tables for compatibility and import. ### conversations Legacy conversation metadata. The business id is `conv_id`; `id` is an autoincrement row id. Important columns include `title`, `provider_id`, `model_id`, generation settings, search settings, `context_chain`, `active_skills`, parent fork fields, `created_at`, and `updated_at`. ### messages Legacy message timeline. - `msg_id`: business id. - `conversation_id`: references `conversations.conv_id`. - `parent_id`: tree/variant parent. - `role`: `user`, `assistant`, `system`, or `function`. - `content`: message content. - `order_seq`: timeline order. - `metadata`, `token_count`, `status`, `is_context_edge`, `is_variant`. ### message_attachments Legacy attachments by `message_id`, `type`, and serialized `content`. Prefer `new_sessions` and `deepchat_*` for current imports. Use legacy tables or `chat.db` only when `agent.db` is missing, an old backup is imported, or the user explicitly wants legacy data. -
sqlite-access.md 4.8 KB
# SQLite Access And Encryption Use this reference before opening `agent.db` or `chat.db`. ## Decision Tree 1. Locate `database-security.json`. 2. If the file is absent or `metadata.enabled !== true`, open `agent.db` as plain SQLite. 3. If `metadata.enabled === true`, open with SQLCipher using the DeepChat SQLite password. 4. If `metadata.passwordStorage === "safeStorage"` and `wrappedPassword` exists, first try an Electron safeStorage helper. 5. If safeStorage is unavailable, decryption fails, or the importing runtime is not Electron, ask the user for the SQLite password and validate it before reading. Legacy `chat.db` is normally unencrypted. If a user supplies an encrypted database explicitly, treat it with the same SQLCipher path. ## Opening Unencrypted SQLite Use a read-only connection when possible: ```sql SELECT name FROM sqlite_master LIMIT 1; PRAGMA quick_check; ``` If the importer sees `file is not a database`, `SQLITE_NOTADB`, or `SQLITE_CORRUPT` against `agent.db`, check `database-security.json` before treating the file as corrupt. ## Opening Encrypted SQLite DeepChat uses `better-sqlite3-multiple-ciphers` and configures SQLCipher compatibility before applying the key: ```ts db.pragma("cipher='sqlcipher'") db.pragma('legacy=4') db.key(Buffer.from(password, 'utf8')) ``` Then validate with: ```sql SELECT name FROM sqlite_master LIMIT 1; PRAGMA quick_check; ``` For other SQLCipher bindings, choose SQLCipher 4 compatible settings that match the binding's equivalent of the `legacy=4` mode. Use parameterized or native key APIs when the library supports them. ## Electron Importer An Electron-based third-party importer has the best chance of using DeepChat's wrapped password. Read the metadata JSON manually, then try `safeStorage.decryptString`. ```ts import { app, safeStorage } from 'electron' import fs from 'node:fs' import path from 'node:path' async function readDeepChatPassword(deepChatUserData: string): Promise<string | null> { await app.whenReady() const metadataPath = path.join(deepChatUserData, 'database-security.json') const raw = JSON.parse(fs.readFileSync(metadataPath, 'utf8')) as { metadata?: { enabled?: boolean passwordStorage?: string wrappedPassword?: string } } const metadata = raw.metadata if (!metadata?.enabled) return undefined if (metadata.passwordStorage !== 'safeStorage' || !metadata.wrappedPassword) return null if (!safeStorage.isEncryptionAvailable()) return null try { return safeStorage.decryptString(Buffer.from(metadata.wrappedPassword, 'base64')) } catch { return null } } ``` If this returns `null`, fall back to a password prompt. SafeStorage blobs are tied to the user's OS security context and Electron's underlying implementation; cross-app or cross-machine decrypt is not a stable public contract. ## Tauri Importer Tauri cannot directly call Electron safeStorage. Prefer this flow: 1. Locate `agent.db` and `database-security.json`. 2. If unencrypted, open with a normal SQLite crate or plugin. 3. If encrypted, ask the user for the SQLite password. 4. Open through a SQLCipher-capable SQLite binding. Standard SQLite drivers will not open encrypted `agent.db`. 5. Optionally spawn a small Electron helper only for safeStorage unwrap, then pass the password back through a local, user-consented channel. Use Tauri or OS keyring APIs only to store the importer's own remembered password. Do not assume they can unwrap DeepChat's Electron safeStorage blob. ## Native macOS, Windows, And Linux For unencrypted databases, use the platform's normal SQLite library in read-only mode. For encrypted databases, use a SQLCipher-capable library and ask the user for the SQLite password unless you deliberately ship an Electron helper. Platform notes: - macOS: Electron safeStorage depends on Keychain-backed OS crypto. Native Keychain access can be app-permission dependent and should not be treated as a stable DeepChat import API. - Windows: Electron safeStorage commonly relies on current-user OS protection. Native DPAPI experiments may work for some blobs, but the blob format and Electron behavior are implementation details. Prefer manual password fallback. - Linux: safeStorage may use libsecret, KWallet, or a weaker backend reported as `safeStorageBackend`. If the user's desktop secret service is unavailable, DeepChat stores metadata in manual mode and the importer must ask for the password. ## Validation Errors - Wrong password usually surfaces as `file is not a database`, `SQLITE_NOTADB`, or a failure reading `sqlite_master`. - A missing WAL file can make recent rows disappear from a copied live database. Re-copy sidecars or ask the user to close DeepChat. - Do not run rekey or migration operations from an importer. DeepChat's own migration flow copies through an attached temp database and updates metadata only after validation.
-
-
SKILL.md 2.9 KB
--- name: deepchat-data-import description: Help developers build third-party tools that import, inspect, migrate, or analyze DeepChat data. Use when Codex needs to work with DeepChat provider configuration, model configuration, MCP/app settings, sessions, messages, legacy chat data, `agent.db`, `chat.db`, SQLCipher encrypted SQLite, Electron safeStorage wrapped passwords, Tauri importers, or native macOS/Windows/Linux data access. --- # DeepChat Data Import ## Overview Use this skill to design or implement importers for DeepChat local data. Treat DeepChat's SQLite schema as an internal but documentable contract: inspect the current schema when precision matters, prefer read-only snapshots, and avoid writing to a live profile. ## Workflow 1. Identify the source: live DeepChat profile, copied profile, sync backup, exported `agent.db`, or legacy `chat.db`. 2. Read [references/data-locations.md](references/data-locations.md) to locate `agent.db`, sidecar files, encryption metadata, and backup paths. 3. Read [references/sqlite-access.md](references/sqlite-access.md) before opening SQLite. Decide whether the database is unencrypted, can be unlocked through Electron safeStorage, or must ask the user for the SQLite password. 4. Read [references/schema-reference.md](references/schema-reference.md) for provider config, settings, session, message, and legacy table relationships. 5. Read [references/import-recipes.md](references/import-recipes.md) when writing extractor code, mapping DeepChat data to another app, or creating a compatibility import. ## Safety Rules - Get explicit user consent before reading local DeepChat data. Provider keys, OAuth tokens, MCP env vars, prompt text, message traces, and chat content may be sensitive. - Never open the active `agent.db` read-write from a third-party tool. Copy `agent.db`, `agent.db-wal`, and `agent.db-shm`, or use SQLite backup APIs through DeepChat itself. - If DeepChat is running, either ask the user to quit it or make a WAL-aware snapshot before import. - Prefer parameterized key APIs for SQLCipher passwords. Do not interpolate passwords into SQL. - Redact secrets by default in logs, telemetry, previews, and generated sample output. - When the schema has changed, inspect `schema_versions`, `sqlite_master`, and the table classes under `src/main/presenter/sqlitePresenter/tables/` before assuming column availability. ## Source Files Use these repository files as the current source of truth when updating the skill or answering version-sensitive questions: - `src/main/presenter/sqlitePresenter/index.ts` - `src/main/presenter/sqlitePresenter/connectionConfig.ts` - `src/main/presenter/databaseSecurityPresenter/index.ts` - `src/main/presenter/sqlitePresenter/tables/*.ts` - `src/main/presenter/agentRuntimePresenter/messageStore.ts` - `src/main/presenter/agentRuntimePresenter/sessionStore.ts` - `src/main/presenter/configPresenter/**`
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.