AgentDB Learning Plugins
Create and train AI learning plugins with AgentDB's 9 reinforcement learning algorithms. Includes Decision Transformer, Q-Learning, SARSA, Actor-Critic, and more. Use when building self-learning agents, implementing RL, or optimizing agent behavior through experience.
Install
npx skills add https://github.com/spencermarx/open-code-review/tree/main/.claude/skills/agentdb-learning
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install spencermarx-open-code-review@llmmart
git clone https://github.com/spencermarx/open-code-review.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole spencermarx/open-code-review collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
AgentDB Learning Plugins
What This Skill Does
Provides access to 9 reinforcement learning algorithms via AgentDB's plugin system. Create, train, and deploy learning plugins for autonomous agents that improve through experience. Includes offline RL (Decision Transformer), value-based learning (Q-Learning), policy gradients (Actor-Critic), and advanced techniques.
Performance: Train models 10-100x faster with WASM-accelerated neural inference.
Prerequisites
- Node.js 18+
- AgentDB v1.0.7+ (via agentic-flow)
- Basic understanding of reinforcement learning (recommended)
Quick Start with CLI
Create Learning Plugin
# Interactive wizard
npx agentdb@latest create-plugin
# Use specific template
npx agentdb@latest create-plugin -t decision-transformer -n my-agent
# Preview without creating
npx agentdb@latest create-plugin -t q-learning --dry-run
# Custom output directory
npx agentdb@latest create-plugin -t actor-critic -o ./plugins
List Available Templates
# Show all plugin templates
npx agentdb@latest list-templates
# Available templates:
# - decision-transformer (sequence modeling RL - recommended)
# - q-learning (value-based learning)
# - sarsa (on-policy TD learning)
# - actor-critic (policy gradient with baseline)
# - curiosity-driven (exploration-based)
Manage Plugins
# List installed plugins
npx agentdb@latest list-plugins
# Get plugin information
npx agentdb@latest plugin-info my-agent
# Shows: algorithm, configuration, training status
Quick Start with API
import { createAgentDBAdapter } from 'agentic-flow/reasoningbank';
// Initialize with learning enabled
const adapter = await createAgentDBAdapter({
dbPath: '.agentdb/learning.db',
enableLearning: true, // Enable learning plugins
enableReasoning: true,
cacheSize: 1000,
});
// Store training experience
await adapter.insertPattern({
id: '',
type: 'experience',
domain: 'game-playing',
pattern_data: JSON.stringify({
embedding: await computeEmbedding('state-action-reward'),
pattern: {
state: [0.1, 0.2, 0.3],
action: 2,
reward: 1.0,
next_state: [0.15, 0.25, 0.35],
done: false
}
}),
confidence: 0.9,
usage_count: 1,
success_count: 1,
created_at: Date.now(),
last_used: Date.now(),
});
// Train learning model
const metrics = await adapter.train({
epochs: 50,
batchSize: 32,
});
console.log('Training Loss:', metrics.loss);
console.log('Duration:', metrics.duration, 'ms');
Available Learning Algorithms (9 Total)
1. Decision Transformer (Recommended)
Type: Offline Reinforcement Learning Best For: Learning from logged experiences, imitation learning Strengths: No online interaction needed, stable training
npx agentdb@latest create-plugin -t decision-transformer -n dt-agent
Use Cases:
- Learn from historical data
- Imitation learning from expert demonstrations
- Safe learning without environment interaction
- Sequence modeling tasks
Configuration:
{
"algorithm": "decision-transformer",
"model_size": "base",
"context_length": 20,
"embed_dim": 128,
"n_heads": 8,
"n_layers": 6
}
2. Q-Learning
Type: Value-Based RL (Off-Policy) Best For: Discrete action spaces, sample efficiency Strengths: Proven, simple, works well for small/medium problems
npx agentdb@latest create-plugin -t q-learning -n q-agent
Use Cases:
- Grid worlds, board games
- Navigation tasks
- Resource allocation
- Discrete decision-making
Configuration:
{
"algorithm": "q-learning",
"learning_rate": 0.001,
"gamma": 0.99,
"epsilon": 0.1,
"epsilon_decay": 0.995
}
3. SARSA
Type: Value-Based RL (On-Policy) Best For: Safe exploration, risk-sensitive tasks Strengths: More conservative than Q-Learning, better for safety
npx agentdb@latest create-plugin -t sarsa -n sarsa-agent
Use Cases:
- Safety-critical applications
- Risk-sensitive decision-making
- Online learning with exploration
Configuration:
{
"algorithm": "sarsa",
"learning_rate": 0.001,
"gamma": 0.99,
"epsilon": 0.1
}
4. Actor-Critic
Type: Policy Gradient with Value Baseline Best For: Continuous actions, variance reduction Strengths: Stable, works for continuous/discrete actions
npx agentdb@latest create-plugin -t actor-critic -n ac-agent
Use Cases:
- Continuous control (robotics, simulations)
- Complex action spaces
- Multi-agent coordination
Configuration:
{
"algorithm": "actor-critic",
"actor_lr": 0.001,
"critic_lr": 0.002,
"gamma": 0.99,
"entropy_coef": 0.01
}
5. Active Learning
Type: Query-Based Learning Best For: Label-efficient learning, human-in-the-loop Strengths: Minimizes labeling cost, focuses on uncertain samples
Use Cases:
- Human feedback incorporation
- Label-efficient training
- Uncertainty sampling
- Annotation cost reduction
6. Adversarial Training
Type: Robustness Enhancement Best For: Safety, robustness to perturbations Strengths: Improves model robustness, adversarial defense
Use Cases:
- Security applications
- Robust decision-making
- Adversarial defense
- Safety testing
7. Curriculum Learning
Type: Progressive Difficulty Training Best For: Complex tasks, faster convergence Strengths: Stable learning, faster convergence on hard tasks
Use Cases:
- Complex multi-stage tasks
- Hard exploration problems
- Skill composition
- Transfer learning
8. Federated Learning
Type: Distributed Learning Best For: Privacy, distributed data Strengths: Privacy-preserving, scalable
Use Cases:
- Multi-agent systems
- Privacy-sensitive data
- Distributed training
- Collaborative learning
9. Multi-Task Learning
Type: Transfer Learning Best For: Related tasks, knowledge sharing Strengths: Faster learning on new tasks, better generalization
Use Cases:
- Task families
- Transfer learning
- Domain adaptation
- Meta-learning
Training Workflow
1. Collect Experiences
// Store experiences during agent execution
for (let i = 0; i < numEpisodes; i++) {
const episode = runEpisode();
for (const step of episode.steps) {
await adapter.insertPattern({
id: '',
type: 'experience',
domain: 'task-domain',
pattern_data: JSON.stringify({
embedding: await computeEmbedding(JSON.stringify(step)),
pattern: {
state: step.state,
action: step.action,
reward: step.reward,
next_state: step.next_state,
done: step.done
}
}),
confidence: step.reward > 0 ? 0.9 : 0.5,
usage_count: 1,
success_count: step.reward > 0 ? 1 : 0,
created_at: Date.now(),
last_used: Date.now(),
});
}
}
2. Train Model
// Train on collected experiences
const trainingMetrics = await adapter.train({
epochs: 100,
batchSize: 64,
learningRate: 0.001,
validationSplit: 0.2,
});
console.log('Training Metrics:', trainingMetrics);
// {
// loss: 0.023,
// valLoss: 0.028,
// duration: 1523,
// epochs: 100
// }
3. Evaluate Performance
// Retrieve similar successful experiences
const testQuery = await computeEmbedding(JSON.stringify(testState));
const result = await adapter.retrieveWithReasoning(testQuery, {
domain: 'task-domain',
k: 10,
synthesizeContext: true,
});
// Evaluate action quality
const suggestedAction = result.memories[0].pattern.action;
const confidence = result.memories[0].similarity;
console.log('Suggested Action:', suggestedAction);
console.log('Confidence:', confidence);
Advanced Training Techniques
Experience Replay
// Store experiences in buffer
const replayBuffer = [];
// Sample random batch for training
const batch = sampleRandomBatch(replayBuffer, batchSize: 32);
// Train on batch
await adapter.train({
data: batch,
epochs: 1,
batchSize: 32,
});
Prioritized Experience Replay
// Store experiences with priority (TD error)
await adapter.insertPattern({
// ... standard fields
confidence: tdError, // Use TD error as confidence/priority
// ...
});
// Retrieve high-priority experiences
const highPriority = await adapter.retrieveWithReasoning(queryEmbedding, {
domain: 'task-domain',
k: 32,
minConfidence: 0.7, // Only high TD-error experiences
});
Multi-Agent Training
// Collect experiences from multiple agents
for (const agent of agents) {
const experience = await agent.step();
await adapter.insertPattern({
// ... store experience with agent ID
domain: `multi-agent/${agent.id}`,
});
}
// Train shared model
await adapter.train({
epochs: 50,
batchSize: 64,
});
Performance Optimization
Batch Training
// Collect batch of experiences
const experiences = collectBatch(size: 1000);
// Batch insert (500x faster)
for (const exp of experiences) {
await adapter.insertPattern({ /* ... */ });
}
// Train on batch
await adapter.train({
epochs: 10,
batchSize: 128, // Larger batch for efficiency
});
Incremental Learning
// Train incrementally as new data arrives
setInterval(async () => {
const newExperiences = getNewExperiences();
if (newExperiences.length > 100) {
await adapter.train({
epochs: 5,
batchSize: 32,
});
}
}, 60000); // Every minute
Integration with Reasoning Agents
Combine learning with reasoning for better performance:
// Train learning model
await adapter.train({ epochs: 50, batchSize: 32 });
// Use reasoning agents for inference
const result = await adapter.retrieveWithReasoning(queryEmbedding, {
domain: 'decision-making',
k: 10,
useMMR: true, // Diverse experiences
synthesizeContext: true, // Rich context
optimizeMemory: true, // Consolidate patterns
});
// Make decision based on learned experiences + reasoning
const decision = result.context.suggestedAction;
const confidence = result.memories[0].similarity;
CLI Operations
# Create plugin
npx agentdb@latest create-plugin -t decision-transformer -n my-plugin
# List plugins
npx agentdb@latest list-plugins
# Get plugin info
npx agentdb@latest plugin-info my-plugin
# List templates
npx agentdb@latest list-templates
Troubleshooting
Issue: Training not converging
// Reduce learning rate
await adapter.train({
epochs: 100,
batchSize: 32,
learningRate: 0.0001, // Lower learning rate
});
Issue: Overfitting
// Use validation split
await adapter.train({
epochs: 50,
batchSize: 64,
validationSplit: 0.2, // 20% validation
});
// Enable memory optimization
await adapter.retrieveWithReasoning(queryEmbedding, {
optimizeMemory: true, // Consolidate, reduce overfitting
});
Issue: Slow training
# Enable quantization for faster inference
# Use binary quantization (32x faster)
Learn More
- Algorithm Papers: See docs/algorithms/ for detailed papers
- GitHub: https://github.com/ruvnet/agentic-flow/tree/main/packages/agentdb
- MCP Integration:
npx agentdb@latest mcp - Website: https://agentdb.ruv.io
Category: Machine Learning / Reinforcement Learning Difficulty: Intermediate to Advanced Estimated Time: 30-60 minutes
Files (open-code-review)
-
SKILL.md 11.6 KB
--- name: "AgentDB Learning Plugins" description: "Create and train AI learning plugins with AgentDB's 9 reinforcement learning algorithms. Includes Decision Transformer, Q-Learning, SARSA, Actor-Critic, and more. Use when building self-learning agents, implementing RL, or optimizing agent behavior through experience." --- # AgentDB Learning Plugins ## What This Skill Does Provides access to 9 reinforcement learning algorithms via AgentDB's plugin system. Create, train, and deploy learning plugins for autonomous agents that improve through experience. Includes offline RL (Decision Transformer), value-based learning (Q-Learning), policy gradients (Actor-Critic), and advanced techniques. **Performance**: Train models 10-100x faster with WASM-accelerated neural inference. ## Prerequisites - Node.js 18+ - AgentDB v1.0.7+ (via agentic-flow) - Basic understanding of reinforcement learning (recommended) --- ## Quick Start with CLI ### Create Learning Plugin ```bash # Interactive wizard npx agentdb@latest create-plugin # Use specific template npx agentdb@latest create-plugin -t decision-transformer -n my-agent # Preview without creating npx agentdb@latest create-plugin -t q-learning --dry-run # Custom output directory npx agentdb@latest create-plugin -t actor-critic -o ./plugins ``` ### List Available Templates ```bash # Show all plugin templates npx agentdb@latest list-templates # Available templates: # - decision-transformer (sequence modeling RL - recommended) # - q-learning (value-based learning) # - sarsa (on-policy TD learning) # - actor-critic (policy gradient with baseline) # - curiosity-driven (exploration-based) ``` ### Manage Plugins ```bash # List installed plugins npx agentdb@latest list-plugins # Get plugin information npx agentdb@latest plugin-info my-agent # Shows: algorithm, configuration, training status ``` --- ## Quick Start with API ```typescript import { createAgentDBAdapter } from 'agentic-flow/reasoningbank'; // Initialize with learning enabled const adapter = await createAgentDBAdapter({ dbPath: '.agentdb/learning.db', enableLearning: true, // Enable learning plugins enableReasoning: true, cacheSize: 1000, }); // Store training experience await adapter.insertPattern({ id: '', type: 'experience', domain: 'game-playing', pattern_data: JSON.stringify({ embedding: await computeEmbedding('state-action-reward'), pattern: { state: [0.1, 0.2, 0.3], action: 2, reward: 1.0, next_state: [0.15, 0.25, 0.35], done: false } }), confidence: 0.9, usage_count: 1, success_count: 1, created_at: Date.now(), last_used: Date.now(), }); // Train learning model const metrics = await adapter.train({ epochs: 50, batchSize: 32, }); console.log('Training Loss:', metrics.loss); console.log('Duration:', metrics.duration, 'ms'); ``` --- ## Available Learning Algorithms (9 Total) ### 1. Decision Transformer (Recommended) **Type**: Offline Reinforcement Learning **Best For**: Learning from logged experiences, imitation learning **Strengths**: No online interaction needed, stable training ```bash npx agentdb@latest create-plugin -t decision-transformer -n dt-agent ``` **Use Cases**: - Learn from historical data - Imitation learning from expert demonstrations - Safe learning without environment interaction - Sequence modeling tasks **Configuration**: ```json { "algorithm": "decision-transformer", "model_size": "base", "context_length": 20, "embed_dim": 128, "n_heads": 8, "n_layers": 6 } ``` ### 2. Q-Learning **Type**: Value-Based RL (Off-Policy) **Best For**: Discrete action spaces, sample efficiency **Strengths**: Proven, simple, works well for small/medium problems ```bash npx agentdb@latest create-plugin -t q-learning -n q-agent ``` **Use Cases**: - Grid worlds, board games - Navigation tasks - Resource allocation - Discrete decision-making **Configuration**: ```json { "algorithm": "q-learning", "learning_rate": 0.001, "gamma": 0.99, "epsilon": 0.1, "epsilon_decay": 0.995 } ``` ### 3. SARSA **Type**: Value-Based RL (On-Policy) **Best For**: Safe exploration, risk-sensitive tasks **Strengths**: More conservative than Q-Learning, better for safety ```bash npx agentdb@latest create-plugin -t sarsa -n sarsa-agent ``` **Use Cases**: - Safety-critical applications - Risk-sensitive decision-making - Online learning with exploration **Configuration**: ```json { "algorithm": "sarsa", "learning_rate": 0.001, "gamma": 0.99, "epsilon": 0.1 } ``` ### 4. Actor-Critic **Type**: Policy Gradient with Value Baseline **Best For**: Continuous actions, variance reduction **Strengths**: Stable, works for continuous/discrete actions ```bash npx agentdb@latest create-plugin -t actor-critic -n ac-agent ``` **Use Cases**: - Continuous control (robotics, simulations) - Complex action spaces - Multi-agent coordination **Configuration**: ```json { "algorithm": "actor-critic", "actor_lr": 0.001, "critic_lr": 0.002, "gamma": 0.99, "entropy_coef": 0.01 } ``` ### 5. Active Learning **Type**: Query-Based Learning **Best For**: Label-efficient learning, human-in-the-loop **Strengths**: Minimizes labeling cost, focuses on uncertain samples **Use Cases**: - Human feedback incorporation - Label-efficient training - Uncertainty sampling - Annotation cost reduction ### 6. Adversarial Training **Type**: Robustness Enhancement **Best For**: Safety, robustness to perturbations **Strengths**: Improves model robustness, adversarial defense **Use Cases**: - Security applications - Robust decision-making - Adversarial defense - Safety testing ### 7. Curriculum Learning **Type**: Progressive Difficulty Training **Best For**: Complex tasks, faster convergence **Strengths**: Stable learning, faster convergence on hard tasks **Use Cases**: - Complex multi-stage tasks - Hard exploration problems - Skill composition - Transfer learning ### 8. Federated Learning **Type**: Distributed Learning **Best For**: Privacy, distributed data **Strengths**: Privacy-preserving, scalable **Use Cases**: - Multi-agent systems - Privacy-sensitive data - Distributed training - Collaborative learning ### 9. Multi-Task Learning **Type**: Transfer Learning **Best For**: Related tasks, knowledge sharing **Strengths**: Faster learning on new tasks, better generalization **Use Cases**: - Task families - Transfer learning - Domain adaptation - Meta-learning --- ## Training Workflow ### 1. Collect Experiences ```typescript // Store experiences during agent execution for (let i = 0; i < numEpisodes; i++) { const episode = runEpisode(); for (const step of episode.steps) { await adapter.insertPattern({ id: '', type: 'experience', domain: 'task-domain', pattern_data: JSON.stringify({ embedding: await computeEmbedding(JSON.stringify(step)), pattern: { state: step.state, action: step.action, reward: step.reward, next_state: step.next_state, done: step.done } }), confidence: step.reward > 0 ? 0.9 : 0.5, usage_count: 1, success_count: step.reward > 0 ? 1 : 0, created_at: Date.now(), last_used: Date.now(), }); } } ``` ### 2. Train Model ```typescript // Train on collected experiences const trainingMetrics = await adapter.train({ epochs: 100, batchSize: 64, learningRate: 0.001, validationSplit: 0.2, }); console.log('Training Metrics:', trainingMetrics); // { // loss: 0.023, // valLoss: 0.028, // duration: 1523, // epochs: 100 // } ``` ### 3. Evaluate Performance ```typescript // Retrieve similar successful experiences const testQuery = await computeEmbedding(JSON.stringify(testState)); const result = await adapter.retrieveWithReasoning(testQuery, { domain: 'task-domain', k: 10, synthesizeContext: true, }); // Evaluate action quality const suggestedAction = result.memories[0].pattern.action; const confidence = result.memories[0].similarity; console.log('Suggested Action:', suggestedAction); console.log('Confidence:', confidence); ``` --- ## Advanced Training Techniques ### Experience Replay ```typescript // Store experiences in buffer const replayBuffer = []; // Sample random batch for training const batch = sampleRandomBatch(replayBuffer, batchSize: 32); // Train on batch await adapter.train({ data: batch, epochs: 1, batchSize: 32, }); ``` ### Prioritized Experience Replay ```typescript // Store experiences with priority (TD error) await adapter.insertPattern({ // ... standard fields confidence: tdError, // Use TD error as confidence/priority // ... }); // Retrieve high-priority experiences const highPriority = await adapter.retrieveWithReasoning(queryEmbedding, { domain: 'task-domain', k: 32, minConfidence: 0.7, // Only high TD-error experiences }); ``` ### Multi-Agent Training ```typescript // Collect experiences from multiple agents for (const agent of agents) { const experience = await agent.step(); await adapter.insertPattern({ // ... store experience with agent ID domain: `multi-agent/${agent.id}`, }); } // Train shared model await adapter.train({ epochs: 50, batchSize: 64, }); ``` --- ## Performance Optimization ### Batch Training ```typescript // Collect batch of experiences const experiences = collectBatch(size: 1000); // Batch insert (500x faster) for (const exp of experiences) { await adapter.insertPattern({ /* ... */ }); } // Train on batch await adapter.train({ epochs: 10, batchSize: 128, // Larger batch for efficiency }); ``` ### Incremental Learning ```typescript // Train incrementally as new data arrives setInterval(async () => { const newExperiences = getNewExperiences(); if (newExperiences.length > 100) { await adapter.train({ epochs: 5, batchSize: 32, }); } }, 60000); // Every minute ``` --- ## Integration with Reasoning Agents Combine learning with reasoning for better performance: ```typescript // Train learning model await adapter.train({ epochs: 50, batchSize: 32 }); // Use reasoning agents for inference const result = await adapter.retrieveWithReasoning(queryEmbedding, { domain: 'decision-making', k: 10, useMMR: true, // Diverse experiences synthesizeContext: true, // Rich context optimizeMemory: true, // Consolidate patterns }); // Make decision based on learned experiences + reasoning const decision = result.context.suggestedAction; const confidence = result.memories[0].similarity; ``` --- ## CLI Operations ```bash # Create plugin npx agentdb@latest create-plugin -t decision-transformer -n my-plugin # List plugins npx agentdb@latest list-plugins # Get plugin info npx agentdb@latest plugin-info my-plugin # List templates npx agentdb@latest list-templates ``` --- ## Troubleshooting ### Issue: Training not converging ```typescript // Reduce learning rate await adapter.train({ epochs: 100, batchSize: 32, learningRate: 0.0001, // Lower learning rate }); ``` ### Issue: Overfitting ```typescript // Use validation split await adapter.train({ epochs: 50, batchSize: 64, validationSplit: 0.2, // 20% validation }); // Enable memory optimization await adapter.retrieveWithReasoning(queryEmbedding, { optimizeMemory: true, // Consolidate, reduce overfitting }); ``` ### Issue: Slow training ```bash # Enable quantization for faster inference # Use binary quantization (32x faster) ``` --- ## Learn More - **Algorithm Papers**: See docs/algorithms/ for detailed papers - **GitHub**: https://github.com/ruvnet/agentic-flow/tree/main/packages/agentdb - **MCP Integration**: `npx agentdb@latest mcp` - **Website**: https://agentdb.ruv.io --- **Category**: Machine Learning / Reinforcement Learning **Difficulty**: Intermediate to Advanced **Estimated Time**: 30-60 minutes
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.