Processing Queue Management
Monitor and manage document processing tasks across all Libraries
Overview
The Processing Queue dashboard provides centralized monitoring and management of all document extraction and embedding tasks across your entire George AI instance.
Access: Admin Panel → Processing Queue
Queue Status Overview
Worker Status
Queue worker is actively processing tasks
Last processed: 5 seconds ago
Management Actions
Worker Control
Start or stop the queue worker that processes tasks
Resume processing pending tasks. Use when queue was stopped for maintenance.
Pause new task processing. Tasks already running will continue to completion.
Failed Task Management
Handle tasks that encountered errors
Re-queue all failed tasks for another attempt. Useful after fixing configuration issues.
Permanently remove failed tasks from the queue. Use for tasks that cannot be fixed.
Pending Task Management
Manage tasks waiting to be processed
Remove all pending tasks from the queue without processing them.
Task Cancellation
Cancel specific processing tasks
Cancel processing tasks for specific files or Libraries. Applies to both pending and actively processing tasks.
Useful when you need to stop processing for specific files that are problematic or no longer needed.
Viewing Task Details
Task Information
Each processing task shows detailed information:
| Field | Description |
|---|---|
| File Name | The document being processed |
| Library | Which Library this file belongs to |
| Status | Current processing state (pending, extracting, embedding, completed, failed) |
| Created At | When the task was created |
| Started At | When processing began |
| Finished At | When processing completed (or failed) |
| Processing Time | Total time spent processing (in milliseconds) |
| Error Message | Error details if task failed |
Filter by Status
Use the status filter dropdown to view only tasks in a specific state (pending, processing, failed, completed, etc.)
Common Management Scenarios
Symptoms: Thousands of pending tasks, very slow processing
Causes:
- Too many files added at once (large crawler run)
- AI Services overwhelmed or slow
- Not enough AI Service servers
Solutions:
- Add more AI Service servers for parallel processing
- Reduce crawler maxPages settings
- Process in batches: stop worker, clear some pending, restart
- Check AI Services health and restart if needed
Symptoms: Spike in failed tasks after changing Library settings
Common Causes:
- Changed embedding model to one that's not available
- Changed OCR timeout to value that's too low
- AI Services temporarily unavailable
Solutions:
- Revert Library configuration changes
- Verify AI Services are running and responsive
- Use "Retry Failed" to reprocess with corrected settings
Scenario: Need to perform urgent system maintenance
Steps:
- Stop the queue worker
- Wait for currently processing tasks to complete (monitor "Processing Tasks" count)
- Perform maintenance
- Restart queue worker
Scenario: Certain files repeatedly fail or timeout
Steps:
- Identify problematic file(s) from failed task list
- Use "Cancel Tasks" to stop processing those specific files
- Review files - are they corrupted, too large, unsupported format?
- Either fix files and reprocess, or exclude them permanently
For permanently problematic files, consider adding crawler exclude patterns to prevent re-ingestion.
Best Practices
Monitor Regularly
Check the Processing Queue dashboard daily to catch issues early. Set up alerts if failed task count exceeds a threshold.
Investigate Failed Tasks
Don't just clear failed tasks—review error messages to identify root causes and prevent future failures.
Be Cautious with Bulk Actions
Actions like "Clear Pending" or "Clear Failed" affect many tasks at once. Consider using Library filtering to limit scope.
Scale AI Services as Needed
If pending task count consistently grows, you need more processing capacity. Add AI Service servers or increase concurrent limits.