Product

Enrichment Queue Management

Monitor and control the enrichment queue that powers AI-driven data extraction from your documents

What is the Enrichment Queue?

The Enrichment Queue processes tasks that extract structured data from documents using AI. When you create a List with fields, George AI automatically enriches your files by extracting relevant information based on your field definitions.

AI-Powered Extraction

Uses language models to extract specific information from documents based on your prompts

Vector Search

Optionally uses semantic search to find relevant content chunks before extraction

Structured Data

Extracts structured information like text, numbers, dates, or boolean values

Understanding Task Status

Status Color Description Next Steps
Pending Blue Task is waiting in queue to be processed Will be picked up automatically by the worker
Processing Purple Task is currently being processed by AI Wait for completion or stop if needed
Completed Green Successfully extracted a value Value is now stored in the list field
Failed Yellow Extracted value matched failure terms Review failure terms or retry with different settings
Error Red Processing encountered an error Check error message and retry or fix configuration
Canceled Gray Task was manually stopped Start new enrichment if needed

Available Actions

Start Enrichment

Purpose: Create new enrichment tasks for files in a list

When to use:

  • New files have been added to a library linked to the list
  • You've updated field configuration (prompt, context, etc.)
  • You want to re-enrich files with updated settings

Options:

  • onlyMissingValues (default: true) - Only process files without existing values
  • filters - Apply field filters to limit which files are enriched
  • fileId - Process only a specific file
By default, only files with missing values are enriched to avoid unnecessary reprocessing

Stop Enrichment

Purpose: Cancel all pending enrichment tasks for a field

When to use:

  • You started enrichment with wrong settings
  • You need to update field configuration before continuing
  • You want to pause enrichment temporarily

What it does:

  • Cancels all pending tasks (not yet started)
  • Does NOT stop currently processing tasks
  • Marks canceled tasks with "canceled" status

Clear Enrichments

Purpose: Delete completed enrichment tasks and optionally their extracted values

When to use:

  • Clean up old enrichment history
  • Remove failed enrichments before retrying
  • Clear specific file enrichments
  • Reset all enrichments for a field

Scope options:

  • By field: Clear all enrichments for specific field
  • By file: Clear enrichments for specific file only
  • By field + file: Clear single enrichment
  • Entire list: Clear all enrichments from all fields
Clearing enrichments removes the extracted values from your list items

Retry Enrichment

Purpose: Re-run enrichment for a specific file and field

When to use:

  • Enrichment failed due to temporary error
  • Enrichment matched failure terms but shouldn't have
  • You've updated the field configuration and want to test on one file
  • Document has been updated and needs re-enrichment

How Enrichment Works

  • Step 1: Task Creation
    System creates enrichment tasks for files in the list based on field configuration
  • Step 2: Content Search (Optional)
    If useVectorStore is enabled, performs semantic search using contentQuery to find relevant chunks
  • Step 3: Context Assembly
    Gathers context from related fields and similar chunks to provide to the AI model
  • Step 4: AI Generation
    Sends prompt with context to the configured language model to extract the value
  • Step 5: Validation
    Checks if extracted value matches failureTerms (e.g., "N/A", "Unknown", "Not found")
  • Step 6: Storage
    Stores extracted value in the list item and marks task as completed or failed

Common Management Scenarios

Many enrichments are failing with "Failed" status

Cause: AI is returning values that match your failure terms

Solution:

  1. Review the extracted values in failed tasks
  2. Check if failure terms are too broad (e.g., "N/A", "None", "Unknown")
  3. Update failure terms in field configuration to be more specific
  4. Clear failed enrichments
  5. Retry enrichment with updated settings
Enrichments are stuck in "Pending" status

Cause: Worker may be stopped or overloaded

Solution:

  1. Check the Processing Queue worker status
  2. Ensure the worker is running
  3. Check if there are many pending tasks in the processing queue
  4. Wait for current tasks to complete or start the worker
Need to re-enrich all files after changing field prompt

Solution:

  1. Clear all enrichments for the field to remove old values
  2. Update field configuration with new prompt
  3. Start enrichment (it will process all files since values were cleared)
  4. Monitor progress in the enrichment queue
Tasks show "Error" status with error messages

Cause: Processing error (API error, model error, etc.)

Solution:

  1. Click on the task to view detailed error message
  2. Common errors:
    • API rate limits - wait and retry later
    • Model unavailable - check AI service configuration
    • Invalid prompt - review and fix field prompt
    • Missing content - ensure file has been processed first
  3. Fix the underlying issue
  4. Retry the specific enrichment task
Want to enrich only files matching specific criteria

Solution:

  1. Set up field filters before starting enrichment
  2. Filters can match on other field values
  3. Start enrichment with filters applied
  4. Only files matching filter criteria will be enriched

Best Practices

Do

  • ✓ Test field configuration on a single file first using retry
  • ✓ Use specific, clear prompts to get better extraction results
  • ✓ Configure failure terms to catch common non-answers
  • ✓ Use context fields to provide related information to the AI
  • ✓ Enable vector search for fields that need semantic understanding
  • ✓ Monitor failed/error tasks regularly
  • ✓ Clear old enrichment history periodically to reduce clutter

Don't

  • ✗ Start enrichment on thousands of files without testing first
  • ✗ Use vague prompts like "extract information"
  • ✗ Ignore failed tasks - they indicate configuration issues
  • ✗ Re-enrich all files unnecessarily (wastes API credits)
  • ✗ Clear enrichments without backing up important extracted data
  • ✗ Enable vector search for simple file property extractions
  • ✗ Use expensive models for simple extraction tasks

Understanding Task Details

Each enrichment task contains detailed information about the processing pipeline. Click on any task to view its full details including input configuration, processing steps, and output results.

Input Data

  • File: Source document being enriched
  • Field: Target field for extraction
  • Library: Library containing the file
  • AI Model: Language model used (e.g., gpt-4, claude-3)
  • Prompt: Instructions sent to the AI
  • Context Fields: Related field values provided as context
  • Content Query: Search query for vector store (if enabled)

Output Data

  • Enriched Value: The extracted result
  • Similar Chunks: Relevant document sections found (if using vector search)
  • Messages: Full conversation with AI model
  • AI Instance: Specific model instance used
  • Issues: Warnings or non-critical problems
  • Error: Error message if processing failed
Example enriched value
"Company acquired in 2021 for $50M"
 
Example similar chunks (vector search)
 
Chunk 1 (distance: 0.23): "...acquisition announced June 2021..."
 
Chunk 2 (distance: 0.31): "...purchase price of fifty million USD..."
 
 
 
  
 
 
 
Next Steps
 
    
Learn About Lists
    
Processing Queue
    
All Documentation
 
 
 
 
 
  
  
 
    
George-Cloud