GetAllDocuments 1.0.0¶

Overview¶

Data Beginner

Description¶

Retrieves all documents (DataSetItems) from every dataset within a specified dataspace. This block aggregates documents across multiple datasets by fetching the dataspace configuration, extracting all dataset IDs, and collecting all items from each dataset into a unified collection.

Configuration Options¶

Name	Data Type	Description	Default Value
dataspace_id	`UUID`	Unique identifier of the dataspace to retrieve documents from. The block will fetch all datasets within this dataspace and aggregate their documents.	None

Inputs¶

Name	Data Type	Description
run	`Any`	Workflow run context object. Used internally for execution tracking and context management. Typically provided automatically by the workflow engine.

Outputs¶

Name	Data Type	Description
documents	`list[DataSetItem]`	Aggregated list of all DataSetItems from every dataset within the specified dataspace. Each DataSetItem contains properties, metadata (createdAt, modifiedAt), and indexText content. Items from all datasets are combined into a single flat list.

Examples¶

# Retrieve all documents from a dataspace
- id: get_all_docs
  uses: GetAllDocuments@1.0.0
  with:
    dataspace_id: "550e8400-e29b-41d4-a716-446655440000"
    run: "Workflow run context"
  outputs:
    documents: all_documents

    

# Bulk document processing workflow
- id: collect_all_docs
  uses: GetAllDocuments@1.0.0
  with:
    dataspace_id: "550e8400-e29b-41d4-a716-446655440000"
    run: "Workflow run context"
  outputs:
    documents: source_documents

# Process each document through conversion
- id: process_documents
  uses: DocumentCreator@1.0.0
  with:
    output_file_type: "pdf"
    title: "Batch_Processed_Documents"
    content: "Reference to each document from source_documents"
  outputs:
    result: processed_documents

# Archive all documents
- id: archive_docs
  uses: FileStore@1.0.0
  with:
    files: "Reference to processed_documents"
    storage_path: "/archive/batch_processed/"
  outputs:
    stored_files: archived_documents

    

# Content analytics across all documents
- id: gather_all_content
  uses: GetAllDocuments@1.0.0
  with:
    dataspace_id: "550e8400-e29b-41d4-a716-446655440000"
    run: "Workflow run context"
  outputs:
    documents: content_corpus

# Analyze document properties and metadata
- id: analyze_metadata
  uses: GetJsonField@1.0.0
  with:
    data: "Reference to content_corpus"
    field_path: "createdAt"
  outputs:
    value: creation_dates

# Extract and process indexText for analysis
- id: extract_text_content
  uses: GetJsonField@1.0.0
  with:
    data: "Reference to content_corpus"
    field_path: "indexText"
  outputs:
    value: document_texts

# Generate summary report
- id: create_summary_report
  uses: DocumentCreator@1.0.0
  with:
    title: "Dataspace_Content_Summary"
    output_file_type: "docx"
    content: |
      # Dataspace Content Analysis Report

      ## Overview
      Total documents analyzed: Referenced from content_corpus count

      ## Content Summary
      Full text content extracted for analysis

      ## Metadata Analysis
      Document creation patterns and timestamps
  outputs:
    result: analytics_report

    

Error Handling¶

DataspaceNotFoundError

Error Code: dataspace_not_found
Common Cause: Specified dataspace_id does not exist or is not accessible through the configuration API
Solution: Verify the dataspace_id exists and is accessible, check API connectivity and authentication

DatasetAccessError

Error Code: dataset_retrieval_failed
Common Cause: One or more datasets within the dataspace cannot be accessed or queried
Solution: Check dataset permissions, verify dataset service connectivity, ensure datasets are properly initialized

ConfigAPIError

Error Code: config_api_connection_failed
Common Cause: Cannot connect to configuration API service to fetch dataspace information
Solution: Verify configuration API service availability, check network connectivity and authentication credentials

FAQ¶

How is this different from GetDataSet block?

GetAllDocuments retrieves ALL documents from ALL datasets within a dataspace, while GetDataSet retrieves specific items from a single dataset with filtering and pagination. Use GetAllDocuments for bulk operations across an entire dataspace, and GetDataSet for targeted queries on specific datasets.

What happens if a dataspace has many large datasets?

The block will retrieve all items from every dataset, which could result in very large result sets. For large dataspaces, consider memory and performance implications. You may want to implement pagination or use GetDataSet with filtering instead for better performance control.

Can I filter the documents returned by this block?

This block does not support filtering - it returns ALL documents from the dataspace. To filter documents, use the results with a Filter block afterward, or use GetDataSet blocks on individual datasets with specific filter criteria.

How are documents ordered in the results?

Documents are returned in the order they are retrieved from each dataset, with all items from the first dataset, then all items from the second dataset, etc. The order within each dataset depends on the dataset service implementation. For specific ordering, process results with a Sort block.

What if some datasets in the dataspace are empty?

Empty datasets simply contribute no items to the final result. The block will continue processing all datasets and return documents from those that contain data. The aggregation handles empty datasets gracefully without errors.