Skip to main content

Document Management API Routes

This document describes the API routes defined in documentRoutes.js for managing document uploads and their integration with the external ingestion service.

Overview​

The documentRoutes.js file defines the API endpoints responsible for handling document uploads within the ChainAlign backend. It orchestrates the process from receiving the file to forwarding it to an external Python-based ingestion service for further processing.

Key responsibilities include:

  • Receiving multipart/form-data file uploads.
  • Utilizing multer for temporary file storage during the upload process.
  • Authenticating requests via the verifyToken middleware to ensure tenant association.
  • Forwarding the document and tenant information to the external Python-based ingestion service.
  • Cleaning up temporary files after successful forwarding or in case of errors.

Key Features and Implementation Details​

fs Module Import​

The Node.js fs (file system) module is explicitly imported to handle file-related operations, such as ensuring the temporary upload directory exists and cleaning up temporary files.

Configurable multer​

multer is configured to use a temporary directory for file storage during uploads. This directory is configurable via the UPLOAD_TEMP_DIR environment variable, defaulting to uploads/. This allows for flexible deployment configurations.

Corrected File Forwarding​

Documents are forwarded to the external ingestion service using axios and FormData. A readable stream is created from the temporary file, ensuring efficient handling of potentially large files. The tenant_id is included in the forwarding request to associate the document with the correct tenant.

Error handling includes robust cleanup of temporary files even if the forwarding process fails.

API Endpoints​

POST /api/documents/upload​

Summary: Upload and process a document for a tenant.

Description: Handles file upload, associates it with the user's tenant, and initiates the processing pipeline (parsing, chunking, embedding) via the external ingestion service.

Security: Requires bearerAuth (token verification).

Request Body:

  • multipart/form-data
    • document (type: string, format: binary): The document file to upload.

Responses:

  • 202 Accepted: Document accepted for processing.
  • 400 Bad Request: No file uploaded.
  • 500 Internal Server Error: Server error during processing or forwarding.