Ragwalla is now generally available! Get started today

OpenAI Files API: Complete Guide to Upload, Manage & Delete Files

The OpenAI Files API allows you to upload, manage, and delete files for use with Assistants, fine-tuning, and vector stores. While powerful, the API lacks built-in bulk operations and visual management tools, making file management challenging for developers working with large datasets.

This comprehensive guide covers everything you need to know about the OpenAI Files API, including practical code examples, best practices, common issues, and solutions for efficient file management.

Table of Contents

  1. OpenAI Files API Overview
  2. File Upload Methods
  3. File Management Operations
  4. Vector Store Integration
  5. File Purposes and Limitations
  6. Bulk Operations Challenge
  7. Best Practices
  8. Common Issues and Solutions
  9. API Reference Examples
  10. File Management Tools

OpenAI Files API Overview

The OpenAI Files API provides endpoints for managing files used across OpenAI's platform. Files can serve different purposes including Assistants API integration, fine-tuning datasets, and vector store content.

Supported File Types and Limits

File Size Limits:
- Individual files: Up to 512 MB per file
- Organization total: 100 GB across all files
- Vector stores: 10,000 files per vector store maximum

Supported File Formats:
- Documents: PDF, TXT, DOCX, MD, HTML
- Code: JS, PY, JSON, CSV, XML
- Data: JSONL (for fine-tuning)
- Images: PNG, JPEG, GIF, WEBP (for vision models)

File Purposes

Files are categorized by purpose, which determines how they can be used:

  • assistants: Files for use with the Assistants API
  • vision: Images for vision model processing
  • batch: Files for batch processing
  • fine-tune: Training data for model fine-tuning
  • user_data: General user data files

File Upload Methods

Basic File Upload

import OpenAI from 'openai';
import fs from 'fs';

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY
});

// Upload a single file
async function uploadFile(filePath, purpose = 'assistants') {
  try {
    const file = await openai.files.create({
      file: fs.createReadStream(filePath),
      purpose: purpose
    });

    console.log('File uploaded:', file.id);
    return file;
  } catch (error) {
    console.error('Upload failed:', error.message);
  }
}

// Usage
const uploadedFile = await uploadFile('./document.pdf', 'assistants');

Upload with Error Handling

async function uploadFileWithRetry(filePath, purpose = 'assistants', maxRetries = 3) {
  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      const file = await openai.files.create({
        file: fs.createReadStream(filePath),
        purpose: purpose
      });

      return {
        success: true,
        file: file,
        filename: filePath.split('/').pop()
      };
    } catch (error) {
      console.error(`Upload attempt ${attempt} failed:`, error.message);

      if (attempt === maxRetries) {
        return {
          success: false,
          error: error.message,
          filename: filePath.split('/').pop()
        };
      }

      // Wait before retry
      await new Promise(resolve => setTimeout(resolve, 1000 * attempt));
    }
  }
}

Batch Upload Multiple Files

async function uploadMultipleFiles(filePaths, purpose = 'assistants') {
  const results = [];
  const concurrency = 3; // Limit concurrent uploads

  for (let i = 0; i < filePaths.length; i += concurrency) {
    const batch = filePaths.slice(i, i + concurrency);

    const batchPromises = batch.map(filePath => 
      uploadFileWithRetry(filePath, purpose)
    );

    const batchResults = await Promise.all(batchPromises);
    results.push(...batchResults);

    // Progress logging
    console.log(`Uploaded ${Math.min(i + concurrency, filePaths.length)} of ${filePaths.length} files`);
  }

  return results;
}

// Usage
const filePaths = ['./doc1.pdf', './doc2.txt', './doc3.docx'];
const uploadResults = await uploadMultipleFiles(filePaths);

// Check results
const successful = uploadResults.filter(r => r.success);
const failed = uploadResults.filter(r => !r.success);

console.log(`Successfully uploaded: ${successful.length}`);
console.log(`Failed uploads: ${failed.length}`);

File Management Operations

List Files

// List all files
async function listAllFiles() {
  try {
    const files = await openai.files.list();
    return files.data;
  } catch (error) {
    console.error('Failed to list files:', error.message);
    return [];
  }
}

// List files by purpose
async function listFilesByPurpose(purpose) {
  const allFiles = await listAllFiles();
  return allFiles.filter(file => file.purpose === purpose);
}

// List files with details
async function listFilesWithDetails() {
  const files = await listAllFiles();

  return files.map(file => ({
    id: file.id,
    filename: file.filename,
    purpose: file.purpose,
    size: `${(file.bytes / 1024 / 1024).toFixed(2)} MB`,
    created: new Date(file.created_at * 1000).toISOString(),
    status: file.status
  }));
}

Retrieve File Information

// Get file metadata
async function getFileInfo(fileId) {
  try {
    const file = await openai.files.retrieve(fileId);
    return {
      id: file.id,
      filename: file.filename,
      purpose: file.purpose,
      size: file.bytes,
      created: new Date(file.created_at * 1000),
      status: file.status
    };
  } catch (error) {
    console.error(`Failed to retrieve file ${fileId}:`, error.message);
    return null;
  }
}

// Check file processing status
async function checkFileStatus(fileId) {
  const fileInfo = await getFileInfo(fileId);

  if (!fileInfo) {
    return 'not_found';
  }

  return fileInfo.status; // 'uploaded', 'processed', 'error'
}

Delete Files

// Delete a single file
async function deleteFile(fileId) {
  try {
    const result = await openai.files.del(fileId);
    console.log(`File ${fileId} deleted:`, result.deleted);
    return result.deleted;
  } catch (error) {
    console.error(`Failed to delete file ${fileId}:`, error.message);
    return false;
  }
}

// Delete multiple files (sequential to avoid rate limits)
async function deleteMultipleFiles(fileIds) {
  const results = [];

  for (const fileId of fileIds) {
    const deleted = await deleteFile(fileId);
    results.push({ fileId, deleted });

    // Brief pause to respect rate limits
    await new Promise(resolve => setTimeout(resolve, 100));
  }

  return results;
}

// Delete files by criteria
async function deleteFilesByPurpose(purpose, olderThanDays = null) {
  const files = await listFilesByPurpose(purpose);

  let filesToDelete = files;

  if (olderThanDays) {
    const cutoffDate = new Date();
    cutoffDate.setDate(cutoffDate.getDate() - olderThanDays);

    filesToDelete = files.filter(file => {
      const fileDate = new Date(file.created_at * 1000);
      return fileDate < cutoffDate;
    });
  }

  console.log(`Deleting ${filesToDelete.length} files with purpose: ${purpose}`);

  const fileIds = filesToDelete.map(file => file.id);
  return await deleteMultipleFiles(fileIds);
}

Retrieve File Content

// Download file content
async function downloadFileContent(fileId, outputPath = null) {
  try {
    const response = await openai.files.content(fileId);

    if (outputPath) {
      // Save to file
      const buffer = Buffer.from(await response.arrayBuffer());
      fs.writeFileSync(outputPath, buffer);
      console.log(`File content saved to: ${outputPath}`);
      return outputPath;
    } else {
      // Return content as text (for text files)
      return await response.text();
    }
  } catch (error) {
    console.error(`Failed to retrieve content for file ${fileId}:`, error.message);

    // Check if file purpose allows content retrieval
    if (error.message.includes('Not allowed to download')) {
      console.log('Tip: Some file purposes (like fine-tune) restrict content download');
    }

    return null;
  }
}

// Check if file content can be retrieved
async function canDownloadFile(fileId) {
  try {
    await openai.files.content(fileId);
    return true;
  } catch (error) {
    return false;
  }
}

Vector Store Integration

Add Files to Vector Store

// Create vector store with files
async function createVectorStoreWithFiles(name, fileIds) {
  try {
    // Create the vector store
    const vectorStore = await openai.beta.vectorStores.create({
      name: name
    });

    // Add files to vector store
    if (fileIds.length > 0) {
      await openai.beta.vectorStores.fileBatches.create(vectorStore.id, {
        file_ids: fileIds
      });
    }

    return vectorStore;
  } catch (error) {
    console.error('Failed to create vector store:', error.message);
    return null;
  }
}

// Add files to existing vector store
async function addFilesToVectorStore(vectorStoreId, fileIds) {
  try {
    const batch = await openai.beta.vectorStores.fileBatches.create(vectorStoreId, {
      file_ids: fileIds
    });

    console.log(`Added ${fileIds.length} files to vector store ${vectorStoreId}`);
    return batch;
  } catch (error) {
    console.error('Failed to add files to vector store:', error.message);
    return null;
  }
}

// Remove files from vector store
async function removeFilesFromVectorStore(vectorStoreId, fileIds) {
  const results = [];

  for (const fileId of fileIds) {
    try {
      await openai.beta.vectorStores.files.del(vectorStoreId, fileId);
      results.push({ fileId, removed: true });
    } catch (error) {
      console.error(`Failed to remove file ${fileId}:`, error.message);
      results.push({ fileId, removed: false, error: error.message });
    }
  }

  return results;
}

// List files in vector store
async function listVectorStoreFiles(vectorStoreId) {
  try {
    const files = await openai.beta.vectorStores.files.list(vectorStoreId);
    return files.data;
  } catch (error) {
    console.error(`Failed to list vector store files:`, error.message);
    return [];
  }
}

Vector Store File Management

// Complete vector store management example
async function manageVectorStore(vectorStoreName, documentPaths) {
  console.log(`Setting up vector store: ${vectorStoreName}`);

  // 1. Upload all documents
  console.log('Uploading documents...');
  const uploadResults = await uploadMultipleFiles(documentPaths, 'assistants');
  const successfulUploads = uploadResults.filter(r => r.success);

  if (successfulUploads.length === 0) {
    console.error('No files uploaded successfully');
    return null;
  }

  // 2. Create vector store with uploaded files
  const fileIds = successfulUploads.map(r => r.file.id);
  const vectorStore = await createVectorStoreWithFiles(vectorStoreName, fileIds);

  if (!vectorStore) {
    console.error('Failed to create vector store');
    return null;
  }

  // 3. Wait for processing to complete
  console.log('Waiting for vector store processing...');
  await waitForVectorStoreProcessing(vectorStore.id);

  console.log(`Vector store ${vectorStoreName} ready with ${fileIds.length} files`);
  return vectorStore;
}

// Wait for vector store processing
async function waitForVectorStoreProcessing(vectorStoreId, maxWaitMinutes = 10) {
  const startTime = Date.now();
  const maxWaitTime = maxWaitMinutes * 60 * 1000;

  while (Date.now() - startTime < maxWaitTime) {
    try {
      const vectorStore = await openai.beta.vectorStores.retrieve(vectorStoreId);

      if (vectorStore.status === 'completed') {
        console.log('Vector store processing completed');
        return true;
      } else if (vectorStore.status === 'failed') {
        console.error('Vector store processing failed');
        return false;
      }

      console.log(`Vector store status: ${vectorStore.status}`);
      await new Promise(resolve => setTimeout(resolve, 10000)); // Wait 10 seconds
    } catch (error) {
      console.error('Error checking vector store status:', error.message);
      break;
    }
  }

  console.error('Vector store processing timeout');
  return false;
}

File Purposes and Limitations

Understanding File Purposes

Different file purposes have different restrictions and capabilities:

const FILE_PURPOSES = {
  'assistants': {
    description: 'Files for Assistants API and vector stores',
    maxSize: '512MB',
    contentDownload: 'Limited', // Some restrictions apply
    formats: ['pdf', 'txt', 'docx', 'md', 'html', 'json', 'csv'],
    usage: 'Knowledge base, document analysis, RAG applications'
  },
  'vision': {
    description: 'Images for vision model processing',
    maxSize: '20MB',
    contentDownload: 'Yes',
    formats: ['png', 'jpeg', 'gif', 'webp'],
    usage: 'Image analysis, vision tasks'
  },
  'batch': {
    description: 'Files for batch processing',
    maxSize: '100MB',
    contentDownload: 'Yes',
    formats: ['jsonl'],
    usage: 'Batch API requests'
  },
  'fine-tune': {
    description: 'Training data for fine-tuning',
    maxSize: '1GB',
    contentDownload: 'No', // Restricted for security
    formats: ['jsonl'],
    usage: 'Model fine-tuning datasets'
  }
};

// Check if file purpose allows content download
function canDownloadByPurpose(purpose) {
  const purposeInfo = FILE_PURPOSES[purpose];
  return purposeInfo && purposeInfo.contentDownload !== 'No';
}

// Validate file for purpose
function validateFileForPurpose(filename, purpose) {
  const purposeInfo = FILE_PURPOSES[purpose];
  if (!purposeInfo) {
    return { valid: false, error: `Unknown purpose: ${purpose}` };
  }

  const extension = filename.split('.').pop().toLowerCase();
  if (!purposeInfo.formats.includes(extension)) {
    return {
      valid: false,
      error: `File type .${extension} not supported for purpose ${purpose}. Supported: ${purposeInfo.formats.join(', ')}`
    };
  }

  return { valid: true };
}

File Processing Status

// Monitor file processing status
async function monitorFileProcessing(fileId, maxWaitMinutes = 5) {
  const startTime = Date.now();
  const maxWaitTime = maxWaitMinutes * 60 * 1000;

  console.log(`Monitoring file ${fileId} processing...`);

  while (Date.now() - startTime < maxWaitTime) {
    const status = await checkFileStatus(fileId);

    switch (status) {
      case 'processed':
        console.log(`File ${fileId} processed successfully`);
        return true;
      case 'error':
        console.error(`File ${fileId} processing failed`);
        return false;
      case 'not_found':
        console.error(`File ${fileId} not found`);
        return false;
      case 'uploaded':
        console.log(`File ${fileId} still processing...`);
        break;
    }

    await new Promise(resolve => setTimeout(resolve, 5000)); // Wait 5 seconds
  }

  console.warn(`File ${fileId} processing timeout`);
  return false;
}

Bulk Operations Challenge

One of the biggest limitations of the OpenAI Files API is the lack of built-in bulk operations. This creates several challenges for developers:

The Problem

// ❌ This doesn't exist in OpenAI API
// await openai.files.bulkDelete(fileIds);

// Instead, you must do this:
// ✅ Manual iteration required
async function bulkDeleteFiles(fileIds) {
  for (const fileId of fileIds) {
    try {
      await openai.files.del(fileId);
      console.log(`Deleted: ${fileId}`);
    } catch (error) {
      console.error(`Failed to delete ${fileId}:`, error.message);
    }

    // Rate limiting delay
    await new Promise(resolve => setTimeout(resolve, 100));
  }
}

Common Bulk Operations

Here are utility functions for common bulk operations:

// Bulk operations utility class
class OpenAIFileManager {
  constructor(apiKey) {
    this.openai = new OpenAI({ apiKey });
  }

  // Delete all files older than specified days
  async cleanupOldFiles(olderThanDays = 30) {
    const files = await this.openai.files.list();
    const cutoffDate = new Date();
    cutoffDate.setDate(cutoffDate.getDate() - olderThanDays);

    const oldFiles = files.data.filter(file => {
      const fileDate = new Date(file.created_at * 1000);
      return fileDate < cutoffDate;
    });

    console.log(`Found ${oldFiles.length} files older than ${olderThanDays} days`);

    const results = [];
    for (const file of oldFiles) {
      try {
        await this.openai.files.del(file.id);
        results.push({ id: file.id, filename: file.filename, deleted: true });
      } catch (error) {
        results.push({ id: file.id, filename: file.filename, deleted: false, error: error.message });
      }

      await new Promise(resolve => setTimeout(resolve, 100));
    }

    return results;
  }

  // Upload directory of files
  async uploadDirectory(directoryPath, purpose = 'assistants') {
    const files = fs.readdirSync(directoryPath);
    const filePaths = files.map(file => path.join(directoryPath, file));

    return await uploadMultipleFiles(filePaths, purpose);
  }

  // Get storage usage summary
  async getStorageUsage() {
    const files = await this.openai.files.list();

    const summary = {
      totalFiles: files.data.length,
      totalBytes: files.data.reduce((sum, file) => sum + file.bytes, 0),
      byPurpose: {}
    };

    // Group by purpose
    files.data.forEach(file => {
      if (!summary.byPurpose[file.purpose]) {
        summary.byPurpose[file.purpose] = { count: 0, bytes: 0 };
      }
      summary.byPurpose[file.purpose].count++;
      summary.byPurpose[file.purpose].bytes += file.bytes;
    });

    // Convert bytes to readable format
    summary.totalSize = `${(summary.totalBytes / 1024 / 1024 / 1024).toFixed(2)} GB`;

    Object.keys(summary.byPurpose).forEach(purpose => {
      const purposeData = summary.byPurpose[purpose];
      purposeData.size = `${(purposeData.bytes / 1024 / 1024).toFixed(2)} MB`;
    });

    return summary;
  }
}

// Usage example
const fileManager = new OpenAIFileManager(process.env.OPENAI_API_KEY);

// Clean up old files
await fileManager.cleanupOldFiles(30);

// Get storage usage
const usage = await fileManager.getStorageUsage();
console.log('Storage Usage:', usage);

Best Practices

File Organization

// Use consistent naming conventions
const generateFileName = (type, date, description) => {
  const timestamp = date.toISOString().split('T')[0];
  return `${type}_${timestamp}_${description}`.replace(/[^a-zA-Z0-9_-]/g, '_');
};

// Example: "knowledge_base_2025-08-05_company_policies.pdf"
const filename = generateFileName('knowledge_base', new Date(), 'company policies');

Error Handling and Retry Logic

// Robust file operation with exponential backoff
async function robustFileOperation(operation, maxRetries = 3) {
  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      return await operation();
    } catch (error) {
      const isRetryable = error.message.includes('rate_limit') || 
                         error.message.includes('timeout') ||
                         error.status >= 500;

      if (!isRetryable || attempt === maxRetries) {
        throw error;
      }

      const delay = Math.pow(2, attempt) * 1000; // Exponential backoff
      console.log(`Attempt ${attempt} failed, retrying in ${delay}ms`);
      await new Promise(resolve => setTimeout(resolve, delay));
    }
  }
}

// Usage
const file = await robustFileOperation(() => 
  openai.files.create({
    file: fs.createReadStream('large-file.pdf'),
    purpose: 'assistants'
  })
);

Rate Limiting

// Rate limiter utility
class RateLimiter {
  constructor(requestsPerSecond = 3) {
    this.requestsPerSecond = requestsPerSecond;
    this.lastRequestTime = 0;
  }

  async wait() {
    const now = Date.now();
    const timeSinceLastRequest = now - this.lastRequestTime;
    const minimumInterval = 1000 / this.requestsPerSecond;

    if (timeSinceLastRequest < minimumInterval) {
      const waitTime = minimumInterval - timeSinceLastRequest;
      await new Promise(resolve => setTimeout(resolve, waitTime));
    }

    this.lastRequestTime = Date.now();
  }
}

// Use with bulk operations
const rateLimiter = new RateLimiter(3); // 3 requests per second

async function rateLimitedBulkDelete(fileIds) {
  const results = [];

  for (const fileId of fileIds) {
    await rateLimiter.wait();

    try {
      const result = await openai.files.del(fileId);
      results.push({ fileId, deleted: result.deleted });
    } catch (error) {
      results.push({ fileId, deleted: false, error: error.message });
    }
  }

  return results;
}

Storage Management

// Monitor and manage storage limits
async function checkStorageLimits() {
  const files = await openai.files.list();
  const totalBytes = files.data.reduce((sum, file) => sum + file.bytes, 0);
  const totalGB = totalBytes / (1024 * 1024 * 1024);

  const warnings = [];

  if (totalGB > 80) { // 80% of 100GB limit
    warnings.push(`Storage usage: ${totalGB.toFixed(2)}GB / 100GB (${(totalGB).toFixed(1)}%)`);
  }

  // Check for unusually large files
  const largeFiles = files.data.filter(file => file.bytes > 100 * 1024 * 1024); // > 100MB
  if (largeFiles.length > 0) {
    warnings.push(`${largeFiles.length} files larger than 100MB detected`);
  }

  // Check for old unused files
  const oldFiles = files.data.filter(file => {
    const fileAge = Date.now() - (file.created_at * 1000);
    return fileAge > 90 * 24 * 60 * 60 * 1000; // > 90 days
  });

  if (oldFiles.length > 0) {
    warnings.push(`${oldFiles.length} files older than 90 days could be cleaned up`);
  }

  return {
    totalFiles: files.data.length,
    totalGB: totalGB.toFixed(2),
    percentUsed: ((totalGB / 100) * 100).toFixed(1),
    warnings
  };
}

Common Issues and Solutions

File Upload Failures

// Diagnose upload failures
async function diagnoseUploadFailure(filePath, purpose) {
  const checks = [];

  // Check file exists
  if (!fs.existsSync(filePath)) {
    checks.push({ check: 'File exists', status: 'FAIL', message: 'File not found' });
    return checks;
  }
  checks.push({ check: 'File exists', status: 'PASS' });

  // Check file size
  const stats = fs.statSync(filePath);
  const sizeMB = stats.size / (1024 * 1024);
  if (sizeMB > 512) {
    checks.push({ check: 'File size', status: 'FAIL', message: `${sizeMB.toFixed(2)}MB exceeds 512MB limit` });
  } else {
    checks.push({ check: 'File size', status: 'PASS', message: `${sizeMB.toFixed(2)}MB` });
  }

  // Check file type
  const filename = path.basename(filePath);
  const validation = validateFileForPurpose(filename, purpose);
  if (!validation.valid) {
    checks.push({ check: 'File type', status: 'FAIL', message: validation.error });
  } else {
    checks.push({ check: 'File type', status: 'PASS' });
  }

  // Check organization storage
  const usage = await checkStorageLimits();
  if (parseFloat(usage.totalGB) + sizeMB/1024 > 100) {
    checks.push({ check: 'Storage capacity', status: 'FAIL', message: 'Would exceed 100GB limit' });
  } else {
    checks.push({ check: 'Storage capacity', status: 'PASS' });
  }

  return checks;
}

File Processing Issues

// Debug file processing problems
async function debugFileProcessing(fileId) {
  try {
    const file = await openai.files.retrieve(fileId);

    const debug = {
      fileId: file.id,
      filename: file.filename,
      purpose: file.purpose,
      status: file.status,
      size: `${(file.bytes / 1024 / 1024).toFixed(2)} MB`,
      age: `${Math.floor((Date.now() - file.created_at * 1000) / (1000 * 60))} minutes`,
      recommendations: []
    };

    // Status-specific recommendations
    switch (file.status) {
      case 'error':
        debug.recommendations.push('File processing failed - check file format and content');
        debug.recommendations.push('Try re-uploading the file');
        break;
      case 'uploaded':
        const ageMinutes = Math.floor((Date.now() - file.created_at * 1000) / (1000 * 60));
        if (ageMinutes > 10) {
          debug.recommendations.push('File has been processing for over 10 minutes - may have failed');
        } else {
          debug.recommendations.push('File is still processing - wait a few more minutes');
        }
        break;
      case 'processed':
        debug.recommendations.push('File processed successfully and ready for use');
        break;
    }

    // Size-based recommendations
    if (file.bytes > 50 * 1024 * 1024) { // > 50MB
      debug.recommendations.push('Large file may take longer to process');
    }

    return debug;
  } catch (error) {
    return { error: error.message };
  }
}

Vector Store Issues

// Troubleshoot vector store problems
async function troubleshootVectorStore(vectorStoreId) {
  try {
    const vectorStore = await openai.beta.vectorStores.retrieve(vectorStoreId);
    const files = await openai.beta.vectorStores.files.list(vectorStoreId);

    const troubleshoot = {
      vectorStoreId: vectorStore.id,
      name: vectorStore.name,
      status: vectorStore.status,
      fileCount: vectorStore.file_counts.total,
      issues: []
    };

    // Check vector store status
    if (vectorStore.status === 'failed') {
      troubleshoot.issues.push('Vector store processing failed');
    }

    // Check file processing status
    const fileStatuses = {};
    files.data.forEach(file => {
      fileStatuses[file.status] = (fileStatuses[file.status] || 0) + 1;
    });

    if (fileStatuses.failed > 0) {
      troubleshoot.issues.push(`${fileStatuses.failed} files failed to process`);
    }

    if (fileStatuses.in_progress > 0) {
      troubleshoot.issues.push(`${fileStatuses.in_progress} files still processing`);
    }

    // Check for empty vector store
    if (vectorStore.file_counts.total === 0) {
      troubleshoot.issues.push('Vector store contains no files');
    }

    troubleshoot.fileStatuses = fileStatuses;
    return troubleshoot;
  } catch (error) {
    return { error: error.message };
  }
}

API Reference Examples

Complete File Management Script

#!/usr/bin/env node

import OpenAI from 'openai';
import fs from 'fs';
import path from 'path';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

// Command line file management utility
class OpenAIFileCLI {
  async list(purpose = null) {
    const files = await openai.files.list();
    let filteredFiles = files.data;

    if (purpose) {
      filteredFiles = files.data.filter(f => f.purpose === purpose);
    }

    console.table(filteredFiles.map(file => ({
      ID: file.id,
      Filename: file.filename,
      Purpose: file.purpose,
      Size: `${(file.bytes / 1024 / 1024).toFixed(2)} MB`,
      Status: file.status,
      Created: new Date(file.created_at * 1000).toLocaleDateString()
    })));

    console.log(`\nTotal: ${filteredFiles.length} files`);
  }

  async upload(filePath, purpose = 'assistants') {
    console.log(`Uploading ${filePath} with purpose: ${purpose}`);

    const file = await openai.files.create({
      file: fs.createReadStream(filePath),
      purpose: purpose
    });

    console.log(`✅ Uploaded: ${file.id} - ${file.filename}`);
    return file;
  }

  async delete(fileId) {
    const result = await openai.files.del(fileId);
    console.log(`✅ Deleted: ${fileId}`);
    return result;
  }

  async info(fileId) {
    const file = await openai.files.retrieve(fileId);
    console.log('File Information:');
    console.log(`  ID: ${file.id}`);
    console.log(`  Filename: ${file.filename}`);
    console.log(`  Purpose: ${file.purpose}`);
    console.log(`  Size: ${(file.bytes / 1024 / 1024).toFixed(2)} MB`);
    console.log(`  Status: ${file.status}`);
    console.log(`  Created: ${new Date(file.created_at * 1000)}`);
  }

  async cleanup(olderThanDays = 30) {
    console.log(`Cleaning up files older than ${olderThanDays} days...`);

    const files = await openai.files.list();
    const cutoffDate = new Date();
    cutoffDate.setDate(cutoffDate.getDate() - olderThanDays);

    const oldFiles = files.data.filter(file => {
      const fileDate = new Date(file.created_at * 1000);
      return fileDate < cutoffDate;
    });

    console.log(`Found ${oldFiles.length} files to delete`);

    for (const file of oldFiles) {
      try {
        await this.delete(file.id);
      } catch (error) {
        console.error(`Failed to delete ${file.id}: ${error.message}`);
      }
    }
  }
}

// CLI usage
const cli = new OpenAIFileCLI();
const command = process.argv[2];

switch (command) {
  case 'list':
    await cli.list(process.argv[3]);
    break;
  case 'upload':
    await cli.upload(process.argv[3], process.argv[4]);
    break;
  case 'delete':
    await cli.delete(process.argv[3]);
    break;
  case 'info':
    await cli.info(process.argv[3]);
    break;
  case 'cleanup':
    await cli.cleanup(parseInt(process.argv[3]) || 30);
    break;
  default:
    console.log('Usage: node file-manager.js <command> [args]');
    console.log('Commands:');
    console.log('  list [purpose]     - List files');
    console.log('  upload <file> [purpose] - Upload file');
    console.log('  delete <fileId>    - Delete file');
    console.log('  info <fileId>      - Show file info');
    console.log('  cleanup [days]     - Delete files older than N days');
}

File Management Tools

While the OpenAI Files API is powerful, managing files through code can be tedious, especially for bulk operations. The API lacks:

  • Visual file browser for easy navigation
  • Bulk selection and deletion capabilities
  • File usage tracking across assistants and vector stores
  • Secure API key management for quick operations

The Solution: AI Files App

For developers who need a visual interface for OpenAI file management, we've built AI Files - a free macOS application that provides:

  • Bulk file operations - Upload, delete, and manage multiple files at once
  • Vector Store management - Visual interface for vector store file operations
  • Secure API key storage - Keys stored safely in macOS Keychain
  • File usage insights - See which assistants and vector stores use your files
  • Drag-and-drop uploads - Simple file management interface

Learn more about AI Files →

The app solves the bulk operations problem highlighted throughout this guide, providing the visual file management interface that the OpenAI platform currently lacks.

When to Use the API vs. Visual Tools

Use the OpenAI Files API directly when:
- Integrating file operations into your application
- Automating file uploads as part of a workflow
- Building custom file management logic
- Processing files programmatically

Use visual tools like AI Files when:
- Managing files during development and testing
- Performing bulk cleanup operations
- Exploring and organizing your file collection
- Quick ad-hoc file management tasks

Conclusion

The OpenAI Files API is a powerful tool for managing files across OpenAI's platform, but it requires careful planning and implementation to use effectively. Key takeaways:

  1. Understand file purposes - Different purposes have different limitations and capabilities
  2. Implement proper error handling - File operations can fail for various reasons
  3. Use bulk operations carefully - Rate limiting and sequential processing are important
  4. Monitor storage usage - Stay within the 100GB organization limit
  5. Consider visual tools - For bulk operations and file exploration, GUI tools can save significant time

Whether you're building automated workflows with the API or managing files visually, understanding these concepts will help you work more effectively with OpenAI's file system.

For complex file management needs, combining programmatic API usage with visual tools like AI Files provides the best of both worlds - automation where needed and visual control for bulk operations.

Need help with OpenAI Files API integration? Contact our team for guidance on implementing robust file management in your applications.