Configure retention policies, automated data deletion, and archival strategies for compliance and storage optimization

Data Retention

Data retention policies help you manage data lifecycle, meet compliance requirements, and optimize storage costs by automatically archiving or deleting data based on age and business rules.

Overview

MongoDash retention policies apply to:

Audit logs - User activity and system events
Query history - Saved queries and execution logs
Dashboard snapshots - Point-in-time dashboard captures
Export archives - Exported data and reports
Session data - Authentication and user session records
Webhook logs - Webhook delivery attempts and responses

Retention policies for workspace data (collections, documents) must be configured directly in MongoDB. MongoDash retention policies apply only to MongoDash-generated metadata and logs.

Default Retention Periods

Data Type	Free	Team	Business	Enterprise
Audit Logs	N/A	7 days	30 days	90 days - 7 years
Query History	30 days	90 days	1 year	2 years
Dashboard Snapshots	30 days	90 days	1 year	2 years
Export Archives	7 days	30 days	90 days	1 year
Webhook Logs	7 days	30 days	90 days	1 year
Session Data	30 days	30 days	30 days	90 days

Enterprise plans can customize retention periods to meet specific compliance requirements.

Configuring Retention Policies

Global Retention Settings

Navigate to Workspace Settings > Data Management > Retention Policies.

Select the data type you want to configure (Audit Logs, Query History, etc.).

Set the retention period:

Active retention: How long data remains immediately accessible
Archive retention: How long data is kept in cold storage
Total retention: Active + Archive period

Configure archival options:

Enable archival: Move old data to cold storage instead of deleting
Archival threshold: When to move data to archive (e.g., after 90 days)
Archive location: S3, Google Cloud Storage, or Azure Blob Storage

Click Save Retention Policy to apply changes.

Data retention policy configuration interface

Per-Collection Policies Business

Create retention rules for specific types of data:

{
  "name": "PII Audit Logs Extended Retention",
  "dataType": "audit_logs",
  "filter": {
    "resource.type": "collection",
    "resource.collection": { "$in": ["users", "customer_data", "payments"] }
  },
  "retention": {
    "active": "1 year",
    "archive": "6 years",
    "total": "7 years"
  },
  "reason": "HIPAA compliance requirement"
}

{
  "name": "Development Query History Short Retention",
  "dataType": "query_history",
  "filter": {
    "connection.environment": "development"
  },
  "retention": {
    "active": "7 days",
    "archive": "0 days",
    "total": "7 days"
  },
  "reason": "Development data not required for compliance"
}

Archival Strategies

Cold Storage Configuration

Archive old data to reduce costs while maintaining accessibility:

Go to Retention Policies > Archival Settings.

Choose your archival storage provider:

Amazon S3 (with Glacier support)
Google Cloud Storage (Coldline/Archive tiers)
Azure Blob Storage (Cool/Archive tiers)
MongoDash Managed Archive

Configure storage credentials and bucket/container details.

Set archival rules:

Compression format (gzip, bzip2, zstd)
Encryption (AES-256 required for compliance data)
Indexing (maintain searchable index of archived data)

Test the archival configuration with a sample export.

Example S3 Archival Configuration

{
  "provider": "aws_s3",
  "bucket": "company-mongodash-archive",
  "region": "us-east-1",
  "credentials": {
    "accessKeyId": "AKIA...",
    "secretAccessKey": "{{secret_key}}"
  },
  "storage_class": "GLACIER_DEEP_ARCHIVE",
  "encryption": {
    "type": "AES256",
    "kmsKeyId": "arn:aws:kms:us-east-1:123456789:key/abc-123"
  },
  "compression": "zstd",
  "path_template": "/{{workspace_id}}/{{data_type}}/{{year}}/{{month}}/{{day}}.archive",
  "metadata_index": true
}

When using Glacier Deep Archive, retrieval can take 12-48 hours. Plan accordingly for compliance audits.

Automated Data Deletion

Deletion Policies

Configure automatic deletion of data that exceeds retention periods:

{
  "dataType": "webhook_logs",
  "enabled": true,
  "retention": "30 days",
  "deletionSchedule": {
    "frequency": "daily",
    "time": "02:00 UTC"
  },
  "safetyChecks": {
    "requireArchival": true,
    "verifyArchiveIntegrity": true,
    "maxDeletionPerRun": 100000
  },
  "notifications": {
    "onCompletion": ["data-admin@company.com"],
    "onError": ["data-admin@company.com", "engineering@company.com"]
  }
}

Deletion Safety Features

Protect against accidental data loss:

Archive verification - Confirm data is archived before deletion
Deletion limits - Cap maximum records deleted per run
Grace periods - Add buffer time before deletion
Deletion logs - Maintain records of what was deleted
Restoration capabilities - Quick restore from archive if needed

Automated deletion policy configuration with safety checks

Compliance-Driven Retention

Handle data subject deletion requests:

Navigate to Data Management > Data Subject Requests.

Click New Erasure Request and enter the user identifier (email, user ID, etc.).

Review the data discovery report showing all data associated with the subject:

Audit log entries
Query history
Dashboard ownership
API keys and sessions

Select data to delete or anonymize:

Full deletion: Permanently remove all data
Anonymization: Replace identifiers with anonymized tokens
Selective deletion: Remove specific data types only

Click Execute Erasure Request and confirm the action.

Download the completion certificate for compliance documentation.

Example Erasure Request

{
  "requestId": "dsr_2024_001",
  "subject": {
    "type": "user",
    "identifier": "user@example.com",
    "userId": "usr_abc123"
  },
  "requestDate": "2024-02-24T10:00:00Z",
  "completionDeadline": "2024-03-25T23:59:59Z",
  "scope": {
    "auditLogs": "anonymize",
    "queryHistory": "delete",
    "dashboards": "transfer_ownership",
    "apiKeys": "revoke_and_delete",
    "sessions": "delete"
  },
  "status": "completed",
  "completedDate": "2024-02-24T15:30:00Z",
  "recordsAffected": {
    "auditLogs": 1247,
    "queryHistory": 89,
    "dashboards": 3,
    "apiKeys": 2,
    "sessions": 15
  }
}

GDPR requires responding to erasure requests within 30 days. MongoDash provides automated workflows to meet this deadline.

Legal Hold and Preservation

Creating Legal Holds Enterprise

Temporarily suspend retention policies for litigation or investigation:

Go to Data Management > Legal Holds.

Click Create Legal Hold and provide:

Hold name and case reference number
Data scope (date range, users, collections)
Reason for hold
Expected duration

Review affected retention policies that will be suspended.

Click Activate Legal Hold to prevent data deletion.

Legal Hold Configuration

{
  "holdId": "lh_2024_litigation_001",
  "caseName": "Company v. Vendor Dispute",
  "caseReference": "CASE-2024-12345",
  "activatedDate": "2024-02-24",
  "activatedBy": "legal@company.com",
  "scope": {
    "startDate": "2023-01-01",
    "endDate": "2024-01-31",
    "dataTypes": ["audit_logs", "query_history"],
    "users": ["employee1@company.com", "employee2@company.com"],
    "collections": ["production_db.orders", "production_db.invoices"]
  },
  "status": "active",
  "affectedRecords": 45678,
  "retentionPoliciesSuspended": [
    "policy_audit_logs_90day",
    "policy_query_history_1year"
  ],
  "estimatedStorageCost": "$127/month"
}

Legal holds override all retention and deletion policies. Data under hold will not be deleted even after retention periods expire.

Monitoring and Reporting

Retention Dashboard

Monitor retention policy effectiveness:

Storage trends - Track data growth and archival rates
Policy compliance - Verify policies are executing as configured
Cost optimization - Identify opportunities to reduce storage costs
Upcoming expirations - Preview data scheduled for archival or deletion

Data retention monitoring dashboard showing storage trends and policy status

Retention Reports

Generate reports on data lifecycle management:

# Generate monthly retention report
curl -X POST https://api.mongodash.com/v1/retention/reports \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "reportType": "retention_summary",
    "period": {
      "start": "2024-02-01",
      "end": "2024-02-29"
    },
    "includeMetrics": [
      "records_archived",
      "records_deleted",
      "storage_saved",
      "policy_compliance"
    ]
  }'

Example Report Output

{
  "period": "2024-02",
  "summary": {
    "recordsArchived": 1234567,
    "recordsDeleted": 456789,
    "storageSaved": "847 GB",
    "costSavings": "$42.35"
  },
  "byDataType": {
    "audit_logs": {
      "archived": 890000,
      "deleted": 125000,
      "policyCompliance": "100%"
    },
    "query_history": {
      "archived": 234567,
      "deleted": 234567,
      "policyCompliance": "100%"
    },
    "webhook_logs": {
      "archived": 110000,
      "deleted": 97222,
      "policyCompliance": "98.5%"
    }
  },
  "exceptions": [
    {
      "dataType": "webhook_logs",
      "reason": "Legal hold lh_2024_litigation_001",
      "recordsPreserved": 2778
    }
  ]
}

Data Restoration

Restoring from Archive

Access archived data when needed:

Go to Data Management > Archived Data.

Search for the data you need:

Filter by date range
Filter by data type
Search by keywords or identifiers

Select the archive file and click Initiate Restoration.

Choose restoration options:

Target location: Active storage or temporary access
Restoration priority: Standard (12-48 hours) or Expedited (1-5 hours)
Access duration: How long to keep data in active storage

Click Start Restoration. You'll receive an email when data is available.

Restoration Costs

Be aware of retrieval costs for archived data:

Storage Tier	Standard Retrieval	Expedited Retrieval
S3 Glacier	$0.01/GB	$0.03/GB
S3 Glacier Deep Archive	$0.02/GB	Not available
GCS Archive	$0.05/GB	$0.10/GB
Azure Archive	$0.02/GB	$0.03/GB

For compliance audits, request archived data 1-2 weeks in advance to avoid expedited retrieval costs.

Best Practices

Retention Policy Design

Align with compliance - Set retention periods based on regulatory requirements
Consider investigation needs - Retain audit logs long enough for security investigations
Balance cost and risk - Archive rather than delete when possible
Document policies - Maintain written retention policy documentation
Review regularly - Audit retention policies quarterly

Storage Optimization

Use compression - Enable compression for archived data
Lifecycle policies - Transition to cheaper storage tiers over time
Delete duplicates - Identify and remove redundant data before archival
Monitor costs - Set budget alerts for storage and retrieval costs

Compliance Readiness

Test restoration - Periodically verify archived data can be retrieved
Maintain indexes - Keep metadata searchable even for archived data
Document exceptions - Record legal holds and policy deviations
Automate reporting - Schedule regular retention compliance reports

What's Next?

Audit Logs - Configure comprehensive activity logging
Compliance Reports - Generate regulatory reports
Billing Management - Monitor storage costs

Data Retention

Data retention policies help you manage data lifecycle, meet compliance requirements, and optimize storage costs by automatically archiving or deleting data based on age and business rules.

Overview

MongoDash retention policies apply to:

Audit logs - User activity and system events
Query history - Saved queries and execution logs
Dashboard snapshots - Point-in-time dashboard captures
Export archives - Exported data and reports
Session data - Authentication and user session records
Webhook logs - Webhook delivery attempts and responses

Retention policies for workspace data (collections, documents) must be configured directly in MongoDB. MongoDash retention policies apply only to MongoDash-generated metadata and logs.

Default Retention Periods

Data Type	Free	Team	Business	Enterprise
Audit Logs	N/A	7 days	30 days	90 days - 7 years
Query History	30 days	90 days	1 year	2 years
Dashboard Snapshots	30 days	90 days	1 year	2 years
Export Archives	7 days	30 days	90 days	1 year
Webhook Logs	7 days	30 days	90 days	1 year
Session Data	30 days	30 days	30 days	90 days

Enterprise plans can customize retention periods to meet specific compliance requirements.

Configuring Retention Policies

Global Retention Settings

Navigate to Workspace Settings > Data Management > Retention Policies.

Select the data type you want to configure (Audit Logs, Query History, etc.).

Set the retention period:

Active retention: How long data remains immediately accessible
Archive retention: How long data is kept in cold storage
Total retention: Active + Archive period

Configure archival options:

Enable archival: Move old data to cold storage instead of deleting
Archival threshold: When to move data to archive (e.g., after 90 days)
Archive location: S3, Google Cloud Storage, or Azure Blob Storage

Click Save Retention Policy to apply changes.

Per-Collection Policies Business

Create retention rules for specific types of data:

{
  "name": "PII Audit Logs Extended Retention",
  "dataType": "audit_logs",
  "filter": {
    "resource.type": "collection",
    "resource.collection": { "$in": ["users", "customer_data", "payments"] }
  },
  "retention": {
    "active": "1 year",
    "archive": "6 years",
    "total": "7 years"
  },
  "reason": "HIPAA compliance requirement"
}

{
  "name": "Development Query History Short Retention",
  "dataType": "query_history",
  "filter": {
    "connection.environment": "development"
  },
  "retention": {
    "active": "7 days",
    "archive": "0 days",
    "total": "7 days"
  },
  "reason": "Development data not required for compliance"
}

Archival Strategies

Cold Storage Configuration

Archive old data to reduce costs while maintaining accessibility:

Go to Retention Policies > Archival Settings.

Choose your archival storage provider:

Amazon S3 (with Glacier support)
Google Cloud Storage (Coldline/Archive tiers)
Azure Blob Storage (Cool/Archive tiers)
MongoDash Managed Archive

Configure storage credentials and bucket/container details.

Set archival rules:

Compression format (gzip, bzip2, zstd)
Encryption (AES-256 required for compliance data)
Indexing (maintain searchable index of archived data)

Test the archival configuration with a sample export.

Example S3 Archival Configuration

{
  "provider": "aws_s3",
  "bucket": "company-mongodash-archive",
  "region": "us-east-1",
  "credentials": {
    "accessKeyId": "AKIA...",
    "secretAccessKey": "{{secret_key}}"
  },
  "storage_class": "GLACIER_DEEP_ARCHIVE",
  "encryption": {
    "type": "AES256",
    "kmsKeyId": "arn:aws:kms:us-east-1:123456789:key/abc-123"
  },
  "compression": "zstd",
  "path_template": "/{{workspace_id}}/{{data_type}}/{{year}}/{{month}}/{{day}}.archive",
  "metadata_index": true
}

When using Glacier Deep Archive, retrieval can take 12-48 hours. Plan accordingly for compliance audits.

Automated Data Deletion

Deletion Policies

Configure automatic deletion of data that exceeds retention periods:

{
  "dataType": "webhook_logs",
  "enabled": true,
  "retention": "30 days",
  "deletionSchedule": {
    "frequency": "daily",
    "time": "02:00 UTC"
  },
  "safetyChecks": {
    "requireArchival": true,
    "verifyArchiveIntegrity": true,
    "maxDeletionPerRun": 100000
  },
  "notifications": {
    "onCompletion": ["data-admin@company.com"],
    "onError": ["data-admin@company.com", "engineering@company.com"]
  }
}

Deletion Safety Features

Protect against accidental data loss:

Archive verification - Confirm data is archived before deletion
Deletion limits - Cap maximum records deleted per run
Grace periods - Add buffer time before deletion
Deletion logs - Maintain records of what was deleted
Restoration capabilities - Quick restore from archive if needed

Compliance-Driven Retention

Handle data subject deletion requests:

Navigate to Data Management > Data Subject Requests.

Click New Erasure Request and enter the user identifier (email, user ID, etc.).

Review the data discovery report showing all data associated with the subject:

Audit log entries
Query history
Dashboard ownership
API keys and sessions

Select data to delete or anonymize:

Full deletion: Permanently remove all data
Anonymization: Replace identifiers with anonymized tokens
Selective deletion: Remove specific data types only

Click Execute Erasure Request and confirm the action.

Download the completion certificate for compliance documentation.

Example Erasure Request

{
  "requestId": "dsr_2024_001",
  "subject": {
    "type": "user",
    "identifier": "user@example.com",
    "userId": "usr_abc123"
  },
  "requestDate": "2024-02-24T10:00:00Z",
  "completionDeadline": "2024-03-25T23:59:59Z",
  "scope": {
    "auditLogs": "anonymize",
    "queryHistory": "delete",
    "dashboards": "transfer_ownership",
    "apiKeys": "revoke_and_delete",
    "sessions": "delete"
  },
  "status": "completed",
  "completedDate": "2024-02-24T15:30:00Z",
  "recordsAffected": {
    "auditLogs": 1247,
    "queryHistory": 89,
    "dashboards": 3,
    "apiKeys": 2,
    "sessions": 15
  }
}

GDPR requires responding to erasure requests within 30 days. MongoDash provides automated workflows to meet this deadline.

Legal Hold and Preservation

Creating Legal Holds Enterprise

Temporarily suspend retention policies for litigation or investigation:

Go to Data Management > Legal Holds.

Click Create Legal Hold and provide:

Hold name and case reference number
Data scope (date range, users, collections)
Reason for hold
Expected duration

Review affected retention policies that will be suspended.

Click Activate Legal Hold to prevent data deletion.

Legal Hold Configuration

{
  "holdId": "lh_2024_litigation_001",
  "caseName": "Company v. Vendor Dispute",
  "caseReference": "CASE-2024-12345",
  "activatedDate": "2024-02-24",
  "activatedBy": "legal@company.com",
  "scope": {
    "startDate": "2023-01-01",
    "endDate": "2024-01-31",
    "dataTypes": ["audit_logs", "query_history"],
    "users": ["employee1@company.com", "employee2@company.com"],
    "collections": ["production_db.orders", "production_db.invoices"]
  },
  "status": "active",
  "affectedRecords": 45678,
  "retentionPoliciesSuspended": [
    "policy_audit_logs_90day",
    "policy_query_history_1year"
  ],
  "estimatedStorageCost": "$127/month"
}

Legal holds override all retention and deletion policies. Data under hold will not be deleted even after retention periods expire.

Monitoring and Reporting

Retention Dashboard

Monitor retention policy effectiveness:

Storage trends - Track data growth and archival rates
Policy compliance - Verify policies are executing as configured
Cost optimization - Identify opportunities to reduce storage costs
Upcoming expirations - Preview data scheduled for archival or deletion

Retention Reports

Generate reports on data lifecycle management:

# Generate monthly retention report
curl -X POST https://api.mongodash.com/v1/retention/reports \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "reportType": "retention_summary",
    "period": {
      "start": "2024-02-01",
      "end": "2024-02-29"
    },
    "includeMetrics": [
      "records_archived",
      "records_deleted",
      "storage_saved",
      "policy_compliance"
    ]
  }'

Example Report Output

{
  "period": "2024-02",
  "summary": {
    "recordsArchived": 1234567,
    "recordsDeleted": 456789,
    "storageSaved": "847 GB",
    "costSavings": "$42.35"
  },
  "byDataType": {
    "audit_logs": {
      "archived": 890000,
      "deleted": 125000,
      "policyCompliance": "100%"
    },
    "query_history": {
      "archived": 234567,
      "deleted": 234567,
      "policyCompliance": "100%"
    },
    "webhook_logs": {
      "archived": 110000,
      "deleted": 97222,
      "policyCompliance": "98.5%"
    }
  },
  "exceptions": [
    {
      "dataType": "webhook_logs",
      "reason": "Legal hold lh_2024_litigation_001",
      "recordsPreserved": 2778
    }
  ]
}

Data Restoration

Restoring from Archive

Access archived data when needed:

Go to Data Management > Archived Data.

Search for the data you need:

Filter by date range
Filter by data type
Search by keywords or identifiers

Select the archive file and click Initiate Restoration.

Choose restoration options:

Target location: Active storage or temporary access
Restoration priority: Standard (12-48 hours) or Expedited (1-5 hours)
Access duration: How long to keep data in active storage

Click Start Restoration. You'll receive an email when data is available.

Restoration Costs

Be aware of retrieval costs for archived data:

Storage Tier	Standard Retrieval	Expedited Retrieval
S3 Glacier	$0.01/GB	$0.03/GB
S3 Glacier Deep Archive	$0.02/GB	Not available
GCS Archive	$0.05/GB	$0.10/GB
Azure Archive	$0.02/GB	$0.03/GB

For compliance audits, request archived data 1-2 weeks in advance to avoid expedited retrieval costs.

Best Practices

Retention Policy Design

Align with compliance - Set retention periods based on regulatory requirements
Consider investigation needs - Retain audit logs long enough for security investigations
Balance cost and risk - Archive rather than delete when possible
Document policies - Maintain written retention policy documentation
Review regularly - Audit retention policies quarterly

Storage Optimization

Use compression - Enable compression for archived data
Lifecycle policies - Transition to cheaper storage tiers over time
Delete duplicates - Identify and remove redundant data before archival
Monitor costs - Set budget alerts for storage and retrieval costs

Compliance Readiness

Test restoration - Periodically verify archived data can be retrieved
Maintain indexes - Keep metadata searchable even for archived data
Document exceptions - Record legal holds and policy deviations
Automate reporting - Schedule regular retention compliance reports

What's Next?

Audit Logs - Configure comprehensive activity logging
Compliance Reports - Generate regulatory reports
Billing Management - Monitor storage costs

Data Retention

Data Retention

Overview

Default Retention Periods

Configuring Retention Policies

Global Retention Settings

Per-Collection Policies Business

Archival Strategies

Cold Storage Configuration

Example S3 Archival Configuration

Automated Data Deletion

Deletion Policies

Deletion Safety Features

Compliance-Driven Retention

GDPR Right to Erasure

Example Erasure Request

Legal Hold and Preservation

Creating Legal Holds Enterprise

Legal Hold Configuration

Monitoring and Reporting

Retention Dashboard

Retention Reports

Example Report Output

Data Restoration

Restoring from Archive

Restoration Costs

Best Practices

Retention Policy Design

Storage Optimization

Compliance Readiness

What's Next?

On this page

Data Retention

Data Retention

Overview

Default Retention Periods

Configuring Retention Policies

Global Retention Settings

Per-Collection Policies Business

Archival Strategies

Cold Storage Configuration

Example S3 Archival Configuration

Automated Data Deletion

Deletion Policies

Deletion Safety Features

Compliance-Driven Retention

GDPR Right to Erasure

Example Erasure Request

Legal Hold and Preservation

Creating Legal Holds Enterprise

Legal Hold Configuration

Monitoring and Reporting

Retention Dashboard

Retention Reports

Example Report Output

Data Restoration

Restoring from Archive

Restoration Costs

Best Practices

Retention Policy Design

Storage Optimization

Compliance Readiness

What's Next?

On this page