Automation
Scheduled Queries
Automate MongoDB queries with cron-based scheduling, result notifications, and data exports
Scheduled Queries Team
Scheduled queries automate repetitive data analysis tasks, enabling you to run MongoDB queries on a schedule and receive results via email, webhooks, or exported files.
Overview
Use scheduled queries to:
- Generate daily reports - Automatically export data for business reporting
- Monitor data quality - Check for anomalies or data integrity issues
- Track metrics - Calculate KPIs and metrics on a regular cadence
- Alert on conditions - Notify teams when data matches specific criteria
- Archive snapshots - Capture point-in-time data exports for compliance
Scheduled queries are available on Team, Business, and Enterprise plans.
Creating a Scheduled Query
Navigate to Queries in the left sidebar and open the Query Editor.
Write and test your query using the query builder or raw MongoDB syntax.
Click Save Query and provide a descriptive name.
Click the Schedule icon next to your saved query.
Configure the schedule:
- Frequency: Hourly, daily, weekly, monthly, or custom cron expression
- Time zone: Set the time zone for execution
- Start date: When to begin running the schedule
- End date (optional): When to stop the schedule
Configure output options:
- Email results
- Webhook delivery
- Export to file storage (S3, GCS, Azure)
- Dashboard widget refresh
Click Activate Schedule to start automation.
Schedule Frequency Options
Predefined Schedules
Quick setup with common patterns:
- Hourly - Every hour at minute 0
- Daily - Every day at a specific time (e.g., 9:00 AM)
- Weekly - Specific day of week (e.g., Monday at 8:00 AM)
- Monthly - Specific day of month (e.g., 1st of month at midnight)
Custom Cron Expressions
Use cron syntax for precise scheduling:
# Every 15 minutes
*/15 * * * *
# Every weekday at 9 AM
0 9 * * 1-5
# First day of every month at midnight
0 0 1 * *
# Every Monday and Thursday at 2:30 PM
30 14 * * 1,4
# Every 6 hours
0 */6 * * *
# Last day of every month at 11 PM
0 23 L * *Use the built-in cron expression builder if you're not familiar with cron syntax. It provides a visual interface to create complex schedules.
Time Zone Handling
Always specify the time zone for your schedules:
{
"schedule": "0 9 * * *",
"timezone": "America/New_York",
"description": "Daily report at 9 AM Eastern Time"
}Common time zones:
UTC- Coordinated Universal TimeAmerica/New_York- Eastern TimeAmerica/Los_Angeles- Pacific TimeEurope/London- British TimeAsia/Tokyo- Japan Standard Time
Query Examples
Daily Sales Report
Export yesterday's sales for morning review:
// Query: Daily Sales Summary
{
"createdAt": {
"$gte": new Date(new Date().setDate(new Date().getDate() - 1)),
"$lt": new Date()
},
"status": "completed"
}
// Aggregation for totals
[
{
"$match": {
"createdAt": {
"$gte": new Date(new Date().setDate(new Date().getDate() - 1)),
"$lt": new Date()
},
"status": "completed"
}
},
{
"$group": {
"_id": null,
"totalSales": { "$sum": "$amount" },
"orderCount": { "$sum": 1 },
"avgOrderValue": { "$avg": "$amount" }
}
}
]
// Schedule: Daily at 8 AM
// Delivery: Email to sales@company.comHourly System Health Check
Monitor for error conditions:
// Query: Recent Errors
{
"level": "error",
"timestamp": {
"$gte": new Date(Date.now() - 3600000) // Last hour
}
}
// Schedule: Every hour
// Delivery: Webhook to Slack if results > 0Weekly User Activity Report
Track user engagement metrics:
// Aggregation: Weekly Active Users
[
{
"$match": {
"lastLogin": {
"$gte": new Date(Date.now() - 7 * 24 * 3600000) // Last 7 days
}
}
},
{
"$group": {
"_id": {
"$dateToString": {
"format": "%Y-%m-%d",
"date": "$lastLogin"
}
},
"activeUsers": { "$sum": 1 }
}
},
{
"$sort": { "_id": 1 }
}
]
// Schedule: Every Monday at 9 AM
// Delivery: CSV export to S3Monthly Data Archive
Create monthly snapshots for compliance:
// Query: Previous Month's Records
{
"createdAt": {
"$gte": new Date(new Date().getFullYear(), new Date().getMonth() - 1, 1),
"$lt": new Date(new Date().getFullYear(), new Date().getMonth(), 1)
}
}
// Schedule: First day of month at 2 AM
// Delivery: JSON export to compliance archive bucket
Delivery Options
Email Notifications
Send query results via email:
{
"delivery": {
"type": "email",
"recipients": [
"team@company.com",
"manager@company.com"
],
"subject": "Daily Sales Report - {{date}}",
"format": "html_table",
"attachments": [
{
"type": "csv",
"filename": "sales_{{date}}.csv"
}
],
"includeChart": true,
"sendIfEmpty": false
}
}Email formats:
html_table- Formatted HTML table in email bodyplain_text- Plain text summaryattachment_only- Results as CSV/JSON attachment only
Set sendIfEmpty: false to avoid notification fatigue when queries return no results.
Webhook Delivery
POST results to external systems:
{
"delivery": {
"type": "webhook",
"url": "https://hooks.slack.com/services/YOUR/WEBHOOK/URL",
"method": "POST",
"headers": {
"Content-Type": "application/json",
"Authorization": "Bearer {{secret:webhook_token}}"
},
"bodyTemplate": {
"text": "Scheduled query completed",
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "*Query Results*\n{{results.count}} records found"
}
}
]
},
"retryPolicy": {
"maxAttempts": 3,
"backoff": "exponential"
}
}
}Supported webhook platforms:
- Slack
- Microsoft Teams
- Discord
- Custom HTTP endpoints
- Zapier
- Make (formerly Integromat)
File Export
Export results to cloud storage:
{
"delivery": {
"type": "file_export",
"destination": {
"type": "s3",
"bucket": "company-reports",
"path": "/scheduled-queries/{{query_name}}/{{date}}.csv",
"region": "us-east-1"
},
"format": "csv",
"compression": "gzip",
"encryption": "AES256",
"metadata": {
"query_name": "{{query_name}}",
"execution_time": "{{execution_time}}",
"record_count": "{{results.count}}"
}
}
}Supported storage providers:
- Amazon S3
- Google Cloud Storage
- Azure Blob Storage
- Dropbox (Business plans)
- SFTP servers
Conditional Notifications
Alert Thresholds
Only send notifications when specific conditions are met:
{
"conditions": {
"resultCount": {
"$gt": 0
}
},
"notification": {
"type": "email",
"recipients": ["alerts@company.com"],
"subject": "ALERT: {{count}} errors detected",
"priority": "high"
}
}Advanced Conditions
Use aggregation results to trigger alerts:
{
"query": [
{
"$group": {
"_id": null,
"avgResponseTime": { "$avg": "$responseTime" }
}
}
],
"conditions": {
"results.0.avgResponseTime": {
"$gt": 1000
}
},
"notification": {
"type": "webhook",
"url": "https://pagerduty.com/webhook",
"message": "Average response time exceeded 1000ms: {{results.0.avgResponseTime}}ms"
}
}Multi-Channel Notifications
Send to different channels based on severity:
{
"notifications": [
{
"condition": {
"results.count": { "$gte": 100 }
},
"channels": ["email", "pagerduty"],
"severity": "critical"
},
{
"condition": {
"results.count": { "$gte": 50, "$lt": 100 }
},
"channels": ["email"],
"severity": "warning"
},
{
"condition": {
"results.count": { "$lt": 50 }
},
"channels": ["slack"],
"severity": "info"
}
]
}Managing Scheduled Queries
Viewing Active Schedules
Monitor all scheduled queries:
- Navigate to Automation > Scheduled Queries
- View list of all active, paused, and disabled schedules
- See next execution time and last run status
- Review execution history and success rates
Execution History
Track query execution over time:
{
"scheduleId": "sched_abc123",
"queryName": "Daily Sales Report",
"executions": [
{
"executionId": "exec_xyz789",
"startTime": "2024-02-24T09:00:00Z",
"endTime": "2024-02-24T09:00:03Z",
"status": "success",
"recordsReturned": 1234,
"executionTime": "3.2s",
"deliveryStatus": {
"email": "sent",
"webhook": "sent"
}
},
{
"executionId": "exec_xyz788",
"startTime": "2024-02-23T09:00:00Z",
"endTime": "2024-02-23T09:00:05Z",
"status": "success",
"recordsReturned": 987,
"executionTime": "5.1s",
"deliveryStatus": {
"email": "sent",
"webhook": "failed_retry_succeeded"
}
}
],
"statistics": {
"successRate": "98.5%",
"avgExecutionTime": "4.2s",
"avgRecordCount": 1050
}
}Pausing and Resuming
Temporarily disable schedules:
Navigate to your scheduled query in Automation > Scheduled Queries.
Click the Pause button to temporarily stop execution.
Add an optional note explaining why the schedule is paused.
Click Resume when ready to reactivate the schedule.
Query Optimization
Performance Best Practices
Optimize queries for scheduled execution:
- Use indexes - Ensure queries use appropriate MongoDB indexes
- Limit result sets - Use pagination or date ranges to limit data
- Avoid complex aggregations - Keep pipelines efficient
- Test execution time - Verify queries complete within timeout limits (default: 60 seconds)
Timeout Configuration Business
Adjust execution timeouts for long-running queries:
{
"schedule": "0 2 * * *",
"timeout": "300s",
"resource_limits": {
"maxMemory": "512MB",
"maxScanDocuments": 1000000
}
}Query Result Caching
Cache results to reduce database load:
{
"cache": {
"enabled": true,
"ttl": "3600s",
"shareAcrossUsers": true
}
}Cached results may not reflect real-time data. Only use caching for queries where slight delays are acceptable.
Error Handling
Retry Configuration
Configure automatic retries for failed queries:
{
"errorHandling": {
"retryPolicy": {
"maxAttempts": 3,
"backoff": "exponential",
"initialDelay": "1m",
"maxDelay": "30m"
},
"onFailure": {
"notification": {
"type": "email",
"recipients": ["admin@company.com"],
"subject": "Scheduled Query Failed: {{query_name}}"
},
"pauseAfterFailures": 5
}
}
}Failure Notifications
Get alerted when queries fail:
- Email notifications with error details
- Webhook to incident management systems
- Automatic pause after consecutive failures
- Detailed error logs for troubleshooting
API Access
Manage scheduled queries programmatically:
# Create a new scheduled query
curl -X POST https://api.mongodash.com/v1/scheduled-queries \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "Daily Report",
"query": {"status": "completed"},
"schedule": "0 9 * * *",
"timezone": "UTC",
"delivery": {
"type": "email",
"recipients": ["team@company.com"]
}
}'# Trigger immediate execution
curl -X POST https://api.mongodash.com/v1/scheduled-queries/sched_abc123/execute \
-H "Authorization: Bearer YOUR_API_KEY"# Get execution history
curl -X GET https://api.mongodash.com/v1/scheduled-queries/sched_abc123/history \
-H "Authorization: Bearer YOUR_API_KEY"Best Practices
Scheduling Strategy
- Off-peak execution - Schedule resource-intensive queries during low-traffic hours
- Stagger schedules - Avoid running multiple heavy queries simultaneously
- Use appropriate frequency - Don't over-schedule if data doesn't change frequently
- Consider time zones - Schedule reports for recipient time zones
Query Design
- Be specific - Use precise date ranges and filters
- Test thoroughly - Run queries manually before scheduling
- Handle empty results - Configure appropriate behavior for no results
- Document purpose - Add descriptions to help team members understand schedules
Monitoring
- Review execution logs - Check for failures or performance degradation
- Monitor delivery status - Ensure notifications are reaching recipients
- Track result trends - Watch for unexpected changes in result counts
- Set up alerts - Get notified of schedule failures
What's Next?
- Webhooks - Configure outbound webhooks for real-time events
- Data Sync - Synchronize data between collections
- API Integration - Build custom automation with the REST API