Advanced Filtering

Advanced filtering techniques enable you to construct sophisticated queries using MongoDB's full range of operators, expressions, and aggregation capabilities within filters.

Overview

Advanced filtering allows you to:

• Use complex operators - Leverage MongoDB's extensive operator library
• Compare fields - Filter documents by comparing field values
• Use expressions - Apply aggregation expressions in filters
• Combine conditions - Build multi-layered logical queries
• Optimize performance - Write efficient, index-aware queries

Query Operators

Comparison Operators

Standard comparison operations:

// Equal to
{ "status": "active" }
 
// Not equal to
{ "status": { "$ne": "deleted" } }
 
// Greater than
{ "age": { "$gt": 18 } }
 
// Greater than or equal
{ "score": { "$gte": 90 } }
 
// Less than
{ "quantity": { "$lt": 100 } }
 
// Less than or equal
{ "price": { "$lte": 50 } }
 
// In array
{ "category": { "$in": ["electronics", "computers", "phones"] } }
 
// Not in array
{ "status": { "$nin": ["archived", "deleted"] } }

Logical Operators

Combine multiple conditions:

// AND (implicit)
{
  "status": "active",
  "age": { "$gte": 21 }
}
 
// AND (explicit)
{
  "$and": [
    { "status": "active" },
    { "age": { "$gte": 21 } }
  ]
}
 
// OR
{
  "$or": [
    { "status": "active" },
    { "status": "pending" }
  ]
}
 
// NOT
{
  "status": { "$not": { "$eq": "deleted" } }
}
 
// NOR (not any of)
{
  "$nor": [
    { "status": "deleted" },
    { "isArchived": true }
  ]
}

Element Operators

Query field existence and types:

// Field exists
{ "email": { "$exists": true } }
 
// Field doesn't exist or is null
{ "deletedAt": { "$exists": false } }
 
// Type check
{ "age": { "$type": "number" } }
{ "tags": { "$type": "array" } }
 
// Multiple types
{
  "value": {
    "$type": ["number", "string"]
  }
}

Array Operators

Filter array fields:

// Array contains value
{ "tags": "featured" }
 
// All values must be in array
{ "tags": { "$all": ["featured", "popular"] } }
 
// Array size
{ "items": { "$size": 3 } }
 
// Array element matches
{
  "orders": {
    "$elemMatch": {
      "status": "shipped",
      "total": { "$gte": 100 }
    }
  }
}

String Operators

Pattern matching and text search:

// Regular expression
{ "email": { "$regex": "^user.*@example\\.com$" } }
 
// Case-insensitive regex
{
  "name": {
    "$regex": "john",
    "$options": "i"
  }
}
 
// Text search (requires text index)
{
  "$text": {
    "$search": "mongodb database tutorial",
    "$caseSensitive": false
  }
}

Field Comparison with $expr

Compare field values within the same document:

// Compare two numeric fields
{
  "$expr": {
    "$gt": ["$spent", "$budget"]
  }
}
 
// Compare with calculation
{
  "$expr": {
    "$gte": [
      "$quantity",
      { "$multiply": ["$minStock", 2] }
    ]
  }
}
 
// Compare dates
{
  "$expr": {
    "$lt": [
      "$expiresAt",
      "$$NOW"
    ]
  }
}
 
// String comparison
{
  "$expr": {
    "$eq": [
      { "$toLower": "$email" },
      { "$toLower": "$username" }
    ]
  }
}

Aggregation Expressions in Filters

Arithmetic Expressions

Perform calculations in filters:

// Discount percentage calculation
{
  "$expr": {
    "$gt": [
      {
        "$multiply": [
          { "$divide": [{ "$subtract": ["$originalPrice", "$salePrice"] }, "$originalPrice"] },
          100
        ]
      },
      20  // More than 20% discount
    ]
  }
}
 
// Total order value
{
  "$expr": {
    "$gte": [
      { "$multiply": ["$quantity", "$unitPrice"] },
      1000
    ]
  }
}

Date Expressions

Filter using date calculations:

// Documents created in the last 30 days
{
  "$expr": {
    "$gte": [
      "$createdAt",
      { "$subtract": ["$$NOW", 30 * 24 * 60 * 60 * 1000] }
    ]
  }
}
 
// Same month and year
{
  "$expr": {
    "$and": [
      { "$eq": [{ "$month": "$date" }, { "$month": "$$NOW" }] },
      { "$eq": [{ "$year": "$date" }, { "$year": "$$NOW" }] }
    ]
  }
}
 
// Business days calculation
{
  "$expr": {
    "$and": [
      { "$gte": [{ "$dayOfWeek": "$orderDate" }, 2] },  // Monday = 2
      { "$lte": [{ "$dayOfWeek": "$orderDate" }, 6] }   // Friday = 6
    ]
  }
}

String Expressions

Advanced string filtering:

// String length
{
  "$expr": {
    "$gte": [
      { "$strLenCP": "$description" },
      100
    ]
  }
}
 
// Substring match
{
  "$expr": {
    "$ne": [
      { "$indexOfCP": ["$email", "@example.com"] },
      -1
    ]
  }
}
 
// Concatenation
{
  "$expr": {
    "$eq": [
      { "$concat": ["$firstName", " ", "$lastName"] },
      "John Doe"
    ]
  }
}

Array Expressions

Filter using array operations:

// Array length
{
  "$expr": {
    "$gte": [
      { "$size": "$items" },
      5
    ]
  }
}
 
// Check if array contains value
{
  "$expr": {
    "$in": ["premium", "$features"]
  }
}
 
// Filter array elements
{
  "$expr": {
    "$gt": [
      {
        "$size": {
          "$filter": {
            "input": "$orders",
            "cond": { "$eq": ["$$this.status", "completed"] }
          }
        }
      },
      0
    ]
  }
}

Complex Query Patterns

Nested Conditions

Combine multiple logical levels:

{
  "$and": [
    { "status": "active" },
    {
      "$or": [
        { "plan": "premium" },
        {
          "$and": [
            { "plan": "free" },
            { "trialEndsAt": { "$gte": new Date() } }
          ]
        }
      ]
    },
    {
      "$nor": [
        { "isDeleted": true },
        { "isBanned": true }
      ]
    }
  ]
}

Geospatial Queries

Filter by location:

// Near a point (requires 2dsphere index)
{
  "location": {
    "$near": {
      "$geometry": {
        "type": "Point",
        "coordinates": [-73.97, 40.77]  // [longitude, latitude]
      },
      "$maxDistance": 5000  // meters
    }
  }
}
 
// Within polygon
{
  "location": {
    "$geoWithin": {
      "$geometry": {
        "type": "Polygon",
        "coordinates": [[
          [-73.99, 40.75],
          [-73.98, 40.75],
          [-73.98, 40.76],
          [-73.99, 40.76],
          [-73.99, 40.75]
        ]]
      }
    }
  }
}
 
// Intersects with area
{
  "serviceArea": {
    "$geoIntersects": {
      "$geometry": {
        "type": "Point",
        "coordinates": [-73.97, 40.77]
      }
    }
  }
}

Full-Text Search

Search across text fields:

// Basic text search (requires text index)
{
  "$text": {
    "$search": "mongodb tutorial beginner"
  }
}
 
// Exact phrase search
{
  "$text": {
    "$search": "\"exact phrase\""
  }
}
 
// Exclude terms
{
  "$text": {
    "$search": "mongodb -sql"
  }
}
 
// With relevance score
{
  "$text": {
    "$search": "database performance"
  },
  "score": { "$meta": "textScore" }
}

Performance Optimization

Index-Aware Queries

Write queries that use indexes:

// Good: Uses index on "status" and "createdAt"
{
  "status": "active",
  "createdAt": { "$gte": new Date("2024-01-01") }
}
 
// Bad: $where doesn't use indexes
{
  "$where": "this.status === 'active'"
}
 
// Good: Equality before range
{
  "category": "electronics",  // Equality first
  "price": { "$gte": 100 }    // Range second
}

Avoid Expensive Operators

Minimize performance impact:

// Expensive: Regex without anchor
{ "name": { "$regex": "john" } }
 
// Better: Anchored regex
{ "name": { "$regex": "^john" } }
 
// Expensive: $where
{
  "$where": "this.price > this.cost * 1.5"
}
 
// Better: $expr
{
  "$expr": {
    "$gt": ["$price", { "$multiply": ["$cost", 1.5] }]
  }
}

Limit Result Sets

Always use pagination:

// Include limit and skip
{
  "filter": { "status": "active" },
  "limit": 100,
  "skip": 0
}

Filter Validation

Schema Validation

Ensure filters match expected data types:

// Type validation
{
  "age": {
    "$type": "number",
    "$gte": 0,
    "$lte": 120
  }
}
 
// Format validation
{
  "email": {
    "$regex": "^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$"
  }
}

Error Handling

Handle invalid filter syntax:

Common Filter Examples

E-commerce Queries

// High-value abandoned carts
{
  "status": "abandoned",
  "$expr": {
    "$gt": [
      {
        "$reduce": {
          "input": "$items",
          "initialValue": 0,
          "in": { "$add": ["$$value", { "$multiply": ["$$this.price", "$$this.quantity"] }] }
        }
      },
      500
    ]
  }
}
 
// Products low in stock
{
  "$expr": {
    "$lt": ["$stock.quantity", "$stock.reorderPoint"]
  }
}

SaaS Metrics

// Churned users this month
{
  "subscription.status": "canceled",
  "$expr": {
    "$and": [
      { "$eq": [{ "$month": "$subscription.canceledAt" }, { "$month": "$$NOW" }] },
      { "$eq": [{ "$year": "$subscription.canceledAt" }, { "$year": "$$NOW" }] }
    ]
  }
}
 
// High-value customers
{
  "$expr": {
    "$and": [
      { "$gte": [{ "$size": "$orders" }, 5] },
      {
        "$gte": [
          {
            "$sum": {
              "$map": {
                "input": "$orders",
                "in": "$$this.total"
              }
            }
          },
          1000
        ]
      }
    ]
  }
}

Data Quality

// Invalid email addresses
{
  "$or": [
    { "email": { "$exists": false } },
    { "email": null },
    { "email": "" },
    { "email": { "$not": { "$regex": "@" } } }
  ]
}
 
// Duplicate detection
{
  "$expr": {
    "$eq": [
      { "$toLower": "$email" },
      "duplicate@example.com"
    ]
  }
}

Best Practices

Query Construction

1. Start simple - Begin with basic filters and add complexity incrementally
2. Test incrementally - Validate each added condition
3. Use query builder - Leverage autocomplete and syntax validation
4. Check indexes - Ensure filtered fields are indexed
5. Monitor performance - Use explain plans to understand query execution

Maintenance

1. Document complex queries - Add comments explaining logic
2. Save as views - Store frequently used complex filters
3. Version queries - Save iterations for rollback capability
4. Review regularly - Optimize queries as data grows
5. Share with team - Document and share query patterns

Security

1. Sanitize inputs - Validate user-provided filter values
2. Limit privileges - Use read-only connections when possible
3. Avoid injection - Don't concatenate user input into queries
4. Log queries - Track what filters are being executed
5. Set timeouts - Prevent runaway queries

What's Next?

• Custom Views - Save complex filters as reusable views
• Query Performance - Optimize query execution
• Aggregation Pipelines - Build complex data transformations