Aggregation Pipelines

Build powerful data transformation and analysis pipelines with MongoDB's aggregation framework.

Overview

Aggregation pipelines process documents through a sequence of stages, transforming and analyzing data at each step.

Pipeline Stages

Common aggregation stages:

$match

Filter documents (like find queries):

{
  "$match": {
    "status": "active",
    "age": { "$gte": 18 }
  }
}

$group

Group documents and calculate aggregates:

{
  "$group": {
    "_id": "$category",
    "total": { "$sum": "$amount" },
    "count": { "$sum": 1 },
    "average": { "$avg": "$rating" }
  }
}

$project

Reshape documents and compute fields:

{
  "$project": {
    "name": 1,
    "fullName": { "$concat": ["$firstName", " ", "$lastName"] },
    "age": { "$subtract": [2024, "$birthYear"] }
  }
}

$sort

Order documents:

{
  "$sort": {
    "createdAt": -1
  }
}

$limit

Limit number of documents:

{
  "$limit": 100
}

$lookup

Join collections:

{
  "$lookup": {
    "from": "orders",
    "localField": "_id",
    "foreignField": "userId",
    "as": "userOrders"
  }
}

Visual Pipeline Builder

Team

Build pipelines visually:

Stage Preview

See intermediate results after each stage:

• Document count at each step
• Sample output documents
• Stage performance metrics

Common Pipeline Patterns

Group and Count

Count documents by category:

[
  {
    "$group": {
      "_id": "$category",
      "count": { "$sum": 1 }
    }
  },
  {
    "$sort": { "count": -1 }
  }
]

Top N Results

Find top 10 items by value:

[
  {
    "$sort": { "revenue": -1 }
  },
  {
    "$limit": 10
  }
]

Calculate Running Total

[
  {
    "$sort": { "date": 1 }
  },
  {
    "$group": {
      "_id": null,
      "transactions": {
        "$push": {
          "date": "$date",
          "amount": "$amount"
        }
      }
    }
  },
  {
    "$unwind": {
      "path": "$transactions",
      "includeArrayIndex": "index"
    }
  }
]

Saving Pipelines

Save pipelines for reuse:

1. Build your pipeline
2. Click Save Pipeline
3. Name and describe the pipeline
4. Access from Saved Pipelines sidebar

Pipeline Performance

Optimization Tips

1. $match early - Filter documents before expensive operations
2. $project early - Remove unneeded fields
3. Use indexes - $match and $sort can use indexes
4. Limit results - Add $limit when appropriate

Performance Metrics

View pipeline performance:

• Execution time per stage
• Documents processed per stage
• Index usage
• Memory consumption

What's Next?

• AI-Assisted Queries
• Query Performance