Mongoose: Sorting, Pagination, and Filterting

Taken by Nikolas

One things that I’ve always thought about as I scroll through my Instagram feed is, how is this information so endless. As I build out my own application, I realized that this isn’t a new feature. It’s been there since the beginning. When we google something, we get millions of results and yet it’s only showing the ten most relevant articles to our search on the first page.

In my journey to create an app that feels as real as any other application, I started to search how people only show the first couple of products as oppose to the entire library.

The Problem

I’m creating a task manager application where a user has many tasks. Each task has two attributes, a description (string) and a completion(boolean). Now the issue occurs when I think about my user using this application for years and potentially having hundreds of tasks. How do I give my user the ability so see just a small portion of it and how would I give them control of their search parameters.

The Solution

So right now I have an endpoint that will show the user all of their tasks. Inside of that endpoint, I’d like to provide some additional information to the query. Something like ‘GET /tasks?completed=true’

Filter

router.get('/tasks', auth, async (req,res) => {
const match = {}
if(req.query.completed){ 
   match.completed = req.query.completed === 'true' 
}
try{
   await req.user.populate({
    path:'tasks', 
    match,
    options:{
       limit: parseInt(req.query.limit),
       skip: parseInt(req.query.skip),
       sort
  }
}).execPopulate()
   res.send(req.user.tasks)
}catch(e){
   res.send(e).status(500)
}
})

Let’s break this down. The information that we want will be found inside our query. Based off of that information, we’d like to use that query to get those task. The first thing we do is create an object called match. If ‘completed’ is part of the query (accessed through req.query.completed) then we’d like to append that key value pair to the match object we created.

In mongoose, I’m using the populate and execPopulate methods instead of querying my database again. The populate method takes an object where we use the match object we created. That tells mongoose, “Hey give me all the tasks that match the completed attribute from the query.

Ex: Get/tasks?completed=true
req.user.tasks = [
{'description': 'Create a medium blog', 'completed': true},
{'description': 'This is an example task', 'completed': true}]

Great, we’re getting back a filtered array of tasks while giving our user the ability to do so.

Sort

Sort will follow a very similar procedure. The sort information will be given to use by the query. We’ll start off by making our object of sort again and another conditional that checks if ‘sortBy’ is part of the query. If it is, we store the information given to us by the query to our sort object where the value is either -1 for descending or 1 for ascending. Just like last time, we’ll use the sort object inside of out populate() method, specifically, inside of our options object which says “Hey, give me all my tasks in ascending or descending order. Take a look at the code below!

const sort = {}
if(req.query.sortBy){
   const parts =  req.query.sortBy.split(':')
   sort[parts[0]] = parts[1] === 'desc'? -1: 1;
}
try{
   await req.user.populate({
      path:'tasks', 
      options:{
         limit: parseInt(req.query.limit),
         skip: parseInt(req.query.skip),
         sort
      }
   }).execPopulate()
   res.send(req.user.tasks)
}catch(e){
   res.send(e).status(500)
}
})
Ex. GET/tasks?sortBy=createdAt:desc
output // [
{description:'Created Last', completed: true},
{description:'Created Second', completed: false}, 
{description: 'Created First', completed: true}]

Now we’ve given our user the ability to get their tasks based on the date their task was created. They can either see their newest information first, or in the order they were created in.

Pagination

Pagination is the idea of showing small snippets of a larger piece. Like I mentioned, google paginates your search so that you don’t see the millions of links that match your search. Instead they only show the first 10 and allow you to check out the next 10 on the next page.

Similar to the last two methods of arranging information, we’ll be getting our information from the query again. This time from two parameters called limit and skip.

Limit is the amount of instances you’ll see in a page. A limit of 3 will only show us a maximum of 3 tasks from the response.

Skip is the amount of tasks you’d like to skip before sending back a response. So you can say skip = 3 which will send a response where the task begins on the fourth task from the database.

Combining these two will allow us to paginate. For example. limit =3 and skip = 3 which is saying give me three tasks at a time, but skip the first three… showing me the next three aka our second page.

try{
await req.user.populate({
   path:'tasks', 
   options:{
      limit: parseInt(req.query.limit), 
      skip: parseInt(req.query.skip),
      sort
   }
}).execPopulate()
   res.send(req.user.tasks)
}catch(e){
   res.send(e).status(500)
}

Notice the new information appended to the options object. We parse the information from the query because everything we pull from the query is a string and this needs to be an integer.

Ex 1. Get/tasks?limit=2&skip=2
//This is essentially saying, give me tasks 3 and 4!
output// [
{description:'This is task 3',completed: true},
{description: 'this is task 4', completed: false}]
Ex2. Get/tasks?limit=3&skip=6
output// [{description:'This is task', completed:true},
{description: 'This is task 8', completed: true},
{description: 'This is task 9', completed: false}]

And there we have it! Three ways of giving our user the ability to filter through the tasks that they are interested in. I hope you enjoy this walk through and understand the idea more so than the code itself as the same ideas are translated for different languages. Cheers!