[bjohansebas]

There is a real need to be able to list routes correctly, as clearly shown in issue (AlbertoFdzM/express-list-endpoints#99, expressjs/express#5961, expressjs/express#5858, expressjs/express#6481, https://github.com/wesleytodd/express-openapi/blob/main/lib/generate-doc.js#L16-L74, #122).
Additionally, I’ve been chatting with a Vercel employee who wants to make it possible to deploy Express servers on Vercel without any special configuration, the only blocker for them so far is that they can’t retrieve a list of the application's routes.

Previously, before the upgrade to path-to-regexp@8 (#117), it was possible to access the regex used for routing. However, this was replaced by the matcher property, which returns a function that extracts the path and parameters. This approach adds extra processing overhead, as it transforms a regex into a function—something unnecessary for such a simple goal. That said, it could still serve as an option for those attempting to list routes.

Currently, route layers have a path property, but it's initialized as undefined because the actual path is only resolved at runtime. This is due to the fact that req.url.pathname isn't defined until the request is received. Once the request arrives and the pathname is parsed (as shown in

router/index.js

Line 237 in 3d574ed

match = matchLayer(layer, path)

), the path property in the layer is then set with the correct value.

Proposed Solution

The solution would be to add a new property to the layer that gets initialized at the time the layer is created, using the path it was registered with. This would allow us to reliably retrieve the original path and make it possible for external tools or modules to list routes correctly.

Additionally, I'm proposing a new function that exposes the current stack's route definitions. This would reduce the need for third-party dependencies,or at least make it easier for those dependencies to support the new version of Express without relying on brittle workarounds.

Examples:

const router = new Router({ strict: true, caseSensitive: true })
const admin = new Router({ strict: true, caseSensitive: false })

admin.use((req, res, next) => {
  // some middleware for admin routes
  next()
})

admin.get('/', (req, res, next) => {
  res.end('Hello')
})

router.use("/admin", admin)

router.all('/:id', function (req, res) {
  res.end('Hello')
})

console.log(router.getRoutes())
// [
//   {
//     name: 'router',
//     path: '/admin',
//     methods: undefined,
//     keys: undefined,
//     router: [
//       {
//         name: 'handle',
//         path: '/',
//         methods: ['GET'],
//         keys: undefined,
//         router: undefined,
//         options: { strict: true, caseSensitive: false, end: true },
//       }
//     ],
//     options: { strict: true, caseSensitive: true, end: false }
//   },
//   {
//     name: 'handle',
//     path: '/:id',
//     methods: ['_ALL'],
//     keys: [{ name: 'id', type: "param" }],
//     router: undefined,
//     options: { strict: true, caseSensitive: true, end: true }
//   }
// ]

If this were to be released, in Express the function would be used like this:

import express from 'express'

const app = express()

app.get('/', (req, res) => {
  res.send('Hello World')
})

console.log(app.router.getRoutes())
// [
//   {
//     name: 'handle',
//     path: '/',
//     methods: ['GET'],
//     keys: undefined,
//     router: undefined,
//     options: { strict: true, caseSensitive: true, end: true },
//   }
// ]

Why not reverse-engineer the path?

In Express 4, there were some hacks based on inspecting the generated regular expressions. But these approaches aren't solid or future-proof. If path-to-regexp changes the regex structure, those solutions break. So it's not a sustainable or effective approach.

Furthermore, there are plans to reintroduce some features to path-to-regexp (pillarjs/path-to-regexp#380, pillarjs/path-to-regexp#379), so it's highly likely that the internal regex structure will change again. On top of that, unlike in Express 4, the generated regex is no longer directly accessible.

close expressjs/express#5961, close expressjs/express#6481, close #122).

[jonchurch]

We need clarification on the exact requirements. Can you have them open a feature request?

There are different things that could be exposed, which of these is the request?

1. what routes did a user register?

• e.g. router.get('/users/:id') returns { path: '/users/:id', methods: ['GET'] }
• This is what most of the linked issues want, for documentation/debugging purposes
• This is what the current implementation in PR provides

2. what will match what the user registered?

• e.g. with strict: false (default) /users matches /users and /users/, with strict: true it matches only /users
• e.g. with caseSensitive: false (default) /Users matches /Users and /users, with caseSensitive: true it matches only /Users
• This is necessary for traffic routing purposes, and what Im assuming vercel wants to do, route paths into the express app without passing through all traffic (assuming here without more info)
• The implementation does not cover this

If the ask is number 2, they would need to decide if they apply the routing rules themselves, tell users which options are supported, or we'd need to enhance the ouput to include either router options or otherwise take options into account.

(v good PR description ❤️)

Edit: More importantly, we'd want to answer this for ourselves if we add this feature. If folks depend on this they'll need to know what information is actually conveyed by the returned mapping. Strings they used, or strings router will recognize.

Current approach flattens the whole stack, so we are losing the nuance that different routers can be configured differently. Sweet spot between the two options could be expressed in output as:

  [
    {
      path: '/api',
      methods: ['GET'],
      options: { strict: false, caseSensitive: false }
    },
    {
      path: '/v1/users',
      methods: ['GET'],
      options: { strict: true, caseSensitive: true }
    }
  ]

[jonchurch]

#174 (comment)

[bjohansebas]

@jonchurch thanks for the quick review! Yes, what they shared with me (and I also asked them to comment here about their needs) is that they want to be able to see the routing structure and then map it to their Build Output API (https://vercel.com/docs/build-output-api).

In my opinion, the solution you're proposing (combining both outputs) is also the right one, as it covers those needs as well as other use cases or certain hosting providers. Plus, it's flexible enough for developers to adapt it to their specific requirements.

[
    {
      path: '/api',
      methods: ['GET'],
      options: { strict: false, caseSensitive: false }
    },
    {
      path: '/v1/users',
      methods: ['GET'],
      options: { strict: true, caseSensitive: true }
    }
  ]
``` ref https://github.com/pillarjs/router/pull/174#issuecomment-3129927730

[wesleytodd]

I left a few comments, but I think we should workshop this just a bit. I think ideally we don't need to do this work when we call routes() but instead we can keep internal track of these in such a way that they are much more easily operated on. That would likely only be possible in the next major, but if we can design this api such that it is our "ideal internal book keeping structure" then we can ease that transition in the next breaking change.

So I think my ask is that we add some usage examples in the main text of the PR (or maybe in the readme as docs in this pr) and then we can propose some small changes to the exposed api that way before we invest too much more work in the internals and tests?

EDIT: also, I was thinking that maybe one good pressure test for the api would be if I tried to integrate it into my openapi generator package. If it works well for that, I think it would work well for all the similar use cases. If we can organize this work I am happy to make it a priority.

[ctcpip]

I have thoughts on this PR, but won't have time to review for a bit. If it's urgently needed, I don't want to be the blocker, but otherwise would like time to review. 🙏

[bjohansebas]

I have thoughts on this PR, but won't have time to review for a bit. If it's urgently needed, I don't want to be the blocker, but otherwise would like time to review. 🙏

It's not entirely urgent, so feel free to do the review whenever you can.

[blakeembrey]

This looks like a great starting point, love it! @jonchurch has some really good feedback to think about here too, re what Vercel may need.

Can Vercel handle the regex support?

[blakeembrey]

For .use, could it make sense to allow consumers to iterate recursively themselves? It's something that could be added in a follow up, but the MVP could be simply { path, method, router }. Every field could also be optional, I guess, since path: undefined with .use(fn), method: undefined when all is used, and router: undefined when no nested router.

[bjohansebas]

The path will never be undefined .use always sets the path to '/' when no path is explicitly provided in the arguments.

[blakeembrey]

That's the kind of thing that should be nailed down for the API, but understood. It probably is reasonable to keep it as / and the object was intended to be hypothetical.

[blakeembrey]

E.g. why / vs ""? Does one make it harder for consumers than the other? How do these interact with internal routing behaviors that aren't being exposed in this API? How much can move these expectations/behaviors to be static instead of magic (e.g. removing the trailing / is the one that comes to mind, people need to know how the package works internally).

[wesleytodd]

Agreed here, lets just simplify this down into the bare minimum api before merging this PR. Then we can take on nesting routers and deeply iterating the stack more directly later if we deem it worth it. We want to unstuck this progress, and I think these details is what I got hung up on last time leading to my original blocking review. Better to land what we can for sure agree on adds value and move this forward.

[IamLizu]

Mostly looks good to me now. However, I wonder if there's possibility of infinite recurrsion at collectRoutes.

Imagine a user acceddientally creates circular route references and then calls to getRoutes would go on forever.

Perhaps there should a protection, maybe check if the current layer is visited in the current branch. Here's a possible sample code,

function collectRoutes (stack, prefix, routeMap, options, visited = new WeakSet()) {
  for (const layer of stack) {
    if (layer.pathPatterns && layer.handle && layer.handle.stack && !layer.route) {
      // Check for circular reference
      if (visited.has(layer.handle)) {
        continue  // Skip circular reference
      }
      
      visited.add(layer.handle)
      collectRoutes(layer.handle.stack, pathPrefix, routeMap, options, visited)
      visited.delete(layer.handle)  // Allow re-visiting in different branches
    }
  }
}

Other than this, I think getRoutes is awesome feature to the router.

Good work @bjohansebas 👏

cc: @blakeembrey @wesleytodd @jonchurch

Update

Just to be safe, @bjohansebas can we add more complex and nested routes in the test often found in real world scenarios to be safe?

[blakeembrey]

Note: options will need to contain end: boolean for .use routes, which are prefix matched vs fully matched.

[nronnei]

@bjohansebas you're my hero; eagerly awaiting this change.

[Megapixel99]

Thank you for your work on this pull request. While I was working on: wesleytodd/express-openapi#77, I had to rely on the changes in this PR to get the full path of the routes.

Would it be possible to expose the middleware stack of each route within getRoutes()? Currently I have to get the middleware stack directly from the router (https://github.com/wesleytodd/express-openapi/pull/77/files#diff-ffcbf454c1de79602c5a1cbbc810c181b8d8552f743952d9049ffc13ca8b1f1eR31-R33) despite using getRoutes() to get the full path of the URL earlier in my code (https://github.com/wesleytodd/express-openapi/pull/77/files#diff-ffcbf454c1de79602c5a1cbbc810c181b8d8552f743952d9049ffc13ca8b1f1eR12)

[bjohansebas]

Yep, that could be added in my opinion. I need to partially modify this PR to adapt it to the idea in #174 (comment), I have it in my priorities, so you’ll need to adapt your PR to that format once I make the change.

[jeffsee55]

👋 we (Vercel) have just added an experimental feature in our Express builder that allows us to extract metadata from the Express app like routes, static assets folders, and view rendering engines so we can build Express apps with first-class features like o11y out of the box.

Here's the initial PR with more details, but at the moment the route metadata is very basic (doesn't capture sub-routers) so I wanted to check in on this PR and ask if there's anything we can do to make route introspection better.

You can try it out by using this repo, there's a Vercel deploy button in the README.

[wesleytodd]

Explicitly removing my blocking review. We need to address @blakeembrey's feedback on simplification before landing, but with my limited availability I don't want to be the one blocking the merge.

Added a few nitpick comments, but once we simplify it down I am good to merge and publish.

[wesleytodd]

Agreed here, lets just simplify this down into the bare minimum api before merging this PR. Then we can take on nesting routers and deeply iterating the stack more directly later if we deem it worth it. We want to unstuck this progress, and I think these details is what I got hung up on last time leading to my original blocking review. Better to land what we can for sure agree on adds value and move this forward.

[fed135]

Hey all! I've been working on express plugins, often needing to monkey-patch internals to get all the routes in a router. I had a different approach to this problem: expressjs/express#7321

I feel that this PR here addresses it in a much cleaner fashion. I'd close mine as redundant but there's one part that is missing in the returned objects for getRoutes: the handler stack. This would be super useful to have, even as a Frozen/Immutable.

For reference, the object I'm emitting in my implementation is the plain Route object as-is.

Route {
    path: '/:foo',
    stack: [ [Layer] ],
    methods: [Object: null prototype] { get: true }
  }

Would there be openness to adding the stack property?

[bjohansebas]

Okay, I had overlooked this. I've added it to my backlog of things to finish

[bjohansebas]

Okay, here are the changes I've made so far and how the API behaves now.

each Layer stores the path it was
registered with in rawPath for arrays, a flat(Infinity) that both
copies (immune to later mutations of the caller's array) and unnests. This
is the only cost added to the hot path: one assignment per layer.

1. Route (layer.route is set) → methods = route._methods(): the
   registered methods in uppercase, including the automatic HEAD for
   GET routes, with the _all sentinel filtered out by
   Route._methods() itself. If the result is empty and the route was
   registered only with .all() → undefined ("matches all methods");
   if it is empty without .all() → [] (a route with no handlers).
2. Mounted router (Array.isArray(handle.stack)) → an entry with
   router: handle and methods: undefined. No recursion: consumers
   recurse themselves by calling router.listRoutes() when available —
   which eliminates the cyclic-router problem by design, and the
   shape-based detection also lists routers from older versions of this
   package.
3. Plain middleware → skipped.

Output: one { path, methods, router } entry per path (array paths
are expanded, each with its own copy of methods)

router.get('/test', h1)
router.get('/test', h2)
router.post('/test', h3)

router.listRoutes()
// [
//   { path: '/test', methods: ['GET', 'HEAD'], router: undefined },
//   { path: '/test', methods: ['GET', 'HEAD'], router: undefined },
//   { path: '/test', methods: ['POST'], router: undefined }
// ]

[logaretm]

Got a few nits/questions only.

[UlisesGascon]

Let's give some time for others to review, but I think the change is solid 👍