Getting enough network redundancy at all PSAPs in NG9-1-1

I often hear complaints that in many PSAPs, there isn’t any way to get enough redundancy of network connections to get a reliable ESInet. Often these are rural PSAPs, or larger ones where network diversity wasn’t even considered when siting the facility. I’m here to tell you that I think you CAN get a fair amount of redundancy, and thus availability, but you, and your vendors, have to be creative.

Clearly, if you have fiber coming into the building, then you start there. What you want, and should have as a top 5 priority in siting a new PSAP, is dual fiber entrances, where there are two ducts on opposite sides of the building that have fiber coming in, and no common path or element on those fibers. That’s tough to get in most rural areas, for sure. But one fiber is much better than no fiber.

But then you take a walk around the building and look up. See what kinds of wires you can spot. If there is copper from the local telco, then you want some: whatever high speed service you can get. But what you really want is more than one service, especially if there is a way to get those service from different COs. I’m not a believer in two of the same kind of service from the same vendor. If that’s all there is, then that’s all you can get, but I think you are better off with different services, and different vendors, or any combination. So, metro ethernet, yes please, DSL, yes, I’ll take it. Lowly T1s, yes, if the price is right. Anything that can get you a megabit or so is good, and more than one is better.

You definitely want a cable modem if at all possible, with enterprise service if you can get it, but whatever you can get from the cable company is great.

You also want as much wireless service as you can get: if 2 or 3 wireless carriers have service that works from your PSAP, get a wireless modem for each of them.

Then, go up on your roof and look out farther: see if there is fiber that is close to, but not in your PSAP. Could you put up point to point radio, microwave or laser, and pipe in some more bandwidth from another source or even just another CO? How far do you have to go to get service from another network supplier? Point to point wireless is getting pretty cheap. You may have some tower access that can help, and even if you need a repeater, that may get you bandwidth when your other connections go down.

I think 8 network connections to every PSAP is definitely not too many, and if there were 4 or 5 different technologies in that 8, that’s good. Sure, you want tens or hundreds of megabits but anything >1 megabit is helpful. One of them might be the “normal” connection that is what gets used most of the time. The others are there for when it dies.

What you may not realize is that IP networks have this very important characteristic: if there exists a path between two points: regardless of who provides the paths, how torturous they are, how many routers they go through, who owns the routers, or what technology underlies the IP connection, the path will be found and used. If there is more than one path, the one that appears to be “best” at the time a packet is sent will be used, and the route for one packet can be completely different from the route the very next packet takes. Network operators can, and often do, screw up routing to prohibit this kind of behavior, but they also can be instructed to let IP do its thing, and then you get data flowing if ANY path exists. Without prior planning. Without prior configuration.

You want that. You want lots of paths in the hopes that whomever you need to send packets to or receive packets from, there exists at least one working path, even in a disaster. That’s why IP network tend to be more reliable than other networks when disaster strikes. If a path exists, it will be automatically found and used.

However, as you are no doubt well aware, every network operator has problems, even the best of them. The recent outage at CenturyLink is an example. Many of their services went down at the same time. You should not look at that as a particular shortcoming of CL. CL is a fine network operator. Some version of what happened could and probably will occur on your prime network operator some day. You really, really, really don’t want to be dependent on one vendor to keep your ESInet and your PSAP running.

Now, to be sure, your NGCS vendor may want to run VPN connections on each of your paths to the ESInet. That’s fine. The underlying IP networks will work the way we want them to.

But there is another part of this that I wish the NGCS operators and the originating service providers would work out. If a call comes from one of the networks that you have a connection to, and that network works well enough that packets can flow from the caller to the PSAP, then, no matter what, I want that call to be able to be answered by you. Most ESInets won’t do that. They need a path from the caller to the NGCS and from the NGCS to the PSAP, and they won’t allow calls to go direct, even when there is no other path. Most of them insist on putting the ESInet’s BCF in the media path, which makes the network connection requirements in disasters even less likely to work. On top of that, the originating service providers often have very restrictive network configurations that wouldn’t allow the kind of path I’m advocating, but I think they should have.

I’m not big fan of “sub-IP” mechanisms like MPLS, and I say that as someone who was present at the beginning of MPLS. It’s not that I think it’s bad, it’s actually great. It’s that the benefits doesn’t justify the cost. I’d rather see your dollars going into more diversity rather than MPLS. One thing to recognize: in many networks, if you get something like metro Ethernet, that service is often built on top of MPLS. You don’t get all the benefits of having dedicated label switched paths, but you get most of them, and metro Ethernet tends to be the least cost way to get bandwidth when there is some decent sized pipe. There is some experience that suggests that at least in some networks, MPLS fails in disasters where basic IP networks keep working. That often has to do with how the MPLS network is configured rather than some inherent defect in the technology.

I expect that most ESInet designers will look at this piece and think I’m nuts. They want to bring two MPLS connections from the same network provider to your PSAP and be done. They would say that trying to deal with 8 different network connections from 7 providers is so much work that it’s not feasible. They would say that each of those connections doesn’t have anywhere near the reliability of their MPLS connection. They will tell you they can’t manage those connections. They would be wrong, of course. Diversity is achievable. Even in the boonies.

Leave a comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.