2 October 2015

Some comments of mine on Twitter (@gmclwill) regarding NBN Co’s  announcement of an RFI  for Deep Packet Inspection (DPI) technology got picked up in some press this week.

So I thought I would put some deeper analysis online to explain my concerns. This is not meant to be critical of NBN Co. Hopefully it will be seen as some helpful feedback to NBN Co and the broader NBN community to ensure all perspectives have been considered.

First of all, what is DPI technology? This technology is widely deployed by internet service providers to help monitor and manage traffic travelling over their networks. It allows the service provider to inspect the packets carrying the internet data and “see” what is in them so as to understand and control what applications are being used (such as web surfing, video streaming, voice calls, peer to peer networking etc).

The technology can also be used for surveillance and recording of these packets. Obviously such surveillance and recording must be permitted by the laws of the land but I won’t be getting into that topic in this post.

The NBN RFI stated the purpose was essentially for “collection of statistics”, “enforcement of … policies” and “automatic triggering of policies”. In summary NBN appear to be looking at DPI technology in order to understand the application traffic on their network and apply different policies based on these applications.

The use of DPI technology has come under much scrutiny because of the debates globally over “Net Neutrality”. This is a concern for many internet users because of the fear that service providers will create “fast lanes” and “slow lanes” for different applications on the public internet based on their commercial needs. The United States communications regulator has recently created new rules after years of debate that seek to prevent this type of behaviour by internet service providers.

Net Neutrality in itself is a deep discussion and I have written about it previously and why Australia has not experienced the sometimes emotional debate that has occurred in other markets. In summary most Australians pay a higher price if they download more data. As a result service providers encourage more data usage to earn more revenue and are not motivated to slow or restrict content. This is especially important if networks are constrained in the amount of data they can carry (eg. mobile networks or legacy non-fibre networks) – the extra fee encourages more investment that brings a return to the service provider and enables better networks to be built, ultimately benefiting end users.

But what about the NBN?

NBN Co, the company building the underlying network infrastructure, is a wholesale only provider of services. It cannot, by law, sell directly to end users. Instead it sells to other service providers who generally provide retail services to end users – so called Retail Service Providers (or RSPs). But despite this NBN Co still has a pricing mechanism that ensures that when more data is transmitted over the network NBN Co receives more revenue. This is done through a Connectivity Virtual Circuit (CVC) that is effectively a virtual pipe that connects RSPs to the NBN. If the end users of an RSP download more data then this pipe needs to expand and NBN Co receives more revenue.

The price of this CVC is controversial but the principle is sound. NBN Co does have higher costs as data usage grows. In the fibre parts of the network this is limited to the electronics that switch and route the data through the network. More data, means more switches means more investment.

So with this in mind why does NBN Co need to deploy DPI technology? Why would they be looking to restrict or slow certain applications?

The reason is that NBN Co is deploying some technologies that require a large step increase in investment to provide more capacity as downloads increase.

This is particularly the case for the satellite and fixed wireless networks. If users download more data then the radio based parts of these networks will become overloaded and congestion will occur. The only way to overcome this is to build more satellites or install more fixed wireless towers – both of which are very expensive. These extra large step changes in investment are something service providers seek to avoid until they absolutely can’t delay any longer. And NBN, being a monopoly, can probably delay this longer than other service providers.

So how should a company like NBN Co manage this situation where congestion is likely. How should it share the limited resource, in this case the radio spectrum amongst the end users, remembering that NBN Co is a wholesale only company. This is where the CVC can also help. NBN Co can put a cap on the size of the total CVC size to ensure that the total bandwidth available through the bottleneck resource (ie. the radio spectrum) is matched to the total amount of CVC bandwidth available to RSPs. This is necessarily an approximation because CVCs do not match on a one for one basis every bottleneck resource (ie. every spot beam in the satellite and every fixed wireless tower). There is actually a benefit in aggregating multiple bottlenecks due to the statistical nature of packet traffic but that involves deeper traffic engineering to explain.

In essence this transfers the bottleneck problem to the RSPs. Some might say this is a cop-out. I would say that the RSPs are in the best position to manage this in a wholesale model. RSPs have to deal with this congestion problem in their own networks anyway (ie. providing sufficient bandwidth to the global internet). Congestion is best managed from an end to end perspective and RSPs are best placed to do it because of the wholesale model.

RSPs also control the end user price. They have the ability to charge higher prices for higher quality (ie. less congested services) and lower prices for more congested services. They can also provide different service levels to different end users. For example, business end users can be charged more for higher quality than consumers. Different CVCs from NBN Co can be used with different ratios of end users to bandwidth for different customer segments. And most importantly they can charge on a variable basis depending on how much data is downloaded through different plans (ie. GBytes per month).

RSPs can also use DPI technology to preference certain applications if that is what is preferred by their customers. For example, RSPs may wish to market particular service plans that give video traffic preference rather than normal web surfing or peer to peer networking. Other innovations will also be encouraged to find ways to reduce the impact of congestion using different technologies.

Lastly, but most importantly, end users have a choice between RSPs as to which ones manage the congestion problem better. If customers are not happy they can change to a different service provider. At the heart of the NBN model is the principle that retail competition should still encourage innovation and increased customer service.

NBN Co is not able to differentiate services or offer choice. It must offer the same service at the same price to all RSPs. This ensures fairness in competition at the retail level and is a fundamental principle of the wholesale NBN model.

NBN Co has clarified that the DPI technology is aimed for use in the new satellite network. To some extent NBN Co already has plans to deploy a form of DPI in the satellite network to minimise delay or latency of packets travelling back and forth to the satellite. This technology called Transparent Performance Enhancement Proxy (TPEP) looks into the packets and pre-fetches internet data before the end user requests it so the data is in the end users satellite terminal (see NBN Co Network Design Rules page 30). But as you will see it is all about improving web surfing latency and not prioritising certain applications.

The big danger is that, if implemented, a DPI tool will become entrenched and used to preference certain applications. NBN Co will be effectively deciding which applications should have preference and which applications should be restricted. All users on the NBN, irrespective of their RSP, will necessarily be subject to the same preferences and restrictions. As a result innovation and end user choice at the retail level is restricted and competition is minimised.

Congestion in satellite and fixed wireless networks is likely given the limited radio spectrum resources. It may also occur in other networks, especially the Hybrid Fibre Coax (HFC) networks being deployed as the HFC network essentially also uses radio spectrum shared by many users to deliver broadband.

In my opinion the use of DPI technology by NBN Co will create a precedent for more control of the end user service experience by NBN Co than is necessary. This restricts the ability for competition and innovation at the RSP level to more efficiently address congestion where it does occur in the NBN wholesale model.

If DPI technology is implemented this will mean less ability for RSPs to differentiate and compete on anything other than price for a “vanilla” NBN Co service. NBN Co will essentially be packaging up a retail product that the RSP can resell rather than supplying a wholesale product for the RSP to add value on top of. This is very different to the originally envisaged NBN wholesale model.