thoughts and observations of a privacy, security and internet researcher, activist, and policy advisor

Friday, March 20, 2009

Deep Packet Inspection: Reading List and Call for Papers

When I started my research project about the governance of Deep Packet Inspection (DPI) almost a year ago, there was basically no social-scientific or even political science literature about it. Some political reporting about it was done by specialized online sources like Ars Technica (hat tip to Nate Anderson for covering the issue so well and early), but all the academic literature on DPI was from some geeks publishing in computer engineering journals. Don't get me wrong, I love geeks, but sometimes they just get lost in the amazing technology options and forget about the political implications.

Times seem to change, and part of the reason for this is a more general awareness of this new technology and its powers. So, for all of you who want to understand more of the DPI debate, and who would be curious to find out how bandwidth management, ad injection, government surveillance, and internet censorship belong together and still often get different rules and regulations, here is a little reading list, in chronological order:
  • Christopher Parsons has published a working paper as early as 2008 for the New Transparency Project overseen by surveillance studies guru David Lyon. The paper is called "Deep Packet Inspection in Perspective: Tracing its lineage and surveillance potentials". Parsons argues that DPI equipment "should be identified as surveillance technologies that can potentially be incredibly invasive". He argues that ISPs "implicitly ‘teach’ their customers norms about what are ‘inappropriate’ data transfer programs, and the appropriate levels of ISP manipulation of customer data traffic."
  • Paul Ohm of the University of Colorado Law School was the first to make the link between the network neutrality debate and the unavoidable privacy invasions that come with any traffic discrimination approach: "The Rise and Fall of Invasive ISP Surveillance". A lengthy, but recommended legal paper that is a good read even for non-lawyers like me.
  • Ben Wagner presented a paper titled "Modifying the Data Stream: Deep Packet Inspection and Internet Censorship" at the 3rd Annual Symposium of the Global Internet Governance Academic Network last December.
  • Joseph Noel, a stock market analyst, has recently published an interesting analysis of the still emerging market for DPI gear. He is guessing that the FCC's decision last year is slowly making clearer where the rules for network management are going, and that this will break the "Traffic Management Deployment Logjam". His recommendations: Cisco Systems - Hold; Procera Networks – Strong Buy; SandVine Corp. - Buy; Allot Communications - Hold. I wonder about all the other DPI vendors, but I also wonder if he knows that the FCC's decision is still being challenged at the U.S. Court of Appeals (DC Circuit).
  • My own paper I presented at the International Studies Association's 50th Annual Convention in February is now available in an updated version: "Global technology trends and national regulation: Explaining Variation in the Governance of Deep Packet Inspection". I go through different use-cases and a few countries and try to explain the variation in DPI governance with the strategic actor setting shaped by each use case as well as with the institutional framework in which the governance debates took place. I also try to lay the groundwork for a "technology-aware policy analysis"-approach to internet governance studies (yes, feedback is welcome!).
  • Chris Riley and Ben Scott of Free Press, not really an academic institution but a lobbying think tank, just published a nice paper about the impact of DPI on Net Neutrality and ISPs' revenue considerations: "Deep Packet Inspection: The end of the internet as we know it?". A good provocative piece that points out potential "winners and losers" in the traffic management arms race (but hell - why did they steal my title?).
  • Nate Anderson again has already written a good summary of the Riley/Scott paper and put it into perspective: "This is the way the Internet ends: not with a bang, but DPI".
Of course, there is a lot more literature around on Net Neutrality, Internet Privacy and other related issues. But the fact that so few researchers have yet even mentioned Deep Packet Inspection or even systematically addressed it is also a sign that many of them are not really aware of the underlying technology trends here.

I would love to see more social-scientific, legal, and philosophical studies on DPI, e.g.
  • from a governmediality or "code is law" perspective, analyzing how the injection of DPI in our technology-mediated environment shapes the way we as Internet users can behave and which choices we have;
  • from a discourse-analytical perspective, tracing the discoursive frames and public perceptions around DPI;
  • from a governance perspective, explaining the variations in DPI governance and regulation from perspectives other than the "interaction-oriented policy analysis"-approach I used for my paper - hey, what about regulatory capture, agenda-setting, new modes of government, or plain old economic pressure?
  • with empirical data from beyond the U.S. or the english-speaking Western world (Wagner tries this, but the sources from China are limited so far);
  • with quantitative data on DPI usage by different ISPs in different countries, linking it with the regulatory and market environment and showing statistically significant links;
  • from a human rights perspective, making clear the possible conflicts of DPI with freedom of speech, freedom of assembly and freedom from intrusion (a.k.a. privacy) online;
  • edited to add: from a legal perspective, analysing the regulations for DPI and related technologies in different countries;
  • edited to add: [fill in your favourite social sciences / humanities / legal and related perspective here].
So, here is my pledge: If I get enough feedback and ideas for possible papers in these or other interesting directions, I promise to you that I will take the task of organizing a workshop or a conference where we can all meet and discuss wildly. How does this sound?


Anonymous Stef said...

Hiho, what about the legal perspective on DPI we just talked about? :-) Don't you want to have a contribution on that?

23/3/09 01:18

Blogger Ralf Bendrath said...

Just added it. ;-) (I assumed it was covered under the human rights perspective, but IANAL.)

This list is just a starter for further brainstorming anyway.

23/3/09 01:32

Anonymous Brett Glass said...

Ralf, your list of papers and resources is extremely one-sided. It seems to include only those authors and organizations which demonize ISPs and are calling for extremely onerous regulation of the Internet.If you consider only highly biased data, it's a foregone conclusion that your work could only result in more of the same and would not be likely to present a fair or reasonable view of the technology. Intellectual honesty demands that you talk to folks other than "astroturf" think tanks (such as Free Press, which carries water for Google) and "network neutrality" activists and lobbyists. Perhaps a real life network engineer or two? Or maybe an ISP who actually has to gather statistics, diagnose network problems, prioritize network traffic, and prevent network abuse?

23/3/09 04:18

Blogger Ralf Bendrath said...

Brett: This is the list of papers known to me that look on DPI from a (more or less) social-scientific / political science perspective. If you have other papers that should be added, I'd be more than happy to list them here, too.

I reject your (implicit) accusation that my own work is "extremely one-sided". My main focus is a neutral view on the political interactions that lead or have lead to DPI governance regimes, be they based on law or self-regulation. Of course I have spoken and listened to DPI engineers, ISPs as well as DPI vendors.

Have you read my paper at all before making these accusations???

23/3/09 16:23

Anonymous Matthias Bärwolff said...


I would suggest you look at the more recent papers from Dave Clark, one of "inventors" [sic!] of the end-to-end arguments, in order to get a more balanced and up-to-date view on the end-to-end arguments. From briefly scanning your 2009 paper I gather that you equate network neutrality and end-to-end arguments (based on the 1999 Lessig book), a common fallacy that Milton Mueller has commented upon back in 2007 (Net Neutrality as Global Principle for Internet Governance, It is rather curious that this myth is *still* an integral part of many rhetorics on network neutrality (see e.g. Lessig's statement at the 2008 FCC hearing).

In particular, the trust-to-trust paper is very good: Clark and Blumenthal (2007) The end-to-end argument and application design - The role of trust (

[W]hat the end-to-end argument asserts is that application-specific functions should be moved up out of the communications subsystem and into 'the rest' of the system. But the argument, as stated, does not offer advice about how 'the rest' should be structured. (p. 2)

I, for one, have come to regard DPI as but one means of injecting application layer functions in the network, and often a sensible one at that (think about network based detections of malicious attacks). After all, end points can (in principle) exert almost arbitrary control over the scope and depth of DPI (by encrypting and multihoming). Plus, given the ossification of IP, there is no imminent danger of the sky falling, at the network layer DPI will be transparent to the end points. Put differently, DPI does not break the IP protocol. And whenever they break upper layer protocols, the ISPs usually shoot themselves in the foot, in no small measure precisely because of end user driven measurement and user empowerment efforts such as nnsquad (see Comcast's TCP RST packets).

The solution is not in outlawing tussles but in providing mechanisms that promote competition, both amongst ISPs, and amongst application developers/providers.


23/3/09 21:02

Anonymous Brett Glass said...


Both the other papers and your own betray a deep, abiding prejudice against the technologies which those desiring to impose onerous regulation upon the Internet label, misleadingly, as "deep packet inspection." (This term is misleading for several reasons. First of all, packets are linear strings of bits; they have no "depth." Secondly, the term "inspection" implies human intervention rather than mechanical pattern matching which is done for the purposes of managing the network and its traffic.)

Your almost Luddite-like condemnation of the technologies you brand with this label is displayed by the fact that the abstract of your paper, right at the outset, states not with a question to be investigated but the following:

If rolled out widely, this technology known as deep
packet inspection (DPI) would turn the internet into something completely new...

In short, before having even discussed the issue, you have already jumped to a false and highly prejudiced conclusion.

You continue:

from the “dumb pipe” principle which Lawrence Lessig has so nicely compared to a
“daydreaming postal worker” who just moves packets around without caring about their

The fact is that the Internet never has been, and never could be, a series of "dumb pipes." The routers that keep the Internet running are, in fact, special-purpose supercomputers, each with more raw computing power than existed in the entire world two decades ago. This is precisely the opposite of "dumb." It is also noteworthy that you are not citing the opinion of someone who is knowledgeable about technology but rather Larry Lessig, who is on the Boards of Directors of two Washington DC lobbying groups which are lobbying, on behalf of Google, Inc., for regulation of the Internet. Hardly a creditable source, and yet you cite his unsupported opinions as gospel both in the abstract and in the body of the paper. It is also interesting that you downplay spying on Internet users by Google (as do Free Press, Public Knowledge, and other lobbying groups which are funded by Google) while demonizing ISPs.

You then continue to use language betraying a strong bias; for example, you state that "Comcast was caught" doing something wrong when in fact the FCC stated in its opinion that it did not know exactly what Comcast was doing!

But perhaps the most chilling element of both your paper and your blog posting is that you are talking about "governance" of a technology. Should we likewise restrict access to CPU chips, because they might be used for a purpose you (or someone ese) did not like?

You would also be much more credible if you included even a few sources who actually understood the technology and who did not condemn or vilify network management technology or ISPs. For example you might try">">


24/3/09 00:51

Blogger Ralf Bendrath said...

Matthias and Brett: Thanks for the links, I look forward to reading the texts behind them.

Matthias: Milton is my boss here at TU Delft, and there does not seem to be a problem on the NN / E2E front yet. ;-) I am looking empirically at how DPI is used, not trying to prescribe any best use-cases or NN definitions.

Brett: You seem to assume I am biased because I quote certain people, while I just try to establish the relevance of DPI there.

I also accept the empirical fact that technology is governed as well as that it has an impact on human life. I was mainly trying to understand why some ISPs use DPI in certain ways and others don't, and what kind of interest constellation and institutional framework may be an explanation for this.

Talking about bias: "DPI" is a common industry term, and it is used by vendors to market their products. If you see me biased because of this, I find it hard to respond.

Please take a step back and try to understand us political scientists, and read the paper again after letting off the steam.

24/3/09 01:39

Anonymous Brett Glass said...


Politicians and political "scientists" (I put the word in quotes because in fact they do not employ the scientific method and are not engaged in the discovery of scientific principles; they therefore are simply not scientists at all) have no business meddling in technology or science. (We have seen the horrible results which occurred when America's Bush Administration did so.) To govern technology -- i.e. knowledge -- is the most onerous kind of censorship, and it is what your paper and the papers in your reading list advocate. There is such an extreme bias in your writings and the ones that you recommend that I have serious doubts as to whether there is any hope of correcting it. I can only hope that I am wrong about this. I also hope that if your papers are peer-reviewed (a process which is common in the actual sciences but is not likely to be rigorous, if it is done at all, in non-sciences), your bias and the absence of any vaguely scientific support for your conclusions will be pointed out. In the event that you are not solely interested in carrying out a political crusade and would like to avoid coming across to your peers as an ideologue rather than any sort of serious academic, I'd be glad to converse with you offline about this topic.

24/3/09 03:47

Blogger Ralf Bendrath said...

Brett, I am afraid this discussion does not make much sense if you continue questioning my academic curiosity and the methods of social and political science in general.

Just a quick remark on governing technology: Mankind has always controlled what people were allowed to do with certain technologies, and how they have to use them. This goes back to the first arms control agreements over the crossbow, continues with rigid and legally reqired safety measures for power plants and other big machinery, and does not end with last mile unbundling regulations like we have in Germany, or higher taxes for cars that emit more carbon dioxide.

24/3/09 14:58

Anonymous Brett Glass said...

Ralf, mankind has always sought to prevent people from actively harming one another, via technology or any other means. But to outlaw technology itself is a travesty. Also, to compare a network monitoring tool to a weapon of mass destruction is silly.

Again, it appears that 90% of the sources you are citing are alarmist and full of misinformation. I certainly hope that you'll correct this.

25/3/09 01:05

Blogger Michael Zimmer said...

Ralf, count me in on the human rights/ethical perspective.

5/4/09 17:46

Blogger Samir Chopra said...

Ralf: Thanks for this page.

With reference to the little discussion thats happening on this page about the evils of regulation, I wonder why it is never appreciated that the combination of technology can be a regulator as well? Couldn't DPI itself be a form a regulation? Of the net, of its users?

Furthermore, the idea that only government can regulate, and private players can't, seems ludicrous to me.

5/4/09 19:40

Anonymous Danielle Citron said...

I would love to participate in the conversation as well! Danielle Citron

8/4/09 12:16

Anonymous Anonymous said...

Here is a hearing statement from the Center for Democracy & Technology on deep packet inspection.

15/4/09 20:33

Anonymous Anonymous said...

Here is recent testimony from the Center for Democracy & Technology addressing deep packet inspection.

On April 23rd, Leslie Harris, President and CEO of CDT testified before the House Energy and Commerce Subcommittee on Communications, Technology and the Internet telling the congressional panel that Deep Packet Inspection (DPI) technologies pose a serious challenge to privacy and the openness and innovation of the Internet. Because all applications of DPI raise serious privacy concerns owing to the interception and analysis that's done on all of a user's Internet traffic, policymakers must carefully consider each use of DPI and balance the perceived benefit against the risks to civil liberties, Harris said. CDT believes that only rare uses of DPI will be acceptable after such examination and then only with additional privacy safeguards including enactment of baseline consumer privacy legislation. At the hearing, Subcommittee Chairman Rep. Rick Boucher (D-VA) restated his intention to introduce a comprehensive consumer privacy bill this year.

24/4/09 15:41


Post a Comment

Links to this post:

Create a Link

<< Home