5 Aug 2011

Best efforts

Author: will | Filed under: best invention, cloud computing, domain, engineer, engineering, LBC, programming, science, software, technology, web standards

The internet is based on unintended and uncertain consequences. And that’s in its design.

Now I’m not talking about how the web escaped from the really useful tool in CERN. I’m not talking about how the parent of the internet, ARPANET is a direct consequence of the assassination of Archduke Franz Ferdinand.* (Mainly because it isn’t++.)

The internet, or rather the Internet Protocol on which the internet runs is based on three concepts:
1) Packet routing; moving small chunks of information around at a time
2) Protocol hierarchy; a careful and clear organization to the structure of the data that moves across the Internet (mostly through Babushka doll like wrappers)
3) Best effort is good enough, which means living with unintended consequences.

At the time, best effort was the exact opposite of how things were done. The network of computers at the time was based on point-to-point circuit-based communications. That meant a dedicated (if sometimes temporary) “physical” connection 100% of the time while connected. This is very similar to the old telephone system, there had to be an actual phone to phone connection for the length of the call. There may be an uplink and downlink satellite connection (or more) along the way, but it was a dedicated connection, and there were a few colleges that have dedicated pipes between them. This dedicated connection granted one thing; 100% guaranteed transfer of data.

There were no packets, routing information wasn’t needed once the connection was established, and every bit was acknowledged since the dedicated connection meant no congestion.

But it was very, very expensive. You needed to have enough modems (remember them) for every call you expected to need at any one time. TCP/IP meant connections could be shared, as the data was split up and routed through, so you needed fewer modems, or routers.

Other network protocols have packets and embedded routing, but best effort is the scary part, because its designed imperfection.

Best effort means firing off the packets dressed in their routing best, and not keeping an eye on them. It means the receiver of the information does not send an acknowledgement for each packet. This actually makes sense, because data is bursty.

While there is data flowing somewhere on the net, some routers are quieter than others, and sometimes a router gets more packets in than its able to send out. So rather than trying to carefully account for every packet (and risk end up exceeding the capacity of the router) it simply drops them. The packets are deleted. Parts of thoughts and dreams vanish. Designed imperfection.

It just drops them, and doesn’t tell anyone (after all if its getting more packets than it can send out, why send out more packets as its already exceeded its limits).

Just remember that this the the technology you are using right now, so you know it works.

So lets say that the middle of a file gets dropped, how come you get the file in the end? Well that’s not something the Internet Protocol worries about. The fixer is part of the TCP, Transmission Control Protocol. TCP rearranges out of order packets, handles duplicates and requests missing packets if they don’t show up after an allotted time. Its part of the Protocol Hierarchy of the three structures of the net, in this case, it repairs damage caused by using the Internet Protocol. It is also why TCP/IP is rarely separated.

So the Internet Protocol looks like a rickety mess which might have the unintended consequence of loosing half your data, but its what we use. And if you can read this… its working.

This is a Loose Bloggers Consortium post on the theme of “Unforeseen Circumstances”; chosen by Gaelikaa. To find out that the others in the consortium think, check out, in alphabetical order: Akanksha (Anki), Anu, Ashok, Conrad, Delirious, Gaelikaa, GrannymarMagpie 11, Maria Silver Fox, Padmum, Noor, Rohit and the GOM of LBC, Ramana Sir.

Thanks to Steve Gibson for the IP description.

* OK, the First World War didn’t necessarily have its root in the assassination, but the First World War begot the defeat of Germany which lead to the Deutsch-mark devaluing and hyperinflation which begot the circumstances for World War 2, which lead to the USA and Russia becoming superpowers, which begot a lot of German scientists working for them, which begot the Space Race and the Cold War and nuclear proliferation which lead to ARPANET that became the internet. Greatly simplified, but a seeming unintended result during these wars, right? Actually no.

++ The legend that the ARPANET was designed to survive a nuclear war is actually false. After it was up and running it was confirmed that the routing mechanism would allow for such a consequence, but it was really created to save money and allow easier access for academics. The legend persists however because of that unforeseen circumstance during its design.

Tags: , , , , ,

6 Responses to “Best efforts”

  1. Delirious Says:

    The internet is truly complex, and difficult for me to understand! But what a great invention! I can barely remember life without it.

  2. Rummuser Says:

    Your contribution to the LBC is a welcome change from what we usually write about. Many of the things that we now take for granted are indeed directly attributable to the IT revolution, most of which was not foreseen. Quite whether all of them have changed our lives for the better is a moot point but, changed they have irrevocably. Even ten years ago, neither I nor anyone close to me would have foreseen my being an avid blogger and using the computer and the internet the way I do. My 94 year father thinks that I am nuts!

  3. Grannymar Says:

    That is an interesting take on the topic, Will. I won’t pretend to understand all of it, but I am learning.

  4. will Says:

    Delirious: The internet barely touched most people outside of major corporations, military and academics until recently. The waves of cultural impact are still washing on our consciousness.

    And more tools are being invented every day. The rise of the smart phones have changed how people use the same network they used 10 or 20 years ago.

    And chances are things will be different again in another 10 years.

    Rummuser: I guess my outlook is different from the rest of the LBC because I don’t want to delve into the areas I see as being private or family. I have a nasty curse of being told some peoples secrets, more than one I’m trying to forget.

    I also really like technology and how things change. I’ve really got to grab my description from Culch.ie; I like to join the dance at the intersection between people, culture and technology, and the floor is freshly waxed.

    On top of that, the solid cultural touchstones in Ireland have turned to sand while I was growing up. The Ireland of my birth is different from the Ireland of my childhood is different from the Ireland of my adolescence is different from the country I live in now. And I suspect to find the same country will be a different place in another 5 years.

    Oh and GM: the internet is best thought of as little ping pong shaped babushka dolls (packets of data wrapped in shells of protocols) being inspected and bated around by a series of cats (called routers) until the right cat finds the ping pong intended for her, there its opened and the contents eaten (well, passed to the person or machine who wanted that bit of data).

    The internet is made of cats, if you don’t believe me, ask 4Chan, Anonymous and Lulzsec. They are the ones who a) bring down governments and b) created “LOLcats and c) hunted down that lady who put the cat in the wheelie bin.

  5. Rohit Says:

    Hi Will, didn’t get a chance to comment earlier though I read this post a few days ago. First of all, a thank you! I always wanted to know how this darn thing internet works…I just love plain language used to explain technical jargon! I feel slightly less stupid now. Most important of all, I like this rather objective theme for a topic that is otherwise highly subjective.

  6. gaelikaa Says:

    Hi, Will Knott!

    I’m gaelikaa of the LBC, who is returning after a bit of a break, so to speak.

    I look forward to visiting your blog often and cordially invite you to mine, whenever you wish, not just on LBC days. You’ve got quite a technical approach, unlike me, it seems. That will broaden my horizons no end.

Leave a Reply