Since I roundly dumped all over the IETF in my last post I’ve been thinking how the IETF could improve. My current best idea is “testing”.
Does it Work ?
When I use routing software, from any source, I have a fundamental distrust that it will work correctly, predictably and reliably because experience shows that bugs are guaranteed in a vendors product, even if that product/software has been in use for decades.
Simply, our suppliers and especially commercial vendors are proven to ship unreliable products.
This has enormous impacts on perception of the value of networking:
- Customer executives don’t trust networks to be reliable which compromises attention, funding and resources.
- Engineers are unable to make reasonable guarantees to the IT team and those who depend on them.
- Raising bug reports as a customer is EXPENSIVE in time and resources. As a reseller, bug reports on big projects can push them into receivership (seen this more than once).
So how do we address this ? We need testing software.
The dirty secret is that open source routing software has roughly same reliability and commercial vendors (just ask around enough engineers who have done both).
Validation and Acid Tests
Why don’t we have this for networking ?
An open source project that could be used to test and validate that BGP is working. Which RFCs are supported in this router ? Is the BGP fully compatible ?
The IETF once depended on rough consensus and running code. That period is over for a number of reasons. Lets replace running code with testing code. For every RFC written, produce a suite of standardised tests that can validate the operation of the RFC when implemented.
Lets take IPv6 ….
IPv6 adoption is a joke. Few people believe that vendors are producing reliable products especially in the area of proxies, firewalls and other middle boxes.
We could improve IPv6 adoption by putting testing tools in the hands of customers. Imagine putting into a tender/supply agreement this product must comply with standard IETF testing suite and then attaching financial penalties or product returns based on non-compliance.
IETF should focus on testing and validation
Could the IETF produce running code in the form of a testing suite ? Well, we already have hundreds of network related open-source projects managed by the Linux Foundation where the “code is the coin of the realm”. Companies of every sort contribute vast resources (money and people) to project.
It would slow and hopefully prevent the crazy RFCs that are being produced right now.
- You shouldn’t be able to produce an RFC that cannot be tested by an external testing suite.
- A software testing suite should be a open source platform that anyone can access
- Customers can use it to validate function and interoperability
- If vendors want to offer enhanced features, then code for the testing suite should accompany the submission. (Proprietary features need to can be testable)
- The testing suite would become a key value proposition of the IETF and critical to infrastructure automation.