Lifelong Learning

Black Boxes, White Boxes

The issue of "black box" vs "white box" testing techniques arises often in the context of testing, as well as in general analysis. The central idea is simple: a black box is known only through its interfaces, while a white box reveals its internals for inspection. In some situations this distinction appears to be useful.

I find that it is much more useful to proceed as if there is no such a thing as a white box - only black boxes. Every black box has yet more black boxes inside it (a subtle point here is that we are allowed and able to look inside a black box to see if there are other boxes inside it  - a nod to the "whiteness" of the box, or to the believers that you need to examine it as a white box - a nod, no more!).

A perspective that everything is a black box forces the examination of anything only from its interfaces. The point is: we can never know what something IS, we can only know how something behaves. If the model is explanatory and predictive enough, for our current purposes, then that's all we need to know, and it is the "right" level to stop at.

Consider a black box A that you can analyze knowing its interfaces. Now you want to get "under the hood" and peek inside. You find two boxes, B and C. What options of analysis do you have? Study how the interfaces of B and C make the bigger interfaces of A, and how B and C interface to each other. Now you are more curious and you want to look inside both B and C. What could you possibly find? Inside B, boxes B1, B2, and B3 with interfaces among them, and external interfaces that add up to higher level interfaces. Same applies to C. You get the picture. Anyone who has ever opened the hood of a car knows the feeling.

The "looking under the hood" metaphor is instructive - and we can stretch it a bit farther. The instant you go under the hood of a car, you're exposed to the dangers of the white box thinking. You're face to face with enemy of all progress - complexity. The complexity in the problem space suddenly jumps from one black box with well understood interfaces to a jumble of black boxes interconnected in many bewildering ways. Welcome to the Hotel California, you are now face to face with the beast. "You can stab it with your steely knives, but you can never kill the beast".

The essence of this problem is granularity. Each box has inside it boxes of finer granularity. Success, or failure, centers around the "right" granularity of analysis. The auto engine has a basic interface - take in gasoline, and produce rotational energy. The "essence" of the engine is encapsulated in two of its components, the piston, and the crankshaft. The rest is detail. The piston converts heat to vertical motion. The crankshaft converts vertical motion to rotational motion. This illustrates the need for a conceptual framework, to conquer complexity. You must postulate a model. No analysis can proceed just by "getting under the hood". Talking to a mechanic may or may not be instructive. A reflective mechanic, like Click and Clack, the Tappet Brothers, are scientists at heart - and damn good mechanics too. Rare combination to find. A model by necessity is a simplification, and risks some inaccuracy, of course. The biggest danger of simplicity is the slide into trivializing reductionism. But the silliness of reductionism is easier to deal with than the obfuscation of complexity.

Information Security Vulnerabilities

From a book on Penetration Testing, Pen Testing for Web Applications.

There are a few information security issues out there that cannot be traced back to bad code, lazy coding, ignorant programming, something having to do with bad software, or bad practices in the creation of software.

Does Intent Matter?

Many times you'd get an email, and you respond to it, only to get a response saying, "that was not my intent". Sometimes you'd read a work (article, book, white paper), or see a movie or play, and you comment on it, only to be told, "that was not the author's intent".

I maintain that intent is totally irrelevant - that is assuming you can determine it - which in most situations is an impossibility (dead author is one example, dishonest author is another). The work, or the act, should, and does speak for itself. Once you put your words out there, they have a life of their own. When they're criticized, it should be understood, that the words, and not the author, is what is being criticized. You'd think that would be a truism. Well, it is not - and that is wrong.

This is not a trivial consideration. Many authors will hide behind the "not my intent" subterfuge. How would you ever find out what their intent was? Why should you care?

That is not to say intent is not of any importance at all. But its importance should be restricted to being merely an additional aid to interpretation. One more way to look at the work, and not the only way. When I see an Almodovar movie, for example, I always like to read or hear his views about what his intent was. This is mostly because he usually has many levels of meanings, that I sometimes miss. Having said that, I still won't accept form the great Almodovar himself the statement, "that was not my intent".  If I had understood a particular movie in a particular way. I would say, "sorry, great one", but your work speaks for itself, and that is what it said to me.

Intent should never matter.

The Power of Questions

If I had my college life to do over, or if I had control over college education, I would like to see the Freshman year in college dedicated to the art of seeking knowledge through the Socratic Dialog. No matter what your level of knowledge is, a Socratic Dialog with a teacher, with a colleague, or with yourself, can help you advance your state of knowledge, and solve problems or probe issues. At this age and time, we all lack knowledge in one area or another, because we all tend to specialize, and each of us is sort of an "island of knowledge". Questions can serve as bridges between these islands. "Peeling the onion", from the consultative approach training, is sort of a form of Socratic Dialog. Socrates was a bit more probing, though. Too probing for his own good, actually. He was sentenced, and put to death for "insufficient piety, and corrupting the youth". A very instructive fate!

Anyway, that was getting too philosophical. Lets get back to pragmatic engineering thinking. A danger of Socratic Dialog is that it can turn philosophical. Unlikely for engineering oriented questioners, but a real danger, nonetheless. We just need to remember that not all questioning is philosophical. Something that Socrates did not seem to realize. But he was not an engineer!

Questions can help individuals, and teams, explore how to approach a problem, either of knowledge or of planning, and problem solving. To make this discussion more concrete, I would like to provide an example. I will attempt to illustrate with a personal experience, I hope will be meaningful to others.

In all honesty this is not how I approached it to begin with. This is how I would like to have approached it. This is in the spirit of Parnas's, "A Rational Development Process, and How to Fake it". Document as if you followed a rational process - even if you did not. But I digress again. This is all a "bloggish" writing style. Quick, spontaneous, and definitely not a research paper.

Say you are asked by your management to explore a topic that you are not very familiar with, that could possibly pay back for the company. Say it is "Design By Contract". What is a good approach to begin the investigation? The topic could be anything: Failure Modes Analysis, High Availability Study, Database Performance, Best Framework for OLTP Applications, Design of a Best Practices Process, whatever.

Note that Socratic Dialog is NOT brainstorming. Very far from it.

Here is one approach to explore a planning a task based on Socratic Dialog.

There are eight basic questions that we would need to be answered before we spend a lot of time, effort, and money. The eight questions relate to the basic elements of thought (See Richard Paul's writings). These eight elements are: purpose, main question, information, concepts, assumptions, point of View, inferences and conclusions, and implications. That is not the only way to structure a Socratic dialog, only one way. I'll write about others in future blogs.

1. What is the fundamental purpose for exploring this topic?

- To survey the field?

- To explore possible benefits?

- To do an in depth study of the best way to bring development teams to adopt DBC and to define concrete practices and techniques to transition teams to DBC and make it a reality in software development practices at VeriSign?

Questions of purpose address the value proposition, and explore possible scopes. The three alternatives listed above define scopes that can vary in level of effort from the production of a one week exploration presentation all the way to several man yeas by several people developing detailed practices, guidelines, and training. The question of purpose must be the first question addressed.

2. What is the main question I am trying to answer. Other questions flow from answers to this. Typically a "question tree" can result - could be useful as an outline or "roadmap".

- What is DBC?

- Is DBC beneficial to adopt? Why, How? What are the costs/benefits?

- What exactly is involved in learning/adopting DBC?

3. What information do I need to answer my main questions?

- What is my "knowledge gap" with regards to this topic? Filling a knowledge gap is called "research". If we have all the information we need, we don't need research.

- How much do I know, how much do I need to know to answer these questions?

- What is the state of the art in this topic? Who has it?

- How applicable is the existing body of knowledge to my purposes? Is it applicable to my domain of concern (Java development of OLTP applications)?

- If it is not, what would I need to do to create "new knowledge"?

4. What are the most basic concepts that I need to explore?

- Contract theory in human interactions. Contract law. Are contracts useful/necessary/valuable in human affairs? How?

- Contract theory for designing software. What exactly are "software contracts?"

- What other concepts are related to the basic concepts?

- What is the "ontology"?

- Contract specification (where, how).

- Contract enforcement (where, how).

- Design methods, design patterns

- Specification methods (requirements, test plans)

- Interface specification. Interface/implementation/runtime relationships.

5. What assumptions am using in my reasoning?

- DBC is valuable.

- DBC is easy to learn/teach.

- DBC is reasonable, pragmatic, relevant (as opposed to unreasonable, academic, irrelevant)

- Developers will "take to it" when they see its benefits.

- Management is predisposed to it and is strongly inclined to support the efforts to make it a reality. To what extent is this true?

6. What different points of view do I need to consider while looking at this?

- Who is the "customer" for this effort? Upper Management, Development Managers, Development teams, or individual developers. Others?

- Developer point of view: Here's what it is, here's how you do it. Here's what it will help you achieve. Here are the downsides.

- Development Manager point of view. Why should I adopt this? Will productivity suffer? Will quality benefit? What are the downsides?

- Upper Management point of view. Are expenditures on exploring, studying, adopting warranted? How much are we willing to spend to make this a reality? Will it pay back?

7. What are my most fundamental inferences or conclusions?

- That's what the research and exploration will uncover. This is "my destination point".

- What have I learned from this exploration? What should I conclude?

- Should I recommend anything? or let the conclusions stand on their own, and decision makers make whatever decisions are appropriate for them?

8. What are the implications of this effort?

- If we complete and publish a best practices paper, what follows?

- Do we need to embark on an adoption effort? Education, evangelization, an adoption plan?

- Implications for operations? Infrastructure?

- How do we track the benefits/costs?

Notes:

- The particular methodology cited here was developed by Dr. Richard Paul, and is described in the book, "The Art of Socratic Questioning", by Richard Paul, and Linda Elder.

- You can learn more about the techniques described here at The Foundation for Critical Thinking.