Some critiques of the Semat initiative

2010/04/24

Given the ambition of the Semat project it would be surprising if it did not cause some controversy. Alistair Cockburn wrote a long critique at http://alistair.cockburn.us/A+Detailed+Critique+of+the+SEMAT+Initiative. Martin Fowler, at http://martinfowler.com/bliki/Semat.html, echoes Cockburn and states that the effort is “fundamentally doomed”.

The issues are complex and we welcome controversy. We regret, however, the strident tone of these comments and their authors’ decision to criticize from the outside rather than helping with Semat’s mission of improving software engineering. This decision is for a large part based on rather unconvincing generalities. For example Fowler’s argument is that “I spent a fair bit of time in the 80’s and 90’s mucking around with this idea. In the end I decided that it was too hard and of limited value.” It is good to know that Fowler gave some thought to the problems of software engineering twenty years ago and gave up, but others might be forgiven for not finding this a good enough reason for the rest of the world to stop looking for solutions.

Fowler writes: “I got the distinct impression that the central thrust of the initiative is to create a software meta-method-kernel – essentially a set of common process elements for software developments that you can rigorously compose into a method for your own project.”  This is a misreading. The Semat kernel is not at the “meta” level; it is a concrete kernel of practices and patterns. The kernel is small, and includes universal elements relevant to all development projects.  It will be used to describe methods.

Cockburn’s critique exhibits a problem that we had foreseen and explicitly tried to forestall: the prima donna phenomenon. In setting up Semat we have contacted thought leaders in software methodology (Cockburn and Fowler included) and asked them to accept that each has a piece of the truth but none has the entire truth. We understand that this is difficult to accept (none of us is particularly modest either), but that accepting to work together regardless of guru status is essential to progress. Cockburn’s main argument is the eight years of carefully constructed literature including his own book, and that his 2002 doctoral dissertation pointed out that ”methodologies will inevitably increase in number, not decrease”. Much as we enjoy Cockburn’s contributions, they are neither the only word nor the last word.

Nowhere do the Semat documents suggest that Semat would result in fewer methods.  The Call for Action’s mention of the “huge number of methods and method variants“; Alistair cites this, but ignores the following comment:  “with differences little understood and artificially magnified.”  The problem is not the large number of methods, but their monolithic nature and the difficulty of comparing and combining them The Semat initiative should in fact make it easier to create new methods out of a smaller number of clearly defined practices.

Cockburn’s complaint also embodies a strangely thin-skinned reaction to suggestions that previous ideas, in particular Agile ideas, may need further refinement. Cockburn objects to our comment on the Agile manifesto, which stated that:

“… the elegantly written foundational document of the [Agile] approach is a “manifesto”, long on emotional appeals in the first person plural and short on facts. This style may be effective to capture attention, but as ideas mature it should yield to more traditional (even if also more boring) forms of argumentation.” (From an article by Ivar Jacobson and Bertrand Meyer)

He states that “The authors once again use precisely the polemical language they decry”. This is a bizarre criticism: first we don’t object to “polemical” language (discussion is a normal part of progress in science and technology), only to assertions that are not backed by facts and theory. Second, the article from which this is extracted was published long before the Semat documents and the first Semat meeting, so it is not like he discovered it after attending the meeting. But third and most importantly, our cited extract is quite laudatory of the Agile Manifesto; it simply points out that manifestos are not enough (a comment that also applies to the Semat Call for Action). It is one of the fundamental premises of the Semat effort that there are no sacred cows: we want to get the best of all existing ideas, agile included, and understand their limitations as well. We fear that Cockburn’s reaction shows a lack of willingness to have any aspect whatsoever of agile methods questioned in any way. That is not justifiable: everyone’s ideas, including ours, Cockburn’s and Fowler’s, should be assessed at face value.

Cockburn is also critical of the Zurich meeting. He complains that no agenda was presented.  Of course the meeting had an agenda; If he had read that agenda, the Call for Action and the vision statement, he would have known the goals and principles of the Semat initiative before he decided to come.  Moreover, He omits one of the main difficulties that this meeting (altogether very successful) encountered. The cause of that difficulty was Cockburn’s own decision, unannounced, to comment on the discussions in real time on Twitter; many of the comments were sharply critical of other participants. One of these participants suddenly found out and was upset at what he viewed as a betrayal of trust. A discussion ensued, moderated by us (we maintained a neutral view); it led first to a decision to allow tweeting and soon thereafter to the reverse decision, as a majority felt the tweeting compromised the quality and integrity of the discussions. This was a side issue, and was taken care of, but it did divert the meeting from its focus and cause tempers to rise for a while.

Cockburn states that the “state of [the research literature] more indicates that we don’t yet understand what is happening on software projects”. This is greatly exaggerated; of course we as a community do not understand everything — if we did, there would be no need for Semat — but a vast amount of literature, including both process-oriented (CMMI etc.), object-oriented and agile sheds considerable light on the reality of software development. The attitude that we don’t understand anything anyway is good for gurus (just as ignorance of medicine is good for witch doctors), but Semat takes a different view: that we know quite a bit already, albeit in a messy and often inconsistent way; that we must organize and classify this knowledge; that we must assess every idea, existing or new, through objective criteria, without fear or favor; and that we must apply our best efforts to solve the remaining open problems. Proponents of agile methods have a major role to play in this process.

— Ivar, Bertrand, Richard

15 Responses to “Some critiques of the Semat initiative”


  1. When I received an email through the SEWorld about SEMAT in Jan 2009, I was elated that somebody has taken notice of the ambiguities and proliferation in the field of software development and has taken an initiative to to do something about it. I immediately signed up and encouraged a few of my friends to do so.

    But as time (about 16 months) elapses, I am not so sure that the initiative is going in the right direction. In my first comment as in subsequent comments on the SEMAT blog, I suggested that we start addressing the definitions. I suggested a definition of “software engineering” too, since it seemed to me that the initiative was about “software engineering”.

    But I have been getting the impression, that we have been using the phrases “software development” and “software engineering” synonymously.

    Is there no difference between the two phrases, namely “software development” and “software engineering” at all? Do they mean the same – I think not.

    I read the vision statement that was the result of the Zurich meeting with keen interest. It seemed to me that the document is addressing “software development”. One encouraging aspect of it was that committees were formed to look into details and come up with results. I fervently hope that those committees would come up with resolutions to the ambiguities of the software world.

    I would once again sincerely submit that definitions (especially for “software development” and “software engineering”) be address first.


    • Hi Murali,

      I would agree that addressing the confusion surrounding these terms is one of the critical first steps to establish context and scope. I would also argue that this is going to take time and reflection. This is because it is laced with the complexity of modern society itself, as software is so pervasive these days. See http://www.fourth-medium.com/wordpress/?p=150 for some reasoning about why this issue is so important. Seems like an age old conflict between that arts and the sciences with engineering stuck right in the middle.

      I wonder whether we are seeing re-incarnation of social activism and labor movements of the past with lobbies like the Software Craftsmanship movement.

      See http://en.wikipedia.org/wiki/Guilds or http://en.wikipedia.org/wiki/Arts_and_Crafts_movement as influenced by John Ruskin for the striking resemblence.

      Mark Kennaley


  2. Hi

    I read the article of Alistair Cockburn pointed out above. IMHO, it is not criticism per se but a lament that things are not going on well at SEMAT – although it is his opinion. It would augur well, if we take it in the right spirit to move forward.

    There is an existing taxonomy document for software engineering terminology by IEEE which we can use as a starting point and refine it.

  3. Kim Gräsman Says:

    Hello,

    I thought one of the most compelling concerns of Cockburn/Fowler’s critiques was that they seemed to think SEMAT was set on formulating a mathematical language for a people-oriented activity.

    Could you follow up on that?

    Thanks!

  4. Andre Cousins Says:

    Hi,
    After reading Alistair Cockburn long critique I have come to the conclusion that someone or some people have upset him. I think the cruxs of this is in his statement “They are free to discuss matters that do not interest others”. I find this statement along with many others in his critique unbeliveable!
    I would hope that you would discuss matters that have no interest to some other people. It is only by diversity of opinions you can hope to solve your goals. SEMAT has attracted a lot of ‘big’ names from many disiplines and the opinions of these individules thankfully are not always going to be the same (or we would already have the ‘answers’ :)). It can be hard when you are passionate about your field to take critique, but this is necessary for the field to evolve. Please lets not resort to ‘Prima Donna’ tatics to try and get our ‘views’ across. Let’s listen to all opinions even ones that may “not interest others”. Often the answers lay in some small details, that holds the key to the whole thing. This small detail can be hiding in any field (even ones not associated to the industry). Indeed you may find the answers lie in the grasps of NONE experts. I will never forget a lession my daughter taught me when she was only 2 years old, that adults missed! Lets look for the answers where ever they lie and not be drawn into blinkered discussions, it is only by open disscussion we will find the answers!

  5. Phil Armour Says:

    One of the big “methods” changes of the last twenty years has been the growing understanding of Systems Development as a social activity rather than some abstracted reasoning process performed by some unspecified and abstracted thinking machines. The books on methods written in the 1980s were mostly concerned with a method’s diagrammatic conventions: whether a system’s transform “process” should be a circle or a rounded square, whether the path element of a directed graph should be a solid or dotted line, have one or two or no arrows and what that convention implies or “means” in systems development. Associated with these methods were some modest attempts to map what are simply drawing conventions onto (some) aspects of the “real world” that the development activity was attempting to replicate in the executable knowledge form called software.

    Most popular books on modern methods at least somewhat address the cooperative and cognitive aspects of building systems, so the social aspect of systems development is now being explored and somewhat codified. What I think we are seeing in the recent exchanges of views on the purpose and process of SEMAT is that systems development *methods development* is also a social (and perhaps sometimes anti-social) activity. Why would we expect otherwise?

    Ivan, Bertrand, and Richard as prime movers of the SEMAT initiative say that they anticipated the “prima donna phenomenon.” While I think the descriptive phrase chosen in this situation is a rather blunt one, the phenomenon is natural.

    In my view, methods are simply frameworks of thought—they are structures that guide the way we understand things. However, as much as they might help us think, they will inevitably also constrain our thinking. When a given thinking or working structure becomes clearly inadequate, we do one of three things: we ignore or discount the inadequacy—we humans have some marvelous mechanisms for totally disregarding whatever conflicts with the way we think. Secondly, we try to extend our current structures to cover the inadequacies, adding elements that somewhat integrate with existing art or reinterpreting the structure’s rules to cover the new conditions. Lastly, and only with great effort, will we disassemble the way we have viewed things and create a new structure. For many people this restructuring is not possible. This difficulty is not simply bloody-mindedness or recidivism; it is simply how we think. When someone has invested energy and innovation in developing a way of thinking not only is there significant pride (and no small amount of ego) wrapped up in the concept, it is the way they think. If my method (read: my way of thinking) is challenged by an alternative method, I will naturally tend to evaluate the capability and ease of use of the alternative method using the criteria of my existing method. Of course, others’ ideas will almost always tend to come up short by this measure.

    Edward De Bono pointed out years ago that, counter-intuitively, smart people tend to not be good thinkers. Since they are smart, they can quickly grasp complex issues and formulate good analyses and good solutions. They then invest energy in refining these solutions, polishing them so they become honed and precise (at least according to their in-built measures). Being smart people, they can persuasively defend their ideas against challenges. Being smart people, they can see the weaker points of others’ methods and articulately confront, contradict, and refute them. De Bono’s point about smart people not thinking well was that this energy goes into reinforcing existing paradigms rather than exploring other ways of thinking. The more that energy is used, the more likely that these ways of thinking will become entrenched. Therefore smart people will tend to not think around the problem but rather zero in on and progressively reinforce the first structure that seems to fit.

    My view of systems development is that it is entirely a cognitive activity—that all models and methods are grounded (only) in human cognitive processes. For instance, I think the idea of objects and classes does not originate in organizations, businesses, systems or in the “real world”; it is simply a convention of the human mind that must classify and group things in order to understand them. Equally, well-known systems heuristics such as information hiding, modularity, coupling, and typing are not “intrinsic” or universal, they are all ways in which we humans try to manage complexity. Most programming languages closely resemble our natural languages in form and syntax simply because that is the way a part of our brains work. Even our hardware design of registers and stacks and memory recapitulate the human brain in concept if not in actual construction. Given that software is just a knowledge medium, the job of systems development is acquire this knowledge and populate the medium and that knowledge is invariably and exclusively anthropomorphic, this is quite reasonable.

    I offer the previous paragraph as my particular world view, knowing that there are many smart people in the business who disagree and think of software development as some kind of external (to the human brain) mechanism that has an existence as an engineering discipline independent of the people applying the discipline. This perspective allows that processes, workflows, methods, and languages can be developed at least somewhat independently of cognitive and cooperative behavior.

    Since this does not fit with my particular way of thinking, I don’t believe it can (of course). But this clash of thinking mechanisms is predictable, inevitable, and necessary. SEMAT is thinking about thinking using the skills and resources of a lot of smart people who have spent their professional lives building, refining, and defending these thinking frameworks. It’s not surprising there are strong opinions and strong feelings about those opinions; there should be.

    • Johnny Says:

      I would argue we need maintain the separation between the definitions of systems development, systems engineering, software development, and software engineering.


    • I think Phil well described the matter. To his three, let me add a fourth and fifth method for dealing with cognitive dissonance which result in restructuring our way(s) of thinking. These may be empirically demonstrated in our own experiences:
      -4 A significant (positive or negative) emotional event. (No need to dwell on those.)
      -5 The acquisition of a new language. While most apparent when shifting natural languages of different origins, consider also the restructured thinking which accompanies a move from, for example, COBOL to java.

      Given that new language is a path to changing minds and that precise terminology is a mark of engineering, I concur with the expression of others recognizing a deep need for stated definitions of ‘software engineering’ and related concepts.

    • Andy Langton Says:

      “My view of systems development is that it is entirely a cognitive activity—that all models and methods are grounded (only) in human cognitive processes.”

      This is not the full story, but sadly far too many software developments limit their ‘pragmatic’ approaches to just this and implementation, and fail to address the need to accurately and precisely communicate all cognitive activities across a wide range of stakeholders. In doing so they fail to understand what is needed to effectively communicate these cognitive activities and the results/decisions to a variety of people with differing technical levels and backgrounds in such a way so they may easily understand. And importantly that the communication medium or artefacts produced have value and are of a greater benefit over the long-term, and not just for meeting short-term objectives.


  6. As an outsider, but critically interested in this discussion, the contention that systems development is an entirely cognitive activity ignores the domain and context of any development.

    Here in the flight software (ground and on orbit) software development world systems development is considered entirely a Systems Engineering activity until we reach the code development process.

    While the processes of “agile” and “emergent” software development has many places it is applicable, there are numerous domains where software development is an “engineering” process in the same manner hardware, firmware and other “ware,” flight platform, propulsion, environmental and life support to name a few – all software intensive.

    The objections of Alistair and others ignores context and domain. Software intensive system where mission and life are critical – a very large portion of software written for money – lead me to believe some have assumed their view of the world is the only view.

    SEMAT is asking critical questions in the “engineered systems” domain, where software is the basis of Mission Success – a banner in our lobby reads “100% mission success.” No incremental releases once you’re on orbit.

    Keep up the good work.

    Glen B. Alleman
    Program Planning and Controls
    Defense Systems Business Sector


  7. […] pm Much has been written about the conflicts surrounding Semat and its kickoff meeting in Zurich (https://sematblog.wordpress.com/2010/04/24/some-critiques-of-the-semat-initiative/).  One of the more heated discussions occurred on the afternoon of the first day related to Theory […]


  8. Refer to http://www.paulemcmahon.wordpress.com May 3rd blog titled, “SEMAT, Theory and Measurement” for a related discussion.

  9. zoldello Says:

    Yeah, yeah, Fowler and others top-names do not want to be part of SEMAT. No big deal! Move on and make them regret. Software Engineering needs discipline and SEMAT is not perfect but is a great step in a good direction. Now, what results do not have so far? That is a more pertinent question-issue.

  10. Ken Magel Says:

    Thoughtful critiques are vital to. a successful endeavor as ambitious as Semat. Semat needs to consider each of the critiques seriously and carefully. Theybare analogous in some ways to having acceptance testing of a software development product. Semat should encourage the formation and participation of a council of skeptics as long as their arguments are based on penetrating consideration of what Semat actually accomplishes and proposes. If Semat does not have an active opposition, it is not sufficiently ambitious to accomplish much of lasting value. We need to encourage and, perhaps, financially support thoughtful, polite critics.


  11. Well, from coaching real people doing real projects in real businesses, I do have to say that what SEMAT is aiming for is a great thing.

    Seeing people all over the place using their own soups of ideas (and cooking them up in the first place) is consuming inordinate amounts of energy for little results.

    There is already proof (EssWork) that the key concepts do pay off and work. So, there will only be more good coming out of SEMAT.

    I do like the works of Cockburn and Fowler a lot. But that are far from having the final word on these matters. They’d better join than not.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: