Welcome!

2009/12/01

Barely a week after setting up www.semat.org, we are close to two hundred people supporting our initiative. More are joining by the hour. The initiative appears ready to mushroom!

We don’t expect SEMAT to solve all the problems of software engineering. It addresses the development of a common knowledge base; other initiatives are needed for example to increase professionalism in the industry and to close the gap between computer science research and software engineering practice. SEMAT is a great start towards recognition and resolution of fundamental problems. We have committed to working together with developers, researchers, methodologists, and leaders at all levels to make the software world better.

The Call for Action points in the direction of a solution by specifying “a kernel of widely-agreed elements, extensible for specific uses.” It avoids prescribing a specific solution, not because none exist, but because none have been widely accepted.

Now the hard work will start. The first step is a vision statement, specifying what we as a group want to achieve in the next twelve months. The three of us will prepare a skeleton of the vision statement, and will submit it to the community so that you can provide your insights.

Please prepare yourself by reading the material on the semat.org site: the two papers and the Questions and Answers. Additional material is also available on the site.

The biggest difficulty will be to achieve consensus without compromising ourselves to death. We can do it!

Now, we open up the floor for you to express your ideas.

— Ivar Jacobson, Bertrand Meyer, Richard Soley

Advertisements

26 Responses to “Welcome!”


  1. Great initiative,

    In my days as a modelling methods consultant I’ve seen my share of different modelling methods. Most of them are indeed basically the same with mostly cosmetic differences.
    After you’ve seen some of these methods its not that hard to see and abstract out the common concepts that appear in each method.
    I suppose the same holds for other types of methods.

    Regards

    Geert Bellekens


  2. No two projects or systems are identical. In fact they vary wildly in terms of various characteristics such as:

    • Systems of Systems defining characteristics:
      • Complexity
      • Evolution
      • Negative Emergence
      • Size
      • Variability
      • Subsystem Autonomy
      • Subsystem Governance
      • Subsystem Heterogeneity
      • Subsystem Physical Distribution
      • Subsystem Reuse
    • Quality Characteristics (i.e., the “ilities”)
    • Programmatic Characteristics:
      • Organizational characteristics (such as types, number, size, management and engineering culture, geographic distribution, staff experience and expertise)
      • Stakeholder characteristics (such as number, type, authority, accessibility, volatility, and degree of trust)
      • Project characteristics (such as contract type, scope, duration, schedule, and funding)

    With all of this variability, no single software or system development method can possibly be optimal for every situation. That is why I am greatly in favor of taking a situational method engineering approach so that the process is appropriate for the specific needs of the organization, stakeholders, system, etc. That is why I also favor capturing industry best practices in the form of reusable method components.

    I hope that we are finally past the period of “My method is better than your method”, regardless of the situation.

  3. Holger Ericsson Says:

    I am honored o be aske. I have lately started aline of progres that we don’t build programmers like we used to. And that i believe that the difference is that we let the new programming advocates get awy with a low knowledge or scientigfica pproach. I belieev tht there are some basic discipleines tht are criticla to scietific and engineering achievement. And we should make sure tht th all understand that these, I can pnly rfer to them as foundations can not be lefty out of our approaches thinking. Thing like forma methods have fallen by the wayside. WHY, our rate of success has not risen.
    I Don’t need a soap box. I just want to feel better and help make things better.


  4. I spent the day with a customer of mine.
    Main recurring challenges:

    * how do we organize our “stuff” ?
    * how do we work together to achieve ?

    The issues are in how to best manage complexity (and avoid the overly complex in the first place).

    I use concept of TEA (The Economy of Attention): there is only so much time to focus during a day, so “what do we focus on to achieve results?” This is the single most important question in today’s world of too much of everything (too many choices, too many techs, too many elements, …). MoA (Management of Attention) could be a great practice to develop BTW.


  5. A welcome initiative!

    I think the first thing we need do is to get general agreement (if possible) that software development is a form of engineering.

    Also, I would suggest we need a clear explanation of what analysis (problem and domain modelling) and design are really about.

    Finally, that all software development involves modelling (e.g. code is a model) even if it is done in one’s head (undesirable but often a reality).

    Just my 2 cents.

    Cheers,
    Ashley.


  6. In many domains, it is impossible to address software without considering hardware and other types of system components. For example, the so-called “software” quality characteristics are actually a function of both software and hardware (and often people) and are therefore system quality characteristics. This places software engineering as a subdiscipline within systems engineering. This raises the question of “What is the relationship between SEMAT and SystemsEMAT?” Just as hardware engineers sometimes ignore software engineering to the disadvantage of the project, system, and its stakeholders, software engineers sometimes do the same. Also, what is the relationship between software engineering and specialty engineering areas such as reliability engineering, safety engineering, and security engineering? While SEMAT should bound itself so that we do not try to drink the ocean, I think we will do ourselves a disservice if we try to look at methods and theory from only a pure software viewpoint. Our methods and theory have to be consistent with and incorporated into systems methods and theory.

  7. Erik Putrycz Says:

    In Donald’s comments one concept worries me.
    He mentions “reusable method components”, so what does SEMAT really intend? Replace many methodologies (most of them are already meant to be customizable) with method components?
    I’m curious to know how “reusable method components” can be used in small shops.
    In my experience, the best pratice is to have “some” methodology but unfortunately this “some” refers to common sense.
    If you build 1000 houses – you will have to follow a methodology by the rule but if you build a single shed in your backyard you will probably only use “some” methodology.
    There is space to refound software engineering but hopefully it won’t be method driven.


    • The nice thing about method engineering is you get the advantages of both standardization and flexibility. The standardization comes from the standard underlying metamodel, ontology, and set of reusable method components. The flexibility comes from selecting only the appropriate method components, tailoring them, and potentially adding new components. The reason it is called Situational Method Engineering is that the idea is to produce a method that is appropriate for the specific situation you face. Clearly, the method you use for building a small simple website is very different from that of building an ultra-large system of preexisting independently governed systems.

  8. Robert W. Stoddard II Says:

    I concur with the sentiments shared. In my role at the SEI of contributing practical definition to CMMI High Maturity process performance modeling with examples from industry, I see a slow but exciting movement towards more predictive modeling of software process performance. We also teach these methods, including experimental design, in our SEI classes: “Improving Process Performance using Six Sigma” and “Designing Products and Processes using Six Sigma”. We are also in our third year of conducting semi-annual workshops on Process Performance modeling and associated business results. The 2010 workshops will be adding a focus on process performance modeling in the CMMI-Services domain.


  9. @Erik Putrycz

    I don’t think that is what they are after.
    If I understood Ivar correctly when he explained this initiative last month at Devoxx in Antwerp, they are looking for those elements that are common in ALL methods, and then have the ability to apply the different ways of doing such a thing to the methodology.
    To go with you analogy of the house building; If you are building a house, whether it be a mansion with 20+ bedrooms, or a backyard shed, you’ll have to make a plan.
    So “Making a Plan” would be the common element here.
    Now the ways to execute this element can differ. For the mansion it will probably include CAD designs, a small scale model, and who knows what.
    For the shed the way to execute “making a plan” could be to sketch something on a paper napkin.

    So each project could choose its own ways to execute the common activities, based on the actual requirements and complexity of the project.


  10. I have a couple of suggestions:

    1. Please let the Semat initiative become a forum for lifting our heads above the level of code. It seems that the solution to every problem in software engineering these days is framed at the level of code: new languages, frameworks, libraries, patterns, or paradigms (such as Aspect Oriented Programming). These solutions quickly drown in their own detail, complexity and idiosyncrasies. Please let us strive to achieve discourse at higher levels of abstraction, to widen participation and achieve a better separation of problem and solution.

    2. Please don’t let the Semat initiative become an “anti-Agile” forum. I think that the Agile movement has made positive contribution to the practice of software development. I believe the core Agile idea, that requirements cannot be known completely in advance, is correct; as is the answer, an approach designed to adapt as requirements and understanding change. Without discipline, Agile can become hacking; but with discipline it is very effective. It is much better than the Waterfall alternative of setting ignorance in concrete at the beginning of the project.

    It is possible that my two suggestions above may be perceived to be in conflict. Perhaps my third suggestion is that we find ways of meeting both of the above at the same time!

    Regards
    Ashley

  11. Holger (Eric) Ericsson Says:

    First let me apologize for my last post. I am horrified that I let that post full of typos and misspellings happen. Please find below a repost that is hopefully corrected and more comprehensible than that piece of writing trash that I allowed to happen through Flu haze/trance.
    I am honored to be asked. I have lately started a line of protest that we/the established practitioners (read old-timers/educators) don’t build programmers like we used to. And that I believe that the difference is that we let the new programming advocates get away with a low base of knowledge or lack of scientific approach. I believe that there are some basic disciplines that are critical to scientific and engineering achievement. And we should make sure that the discipline is applied I can only refer to them as foundations which should be automatic and not be left out of any disciplined thinking. Thing like forma methods have fallen by the wayside. WHY, our rate of success has not risen.
    I don’t need a soap box. I just want to feel better and help make things better. Well let’s try that post again . And hopefully the moderators wll have mercy and delete the last post.


  12. What amazing timing for this movement. Ironically, I am about to release a book in February 2010 titled “SDLC 3.0: Beyond a Tacit Understanding of Agile”. It is based on 20 years of practitioner and consulting experience on the ground.

    In this work I articulate a few things that may be of value/interest to this initiative…

    1) Many branches of reflection and method capture (what I call SDLC 2.0 – the iterative method wars) emerged from the early “accidental waterfall”. The degree of branching is a little ridiculous now, and we know what happens in software configuration management when we favor isolation too much over integration.

    2) These branches need to merge/integrate around the concept of a practice=pattern and hopefully revive pattern orientation as it relates to software delivery. The rampant branding wars must stop, as it makes IT look ridiculous to the business for which IT serves.

    3) The 1.0 era (waterfall) needs to retire; from a systems/control theory perspective it is open loop, unstable, and is inconsistent with the nature of tacit knowledge.

    4) What to do with Lean :o) After all it’s origins predate all other fads or merger attempts.

    5) SDLC/IT Delivery is only one mission within the “IT Business”. It needs to integrate into the core value-stream and not be articulated unto itself.

    Also in the book I discuss at least part of the theory that this group strives for, as the answer has been around for quite a long time. From
    the early pioneering work of Jay W. Forrester (IEEE Medal of Honour recipient) and the MIT System’s Dynamics group, I believe the unifying basis for all software delivery dynamics challenges can be found.

    Check out http://www.fourth-medium.com/wordpress for some recent blog posts on this history, and what can be abstracted from ALL of this experience, described as a “Complex Adaptive System of Patterns”.

  13. Prabhakar Karve Says:

    I feel it is a great idea to find common ground amongst different methodologies. I have often been disturbed by the ‘war of words’ between agilists and non-agilists. I feel it is in everybody’s interest to dispassionately understand each others’ point of view as well as what works under different situations.

    But it is important that we don’t stop at just identifying various elements of the ‘kernel’. We must come up with the ‘essence’ of each element regardless of the methodology or situation.

    I am really looking forward to something worthwhile coming out of it. It will greatly benefit the whole community.

  14. Winifred Menezes Says:

    Hello,

    Excellent – this is an initiative I support and will talk about.

    However there is one answer in the FAQ section that I wonder about. The answer to the last question is
    A number of people have superb theories, sometimes backed by successful experiments; no doubt yours is one of them. Unfortunately, no single approach has garnered broad acceptance in industry and academia. Like other engineering fields, ours needs a common framework with which everyone agrees. We hope you will contribute your ideas.
    My thoughts: Given 1) software is used to solve a wide variety of problems, and 2) software engineering has elements of engineering, but is also dependant on social interaction, team dynamics and similar “soft issues”; can there really be a single broadly accepted approach? Shouldn’t there be a set of approaches from which the approachs/methods/techniques most suited to the problem at hand are chosen?

    I would contend that we have a common framework of sorts already, though we may not agree on the nomenclature. To develop software the following sequence is followed sometimes once through (waterfall), sometimes with feedback loops, sometimes in a number of interations (of varying length), sometimes in parallel – but the following is performed: some amount of requirements/needs elicitation/whatever you call it; some amount of design (with varying detail), coding and lastly testing and correction – followed of course by some variant of a “release”. We know that humans make errors. Ideally these errors should be found as soon as possible after they have been made. And we would like to prove (as conclusively as possible) that the designs and code we have are error free.

    Maybe I am naive …… so it would be very interesting to hear or read the views from the august list of signatories.

    Winifred


  15. While reading the comments, I noticed that the SEMAT Blog website suggested a potentially related Blog concerning the 1968 NATO Software Engineering Conference. I have never seen an automatically-generated link be so apropos. It is a cautionary tale that we should all read — the one paper presented at the conference, but that was excluded from the conference proceedings:
    http://homepages.cs.ncl.ac.uk/brian.randell/NATO/NATOReports/index.html#Appendix

    Although I strongly support the ideals of this initiative, we must be careful if we do not want that paper to be about us.

  16. Tom Anger Says:

    Just read the article about SEMAT in Dr.Dobbs. Having been taught CS out of the Math department(have the 1st edition of the 3 volume Knuth series), I don’t know what software engineers are being taught today. I do know that I have been directed to chase more fads than I care to recall. Every fad basically said the same thing – reduce design time, reduce development time, reduce integration time, reduce the number of bugs generated, and produce re-usable code.

    I would love to see software engineer defined, universal standards developed. I loved the article, and will keep returning to the site, if only to diffuse my own doubts – herding software designers has a lot in common with herding cats.

  17. John D. McGregor Says:

    My reading of the original post by Ivar, Bertrand and Richard led me to believe this effort is intended to identify fundamental principles. In the postings I see things like “what is common among all methods.” I would assume we would keep an open mind and conduct experiments to identufy concepts. I am not certain that we yet know how to engineer software so looking at existing methods may not yield new information. It might yield what some call “best practices” but I hope this effort will investigate ways to identify what are truly best practices not just the best of what exists. Having said that I believe that one avenue of investigation regards how to specify the context in which software engineering practices are applied. Once we have that we can begin to evaluate differences among methods holding the context constant.

    I have added a brief mention of SEMAT in the January/February Strategic Software Engineering column in JOT. I look forward to much activity and much progress.

  18. Adrian Tawse Says:

    For goodness sake somone shoot ‘Agile’ I am currently watching a geat company fling itself off a cliff because someone thougth ‘Agile’ was ‘Cool’. It is just a quicker way to fail. It is a refuge for the weak minded. There is no alternative to design first code second.


    • Hi Adrian,

      While Agile can be described as a movement and a brand, and although this initiative wishes to stop the fads, various practices that were popularized by this community are quite sound. Its just that some would like a little more meat behind the “science” of why they work. I wouldn’t let some of the fringe elements discourage your firms usage of some of the basic tenets – especially the effects of negative feedback relating to emerging an understanding of scope due to the nature of tacit knowledge. And high-stakes design is part of Lean/Agile if it is warrented (kentou in the world of Lean). But unfortunately some in the community will paint with broad strokes and their belief structure will suggest a low level of abstraction is always the best way to reason about complexity. I don’t think the founders believed in such purist interpretations – I think actually they were quite pragmatic, but also recognized “muda” exists in the mis-application of good practices like MDD or DDD. Practice=pattern and are by definition contextual. And I believe they recognized that sometimes “architectural work” does exists, sometimes it doesn’t when you look at the original pattern language articulations.

      Perhaps Alistair or Robert or Ken might be able to further elaborate…

      Mark


    • Let me be crystal clear: “shooting agile” is NOT our intent. On the contrary, there is a tremendous amount to learn from agile practices and patterns. We can also learn a lot from the agile community’s successes. Agility is here to stay.

      • Adrian Tawse Says:

        This firm started by persuading itself that it did not have the time to analise requirements, design and code in that order. All the requirements were stated, just not in a structured form. They are now in the process of redesigning and recodeing large parts for the third time, not because requirements have changed but because important requirements were not considered first or even second time round. They are over budget an in danger of hitting late delivery penalty clauses. Agile is in the process of killing this company. I am sure that in the right hands Agile can be made to work. However it requires great skill to know just how much requirements analysus and design is sufficient. The problem here is lack of design skill, of too much bad design. To these people Agile is an excuse for incompetance, it is an invitation to suicide, it is the fugu fish of software.

  19. Holger (Eric) Ericsson Says:

    Let me be clear. I think the key is to subject these critical SE efforts epecially design to go through the same thinking, verification that we required of other R&D dvelopment effort. We should not require any less discipline in birthing these products and our ideas read designs are products as we do for anything else that arises from the lab. Look at we approached required from the work at Los Alamos( maybe a bad choice) nbut i don’t think so. Let’s teach them how to think and be disciplined. We are losing the academic discipline that we had in the fifties an sixties. Albeit a very slow pace, thta we don’t want to go back to.

  20. Troy Says:

    First let me state that I am happy to see so many prominent members of the software community come together to discuss how to create software I am sure regardless of the outcome something interesting will be produced. However I am critical of this effort especially the following

    “Software engineering is gravely hampered today by immature practices. Specific problems include:

    The prevalence of fads more typical of fashion industry than of an engineering discipline.
    The lack of a sound, widely accepted theoretical basis.”

    I ask compared to what ? To math, physics, psychology or economics you will find they all have their crackpot theories too you will find that two experts often do not agree.

    “The lack of a sound, widely accepted theoretical basis.”

    This sound like a search for a Grand Unifying Theory for software engineering.

    Other disciplines have also sought this and failed math had its Principia Mathematica. Physics has more then one Grand Theory of Unification none of them proven or accepted.

    I see no evidence that a Grand Unifying Theory exists in any field why should it exist in our field.

    I do not think we are worse than any other discipline there are astounding failures and successes in all fields. An example the contractor renovating my basement is currently 300% over schedule that he set out for himself. I am only usually 50% over schedule that is dictated to me.

    Most software disasters are based on unrealistic expectations most people know they can’t afford to build a skyscraper. Most people think you can hire some inexperienced kid at low cost to build a piece of complicated software and that somehow it will work out ok. No mater how many times they fail at doing this. They will say if only we had the better engineering processes that would allows us to take a monkey and create a software.

    There are No best practices just practices that are good in a context. I am not sure why people are so unhappy with that situation or why they expect that it should be different. If software was easy I would be out of a job so thank God it isn’t easy so I can get paid to do what I like to do. Imagine what a boring place this world would be if there is only one way to do things. Can’t appreciate quality without something to contrast it against.

    I don’t think there is a software crisis. People make good free open source software on a regular basis. Quality cost in any field you want a higher quality home be prepared to pay if you want it cheap builders can do that too they will cut corners. You see the same in software all the time mostly the cutting corners part.

    Anyways I don’t think you will solve this problem if it is even really a problem many have tried before you they were also very bright.

    Most people are not very good at what they do, doesn’t matter if there lawyers, doctors, engineers, construction workers etc this is fact that will always be with us no grand theory going to change that.

    Well I had my little tirade/rant lunch is over back to work I need to work to these tight deadlines.

    Troy


  21. A Manageable Infinity of Methodologies

    I assert that:

    We don’t need a new methodology, we need a means of combining practices into a custom methodology that is tailored to the enterprise under consideration.

    It should be possible to evolve a tailored methodology easily in the light of new information.

    The SEMAT meta process kernel gives us what we need to build custom methodologies. The kernel is populated with ‘practices’. The populated process kernel is the enterprise practice (EP) for that organisation. The meta kernel is made up of ‘concerns’. Each concern is populated by a concern practice (CP). A CP is broken down into practice activities (PA), these are things that are done within the CP.

    Some practices will work better together than others. With some practices there will be a more natural fit.

    Some practices will be more effective/successful than others. Some practices will be more effective/successful in certain situations than others.

    Some concerns are more ‘pressing’ than others. Some concerns may have practices that are more ad hoc based on principles, such as the Team concern which may be loosely defined as guided by Agile principles of mutual respect.

    It becomes possible to define the interface between concerns. Every concern would have an interface with every other concern.

    A taxonomy of enterprise types based on their relationship with software will inform their enterprise practice.
    • Sealed Packaged solutions e.g. Microsoft, Apple
    Build software products and sell them.
    • Unsealed Package solutions e.g. Siebel, SAP
    Build software products, sell them and configure them
    • Managed services e.g. PatientKeeper,
    Build a single software service and sell access to the service.
    • Information provider: e.g. Reuters, The Guardian, Google
    Build and platform that gives access to valuable information.
    • Professional services, (incl. Government, Manufacturer/Supplier): e.g. Aviva, Barclays, Lloyds of London
    Provide banking, insurance, order processing, manufacturing support, government services etc. and use IT to support the business.
    • 3rd party supplier: e.g. mPhasis, Tata, Logica
    Build software and services on behalf of other professional services or information providers.
    How core software is to what the organization does is going to affect the way an organization configures itself with respect to software engineering. It stands to reason that what the organization does will affect how much energy they put into the business of building software.
    The ‘Universals’
    If you want to build software, what are the kinds of things you need to worry about? Have you got all the bases covered? That’s the question the group ‘Software Engineering Methods and Theory’ (SEMAT) have been asking themselves. SEMAT aim to come up with a design for a meta-method-kernel. This is perhaps not a fundamental theory of software engineering [1], it is however a useful thing to have [2] because everybody building software should have thought about the tasks they are going to be faced with and can at least make a rational decision on what practices they should adopt. The decision they make will be affected by the type of organisation they are and the centrality of software to their core mission.

    Therefore, my assertion is that one really useful purpose to which the SEMAT work can be put is to allow for the tailoring of process methodologies based on existing practice families. Practices may come and go, but the meta-kernel makes the theory flexible and practical.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: