A Major Milestone: On the way to a new standard

2011/07/13

Semat Blog Release by Shihong Huang

We are now happy to declare that Semat has reached a milestone in its history. The OMG RFP (Request for Proposal), entitled “A Foundation for the Agile Creation and Enactment of Software Engineering Methods”, was approved by OMG on June 24, 2011. The RFP is heavily inspired by the work of Semat.  Becoming adopted as part of a standard body is a great achievement. The full version of OMG RFP can be found in [1].

1. Challenges we face

The number of people developing software is exceeding 10 millions.  Some sources (for instance Evans Data Corporation) estimated in 2010 the number was 20 millions.  Most of these people have no clearly defined method to follow; they just do it.  Most of them think they know why, what and how they work. So even if not explicitly defined, they have a method.  However, improving these methods to help developers build software better, faster and with happier clients and developers, we need to better understand what software engineering is.  This is why Semat was created with the vision to refound software engineering as a rigorous discipline based on a solid theory, proven principles and best practices.

Some Semat theses include:

(1)        Every method is just a composition of practices, either human- or technology-related.

(2)        Practices are reusable over many application areas and technologies.

(3)        Underneath all practices is a small kernel of universals, things that we always have, do or produce when building software.

(4)        Methods and practices are described by a light Language.

Could we address the need of millions of practitioners, could we bridge the vast chasm between academic research and industrial practices? Today, by reaching this milestone of Semat, we are more confident than ever – yes, we can.  However, it won’t be easy.  We need to create something fundamentally new.

2. Two principles we follow

First, we will rely on two leading principles – ‘separation of concerns’ and ‘agile in everything’, inspired by which some distinctive features have been found.  We believe these features are essential to what we would like to achieve.

2.1. Separation of Concerns

A key principle of the Semat initiative is the principle of Separation of Concerns (SoC) (http://en.wikipedia.org/wiki/Separation_of_concerns).  It penetrates much of what we do. It allows us to specify a core, and then through extensions without changing or complicating the core, add what is needed for the more specific cases.  In this way we can be inclusive of all relevant work in software engineering and not excluding anything that is or will be beneficial to someone.  Below are some examples of the application of this principle, more to come later.

(1)   It separates at least two different views of methods: the practitioners’ view and the method engineers’ view. The primary users of methods are project practitioners (developers, testers, project leads, etc.).  We should target the practitioners uncompromisingly. Through extensions the result will also support method engineers efficiently without complicating its usage for the practitioners.

(2)   It separates the essentials from the non-essentials, such as key guidance from detailed guidance, or explicit knowledge from tacit knowledge.

(3)   It separates the generalized definitions of terms from specialized definition details, allowing for the inclusion, rather than the exclusion of earlier work on methods.

2.2. Agile in everything

Being agile and light-weight is the key to be successful. We should be agile in whatever we do, nothing else would be acceptable. In a separate blog we have made an attempt to capture what we mean by ‘agile in everything’.  Please, read and give your comments and suggestions.

3. Three distinctive features we identify

In our previous blog “The key points of the OMG RFP draft: call for a widely accepted kernel” (http://sematblog.wordpress.com/), we highlighted three striking features of this initiative:

(1)   Finding a kernel

It emphasizes that the RFP is not creating a new method; instead, it is to build a foundation that “consists of a kernel of software engineering domain concepts and relationships that is extensible (scalable), flexible and easy to use”.

(2)    Practitioners focused

The kernel has to be agile and lightweight to be successful. It focuses on people who do the work: the practitioners (e.g., analysts, developers, testers, etc). This foundation is created by practitioners, and serves the practitioners. Without getting the practitioners to adopt the result of this initiative, it will frankly just be an intellectual exercise.

(3)   Focus on the usage of methods, not the definition of them

The priorities of practitioners are to actually develop software and to get the job done. They can do this without a lot of descriptions.  This is why our language must primarily focus on method usage, whereas method definition is a secondary objective. The use of a method can be defined as the carrying out of that method in the context of a specific project effort.

4. Call for participation

Getting the RFP approved by OMG was one of the major milestones of Semat. Quoting Churchill: “Now this is not the end. It is not even the beginning of the end. But it is, perhaps, the end of the beginning.” Now we need to create something that will go beyond anything previously done by any standards body working with methods: getting the standard adopted by the broad developer community.

This is a challenge that cannot be overestimated.  This requires new fresh ideas that are not typical for standard bodies and methods work.  Fortunately, the Semat teams have several such new ideas. ‘Separation of concerns’ and ‘agile in everything’ will guide us, but more is needed.  Thus we would like to have more very talented people involved in our work.

Please let’s us hear from you – your feedback and comments on our blogs. Your involvements are critical in making a difference to the community.

Reference:

[1]   OMG RFP “A Foundation for the Agile Creation and Enactment of Software Engineering Methods”, online at

http://www.semat.org/pub/Main/WebHome/Foundation_for_SE_Methods_RFP_ad-11-06-26.pdf.

(Reading tips: You can skip sections 1—5, since they just introduce general rules for OMG proposals and standards; you can start reading section 6, since the actual technical content starts here.)

About these ads

6 Responses to “A Major Milestone: On the way to a new standard”

  1. Marcus Andersson Says:

    Hi there, method architects.

    Not following your recommendation, I read chapter 2 of the RFP where the term platform is used in violation with its definition. You have to decide if a platform is a “set of subsystems/technologies” or a “set of interfaces and usage patterns” and stick to it. These two definitions can’t possibly be the same.

    Let’s investigate the two definitions. If a platform is a set of subsystems, then everything is a platform. The definition is not very useful because it does not convey any information about the set of subsystems in question. The interesting property is if a set of subsystems is interchangeable with another set of subsystems from the user’s point of view.

    If a platform is a set of interfaces, then we can define an “implementation” to be the set of subsystems that realize the platform. This definition is much more useful, and I think this is what you want. I will stick with this definition from now on.

    Then I get to the platform independent model, which can’t be effective if it doesn’t depend on at least one platform. No one will be able to figure out what it means or does, because it is free floating in a vacuum. Any model will be independent of all platforms out there, except the one it depends on. The property of independence is not a property of the model itself. It is a relation between models and platforms. But now I feel the word model is a bit awkward to use in this context. Why not replace it with system instead. Then a model can be defined as an incomplete description of some aspect of a system. If the model would be a complete description of a system, it would be the system itself, and therefore unnecessary to define as a separate term at all.

    Another term that is tempting to define would be “implementation independent system”, but this is also not a useful definition, because it follows from the definition of platform that all systems are implementation independent. If not, the description of the platform in question is incomplete.

    A mapping is one kind of platform implementation.

    Then I get to chapter 6, and think you should stop for a while and consider the fractal properties of the things you try to define. There exists many such clusters of terms, where each term has a property of relative scale that always seems to break down when scrutinized. I find subsystem, system and enterprise system architecture to describe structures. I find task, endeavor and project to describe effort. I find activity, method and practice to describe repeatable and effective effort. A fractal is an object where the structure of the parts are indistinguishable from the structure of the whole. It is not possible to infer from your surroundings at which scale you operate. The terms defined should work equally well on any scale. It also makes it unnecessary to define terms for more than one level, as the same terms can be applied recursively to form a structure of any complexity. A small effort will require a small method and result in a small system. A large effort will require a large method and result in a large system. All terms should be applicable both in the small and in the large. A fractal approach is what gives you extensibility.

    Then, in analogy with how systems are built up by subsystems connected via interfaces, a method should be built up by submethods connected via interfaces. The big work here is to define what a method interface actually is. The interface description needs to be formal to enable tool support for method composition. A method on the other hand, only have to be english text, but I can easily imagine that over time, repetitive and well defined methods are formalized to automate parts of the software engineering effort.


  2. It is great news that Semat is moving towards a standard and the draft RFP inspired by Semat has been approved by OMG.

    While most of the prevailing software engineering methods and methodologies have adopted an approach of suggesting / prescribing a specific method and assuming / evangelizing that it is applicable to a wide variety of situations.

    It is heartening to see Semat take a diametrically opposite route. Rather than specific methods it is creating a small kernel of universal elements. And it understands that the software practitioners face a variety of situations in their day to day working and need help in terms of generic patterns and guidelines to help them incrementally evolve their current way of working. I feel the software practitioners would greatly welcome such an approach. It will also result in a thriving software community where the members share with each other what works for them in which situations and over a period time help build a great body of knowledge.

  3. pemcmahon Says:

    Please see my recent blog on Semat titled, “What Does Lean Six Sigma And Semat Have in Common?” at paulemcmahon.wordpress.com


    • Hi Paul, A great blog. I was thinking that maybe instead of just making a reference you could copy your blog into the Semat blog, saying that this blog also appeared in your own blog. Maybe some adjustments, but not much

      I signed up for your new blogs

  4. pemcmahon Says:

    The following blog also appears in my blog at paulemcmahon.wordpress.com:

    What Does Lean Six Sigma and Semat Have in Common?

    I recently went through an intensive ten week training course followed by a five hour examination to achieve my Lean Six Sigma Black Belt Certification. Why I decided to invest the time to achieve this certification might not be why you think. To help explain I need to first talk about another initiative I have blogged about in the past– Semat (www.semat.org).

    I have been involved in the Semat initiative for almost two years. Because all of us working on Semat are volunteers our progress has at times been slower than what we would like, but it is currently on the upswing. For those who haven’t been following Semat closely, we are building a kernel– or what we refer to as the “common ground”– for software engineering. We realize we will only succeed if what we produce achieves wide acceptance. Our current focus is on preparing our response to the OMG FACESEM RFP (A Foundation for the Agile Creation and Enactment of Software Engineering Methods). http://www.semat.org/pub/Main/WebHome/Foundation_for_SE_Methods_RFP_ad-11-06-26.pdf.

    Some have questioned the value of spending time on the Semat initiative. As an example, I recently heard from a colleague who works for a large US defense company that their OMG representative questioned the value of Semat because they didn’t see any “beef” in what we are doing. If you attended our kernel working group meeting last Sunday you might understand this comment. Last Sunday we initiated a high level review of our first seven kernel elements (Opportunity, Stakeholders, Requirements, Software System, Team, Work, and Way of Working).

    Small Semat sub-teams have been working on descriptions, states and checklists for each of our kernel elements. We are now stepping back and looking for consistency across these descriptions, and part of this effort includes building a small, but commonly agreed to, glossary. One goal is for these kernel elements to be easily understood and to use a consistent vocabulary. We plan to take our work out to the broader community to get more feedback. This is critically important to ensure wide acceptance.

    Just to give you an example from last Sunday’s discussion– It was observed that in our initial kernel descriptions many words that were used seemed to by synonyms of others. One example we discussed were the three terms– problem, obstacle and issue. After lengthy discussion it was agreed that the way these terms were being used in our kernel description documents that they were essentially intended to mean the same thing. During this discussion it was also pointed out that the word issue, and problem often had a specific meaning inside certain organizations.

    One of our goals is to try wherever possible to avoid terms that may carry certain baggage or lead to misunderstanding. In this case we chose the term obstacle and initiated an effort to remove the terms issue and problem replacing them where it made sense with our preferred new glossary entry obstacle. What we are doing should not be viewed as trying to ban words within Software Engineering. We anticipate that specific organizations and specific software engineering segments will extend the kernel and its glossary to fit their needs.

    What we seek is not to limit, but rather to provide a small common ground starting point that we can all agree to. To keep this small and consistent we spend an incredible amount of time in our working groups looking for things to throw out of the kernel, including words that that don’t add value or words that serve to confuse rather than clarify. As a result, our goal could be viewed as eliminating “beef” that fails to stand up to our criterion of being universal to all software engineering efforts.

    Now I can understand how some might question if it is really worth the effort to spend so much time worrying about these common everyday words we use. Some might ask, isn’t our real goal to write high quality software?

    The answer is yes, our goal is to write high quality software. But those of us who have been working with software for a long time have learned that the most direct path to high quality software requires that we first know our goal, second we must know what must be done to achieve that goal, and third we must know how we will measure ourselves to determine if we have succeeded.

    We have learned that not hitting our goal too often results from unclear objectives and conflicting stakeholder needs and expectations. One way to help us address this frequent trouble spot is through a simple common language and small glossary of terms we can all agree to as a starting point.

    I know it may sound trite, but when software efforts fail it still often comes down to the same issues it came down to 40 years ago — miscommunication of requirements and miscommunication of stakeholder expectations.

    So what does all this have to do with why I invested time in becoming a certified Lean Six Sigma Black Belt? Lean Six Sigma emphasizes the importance of first understanding where the pain is, or stated differently; What is the real problem we need to solve? It second focuses on understanding the customer needs and how to translate those needs into critical to quality requirements which can me measured. It next focuses on quick-wins and getting rid of waste. To do this we make sure we understand each step of the process and how to distinguish value added steps from non-value-added steps specifically in the eyes of the customer.

    Lean techniques drives an organization to look hard inside their operation not just at how to fix defects once they occur, but how to keep them from occurring in the first place. Lean Six Sigma teaches us to look hard at our personal work habits, like cleaning up our work area making it easier to find what we need when we need it. Last Sunday our Semat volunteers spent a good deal of time cleaning up the language we use to express what we do. We are looking to eliminate non-value-added effort wherever it exists. We are working to make it easier to communicate accurately the first time.

    I didn’t decide to become a certified Lean Six Sigma Black Belt because I wanted to learn a new fad. I did it because I wanted to learn something old that we all know, but too often forget. Start by simplifying what you do. Eliminate waste wherever you find it. Eliminate anything that doesn’t contribute directly to your goal. Where is the “beef” you ask? Quality is not measured by how much beef we have. It is measured by how much lean we have and whether our customers are delighted with the value they receive from our products and services.

    Paul E. McMahon
    Principal, PEM Systems
    Author: “Integrating CMMI and Agile Development”
    Certified Lean Six Sigma Black Belt
    Phone: 607-798-7740
    Web: http://WWW.PEMSystems.com
    Email: PEMcMahon@ACM.ORG


  5. I’m having troubles seeing where this initiatives fits in along side CMMI. Many developers might want to know that as well before aligning themselves with the initiative. I still see that the initiative can benefit from the Lean Startup principles.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 50 other followers

%d bloggers like this: