As of today, much like a few years ago, it remains challenging to comprehend the reality of the software industry and the engineering behind it. Why is it so difficult to establish and stabilize a company in our industry, dedicated to software development? Why is maintaining cash flow in such companies challenging? Why do more and more software companies resort to alternatives like OUTSTAFFING (simply renting the human talent that has been hard to acquire) to provide financial support to the company? I often ponder these questions on my way to work, and it’s easy to lose sight of them in my routine as an analyst. However, they represent the reality for those who have taken on the challenge of building a company in an industry that is barely 70 years old.

Even though industry questions get lost amidst my tasks, as an ordinary software engineer, I am not exempt from contemplating other uncertainties. Why do I recall so little of the theoretical foundation of my profession? Why did I study 4 semesters of physics and 5 semesters of theoretical calculus in my university career? Why did my career truly become constructive when practice was included? Why, upon entering my first job, could I salvage much from the practical aspects learned in my career and so little from the theory? All these questions highlight the reality of software engineering, sometimes making it seem like we try to shape ourselves as engineers by following the learning path of other engineering disciplines, learning a bit of everything but losing sight of a common underlying thread (an identity) that we have not defined as an industry. Nevertheless, we shouldn’t be too harsh on our discipline or ourselves as practitioners. As mentioned initially, this discipline has only around 70 years of history (considering its real emergence during the Second World War) and has still propelled humanity into a new era.

As engineers, we should adhere to rigorous, disciplined, and professional practices for software development (similar to those observed in civil engineering). However, what we have truly embraced as Software Engineering is a set of practices borrowed from external disciplines such as project management, design, and quality control. Initially, the idea was to treat software as a manufactured product, built on a production line, applying engineering in phases familiar to all software practitioners: requirements gathering, analysis, design, construction, and testing. This manufacturing perspective makes sense in other disciplines where the initial phases of work have strong theoretical and practical foundations, making the initial effort in the process worthwhile. Unfortunately, this is generally not the case in the software industry. This has led to the devaluation of the work of developers, who are the ones truly making the software function, regardless of what the initial blueprints or designs assert. Given this, there is ongoing debate in today’s world about whether what we do in software engineering should be treated as an art. While for many, this might be seen as a reason for celebration, what is really being asserted is that software engineering is still an artisanal process, leading us to question whether we are engineers or artisans. SOFTWARE DEVELOPMENT, as such, is an artisanal discipline, built on the experience of software masters, guiding the community to construct better solutions with greater quality. Additionally, agile practices for software development have made it possible for large-scale solutions not to lose quality in this artisanal process.

However, as an artisanal process, this can only take us so far. To explain this, let’s draw a parallel with construction engineering.

La Sagrada Familia, Barcelona, España. Créditos foto: de Manuel Torres Garcia

We know of many wonders in the world built by great artisans, such as medieval castles, beautiful cathedrals, or large statues. Unfortunately, these grand works were incredibly costly and took a considerable amount of time to finish. Some have not been completed even after more than a century, like the Sagrada Familia Cathedral in Barcelona, and some ended in disasters after a few years. On the other hand, modern constructions like skyscrapers take much less time to build, and the likelihood of them ending in disaster is minimal. This is all thanks to the development over the years of a genuine engineering approach. All modern constructions, from the smallest house to the largest building, have solid foundations in material science and structural theory. Professionals in the field use these theoretical foundations as a primary basis for disciplined and careful design of the structures they wish to build. While they may occasionally fail, this same theory guides the professional (civil engineers or architects) to find the cause of the error and the solution to it. Construction engineering, besides being an excellent parallel to our discipline, is a perfect example of how an engineering field combines an artisanal process with a solid theoretical foundation, making it possible to effectively educate new professionals in creating new structures and solving problems related to them. This perspective gives us an insight into the reality of software engineering, which, from this standpoint, is not truly an engineering discipline.

So, how should we proceed? Perhaps we should skip the part where we think about reclaiming our money from our respective universities and focus on the challenge ahead: restructuring software engineering.

A few years ago, some organizations in the software industry took on this challenge through the creation of the SEMAT (Software Engineering Method and Theory) community. It has the mission to re-establish software engineering based on sound theory, proven principles, and best practices. Some of its members include ABB, Ericsson, Fujitsu UK, Huawei, and TCS, along with universities such as CARNEGIE MELLON, and with the support of the OBJECT MANAGEMENT GROUP (OMG), and individuals like IVAR JACOBSON, BERTRAND MEYER, and RICHARD MARK SOLEY. SEMAT aims to support the creation of a common ground, a core, or a foundational pillar for software engineering. Currently, SEMAT has seven chapters working globally to establish among different software communities the core of software engineering, the principles tested over the years, and the best practices of today’s software factories. To achieve this, SEMAT has created Essence, defined as a first approach to the standardization of software engineering, i.e., the common ground, the core that we had not defined for a long time. This standard allows teams to understand and visualize the progress and status of work done in software engineering, regardless of the method or practices used. Moreover, it enables these teams to create working methods based on their best practices, contributing to the re-foundation of software engineering. Essence basically consists of defining three elements:

• A means of measuring the progress and health of an effort.
• A categorization of the activities necessary to progress an effort.
• A set of competencies necessary to carry out these activities.

With these elements in mind, Essence focuses on the importance of understanding how an effort is progressing. For this, Essence defines seven dimensions or alphas to measure this progress: Opportunity, StakeHolders, Requirements, Software System, Work, Team, and Way of Working (methods).

Each alpha has a set of states that establish points along the dimensions of the progress of the effort. Each state has a checklist with sub-states, helping to determine the current location of an effort in the context of a specific alpha and establishing the sub-states necessary to move to the next state. A more tangible example of this is a set of cards, easily managed by a software engineering practitioner, thus keeping the effort’s status up to date.

This is a brief overview of the results of the work carried out by SEMAT with the aim of restructuring software engineering. All this information can be expanded through the Essence documentation, which is freely accessible on the community’s website WWW.SEMAT.ORG. As we can see, the software engineering community worldwide has understood the problem initially posed in this article, and efforts are being made to establish the foundations of our work as engineers. It’s worth noting that SEMAT’s work is still in progress; Essence is just a first step towards the restructuring of software engineering. There might be paradigms that, in the eyes of any of us, have not been included in this common ground, and thus, we could contribute to this initiative. Additionally, we must remember that SEMAT works on three main elements for this restructuring: a solid theory or common ground, proven principles, and best practices. The latter two are still being studied by researchers in this community, and as such, practices and methods are born every day with our work. Therefore, one way to contribute to this initiative could be to cultivate these practices and methods in the common ground of Essence and share our findings with the software community. As a professional not dedicated to research, all I have for now is gratitude to the SEMAT community for this initiative, a community of software professionals currently contributing with their work and experience to the restructuring of software engineering, or as Ivar Jacobson stated, “… Or, perhaps, founding it for the first time.”

• Dr. Ivar Jacobson, Dr Pan-Wei Ng, Paul McMahon, Ian Spence, Svante Lidman, P. 2012. The Essence of Software Engineering: The SEMAT Kernel; http://queue.acm.org/detail.cfm?id=2389616. Dr. Ivar Jacobson, Dr Pan-Wei Ng, Paul McMahon, Ian Spence, P. 2014. Major-league SEMAT—Why Should an Executive Care; http://queue.acm.org/detail.cfm?id=2590809.