Evaluating Software Architectures: Methods and Case Studies

The authors provide an in-depth treatment of three methods forevaluating software architectures, all of which were developed at theSoftware Engineering Institute with involvement by the authors. Themethods examined are:(1) ATAM (Architecture Tradeoff AnalysisMethod)(2) SAAM (Software Architecture Analysis Method)(3)ARID (Active Reviews for Intermediate Designs)Each of the aboveaddress software evaluations in increasing levels of detail, with thebook's main emphasis on ATAM.What makes this book so valuable isthe fact that you can learn much about developing softwarearchitectures from the criteria with which they are evaluated. Forexample, the discussion on quality attributes is eye-opening becausewhat architects consider to be well formed quality attributes areusually too vague to properly evaluate, resulting in ill definedarchitectures in the first place. Knowing how to evaluate thearchitecture will provide the keys for defining a solid architecture.More important is the way the authors define the outputs of thearchitecture evaluation, which gives the practicing architect aframework for design that fully meets the evaluation criteria. Thenet result is that a defined architecture will unambiguouslycommunicate the design to the development team, as well as to the QAteam.I especially like the business oriented approach thataddresses the costs and benefits of evaluation, the three approachesfrom which to choose that best meets technical and business goals, andthe case studies that support each of the approaches. Another strongpoint about this book is architecture is also evaluated withproduction in mind. Too many books only consider architecture fromthe development point of view, or in rare cases, from development andQA points of view. The evaluation techniques in this book extend tosupport and maintenance. The authors make selection of the besttechnique easy by comparing them in Chapter 9, and provide an approachto implement evaluations in Chapter 10.If you're an architect I also recommend augmenting the excellentmaterial in this book with Design and Use of Software Architectures byJan Bosch , which gives an alternate method to ATAM that is morecomplete in many respects. Even if you espouse Bosch's approach,however, the approach and techniques given in Evaluating SoftwareArchitectures: Methods and Case Studies are complementary. I personallyrecommend both books and assign equal value to them.

What this book does, it does very well. It presents three techniques for reviewing the suitability of a software architecture. The presentation style is clear, complete, and reasonably frank about the problems an architecture evaluator is likely to encounter.The oldest of the three techniques presented is SAAM, the Software Architecture Analysis Model. It's primary goal is to determine how well a system's structure addresses the technical requirements of the application, and its probable success at addressing future changes of requirements.ATAM, the Architecture Tradeoff Analysis Method, descends from SAAM but is far more complete. It starts upstream of the requirements, at the business model behind the application, then moves forward methodically through the top-level design. At each step, reviewers update the list of technical risks and non-risks (relatively safe items). ATAM is open-ended, in the sense that the project's own goals define the specific measures of quality that apply - it doesn't force-fit every project onto one Procrustean axis of measure.If ATAM is SAAM grown large, then ARID (Active Reviews for Intermediate Design) is SAAM scaled down. Where ATAM and SAAM address strategic issues about complete systems, ARID incorporates tactical information about specific design issues. It's not as narrow as standard design review techniques, but not as broad as an architecture review.ATAM is the main focus of the book, with more pages than SAAM and ARID combined. All three are described in full detail, however. The authors identify the specific skill sets, roles, and responsibilities that must be involved at each step. They present checklists for eliciting the kinds of information needed, even specifics of meeting agendas and meeting room equipment.That creates my second impression of this book: I was very disappointed. This book is for meeting organizers, and deals very little with technical specifics. That is not at all what I hoped for. It is not the fault of the book that it fails to meet my expectations. In my present work, however, the authors present just about nothing to enhance my project's technical content.This is a process book. It seems to be a good one. It takes what works in other design review methodologies, then expands that to the highest level of the software project. It gives enough detail that you can tune specifics of the process to specifics of your project. Still, it's just a process book.

It could be updated. It is on a 12th printing, and some of the content is showing its age. However, on the whole it provides a viable set of methods for evaluating software architectures from SEI - that is the purpose, after all. I recommend reading this book before the "green book" which is the Documenting Software Architectures book, only because that is the order in which they appear to have been authored. If you read this book second, some of the architectural style information which needs updating will annoy you. If you have to choose between the two books, I would recommend the Green Book.

Use of basic tools for trading off different aspects of an architecture are a necessary part of a software engineer's skill set. Good solid work comes from SEI and this book is no exception.

Like two other reviewers, I also found a lack of substantive technical content. The book does delve into great detail about the ATAM process, down to listing the various roles like timekeeper and questioner, and their responsibilities. Fair enough. And for this, there is indeed plenty of content for setting up and running the process.But try as I might, I could not get a firm grasp on how to actually choose between two [or more] software architectures. The technical examples cited in the text were invariably too skimpy for serious consideration. Perhaps the book would be enhanced by several solid, detailed examples and how to choose in those examples.

Very Good.