The research paper published by IJSER journal is about Quantitative Metrics for validating the effectiveness of the Model based approach for indigenously developed SWS/AIC system 1

ISSN 2229-5518

Quantitative Metrics for validating the

effectiveness of the Model based approach for indigenously developed SWS/AIC system

Manju Nanda, Chinmayi S Jamadagni

Abstract— The aim of this paper is to validate the effectiveness of model-based approach for the indigenously devel- oped stall warning and aircraft interface computer system (SWS/AIC) by generating the software engineering process metrics and the development of the empirical relationship between the conventional and the model-based approach. The quantitative metrics for software analyzability, changeability, testability, stability, traceability, safety compliance, reliability, design time, de- bug time, upgrade time, reusability, readability, maintainability, modularity, reachability and availability is derived and generat- ed for the two approaches to demonstrate the effectiveness of the model-based approach. The empirical relationship developed helps in analyzing the reduction in effort for development of safety critical software using model-based approach.

The metrics generated and the empirical relationship derived between the two approaches proves the effectiveness of the model-based approach over the conventional approach. The results of this work are encouraging for incorporation of the model-based approach for the design, development and verification and validation of safety critical systems.

Index Terms— Formal methods, Model based approach, Verification and Validation, Safety critical systems, Metrics, Stall Warning System.

1 INTRODUCTION

——————————  ——————————

Development of critical systems for i.e. the aeronautics or automotive industry requires a strict interdisciplinary ap- proach and conformance to standards and specifications in order to ensure safe systems, since failures are often cata- strophic and with loss of life as a consequence.

The development of embedded systems with real-time and other critical constraints raises distinctive problems. In partic- ular, development teams have to make very specific architec- tural choices and handle key non-functional constraints relat- ed to, for example, real-time deadlines and to platform param- eters like energy consumption or memory footprint. The last
implementation, testing, maintenance and evolution. The use of mathematical techniques reduces the possible personal in- terpretation.
The paper discusses a pioneering framework in mak-
ing the engineering process effective. The framework includes model based approach for the design life cycle and demon- strates the effectiveness of the approach by generating metrics. The approaches adopted for the comparative case study are conventional and model based. The design cycle for the con- ventional and model based approaches is as shown in Fig 1.

Fig 1

few years have seen an increased interest in using model-
based engineering (MBE) techniques to capture dedicated ar-
chitectural and non-functional information in precise (and even formal) domain-specific models in a layered construction of systems. MBE techniques are interesting and promising because they allow to capture dedicated architectural and non- functional information in precise (and even formal) domain- specific models, and they support a layered construction of systems, in which the (platform independent) functional as- pects are kept separate from architectural and non-functional (platform specific) aspects, where the final system is obtained by combining these aspects later using model transformations.
Model based engineering approach is the formalized application of modeling support system requirements, design, analysis and V&V. Mathematical rigor enables users to analyze and verify these models at any part of the program life-cycle: requirements engineering, specification, architecture, design,

http://www.ijser.org

Document-centric approach

REQUIREMENTS DESIGN

IMPLEMENTATION

TEST

Model-centric approach

REQUIREMENTS MODEL

DESIGN AUTO CODE

SYSTEM TEST

Model

TEST

The research paper published by IJSER journal is about Quantitative Metrics for validating the effectiveness of the Model based approach for indigenously developed SWS/AIC system 2

ISSN 2229-5518

Model-Based Design [3] with automatic code genera- tion is an important and established technology for develop- ing aerospace embedded control systems. Early verification,
validation, and test of models and generated code using soft- ware tools with accompanying workflows are increasingly used. Model-based design provides numerous advantages over the traditional design approach. Using the model-based approach, you reduce the risk of mistakes and shorten the de- velopment cycle by performing verification and validation testing throughout the development instead of only during the final testing stage. Design evaluations and predictions can be made much more quickly and reliably with a system model as a basis. This iterative approach results in improved designs,

28 VDC Ps Pt Tt

28 VDC ELEC.

SYSTEM

ADCU & AHRU

RAD ALT

ARINC-429

PILOT’S CONTROL WHEEL

SWS/AIC POWER SUPPLY

COPILOT’S CONTROL WHEEL

ARINC-429

ARINC-429/DISCRETE

EADI/PFD (PILOT SIDE) WITH EFIS

& ALT SELECT CAUTION WARNING

both in terms of performance and reliability. The cost of re- sources is reduced, because of reusability of models between design teams, design stages, and various projects and the re- duced dependency on physical prototypes. Development er- rors and overhead can be reduced through the use of automat- ic code generation techniques. These advantages translate to more accurate and robust control designs, shorter time to market, and reduced design cost.

2 STALL WARNING SYSTEM AND AIRCRAFT INTERFACE COMPUTER SYSTEM

The system under consideration is the Stall Warning System used in aircrafts. The purpose of the SWS/AIC system is to provide stall warning whenever the aircraft approaches stall angle of attack, display continuously the angle of attack information on the primary display, provide interface between Caution warning panel (CWP) and systems which require an interface for CWP and provide pitch trim function and moni- toring. The stall warning system is designed and modeled using both conventional process and model process and the metrics obtained are compared and analyzed in order to ob-

28 VDC Ps Pt Tt

FLAP SYSTEM

LG UNIT

FUEL SYSTEM

ADCU & AHRU

AOA SENSORS RT & LT THROTTLE

LEVER POSITION

DISCRETE INPUTS (WOW, AOA Heat. ETC.)

DUAL SWS/AIC

PROCESSORS (68060/68360)

 INPUT SIGNAL MANAGEMENT

 STALL ALGORITHM PROC.

 AIC PROC

 OUTPUT SIGNAL MGT.

 SERVO & MON.

CONTROL

 BIT

HYDRAULIC PRESS SENSOR

& ON/OFF SWITCH

Fig 2

COMMAND ENGAGE/DISENGAGE

SERVO COMMAND ENGAGE/DISENGAGE

ARINC-429

SWITCHES AND INDICATORS IN COCKPIT

SYSTEM

PITCH TRIM ACTUATOR SHAKER ACTUATOR

EADI/PFD (COPILOT SIDE) WITH EFIS

tain the footprints and figure of merit.
The model based approach [4] allows engineers to de-
sign embedded systems and simulate them on their desktop environment for analysis and design. Model-Based Design provides a variety of code generation capabilities that teams use to generate source code for many purposes including sim- ulation, rapid prototyping and hardware-in-the-loop testing. Model-Based Design promotes a requirements-oriented pro- ject view and greater integration and reuse between conceptu- al and detailed modeling and design work. The block diagram of the SWS/AIC is depicted in Fig 2.
The model based formal implementation of the stall warning system is done using Mathworks toolset (R2010a)[1]. After creating the model, it has to be tested extensively to en- sure that model is identical to the legacy source code. So the model validation and comparison of its outputs with the lega- cy source code becomes an important task in MBSE. MBSE uses a V- Model / Life Cycle for the model creation and its val- idation The Matlab/Simulink model of SWS is depicted in Fig
3. The SWS modeled in Simulink is simulated to check for
functionality and then auto code is generated for the model. The auto code generated is compared with the manual code, thus highlighting the advantages of model based approach over the conventional approach. The model is then verified using Simulink Design Verifier. The SDV generates Auto test cases for coverage of the model.

http://www.ijser.org

The research paper published by IJSER journal is about Quantitative Metrics for validating the effectiveness of the Model based approach for indigenously developed SWS/AIC system 3

ISSN 2229-5518

Fig 3
The Stall warning system module is subjected to Sim- ulink Design Verifier which checks the model for compatibility and generates test cases for the functionality. The SDV in- cludes formal prover engine which proves the properties of
the model. The SDV report for the Landing gear module along with coverage metrics and test cases are discussed below. Fig 4 depicts the test unit for the landing gear module. The inputs from the harness unit are given to the test unit in the form of signal builder as shown in Fig 4. Depending upon the module, the SDV log gives compatible, partially compatible and in- compatible results. The landing gear module taken into con- sideration is compatible with the SDV. The SDV generates a test unit ( module that is compatible with the SDV) for verification purposes to which inputs are provided through signal builder block. The signal builder block serves as a tool for generation of test cases i.e. auto test cases for coverage analysis. The Test case explanation in document format can also be obtained from the tool. Once the auto test cases are obtained, they are run to generate the coverage report for fur- ther analysis.
Fig 4

3 SYSTEM PROPERTY METRICS

The performance metrics [2] for analyzing the system design and the process are carried out on the SWS/AIC sys- tem. The system is first developed using the conventional document centric approach and later the model based formal approach. The SWS/AIC system is modeled using the Sim- ulink 2010a toolset. The auto code generated is compared with the manual code generated using the conventional ap- proach. The two processes are compared and the metrics ob-

http://www.ijser.org

The research paper published by IJSER journal is about Quantitative Metrics for validating the effectiveness of the Model based approach for indigenously developed SWS/AIC system 4

ISSN 2229-5518

tained are shown in TABLE I. Descriptive computation and the comparisons of the two approaches are shown in TABLE II. The system property metrics proposed and analyzed for the
case study are defined as follows and their interrelation is de- scribed formally as empirical formulae. The relationship be- tween the System Property Metrics and the performance met- rics can be described in the tree diagram with weighted meth- odologies in Fig 5.

Metrics Definition:

Reliability of a system can be defined as its ability to perform a given trial or probability that an item will last for a given period of time.[5]

Reachability can be attributed to analyzability and

traceability of a system .Hence an empirical relationship ex- ists between the former and later metrics.

Availability is attributed to analyzability and stabil- ity of a system.

Maintainability of a system is dependent on its

changeability, modularity, traceability, design time and up- grade time.

Safety is the most critical metric and cannot be com- pensated for in any approach. Safety critical systems are de- fined by this metric based on certain standards namely DO178B, DO178C etc... Safety metric is attributed to the testa- bility and modularity towards fault tolerance of a system.

Reusability defines the no of files that are being re- used during simulation and code generation.

Readability includes clarity in interfaces, uniformity

in appearance, coding and documentation.
A system is said to possess changeability if it is flexi-
ble, adaptable, scalable and modifiable [15].

Analyzability of a system is defined from the effort required to detect deficiencies and to modify it [6][7].

Testability of a system is satisfied if it is controllable, observable, isolatable, understandable, and automatable and offers heterogeneity [8].

Stability is defined as the ability of the system not to

hang, not to lose data, not to disrupt system functionality and
be predictable [16].

Modularity of a system defines its level of independ-

ence [11][12][13][14].

TABLE I

http://www.ijser.org

The research paper published by IJSER journal is about Quantitative Metrics for validating the effectiveness of the Model based approach for indigenously developed SWS/AIC system 5

ISSN 2229-5518

PARAME- TER

CONVENTIONAL AP- PROACH

FORMAL APPROACH

There is a 32 % reduction in the number of lines of code compared to the conventional approach. This is a consid-

Approach Document based Model based

erable reduction factor when the complexity of the system

Readability Textual; Interface comply coding standards and design styles; documen- tation complies in house documentation stand- ards

Modular ; Interface com- increases. The effort for programming also reduces; hence de-

ply coding standards and signers can concentrate more on the other phases of the life

design styles; MAAB style cycle. From the data obtained from the case study, comparison

chart is developed for the property metrics in order to obtain
the improvement in the approach involving the formal meth-
ods as compared to the conventional approach.

Changeabil-

ity

Modification done at all

levels of design ; Scala-

ble : more effort; Manual

Modification done at top

level of the design

;Scalable by tool ; Auto- mated

Testability Driven by impact analy- sis; Code review; Func- tional isolation ; Manual

Driven by functionality of the model; Module isola- tion; Auto code generation

; Tool dependent

100%

80%

60%

appr oach

Analyzabil- ity Traceability

[9]

Manual ; Code reviews; Impact analysis Requirements  Design

Code Report ; trace-

ability matrix generated

manually

Automated; Tool depend- ent

Require- mentsModeldesign codeReport ; traceability matrix generated by tool

reliabilityreachabilitayvailabilmityaintainabilitysafety

appr oach

Stability Stable Stable

FIG 6

Modularity Level of independence based on architecture ;

Level of independence based on modules; Interac-

From the above chart (Fig 6) it can be observed that

Interaction by means of

drivers; Specification based on configuration files; Coupling on control and data coupling

tion by means of function- the model-based approach provides an improvement of alities of models; Specifi- 16.34% in the proposed system property metrics keeping the cation based on inputs reliability factor intact. This further helps in obtaining an

;control and data coupling effective systems engineering framework that integrates for-

mal methodologies. The effect of the weighted methodologies

Safety Safe Safe

Design time More ; manual Less ; automated

on the system property metrics for both the approaches is de- picted in the charts given below. (Fig 7 a and Fig 7 b)

Verification

&Validation

Done at the end of the

cycle

Can be done at the start of

the cycle

Test cases Manual ; depends on functionality

Tool generated ; optimized

RBD

TABLE II

From the case study implemented, an empirical relation- ship is deduced between the two approaches.
 Size of manual code ≥ 2 * (size of auto code generated)
 Commented lines = exp (executable lines of code)
 System property metrics for formal approach = 1.258 * ( conventional approach)
 Development Effort is calculated from the statistical
model formula:

100%

80%

60%

40%

20%

reliability reachability availability maintainability safety

analyz

ability

tracea bility

st abilit

modul

arity

change ability

test abi

lity

E = 5.2 * L^ 0.91;

L is the no of lines of code in thousands. (The formula is derived by Walston and Felix with a =5.2 and b=

0.91, constants obtained by regression analysis)

Fig
7(a)

http://www.ijser.org

The research paper published by IJSER journal is about Quantitative Metrics for validating the effectiveness of the Model based approach for indigenously developed SWS/AIC system 6

ISSN 2229-5518

RBD

100%

80%

60%

40%

20%

reliability reachability availabilitymaintainability safety

Fig 7(b)

analyza bility

traceab ility

stability

modula rity

change ability

testabil ity

ACKNOWLEDGMENT

We acknowledge Director CSIR-NAL to carry out this work.

REFERENCES

1. CONTROL ALGORITHM MODELING GUIDELINES USING MATLAB®, Simulink®, and Stateflow® Version 2.1 MathWorks Automotive Advisory Board (MAAB) July 27th, 2007.

2. “Comparison of software metrics tools” by Rüdiger Lincke, Jo- nas Lundberg and Welf Löwe. Software Technology Group, School of Mathematics and Systems Engineering, Växjö Univer- sity, Sweden.

3. “Measuring productivity and quality in model based design”:

excerpt from MATLAB digest; March 2006

4. “Model Based Design for DO178B with qualified tools” by Tom

Errkinen and Bill Potter; Mathworks Inc.

5. “A new approach to system reliability” by Go pal Chaudhri,

Kuolong Hu and Nader Afshar. (IEEE transactions on Reliabil- ity : March 2001)

4 CONCLUSION AND FUTURE WORK

Using Model-Based Design, verification and valida- tion activities occur throughout development. A number of new technologies have been introduced that assist with early model verification such as requirements traceability, model checking, model coverage, formal methods, and test case gen- eration. Continuous verification and validation of require- ments throughout the design life cycle reduced errors and development time.
The results obtained from the work not only helped in
deriving an empirical relationship between the model-based approach and the conventional approach but also highlighted its advantages over the conventional approach. The applica- tion of the model based approach in safety critical domain has proven to be effective and can be extended to more critical functionalities in the domain. The same approach can be im- plemented at design level which encourages V&V at the top most level of the design life cycle, thus ensuring correctness of the system right at the start of the life cycle. Also there are oth- er commercially available tools that support model based de- velopment involving formal techniques apart from Math- works. The other available tools can also be used for imple- menting the approach and a comparative analysis of tools can be done in order to find an effective tool for the particular ap- plication.

6. “Quantifying the analyzability of Software Architectures” by Er- ic Bouwers, Jos´e Pedro Correia, Arie van Deursen and Joost Visser (Delft University of technology ,Delft , Netherlands)

7. www.arisa.se/compendium/analyzability

8. IEEE standards glossary of software engineering technology

9. “On traceability for safety critical systems engineering” by Dr

Paul Mason ( Shinawatra University, Thailand)

10. “Creating Safety Requirements Traceability for Assuring and Recertifying Legacy Safety-Critical Systems” by Janice Hill and Scott Tilley ( IEEE International Requirement Engineering Co n- ference 2010)

11. “Modularity in Design of Products and Systems” by Chun-Che

Huang and Andrew Kusiak ( IEEE 1998)

12. Using Software Architecture Techniques to Support the Modu- lar Certification of Safety Critical Systems” by Tim Kelly (Un i- versity of York, UK)

13. “Meaning , Memory and Modularity” by Stephen Crain , Weijia Ni , Donald Shankweiler , Laura Conway and David Braze (University of Maryland and University of Connecticut)

14. “Measuring Software Design Modularity” by Yuanfang Cai and

Sunny Huynh (Drexel University , Philadelphia)

15. “Defining Changeability: Reconciling Flexibility, Adaptability

,Scalability, Modifiability and Robustness for maintaining Sys- tems Lifecycle value” by Adam M Ross , Donna H Rhodes and Daniel E Hastings ( Massachusetts Institute of technolo- gy,Cambrige)

http://www.ijser.org

International Journal of Scientific & Engineering Research Volume 3, Issue 12, December-2012 7

ISSN 2229-5518

16. "Stability Monitoring and Analysis of Learning in Adaptive Sys tems" by Edgar Fuller, Bojan Cukic,Martin Mladenovski and Sam path Yerramalla (West Virginia University)

IJSER lb)2012

htt p://www .'lser. ora