Recommendation-Based Conceptual Modeling and Ontology Evolution Framework (CMOE+)

Within an enterprise, various stakeholders create different conceptual models, such as process, data, and requirements models. These models are fundamentally based on similar underlying enterprise (domain) concepts, but they differ in focus, use different modeling languages, take different viewpoints, utilize different terminology, and are used to develop different enterprise artifacts; as such, they typically lack consistency and interoperability. This issue can be solved by enterprise-specific ontologies, which serve as a reference during the conceptual model creation. Using such a shared semantic repository makes conceptual models interoperable and facilitates model integration. The challenge to accomplish this is twofold: on the one hand, an up-to-date enterprise-specific ontology needs to be created and maintained, and on the other hand, different modelers also need to be supported in their use of the enterprise-specific ontology. The authors propose to tackle these challenges by means of a recommendation-based conceptual modeling and an ontology evolution framework, and we focus in particular on ontology-based modeling support. To this end, the authors present a framework for Business Process Modeling Notation (BPMN) as a conceptual modeling language, and focus on how modelers can be assisted during the modeling process and how this impacts the semantic quality of the resulting models. Subsequently, a first, large-scale explorative experiment is presented involving 140 business students to evaluate the BPMN instantiation of our framework. The experiments show promising results with regard to incurred overheads, intention of use and model interoperability.


Introduction
Conceptual models are used by enterprises to describe formal aspects of the physical and social world for the purpose of communication and understanding (Mylopoulos 1992). As the various stakeholders of an enterprise have different backgrounds and knowledge, they each use different modeling languages in order to achieve their specific goals. This results in conceptual models (e.g., requirements, data, process models) that are not interoperable and hard to integrate (Hahn 2005;Hofferer 2007;Becker et al. 2009b).
To solve this model interoperability problem, researchers from different fields have proposed using ontologies, albeit in distinctive ways. One research line proposes enterprise ontologies (e.g., Uschold et al. 1998;Geerts and McCarthy 1999), which describes shared concepts and relations across enterprises -to promote model interoperability. Enterprise ontology facilitates the modeling process by suggesting a limited set of enterprise concepts and relationships. However, it also constrains the freedom of the modeler, who is obliged to use generic ontological enterprise elements instead of well-known, conventional terms within his/her enterprise. Another downside is that the specifics of the particular enterprise and its domain may not be reflected in the generic enterprise ontology.
A second research line uses an ontology that is specifically developed for a particular enterprise, sector or application. This ontology is used to either suggest labels for the model elements (Delfmann et al. 2009;Becker et al. 2009b), annotate the model elements (Born et al. 2007;Thomas et al. 2009), or achieve a combination of both (Di Francescomarino et al. 2011). In this case, the ontologies describe the concepts, relations and axioms that are typical of and shared within a particular enterprise; they should therefore be considered enterprise-specific ontologies (ESOs). The main benefits of this approach are that the ontology can be fine-tuned to the specific enterprise-context and, as opposed to most enterprise ontology approaches, no custom modeling elements or language are imposed. The drawbacks are the lack of guidance during modeling and the additional effort required (as annotations are mostly added after model creation), as well as the fact that the ESO quickly becomes extensive and complex, and therefore difficult to manage, keep up-to-date and use.
In this article, we present a novel, holistic approach to assist conceptual modelers within an enterprise in creating semantically annotated, better interoperable and integrable models by means of an ESO. At the same time, this ESO is maintained and developed in order to reflect the evolving enterprise. Essentially, we propose a generic framework called CMOE? (Recommendation-based Conceptual Modeling and an Ontology Evolution Framework) that puts the enterprise's knowledge encoded in the ESO to good use: we use it to recommend relevant concepts and relationships to the modeler which can be used as labels for a model element, and to automatically semantically annotate the models by means of the chosen ESO concepts/ relationships. Furthermore, the ESO evolution process is steered by the feedback we collect on the use of modeling suggestions. CMOE? thus establishes a symbiotic relationship between conceptual modeling, on the one hand, and ESO maintenance and evolution, on the other. With CMOE?, we manage to overcome the drawbacks of both above-mentioned research lines by combining their advantages. Firstly, we recognize that a well-developed, up-to-date ESO is beneficial for enterprises: apart from contributing to the resolution of interoperability issues, it also serves as a knowledge base incorporating concepts and relations that are used throughout the enterprise. Secondly, we acknowledge that enterprises already have a way of working and that certain workflows, preferred modeling languages and artifacts, or IT tools are already in use. Our framework therefore does not impose new working procedures or a rigid, generic ontology or custom modeling language, but instead is designed to support existing, well-known modeling approaches. Thirdly, we recognize that the ESO will contain a large number of concepts and that, as a consequence, a recommendation mechanism is needed to keep the effort involved under control. We therefore believe that the presented framework incorporates a tangible contribution to the state-of-the-art in the field.
As mentioned, CMOE? is a generic framework: it defines and implements our modeling method's workflow, along with common functionalities (e.g., recommendation functions, semantic annotation mechanisms, feedback capturing), and it may be instantiated and further specialized to support different concrete modeling languages. In this article, we present one such concrete (partial) instantiation, CMOE?BPMN, which provides recommendationbased modeling support for business process modeling (BPMN). Finally, using an extensive explorative experiment, we evaluate the presented framework, and discuss its impact on the semantic quality of the resulting models, the model interoperability, the time and effort required, their usefulness, and community acceptance.

Related Work
Existing ontology-based approaches to enhance model interoperability can be classified along two dimensions: (1) approaches that indirectly promote interoperability by means of the modeling language, versus approaches that directly impact on the conceptual model itself (Hofferer 2007), and (2) approaches that enforce interoperability while creating the model (i.e. avoiding model variations), versus those that create interoperability after the model is created (i.e. managing model variations) (Becker et al. 2009b). These dimensions will be used to review the relevant literature below (see Fig. 1).
Within the UEML (Unified Language for Enterprise Modeling) project, the constructs of different conceptual modeling languages are mapped to an intermediate language, which has its origin in the Bunge Wand Weber ontology. Next, these ontological mappings are used to create interoperability between models . The Enterprise ontologies mentioned in the introduction (Uschold et al. 1998;Geerts and McCarthy 1999) are mostly used to develop an enterprise modeling language which is immediately applied during the creation of the model. The work of Becker et al. (2009a), which is based on the ideas of Pfeiffer (2007), uses a domainspecific modeling language to constrain modeling choices, aiming to avoid model variations and promote interoperability.
Approaches that focus directly on the model, as our approach does, use either ontology annotation or matching techniques. For instance, the approach proposed by Born et al. (2007) and Di Francescomarino and Tonella (2009) considers the process model as given and includes an easyto-use mechanism to annotate these models with elements of an ontology. Another example is the work of Pittke et al. (2013), which focuses on locating inconsistencies within model repositories by identifying synonyms and homonyms by means of matching techniques. As a third example, Becker et al. (2009b) and Delfmann et al. (2009) force the modeler to use naming conventions while s/he adds labels to the model. These naming conventions have their origin in a set of domain terms and phrase structures, and are validated with matching techniques.
What is important to note is that in the process modeling domain, semantically enriched process models are not only used to promote interoperability between process models. They can also be used to automatically analyze business processes (Becker et al. 2010;Fill 2011aFill , 2012 or as semantically enriched, machine-readable process specifications for a semantically enhanced process engine (Hepp and Roman 2007;Leutgeb et al. 2007). As a consequence, different authors have proposed languages or frameworks that support adding ontological annotations to process models (Thomas et al. 2009;Fill 2011b) or allow transforming a process model into a semantic business process (Hepp et al. 2005;Abramowicz et al. 2007;Cabral et al. 2009). CMOE?, the framework described in this article, is classified as an Exaptation in the design science research knowledge contribution framework of Gregor and Hevner (2013), in the sense that known solutions are adapted to a new problem context. With respect to using known solutions, CMOE? falls in the bottom right classification: during model creation, it (automatically) semantically annotates model elements. CMOE? additionally addresses the problem of finding the correct ontology concept to annotate with, hereby recognizing the sheer number of concepts typically present in a domain or enterprise ontology. To this end, recommendation mechanisms are proposed to rank ontology elements according to different criteria (see Sect. 3.3) and recommend these to the user during modeling. As such, no restrictions regarding modeling language, structure of models, or use of labels are imposed; instead, the user is guided towards consistent and correct use of terminology within the enterprise ontology. In contrast to related work, where in some cases smallscale validations were performed, we present a large-scale experiment to evaluate various aspects of the presented approach (see Sect. 5). Although this is not the main focus of this article, it is noteworthy that CMOE? also supports the evolution of the ontology and in fact uses feedback gathered during recommendation-and ontology-assisted modeling to help develop the ontology.

Recommendation-Based Conceptual Modeling and Ontology Evolution (CMOE1) Framework
The CMOE? framework was conceived through the Design Science Research Methodology (DSRM) (Hevner et al. 2010), a sound theoretical framework that guides design research and aims at constructing artifacts that solve real-world problems. CMOE? is one of these artifacts, and is represented in Fig. 2. The java implementation of the CMOE? framework is publicly available (Gailly 2016). It consists of two cycles, the Conceptual Modeling (CM) and Ontology Engineering (OE) cycle, and establishes a symbiotic relationship between these. This paper describes the development and evaluation of the ontology-assisted modeling part of CMOE?; the ontology feedback and evolution part will be the subject of a forthcoming publication. The next subsections give a detailed description of the ontology setup, the ontological analysis of the modeling languages, the ontology storage, the recommendation services, and the model creation phases of the CMOE?.

Ontology Setup
The OE cycle commences with the Ontology Setup phase, in which the enterprise decides which ESO it will take as a starting point. The ESO can be created by means of an existing ontology engineering method (for an overview, see Additionally, the enterprise may start from an existing domain ontology that covers the business domain (e.g. the Resource Event Agent Enterprise ontology by Geerts and McCarthy 1999 or the Enterprise Ontology by Uschold et al. 1998) and that is gradually transformed into the ESO. Once developed, the ESO needs to be grounded in a core ontology according to good ontology engineering practice (Guarino 1998). A core ontology describes universally agreed upon, high-level concepts and relations, such as objects, events, or agents (Guarino 1998), and thus provides well-founded semantics, facilitates data integration across different (sub-) domains, and forms the basis for subsequent interoperable application building. CMOE? does not prescribe a specific core ontology, yet we recommend and provide support for the Unified Foundational Ontology (UFO) ) since re-usable analyses of conceptual modeling languages are available in Fig. 2 Recommendation-based CMOE? framework the literature. Different approaches and tools are available to ground the enterprise-specific ontology in a core ontology. For instance, core ontology patterns can be used to develop or analyze ontologies (Blomqvist 2005;Ruy et al. 2015). Other useful tools for ontology engineers are ONTOCLEAN (Guarino and Welty 2002) and OntoUML , which can be used to evaluate the grounding of ontology concepts in the core ontology.

Ontological Analysis of the Conceptual Modeling Language
The first phase of the conceptual modeling cycle is another initialization phase, in which an ontological analysis is performed for the target conceptual modeling language(s) used in the enterprise. Different authors have proposed methodologies and frameworks to achieve this (Evermann and Wand 2005;Harzallah et al. 2012;Guizzardi 2013). The purpose of these methodologies and frameworks is (1) to provide a rigorous definition of the construct of a modeling languages in terms of real-world semantics, (2) to identify inappropriately defined constructs, and (3) to recommend language improvements which reduce a lack of expressivity, ambiguity, and vagueness (Almeida and Guizzardi 2013). In CMOE?, the goal is not to improve the language itself, but to relate the constructs of the conceptual modeling language to the core ontology selected in the ontology setup phase. These connections can later on be exploited in the conceptual modeling recommendation service (see Sect. 3.4). Over the years, different conceptual modeling languages have been analyzed with, for example, Bunge-Wand-Weber (e.g. UML class diagrams in Opdahl and Henderson-Sellers 2002) and UFO (e.g. BPMN in Guizzardi and Wagner 2011). Although the added value of these ontological analyses have generally been accepted, their translation into conceptual modeling practice has been limited. While CMOE? does not prescribe any particular core ontology, it does currently support ontological analyses using UFO or BPMN (see Sect. 4 for more details) and i* (not reported here).

Ontology Storage
Efficient ontology storage is essential in order to easily query and update the ontology and ensure efficient recommendation services. Based on our extensive experience with implementing the framework for BPMN and i*, CMOE? currently supports the Web Ontology Language (OWL) 1 as ontology representation language for various reasons. First of all, it is a generally accepted (W3C) ontology language standard, supported by most ontology engineering tools (e.g. Protégé) and with APIs for various programming languages. In addition, OWL 2.0 supports punning, which is heavily used in our approach (see further in this subsection) (Grau et al. 2008). Finally, OWL offers highly optimized storage media, such as the Stardog semantic graph database, 2 which is used as storage medium in CMOE?. This database was selected for ontology storage in CMOE? because of its support for OWL 2.0, excellent access and querying performance, and support for Java, which is also used by our Eclipse-based modeling tools. Another advantage of Stardog is that it makes CMOE? ready for a future production-level implementation, as it is specifically optimized to handle huge, highly interconnected datasets.
The Stardog Database consists of different interconnected OWL ontology files. Figure 3a gives an overview of the different ontology files and their relationships, while Fig. 3b

Recommendation Services
Based on the above-mentioned stored ontologies, the recommendation services determine what ESO concepts are suggested to the modeler. For each ESO concept, each recommendation service calculates a recommendation score between 0 and 1, with respect to a modeling element added by the modeler. The final relevance score is a weighted average of all individual recommendation scores, creating a (weak) ranking for suggested ESO concepts (see Sect. 3.5). Consequently, ESO concepts are ordered according to relevance, which is essential to help modelers find appropriate concepts quickly, as the ESO rapidly becomes large and complex. CMOE? supports three recommendation services: Fig. 3 Ontologies CMOE? framework 1. The model language recommendation service deduces recommendations based on an ontological analysis of the conceptual modeling language: given a modeling language construct, its associated CoO concepts are derived using ontological analysis mapping and then compared with ESO groundings in CoO concepts. The pseudo code is given in Listing 1. First, a working ontology is considered, merging a selection of ontologies that are available in the framework (line 2). Next, the ontology reasoner is used to extend the ontology with assertions. This is accomplished with both the classification mechanism and realization mechanism of the reasoner (line 3). Here, the added ontology assertions have their origin in the equivalence relations that are defined in the CoO-MLO file, and will result in classifying some of the ESO concepts as individuals of the MLO constructs. After this, the SPARQL query service of the reasoner is used to create a collection of ESO concepts that belong to the type of the modeling language construct that is given as input (line 4 and 5).
The FOR EACH block starting in line 6 is a consequence of the punning mechanism. It uses the SPARQL query service of the reasoner to add the subclasses of the existing ESO concepts candidates (lines 7-9). Finally, the IF-ELSE block of Line 11 checks whether the ESO concept that is given as input of the algorithm is a member of the created ESO candidates set. If this is the case, the algorithm returns the (individual) recommendation score 1; if not, 0 is returned.
Listing 1 Pseudo-code Model language recommendation service It is important to note that in Listing 1, for the sake of simplicity and clarity, we describe the recommendation service that calculates the relevance score for one particular ESO concept. In our implementation, such a relevance score is calculated for all ESO concepts, hereby caching static intermediary results (e.g., ESO candidates) for efficiency.
2. The label-based recommendation service uses the ESO and natural language processing techniques (i.e. string and synonym matching) to give a relevance score to an ESO concept based on lexical distance of the concept name (and all its synonyms) and the label that is entered by the modeler. Listing 2 presents the pseudo code. In line 3, the string matching score is calculated using Jaro-Winkler distance (Winkler 1990) between the label that is entered by the modeler and the label of the ESO concept. Line 4 of the algorithm creates a collection of synonyms for the label of the ESO concept using WordNet (Miller 1995). This collection is used by the FOR EACH block (line 5), which calculates the string matching score between the entered label and every synonym from the collection. The FOR EACH block only remembers the highest matching score. Finally, line 8 returns this stored matching score, which is the (individual) recommendation score.
Listing 2 Pseudo-code label-based recommendation service 3 The rule-based recommendation service uses the rules specified in RulesO to identify suggestions for labels of modeling element added by the modeler. Listing 3 presents the pseudo code. The algorithm starts with creating a new modeling element (see line 2) which corresponds to the model element that is currently selected by the modeler and which is not yet annotated.
To ensure that the recommendation service takes this element into account, the element is added to an updated version of MoO (i.e. MoO'). Next, similar to the model language recommendation service, the algorithm assembles a new working ontology, which is extended with assertions by the reasoner (see lines 4 and 5). Compared to the model language recommendation service, the rule-based recommendation service also uses the RulesO and MoO' as input, which are used by the rules reasoning service of the reasoner to add new suggestions (in the form of asserted semantic annotations) for the currently selected model element.
After reasoning, the algorithm creates a collection which contains all ESO concepts for which the reasoner identified a potential semantic annotation for the new element. If the ESO concept that is given as input of the algorithm is an element of this collection, the algorithm returns 1 as individual recommendation score; if not, 0 is returned.
Listing 3 Pseudo-code rule-based recommendation service

The Conceptual Model Creation Phase
In the Conceptual model creation phase (CM cycle), the modeler is presented with an ordered list of ESO recommendations, based on the selected modeling language construct and the label entered. The (weakly) ordered list is calculated through a (configurable) weighted average of individual recommendation service scores, which determines the order in which the ESO concepts are presented to the modeler. The modeler is free to accept or discard a recommendation. If s/he accepts a recommendation, the selected model element is automatically annotated with the corresponding ontology concept, and the label of the modeling construct that is added is updated with the name of the selected ESO recommendation. CMOE? currently supports semantic annotations using OWL. In line with Thomas et al. (2009), the ontology annotation is stored in the MoO by adding an assertion of the semantic annotation object property between the MoO OWL individual and the ESO OWL individual. Additionally, during modeling and while the process of either adopting or discarding recommendations, feedback is gathered and stored in a log file. This log is stored in the mxml format which means that it can be processed by the ProM process mining tool. 3 The events that are stored in the log are (1) the generation of recommendations for the label entered, (2) acceptance of a recommendation by annotating the model, and (3) deletion of model annotation.

Recommendation-based Business Process Modeling (CMOE1BPMN)
To demonstrate that the CMOE? framework is a feasible, adequate and efficient solution for the presented problem, it was instantiated for process modeling by means of BPMN.
Consequently, we will now move on to describe the CMOE? recommendation-based business process modeling implementation (i.e. CMOE?BPMN) that uses, specializes and extends CMOE?'s generic functionality. The CMOE?BPMN implementation is an Eclipse plugin which can be downloaded from GitHub 4 and is shown in Appendix D (available online via http://springerlink.com). By means of the eclipse plug-in extension point mechanism, the CMOE?BPMN plug-in extends the Eclipse BPMN2 modeler 5 with two views and a preference page. BPMN2 Modeler is a graphical modeling tool which is built using Eclipse Graphiti in combination with the BPMN 2.0 EMF meta-model. Graphiti is an Eclipse-based graphics framework that enables the rapid development of diagram editors starting from an EMF meta-model. The implementation of the ontology storage and the recommendation services are described in more detail below.

Ontology Storage
The ontologies used for CMOE?BPMN, along with some ontologies that will be applied in our case study (see Sect. 5), are the following: • The Unified Foundational Ontology (UFO) was selected as a core ontology (i.e. CoO). UFO has different layers, of which only those elements are selected which are relevant in the context of process modeling for this instantiation of CMOE?. A short description of UFO can be found in Online Appendix A; for a full explanation, we refer to Guizzardi et al. (2015). The OWL formalization of UFO is available online. 6 • In the demonstration, an existing OWL ontology from the financial domain is selected as enterprise-specific ontology (i.e. ESO). The ESO concepts are formalized as both OWL classes and OWL individuals, as outlined in Sect. 3.3. Throughout this paper, ESO concepts are denoted in italics. The mappings between ESO concepts and UFO are presented in Online Appendix B, and were obtained using the description of the ESO concepts and their intent. For example, ESO Pro-ductRateApplication is defined as applied interest rate. This implies that ProductRateApplication is a quality of object type Product. An ESO Loan is intended to relate a Customer to the Branch s/he took a loan from. Therefore, Loan is an instance of the UFO Relator universal relating Customer and Branch. ESO LoanAp-plicationAccepted is an event representing the acceptance of loan application, thus instantiating an Event type in UFO. The OWL formalization of the bank ontology is available online. 7 • The used BPMN ontology (i.e. MLO) is an OWL translation of the meta-model shown in Fig. 4, and is based on the original OMG BPMN standard (OMG 2011). In this paper, we extend OMG meta-model based on the observation that different authors advise BPMN modelers to follow the pattern ''verb noun'' when they specify the name of a task (Delfmann et al. 2009). The OWL formalization of the BPMN metamodel is available online. 8 • The mappings between UFO and BPMN (i.e. CoO-MLO) are based on the ontological analysis provided by (Guizzardi and Wagner 2011  The recommendation services are used by the BPMN editor to arrange the ESO concepts in the ontology property view (see Fig. 5), which is implemented following the Model-View-Controller pattern. The controller of the ontology recommendation view updates the associated view every time the modeler selects a model element on the canvas. The CMOE?BPMN tool contains a second view, which is used to give more detailed information about the selected ontology recommendation. The controller of the ontology property view updates the associated view when the modeler selects an ontology recommendation.
CMOE?BPMN uses the OWL API 10 to implement the different recommendation services, and the HermiT reasoner (Glimm et al. 2014), included in the OWL API, is used for querying and reasoning. The label-based recommendation service uses CMOE?'s support for the Jaro-Winkler distance (Winkler 1990) to compare Strings and WordNet (Miller 1995) to determine synonyms (see Listing 2). In some cases (i.e. for BPMN tasks, sub-processes, events, and conditional gateways), the label is pre-processed. For this purpose, the Stanford Parser 11 is applied to tokenize the labels.
Using the rule-based recommendation mechanism, BPMN-specific recommendation rules (i.e. RulesO) were added in CMOE?BPMN. The rules that were used in the experiment (see Sect. 5) are listed in Table 2; a full specification can be found online. 12 In future research, we plan to investigate in more detail which kind of rules may be useful to add to this recommendation service.

Experimental Design
Using an identical case description (see Online Appendix C), modelers were asked to create a BPMN model. Three different treatments were applied: Treatment 1 assists modelers with CMOE?BPMN as described in Sect. 4; Treatment 2 provides modelers an alphabetically ordered list of ESO concepts, without relevance ordering, so that the modeler needs to find relevant ESO concepts him/ herself (see Online Appendix D 13 ); Treatment 3, as a baseline, does not provide any modeling support (i.e. regular BPMN modeling). Where relevant (Treatment 1 and 2), the modeler was asked to annotate the modeling element with ESO concepts. The BPMN modeling tool described in Sect. 4 was used to conduct the experiments. An additional view was developed for Treatment 2 to support only alphabetical ordering of ESO concepts (without recommendations), and for Treatment 3 the recommendations view was disabled. The participants of our experiment were 140 university students at the master level, who were acquainted with BPMN because they took a mandatory Business Process Management course. The subjects were distributed randomly across the three treatments: 47 for Treatments 1 and 2, and 46 for Treatment 3. Every group was given a tutorial explaining the tool and the required actions during the experiment.

Experiment Measures
In Moody's Method Evaluation Model (MEM), the impact of using the method on performance, user perception and intention of use is measured, thus assessing the acceptance of future practitioners. Applying MEM to CMOE?BPMN resulted in six variables to be observed during the experiment: semantic quality, interoperability, time, perceived ease of use, perceived usefulness, and intention of use. These dependent variables were operationalized in the Cheetah experimental platform (Pinggera et al. 2010), which makes it possible to collect answers for the pre-and post-survey (see Online Appendices E and F), collect the created models and record the time spent on each task.
The first variable, semantic Quality (SQ), was measured by verifying validity (i.e. is every statement in the model correct with respect to the case description?) and completeness (i.e. does the set of all statements completely cover the case?) (Lindland et al. 1994). To measure validity and completeness, for every model, the number of invalid and missing statements were counted, respectively, in comparison with a reference model created by a team of three BPMN modeling experts (Online Appendix G).
The second observed variable was interoperability (I). CMOE?BPMN was expected to enhance interoperability across models (1) by providing ESO-based recommendations and automatically annotating BPMN labels, which promotes the reuse of ESO concepts in model element labels, and (2) by consistently recommending the same ESO concept for similar labels, which promotes model consistency and thus interoperability. The degree of model interoperability was measured by counting the number of annotations in every model (Treatment 1 and 2). In addition, to verify consistency, the variation in labelling of modeling elements with the same underlying meaning was assessed by examining the distribution of labels of such elements across different models of one treatment (all treatments).
The third observed variable was time spent for creating the model (T). The aim was to determine if time overhead was incurred by turning to vocabulary support or not. In our experiment, time was measured by the Cheetah This rule indicates that when the modeler creates a pool construct, the UFO object types, which are related to UFO object types that have previously been used to semantically annotate another pool in the model, will be suggested by the rule recommendation service When a message construct is created that results in the transmission of a message between a activity of a pool and another pool, the suggestions are UFO relators mediating material relations that connect objects that in turn annotate the noun of the task and the ontology annotation of the pool, respectively platform, starting when the participants began model creation, and stopping when the final model was uploaded. All other variables were measured using a post-experiment survey (see Online Appendix F). The perceived ease of use (PEOU) and perceived usefulness (PU) of the method were measured by adapting the generally accepted measurement scales of Davis (1989), with three different questions. Intention of use (IU) was measured by means of two questions in the post-experiment survey. All answers were provided on a Likert scale from one (strongly disagree) to five (strongly agree).

Experimental Results
Before analysing the results, we performed a pre-selection of models based on syntactic quality: models with more than two mistakes against the BPMN specification were discarded to eliminate qualitatively insufficient models 14 and reflect a real-life setting in which syntactically incorrect models are improved before acceptance or discarded.
For the retained models, we analyzed the results for the six variables prescribed by MEM. Statistical significance was tested using the Mann-Whitney test for SQ, PEOU, PU, IU as they are ordinal variables, and for T and I as they are not normally distributed continues variables. Normality of the distribution was tested with Kolmogorov-Smirnov and Shapiro-Wilk tests. Statistical significance of label distribution among models was evaluated using Chi square analysis to determine the likeliness of the observed label distribution occurring by chance, independently of the treatment. For all test, the results were considered statistically significant if the p value was\0.05. In all tables, only statistical significant results are explicitly denoted; all other differences were not statistically significant. Table 3 shows the results of the Semantic Quality (SQ) evaluation. We found no statistically significant difference between the treatments for validity, and thus conclude that ontology support does not decrease validity. For semantic completeness, we found no statistically significant difference between Treatment 1 and Treatment 2, yet both performed significantly worse than Treatment 3. Observation during the experiments indicated that participants from Treatment 1 faced some technical issues with the tool, which could have caused them to concentrate more on the functioning of the tool itself, rather than producing a complete model. Furthermore, the tutorial participants received was focused on vocabulary support, which may have caused them to perceive the experiment as a test in vocabulary usage, relaxing their focus on the modeling and model completeness. These possible influences should be eliminated in follow-up experiments.
The results for Interoperability are shown in Table 3 (number of annotations) and Tables 4 and 5 (naming variation). Considering average and median percentages of annotated modeling elements per treatment (Table 3), roughly 70% of BPMN elements were annotated with an 14 Note that the reference model corresponding to the case study only contains 14 BPMN constructs; more than two errors is thus high and indicates poor model quality. ESO concept. Overall, CMOE?BPMN (Treatment 1) performs slightly better than the other two, but the observed differences were not statistically significant. The number of fully annotated models for Treatment 1, however, is more than twice the number for the other treatments. We can therefore conclude that, if given the possibility, modelers annotate a large portion of their modeling elements, thus increasing model interoperability. Furthermore, customized recommendations, as provided by CMOE?BPMN, increase the number of fully annotated models.
Considering the consistency of labels, Table 4 presents the results of naming distribution across models for elements referring to a customer (i.e. a single BPMN pool), whereas Table 5 shows the results for three different modeling elements featuring loan application [i.e. a start event (loan application received) and two different end events (loan application rejected; loan application accepted)]. Multiple instances of the same event, or an event and a task with the same meaning were not counted. In the first column, we also denote the theoretical maximum number of uses, not counting any models that lack an individual modeling element. We can observe that for ''customer'' (Table 4) and ''loan application'' (Table 5), Treatments 1 and 2 performed statistically significantly better compared to Treatment 3: the label corresponding to an ESO concept was used in around 85% of the cases, while results were more dispersed without vocabulary support. With vocabulary support (i.e. Treatments 1 and 2), modelers thus consistently opt for the correct underlying ESO concept, which more clearly corresponds with the underlying business domain and increases the consistent use of labels. Overall, we can conclude that vocabulary support improves interoperability.
Considering time, Table 6 shows the average and median time needed to create the model for every treatment. No statistically significant differences were found between the different treatments. Vocabulary support therefore does not incur time overhead during model creation, although the participants were not trained in using a vocabulary and had to deal with the overhead of searching through the ESO and selecting concepts as labels for modeling elements (rather than freely writing a label).
The results for Perceived ease of use (PEOU), Perceived usefulness (PU) and Intent of user (IU) are summarized in Table 7, presenting averages of the post-survey Likert scale scores (1-5), in which a lower score is better for PEOU, and a higher score is better for PU and IU. The results show that for PEOU, Treatment 3 scores statistically significantly better -albeit only slightly -than Treatment 1. Regarding PU, Treatment 3 scores slightly better (statistically significant) than Treatment 1, and Treatment 2 scores slightly better than Treatment 1. For PEOU and PU, according to average and mode values, the differences are very small. Vocabulary support in itself was considered useful, as demonstrated by the higher PU score for Treatment 2 compared to Treatment 3. As hinted by informal user feedback, we see two explanations for the slightly   worse user perception of Treatment 1. First, the previously mentioned technical problems were cited as the main cause of annoyance. Given the minimal differences, avoiding these would probably bring scores to a similar level as Treatment 3. Second, in Treatment 1, participants indicated that the re-arranging of the list of suggestions for every modeling element according to relevance was annoying. Future work should test solutions that maintain the order of the suggestion list in Treatment 1, but indicate relevance in an alternative way (e.g. using colour coding). Given that the differences in PEOU and PU were minor, and taking into account the solvable technical difficulties with Treatment 1, we carefully conclude that there is no considerable additional frustration or errors accompanying the added vocabulary support to the modeling task. Finally, results for Intention of use (IU) (see Table 7) do not imply any statistical significant difference. To summarize, supplying a modeler with ESO support has two main benefits: (1) it increases model interoperability by linking elements of the models with appropriate ESO concepts via annotations, and (2) it greatly enhances the consistency of labelling modeling elements, as the same label -and annotation with underlying ESO concept -is used for elements with intrinsically identical meaning. Furthermore, this experiment has demonstrated that the additional information and burden to find and select suitable ESO concepts during modeling does not require extra time, and does not impact on the modeler's acceptance of the modeling setup, nor does it have a negative influence on the validity of the models. However, the models created with vocabulary support were not as complete as those created by means of Treatment 3. This can be attributed to the fact that participants concentrated on finding the appropriate vocabulary rather than on creating complete models. The user perception of our method was slightly worse compared to regular modeling. Feedback in the postsurvey indicates that this was probably caused by technical problems with the tool. For user perception, keeping a stable order in the suggestion list may have a positive influence for CMOE?BPMN. These issues will be tackled in follow-up studies.
Finally, although vocabulary support has proved to be useful, the differences between CMOE?BPMN and (only) vocabulary support are mixed. Some positive effects of the alphabetically ordered vocabulary (Treatment 2) may be neutralized or reversed when a larger, more complex model and a more extensive ESO are used, as a greater variety of ESO concepts needs to be found in a larger amount of ESO concepts. The above-mentioned improvements to our method are expected to further tilt the scale in favor of CMOE?BPMN.

Conclusions and Future Work
This article introduces the recommendation-based conceptual modeling and ontology evolution framework (CMOE?), with two main objectives: (1) to solve the interoperability problem across models by facilitating the creation of different types of conceptual models based on concepts from the ESO, and (2) to stimulate ESO evolution based on conceptual modeling feedback. The ESO documents and disambiguates the terms used within the enterprise and the relations between those terms, and is thus perfectly suited as a semantic basis for model creation in order to improve model interoperability and enable automatic integration and querying across models. On the other hand, the framework exploits valuable information generated during model creation to maintain and allow the ESO to evolve, keeping it in sync with newly emerging and evolving needs of the enterprise. As such, the framework establishes a symbiosis between conceptual modeling and ontology evolution within an enterprise.
The framework is instantiated for the BPMN modeling language in a recommendation-based process modeling method (CMOE?BPMN). This instantiation focuses on the modeling aspect of our framework, and shows how the ESO can be used during BPMN model creation to generate recommendations and annotate BPMN models. CMOE?BPMN supports setting up the ESO, analyzing the selected modeling language, developing recommendationbased services, and extracting feedback. It was Table 7 Post-survey results for ease of use (PEOU), usefulness (PU), and community acceptance (IU); cell values denote a Likert scale value (1-5), with 1 being best and 5 worst for PEOU, and 5 best and 1 worst and for PU and IU implemented as a plug-in that extends the Eclipse BPMN2 modeler, and was validated in an extensive exploratory experiment including 140 business students. The experiment showed some promising results: the use of an ESO vocabulary during modeling indeed results in more consistent labelling of modeling elements and does not incur any time overhead. What is more, users have the intention to use the method. Improvements can be made regarding user perception, which currently shows mixed signals, and model completeness, which could be improved as far as complete models are concerned. Future research will aim at improving CMOE?BPMN and the associated modeling tool to obtain better perceived usefulness and model completeness. If technical problems with the tool are overcome, order-invariant label suggestions are provided and more complex models and ESO are used, we expect the recommendation-based modeling method to be more advantageous than vocabulary-assisted modeling. On a broader scale, we have now finalized the instantiation of our method for requirements engineering using i* (Yu 1997), thus proving its wider applicability. Experiments to validate the i* instantiation are underway. Finally, we aim to exploit the modeling feedback, which has already been gathered and (manually) verified to be useful, in a more formal framework, through a communitybased ontology evolution approach.