Attitudes, perceptions and prospectings on mobile information literacy training: Design and validation of the MOBILE-APP questionnaire

The basic aim of this paper is to outline the process of designing and validating an instrument for measuring teachers’ perceptions regarding the importance of using mobile technologies in the teaching-learning of information competencies (MOBILE-APP). Validation was carried out by administering the instrument to a pilot group of teachers, all of whom were experts in new technologies selected from the Information and Communication, Business and Education degrees, together with a rubric to analyse the quality and relevance of the content, as well as the wording of the items. Quantitative and qualitative methodologies were combined (statistical and descriptive analyses). The results from the quantitative-qualitative analysis of the evaluation rubric and the psychometric analysis of the items show the strengths of the questionnaire and the possible improvements that could be made to optimise it, which were focused on small adjustments to some of the categories and the reordering of several items. The quantitative analysis shows a high degree of internal consistency, thus ensuring the usefulness and applicability of the instrument for evaluating teachers’ perceptions regarding the use of mobile technologies in the teaching-learning of information competencies. The validity and reliability of the instrument mean that it can be transferred to other academic settings as a platform for future diagnostic studies that allow advances to be made in teaching innovation using mobile technologies, while at the same time permitting it to be reviewed and updated. The innovative element of the MOBILE-APP questionnaire lies in the interrelation between the mode of learning (mobile learning) and the information literacy competencies.


Introduction
The technological advances that have taken place over the last two decades have brought with them a considerable shift in perspective as regards the way we relate to and access information, and thus, there is a strong need to consider them in the context of information literacy. This fact has had direct consequences on teaching-learning processes, in terms of both the methodologies used and their adjustment to meet new academic contexts and needs. The integration of mobile technologies has become an increasingly common part of day-to-day reality within the classroom. Keengwe and Bhargava (2014: 737) highlight the fact that 'the application, implementation, and design of mobile technology in the global educational context pose technological and socio-cultural challenges'. There is a direct correlation between the socio-cultural change resulting from these technologies and the need to adapt educational processes to this new context. The most recent generations of students, often called millennials, digital natives (Rossing et al., 2012) or even the 'Google generation' (Kiviluoto, 2015: 308), clearly integrate social networks, mobile resources and applications within their everyday lives. The boundaries between the different spheres of their lives are not clearly delimited. Technology is there, always present. In Spain for example, a recent report drawn up by the Fundación Telefónica (2017) reveals that youngsters between 15 and 24 years of age are essentially 'mobile first' or 'mobile only', meaning that their smartphone is the main or even the only device they use to consult and manage information. This explains why the integration of mobile technology, applications and even social networks is becoming increasingly more frequent in teaching (Tess, 2013), and how mobile social networking sites may be used as a platform that facilitates learning (Wong et al., 2015). Moreover, it is quite clear that teachers are increasingly more aware of the fact that resources, ease of access and the possible means of interaction increase significantly thanks to mobile applications, and their immediacy and ubiquity (Chen and Denoyelles, 2013;Gikas and Grant, 2013). There can be no doubt about the usefulness of mobile phones, although their generalised use in academic contexts can entail a series of advantages and drawbacks. In any case, as stressed by Keengwe et al. (2009: 333), 'integrated appropriately, mobile devices could help students acquire the skills needed to survive in a complex, highly technological knowledge-based economy'.
Competency-based training is fundamental within the context of today's society and it is one of the requirements of most educational systems. It implies the return to constructivism, based on the integration of technology. That is why mobile devices and their implications (accessibility, ubiquity and immediacy) turn into an indispensable tool (Cochrane and Bateman, 2010). Mobile technology and ubiquitous teaching can be a basic educational tool in the process of knowledge construction, in which the student becomes the main player (Kearney et al., 2012). We are witnessing a paradigm shift in higher education (Cochrane, 2014), which, in spite of the generation gap that it may highlight between teachers and students, in fact becomes an opportunity to establish encounters and build bridges. Dalton (2012) shows that the use of mobile devices and the ease of access to information have had a considerable impact on students, since less effort is required and, therefore, there is a considerable change in the competencies that the students start out with and how they are managed.
Reducing the need to memorise information, for example, means that this ability is relegated to the background. Hence, the role of the teacher, as a guide in the process of acquiring the tools and skills needed for searching, selecting and critically analysing information, becomes a priority. In any case, it can be observed that teachers' predisposition towards the introduction of mobile phones in the educational process of higher education varies (Park et al., 2012). The same advantages stemming from both ubiquity and immediate accessibility are viewed partly with scepticism and mistrust, since a critical attitude towards information sources seems to be lacking, while at the same time an authentic disinformative paradox can be perceived in the midst of the digital and ubiquitous era (Kiviluoto, 2015). Likewise, we must bear in mind, as Van Deursen and Van Dijk (2014) and El-Hussein et al. (2010) point out, that the true distance is not so much due to generational reasons or roles (in this case, those of learner and instructor), but to other factors such as education, family, previous experience with the use of technologies, purchasing power and, hence, social status. From this perspective, we can draw three implications: 1. The use of mobile devices in the teaching-learning process and, therefore, the acquisition of information competencies is an unstoppable reality. 2. Mobile devices are useful instruments that offer a wide range of possibilities. As underlined by Gikas and Grant (2013: 18), 'mobile devices offer a variety of ways to learn, communicate and collaborate'. 3. Priority should be given to the analysis of attitudes towards the use of mobile devices and their relationship with the acquisition of information competencies within the higher education setting.
In line with the above, this study describes the procedure used to design and later validate the MOBILE-APP questionnaire for measuring the Teachers' perception regarding the importance of using mobile technologies in the teaching-learning of information competencies (Mobile -Information -Literacy -Education -Attitudes -Perceptions -Prospectings).
The specific aims of this study are as follows: 1. to design an online questionnaire with which to measure teachers' perceptions regarding the use of mobile technologies in the teaching-learning of information competencies; 2. to analyse the validity of the questionnaire, in terms of its content and applicability; 3. to examine the reliability of this instrument; 4. to study the feasibility and usefulness of the questionnaire.

Methodological design of the questionnaire
The questionnaire was produced as an ad hoc instrument for use as the primary means of collecting data. It was created taking into consideration the following aspects and procedures: 1. Reflection upon the real situation, regarding the use of technologies, within the academic setting and in relation to the information competencies, bearing in mind the following elements: First, the increasingly common implementation of digital and mobile technologies in teaching-learning processes may be taken into account, together with the unstoppable process of change in devices and applications.
In this context, teachers are increasingly concerned about the divergence that exists between students and teachers and about how students seldom tend to view digital and mobile technologies as a teaching tool rather than seeing them as elements that often contribute their infoxication, or information overload, and dispersion. It is also important to bear in mind the disinformative paradox of those who, whilst having direct and immediate access to sources of information, are increasingly devoid of the capacity for analysis and critical thinking. All in all, both teachers and students are concerned about the use of technologies in education. In this respect, the study has taken into account the viewpoints of teachers and students as regards the aspects shown in Table 1. 2. A series of tests were conducted to produce this instrument. Particularly, a number of measurements of the use of technologies in the higher education setting were carried out. These tests, which are used in different subjects in the education degrees at the University of Granada to measure the inclusion and generalisation of technologies in the university classroom, were taken as a starting point and basis for the realistic design of the MOBILE-APP questionnaire. 3. Prior examination and diagnosis of the state of the question among teachers, by attending educational innovation conferences, informal interviews and systematic-descriptive observation, with the aim of collecting data and updating information. 4. Analysis of other previously validated questionnaires relating to the topic addressed in this study. As a result, the following material was reviewed: a. The questionnaire proposed by Su and Cheng (2015), which is contextualised within a setting that is inevitably conditioned by the presence of mobile devices and ubiquity, presents the link between apps, integration of mobiles, gamification, motivation and academic achievement. b. The questionnaire by Vázquez-Cano (2014), which focuses on distance learning and specific applications for the different subjects in a higher education setting and analyses students' attitudes towards mobile learning (m-learning). c. The questionnaire put forward by Yang (2013), which quantitatively analyses expectations regarding functioning, social influence, adoption of mobile teaching and its positive consequences in higher education. d. The questionnaire of Uzunboylu and Ozdamli (2011), which they use to analyse teachers' perceptions of m-learning by means of an estimation scale. e. The proposal of Kallaya et al. (2009), who presented the validation of a scale on which the aims that are set are combined with the probability of m-learning being accepted, the effect of this latter in higher education and the different attitudes towards this fact. f. The work by Wang et al. (2009), who performed a quantitative analysis on a population within

Indissoluble link between access to sources of information and use of mobile devices
Perception of a progressive decrease in students' information competencies Perception of certain reluctance on the part of teachers towards including them in teaching methodologies Different opinions about the use of devices and apps in the classroom

Self-perception of very limited information literacy
Negative reaction towards the use of mobile devices from one sector Perception of a generational digital divide Perception of a link between the ubiquity and immediacy of access to information and students' disinformation the higher education context to create a scale that can be used to investigate the determining factors in the acceptance and integration of m-learning and the possible characteristics determining differences, such as gender or age. g. The experience accumulated by the authors in the design and development of other tools for evaluating information competencies, such as INFOLITRANS (Pinto and Sales, 2008;Pinto et al., 2010), which is a tool focused on the self-evaluation of the information competency of students of translation and interpreting, enabling the user/student not only to measure his/her different information capacities and aptitudes but also to diagnose and solve problems detected in the development of his or her informational training; IL-HUMASS (Pinto, 2009), a critical survey developed for a set of different degrees in social sciences and the humanities that analyses four categories (information search, assessment, processing and communication/dissemination) and three self-reporting dimensions (motivation, selfefficacy and favourite source of learning); and EVAL-CI (Pinto and Fernández-Pascual, 2014;Pinto et al., 2016), which was designed to evaluate students' objective knowledge regarding information competencies. h. Moreover, the Framework for Information Literacy for Higher Education (ACRL, 2015), an essential proposal that rethinks the way the teaching of IL is approached today, was also revised. Fifteen years after the publication of the Information Literacy Competency Standards for Higher Education (ACRL, 2000), the ACRL put forward this updated, meticulous and novel review by publishing the Framework, intentionally called so because 'it is based on a cluster of interconnected core concepts, with flexible options for implementation, rather than on a set of standards or learning outcomes, or any prescriptive enumeration of skills' (ACRL, 2015).
At the same time, there are several factors that were taken into account when it came to drawing up the present questionnaire: 1. Originality factor. After a review of different questionnaires that have already been applied, the aim was to ensure this instrument achieved the required degrees of novelty and originality. The innovation it offers results from the conjunction of the two aspects pointed out earlier, that is, the generalised use of mobile devices in the teaching-learning process (learning methodology) and the acquisition of information competencies. The thorough review of the literature and instruments published to date that was carried out provided evidence of one of the two aspects but not a combination of both. For that matter, this tool provides a new approach, since it provides information regarding teachers' opinions and views on these particular issues. 2. Applicability factor. Efforts were made to ensure that in both the dimensions and the items that make up the questionnaire their implementation was realistic and that the questionnaire could be applied to the greatest possible number of university contexts. In this regard, the current questionnaire offers an innovative and transferable proposal. 3. Contextualisation factor. In view of how quickly devices are being included in the higher education setting, both the local context and the current time have been taken into account. 4. Susceptibility to change factor: spiral of obsolescence. The rapid appearance of devices and new applications means that previous ones quickly become obsolete. Accordingly, efforts were made to draw up an instrument with a diachronic projection that could be regularly reviewed to check whether it was still up-to-date. 5. This instrument takes into account new approaches that suit to models according to the new informational and learning behavior developed by students.
After making the decisions about the dimensions selected for analysis (level of quality, clarity, usefulness and perceptions/opinions), a list of easy-to-understand items associated to each dimension was drawn up. Emphasis was placed on ensuring that the maximum internal coherence and the relational coherence between each of the aspects were maintained. Once drafted and formatted, tests and reviews were conducted before a pilot study and validation were carried out with a group of selected members of academic staff (non-probabilistic/intentional sample).
The version of the scale initially proposed consisted of 33 items grouped in five categories:

The informationally literate university…
This first block of the questionnaire addresses a descriptive and introductory dimension of the subject of study: information competencies within the higher education context. With this aim, the teachers were asked about their perception of the relevance of information literacy, its role in promoting critical and reflective training as well as lifelong learning, whether they consider that it facilitates the teaching-learning process and, as a link and a way of leading onto the specific subject of the questionnaire, whether they consider that teaching will be of an increasingly more ubiquitous or blended nature.

An informationally literate person is one who…
The second dimension of the questionnaire focuses on information competencies, to allow the teachers to rate their relevance. these competencies, which are fundamental in all the literature on information literacy, refer to the capacity to identify informational needs, knowledge and handling of relevant and quality sources of information. It also implies the effective use of a range of informational resources, the capacity to issue critical judgements on the sources, the ability to cite the sources of information used in an appropriate manner, and the capacity to undertake rigorous and relevant dissemination of the information.

Rate the following statements
In this third section, we thought it was especially interesting for the teachers to evaluate the Frames from the Framework for Information Literacy for Higher Education (ACRL, 2015). To this end, we asked the teachers for their perception about the six frames proposed by the Framework. We used the wording employed by ACRL in its proposal, to gather information about their perceptions with respect to: (a) the concept of authorship and the relevance of the context in which it is framed; (b) the idea that the creation of information is a constant process; (c) the fact that information has a value; (d) the idea that research involves questioning; (e) the fact that knowledge is a dialogue; (f) and the idea that searching for information is an exploration that requires a strategy.

I use mobile technologies in my teaching process…
The fourth block of the questionnaire focuses on gauging the teachers' perceptions regarding the use of different types of mobile technologies in their teaching. With this aim, it included items about whether they consider that such technologies can make the job of teaching easier, adapting to teaching innovations, students' motivation and the integration of facilitating tools. At the same time they were asked about the ways in which they update their knowledge of mobile technologies, the type of mobile tools they use in their teaching and what tasks they use them for.

ICT and students
The fifth and last category of the questionnaire addresses teachers' perceptions as regards students' use of mobile technologies. The items seek to gather information about their perception of what tasks students use mobile devices for, whether they consider that digital natives think that mobile technology affords them access to all the information they need, whether they think that immediate access to information diminishes students' capacity for critical thinking as regards the use and management of information, whether they believe that the use of technology results in students' not valuing the importance of citing sources in an adequate manner and, finally, whether they consider that students acknowledge the fact that mobile technology has led to their being more distracted in class.
The method chosen to measure their responses was a five-point Likert scale, ranging from 'Totally disagree=1' to 'Totally agree=5'. The questionnaire is available in web form at: http://infocompetencias.com/cuestionarios/ profesores/

Validation of the questionnaire
The next phase of the research was focused on validating the instrument. It must be borne in mind that in this crucial phase of the evaluation of the questionnaire the following axes were taken into account: a. Content validity: in this regard both the appropriateness and relevance of the subject of study that we have proposed were taken into account; b. Validity of the external aspects: clarity and adequacy of the vocabulary, style, clarity of wording, register and accuracy.
This task was carried out by addressing the following specific aims: 1. To determine whether the items, which evaluate the teachers' perceptions about the importance of using mobile technologies in the teaching-learning of information competencies, are appropriate to measure the characteristics of the dimensions under consideration. 2. To determine whether the items that make up the scale are adequate, exhaustive and sufficiently representative of the dimensions under study. 3. To estimate the capacity of the instrument to discriminate among groups of teachers who express different opinions.
An evaluation rubric was designed to address the first two aims. To fulfil the third aim, analyses were performed of the inter-item statistics and reliability using the Cronbach's alpha statistic, and the latent structures were also investigated by means of exploratory factor analysis applied to the answers gathered from the pilot sample of teachers (Brown, 2002;Sijtsma, 2009). The drafting of the questionnaire and its validation were carried out during the year 2017. Both the instrument and the evaluation rubric were administered to a group of experts selected between November 2017 and January 2018.

Profile of the sample
The sample consists of 43 teachers with long teaching experience (see Table 2) in the areas of Information and Communication, Business, and Education, all being experts in ICTs.
The mean age of the group is 41.5 years, with values ranging from 25 to 59. A little under half of them (46.2%) are women.
Of all the experts surveyed, 82% stated that they taught in the classroom, 12.9% did so by means of blended learning and only 5.1% said that they taught online. Most of the teachers in the sample (82%) taught undergraduate courses, 15.4% taught undergraduate and Master's degree courses and 2.6% gave classes only in Master's degree courses. Slightly more than half of the teachers (51.3%) have more than 10 years' teaching experience (Table 2).

Evaluation rubric
Design. The evaluation rubric makes it possible to analyse, in a quantitative and qualitative manner, the quality and coherence of the questionnaire from the point of view of the experts-teachers. We designed an evaluation rubric that uses the same 1-5 Likert scale that is employed in the questionnaire itself, beyond the basic dichotomous scale Yes/No that would allow us to calculate the concordance index among experts, but would not allow us to calculate in an accurate way their degree of agreement, the questionnaire quality level and the clarity and usefulness of the proposed items (Weir, 2005). Moreover, this design does not dismiss the possible variability of the experts, which may create some bias towards one or several items of the questionnaire (Escobar Pérez and Cuervo-Martínez, 2008).
On the scale referring to the rubric, 1 means that there is no match between the item of the questionnaire and its subject, whereas 5 would indicate a perfect match between the two: 1. Very low, 2. Low, 3. Medium, 4. High, 5. Excellent.
Quantitative analysis of the rubric. The results obtained with regard to the general evaluation of the questionnaire are as follows: a. Regarding personal details and instructions According to the experts' evaluation, the sections are well designed and instructions are clear and legible (Table 3).

b. Regarding general coherence and quality
The experts who evaluated the questionnaire rated the coherence and quality very positively. There are mean scores above 4.10 and low dispersion in their criteria, given that the standard deviations do not reach a value of one point (Table 4). Overall quality scores 4.18 points (on the 1-5 Likert scale).
c. Regarding the clarity/ usefulness of the items In the following we provide a two-dimensional analysis of the clarity and usefulness of the 33 proposed items. The descriptive results shed light on which items offer the greatest clarity and/or relevance, and those that need to be reconsidered (see Table 5, for categories, and Table 1A-Appendix 1).
In sum:

Category 1: The informationally literate university…
The score in the overall evaluation of clarity and usefulness was 4.22 points. The item with the best rating in the clarity dimension is item 3 -Fosters lifelong learning (mean_c=4.36), and the one seen as being most useful is item 2 -Promotes the reflective and critical training of the educational community (mean_u=4.32). Generally speaking, there is less dispersion in the usefulness dimension.
The most poorly rated item is number 5 -Assumes that teaching will become of an increasingly more ubiquitous or blended nature. The experts consider this category as the one that was designed the best. The scores are all well above 4 points. The item with the best score in terms of clarity is number 9 -Evaluates sources of information in an analytical and critical way (mean_c=4.72), and the most useful item is number 7 -Is familiar with and uses relevant and highquality sources of information (mean_u=4.49). In general, less dispersion was found in the clarity dimension.

Category 3: Rate the following statements
In this block, item 17 -The search for information is a strategic exploration, obtains the highest score in terms of both clarity and usefulness (mean_c=4.36, mean_u=4.26, respectively). Dispersion stands at around 1 point and does not display a clearly differentiated profile between the two dimensions. The most poorly rated item is number 12 -Authorship is constructed and contextual.

Category 4: I use mobile technologies in my teaching process…
According to the experts' opinions, this would be second most highly rated category. The most highly rated item on clarity is number 20 -As a way to motivate students (mean_c=4.56), and the one which presents the greatest usefulness is number 24 -The virtual campus/classroom is a basic tool in my teaching methodology (mean_u=4.58).
Again, the low dispersion found does not reveal any clear differentiating profile between the two dimensions. The most poorly rated item on clarity is number 28 -The future of teaching cannot be conceived separately from the setting (mean_c=3.90).

Category 5: ICT and students
In this case, the maximum score on clarity and relevance goes to item 29 -My students usually take notes or search for information using mobile devices (laptops, tablets or smartphones) (mean_c=4.54, mean_u=4.34). The most poorly rated dimensions are: (a) the clarity of item 31 -Immediate access to information leads to a lower capacity for critical appraisal when it comes to selecting verified documents and information, and (b) the usefulness of number 33 -Students acknowledge the fact that mobile devices have led to their being more distracted in class. Then we will study the internal correlation that exists between the clarity of the items in the same block and, analogously, the correlation between their usefulness (R ci,cj ,R ui,uj , con i≠j, respectively). Furthermore, we consider the cross-correlation between clarity and usefulness of the same item, R ci,ui .
In the first and fourth categories, there is a significant Pearson correlation (Norman, 2010) between the clarity presented by the items in each block. We also find significant correlations between the usefulness of the items. The In the other categories, the correlations among the ratings for clarity are weak and non-significant in most cases. In contrast, usefulness does reveal significant correlations. The highest correlations are found between the items: u6 -Knows how to identify his or her information needs and u7 -Is familiar with and uses relevant and high-quality sources of information (R u6, u7 =.735); u15 -Research involves questioning and u16 -Learning is a dialogue (R u15, u16 =.752); u30 -Digital natives think that mobile technology affords them access to all the information they need and u31 -Immediate access to information leads to a lower capacity for critical appraisal when it comes to selecting verified documents and information (R u30,u31 =.890).
In sum, somewhat surprisingly, we find a low crosscorrelation on the binomial 'clarity/usefulness', in contrast to what might be expected.
Qualitative analysis of the rubric. With the aim of determining some aspects that are not covered by the questionnaire, an open-ended question was introduced at the end of each of the dimensions of the rubric to allow respondents to note possible suggestions or proposals for improvement. As pointed out by Morgan (1998: 362), combining quantitative and qualitative analyses is very useful because 'it relies on the principle of complementarity'. Hence, after performing the analysis of the quantitative dimensions, the next step was to examine the qualitative aspects included by the experts in their evaluation of the instrument. No significant suggestions were given in the sections Personal Details and Instructions. Contributions/recommendations highlighted by the teachers/raters who filled in the rubric and the improvement measures proposed by the research team that developed the questionnaire and drafted the present paper are outlined ( Table 6).
The absence of any suggestions from most of the experts shows that coherence, comprehension and rightness are perceived in each category and its items.

Internal consistency and reliability of the questionnaire
As measures to evaluate the internal consistency and reliability of the MOBILE-APP questionnaire, the main descriptive measures of the responses, the Cronbach's alpha values and the item-total correlation indices are analysed. To determine the structure and arrangement of the latent categories an exploratory factor analysis is applied.
The descriptive analysis of the responses given by the sample of teachers who participated in the evaluation of the instrument is shown ( Table 7). The validity of an instrument refers to the extent to which the instrument actually measures what we wish to measure. The overall Cronbach's alpha of the MOBILE-APP questionnaire was 0.816, which indicates a high level of reliability and internal consistency (George and Mallery, 2003: 231;Gliem and Gliem, 2003;Loo, 2001).
In order to evaluate the discriminatory power of the proposed items, it is necessary to calculate the Pearson All the proposals for improvement were evaluated and taken into account.
correlation (O'Rourke and Hatcher, 2013), the scores of the N subjects on each item and the X scores on the total of the test. This item-total correlation index is based on the fact that an item will be discriminatory if the respondent that scored high on it also scores high on the rest of the scale without taking into account the item that is being scored. A high positive correlation is therefore expected between the scores on the item and the total score on the scale. On average, 73% of the items exhibited a strong score reliability (see Table 2A -Appendix 2). If an item is removed, the Cronbach's alpha values do not significantly improve the scale and therefore all the items are relevant. The item-total correlation indices offer values above 0.35 on 27 items, and can therefore be considered as acceptable (Cohen et al., 2013). The other nine items, whose item-total coefficients yield values below 0.35, must be reformulated in order to improve their discriminatory power. In view of these results, the research team analysed the weaknesses found in order to put forward proposals for improvement. After evaluating the suggestions and reaching a consensus, a second version of the instrument was proposed.
Finally, the latent structure is determined to check whether the scale displays multidimensionality by means of an exploratory factor analysis that allows the relations among the variables to be used to identify the latent categories present in the dataset. The Kaiser-Meyer-Olkin (KMO) measure of adequacy is 0.62 and therefore exceeds the minimum value required for the factorial analysis to be considered as adequate, despite the small size of the sample (Williams et al., 2010).
In order to determine the optimum number of factors, it can be observed how the sedimentation graph displays an inflection point in 6 factors (Tavakol and Dennick, 2011). These six factors have an explained variance of 71.89% (Table 8).
In view of these results, categories 1 and 2 are consolidated (1. The informationally literary university…, and 2. An informationally literate person is one who…), which are replicated in factors 4 and 5 (F4: Informationally literate university, and F5: Informationally literate person). To reinforce the consistency of the other categories, several items of the questionnaire were reordered and associated, with three subcategories being defined in block 4 -I use mobile technologies in my teaching process: 4.1 Motivation, 4.2 Training and Projection, 4.3 Tools. With these modifications the final version of the instrument was obtained (Table 3A -Appendix 3).

Discussion and conclusions
Our original basic aim was to develop an instrument and validate it. On the one hand, the reliability demonstrates the stability of the elements that are measured. On the other hand, the internal consistency can be directly related to the validity of the results and their interpretation, which ensure the usefulness and applicability of instrument (Kimberlin and Winterstein, 2008). Following its initial analysis and the experts' evaluation, the questionnaire is seen to have a high internal consistency (Cronbach's alpha: 0.816), which means it can be considered an instrument that is valid and applicable to a broad population with a wide range of academic profiles, as well as to other scientific domains.
As has been pointed out earlier, the originality of this instrument is focused on the conjunction of two key aspects: the generalisation of the use of mobile devices as a learning methodology, and the acquisition and development of information competencies. Thus, it addresses the technological and informational but, at the same time, also the emotional and systemic perspectives, as it takes quality into account, based on the satisfaction and success of the user (in this case, the teacher). In this regard, the questionnaire proposed here is grounded in the philosophy of the EFQM (European Foundation for Quality Management) Excellence Model, which as noted by Brusoni et al. (2014: 9), 'establishes broad criteria, which any organisation can use to assess the progress towards excellence'. In fact, we have already used this quality model in evaluation questionnaires developed and validated in previous research projects, such as INFOLITRANS (Pinto and Sales, 2008;Pinto et al., 2010).
If we take into account other instruments that had previously been published, including both those we took as a starting point and those we use as the basis for discussion, we have to underline the fact that there are questionnaires Hence, for example, MacCallum and Jeffrey (2014) address the impact of teachers' adopting mobile devices in three variables: anxiety, efficacy and digital literacy. In this regard, some divergences can be found in the items referring to the literacy dimension and in efficacy, especially in the references that refer to making the teaching process more motivating (factor 4 of MacCallum and Jeffrey's instrument and items 20-21 included in the fourth dimension of the MOBILE-APP instrument). In any case, we have to point out that they are general items and can be applied to a wide range of contexts, and so convergences are inevitable. The two instruments (MacCallum and Jeffrey, and MOBILE-APP) differ in terms of purpose and the aspects that they combine.
In a similar vein we find the instrument developed by Ismail and Azman (2013), which attempts to analyse the attitudes and training of teachers to deal with the introduction and generalised use of mobile devices in the classroom. This study, however, evaluates perception of members of teaching staff, but not in higher education. In any case, the items of its fourth factor are somewhat similar to the fourth and fifth dimensions of our instrument although in the first (Ismail and Azman) they are of a general nature and in the second (the questionnaire we validated, MOBILE-APP) the dimensions are broken down into a greater number of items, which provide more information for the aims initially established and for the level of education which it targets.
Likewise, and applied to the specific context of Jordan, Khwaileh and AlJarrah (2010) validated an instrument that was focused on the advantages and disadvantages of using mobile teaching, based on the perceptions of graduates of the Faculty of Educational Sciences. However, this study touches on information competency, without going into it in any depth; is based on the perceptions of graduates, although not of teachers; and was conducted in 2010, when m-learning was still in the inception stage. This gives Khwaileh and AlJarrah's questionnaire an innovative but prospective nature. Although the MOBILE-APP instrument presents that projection, as it has been designed bearing in mind the possibility of updating it and its adaptability to future scenarios, it has been validated at a time when the use of mobile devices in classrooms is not a reality that may occur sometime in the future, but is already present now. There are some slight similarities between the two questionnaires in terms of the general content but important differences due to the original and unique nature of each of them.
We consider the role played by the experts surveyed to evaluate this tool instrument as crucial, not only as the basis and foundation of the validation but also as a means to propose possible improvements to optimise the questionnaire. In this respect, the rubric included qualitative aspects, which allowed the experts to raise some suggestions. These descriptive contributions chiefly addressed the inclusion, comprehension and/or clarification of certain items. Although the contributions were scarce, they were incorporated into the final version after comparing and crossing them with the quantitative results. In fact, in this sense we are in line with authors such as Clark et al. (2008) and Hussein (2015), who consider qualitative contributions as indispensable for the development of an instrument of this kind.
Likewise, and given the dynamic nature and rapid obsolescence of mobile technologies, we also took into account the possibility of reviewing and adapting the questionnaire through constant updates, as we set out in the aims of this work. As a result, the questionnaire is a simple and highly reliable tool for obtaining information about large populations.
The contribution of the MOBILE-APP questionnaire, as mentioned above, is based on the ability to combine the perception of the introduction of mobile technologies in the teaching-learning process with the acquisition of information competency, that is, the interrelationship between the mode of learning (mobile learning) and the information literacy competencies. Although some instruments had previously been validated and published, and there are similarities between some of the items they include, they are very small and occur, above all, in the intuitively more general questions, as we have explained earlier. We therefore believe that the validity of the instrument, together with the meticulousness of the items that make up each of the dimensions of the instrument, have been made clear. We consider that the questionnaire may be easily implemented, it is realistic and describes the current context of Higher Education, where teachers have to move with the times and adapt to their students' demands and also to the surrounding society. Furthermore, another innovative aspect, in our opinion, is that the MOBILE-APP questionnaire may be revised to be adapted to other contexts and other education levels, such as Secondary Education. All in all, MOBILE-APP is a flexible tool for our times.
After analysing both the validity processes in terms of the content and form of the instrument and the latent structures and reliability of MOBILE-APP, we can conclude that the questionnaire is a useful instrument for determining teachers' perceptions of the relevance of using mobile technologies in the teaching and learning process aimed at the acquisition of information competencies. We will focus on this in future research, using the MOBILE-APP questionnaire to conduct diagnostic studies that can be used as the basis for proposals for educational application and innovation.
Appendix 3 shows the final version of the questionnaire after incorporating the quantitative data highlighted by the experts.

Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.   (26) 25. I use an app on my mobile to interact with my students so as to be able to provide them with academic information that may be of use to them (27) 26. The virtual campus/classroom is a basic tool in my teaching methodology (24) 27*. The incorporation of online platforms and resources has transformed the panorama of higher education (25) 28. The future of teaching cannot be conceived separately from the setting ICT and students 29*. How often my students usually take notes or search for information using mobile devices (laptops, tablets or smartphones) (scale from 1 to 5: never, frequently, often… always) 30*. Mobile devices have led to students' being more distracted in class (33) 31. Digital natives think that mobile technology affords them access to all the information they need (30) 32*. Immediate access to information diminishes students' capacity for critical thinking (31) 33. The use of ICTs results in students' granting less importance to citing sources (32)