General Trends of the Studies about the Sustainability of the Technology Usage in Education : A Thematic Content Analysis Study

The aim of this study is to determine general tendencies through bringing a cumulative perspective to research on sustainability of technology usage in terms of education. In this respect, certain articles were reviewed in SSCI and CPCI-SSH citation indexes located at the ISI Web of Knowledge database. A total of 87 full-text articles were filtered according to the purpose of the study and they were analyzed through meta-synthesis (thematic content analysis) according to the types of theories or models they based on, names of the journals they were published in, along with their years of publication, number of citations, research groups, examined technology and learning environment, data analysis technique, structures with significant or insignificant effects and dependent variables to be explained. Research findings display that the most frequently used model is the Information Systems Continuance Model. While a significant increase is observed in the number of publications in recent years, the three variables with the most frequently researched effects on dependent variables are found to be satisfaction, perceived usefulness and perceived ease of use. In research studies where continuance intention was focused for prediction, variety of independent variables due to rich theoretical basis has been of interest. In the light of the data obtained, certain suggestions have been developed. Conclusions of this research are believed to have theoretical contributions to the literature by displaying the current situation.


Introduction
More intensive utilization of technology in teaching and learning processes day by day has encouraged researchers to focus on various perspectives during their field studies. It is observed that certain studies were conducted through determined technologies and analyzed how these technologies could be integrated into teaching and learning processes, affecting factors, potential complications and mutual effects (Gülbahar, 2007;Kopcha, 2012;Liu, 2011). Some other studies are also observed to place the users in the center, concentrating on how certain technologies were based on and utilized by users

Self Determination Theory
Another theory favored in determining internal variables regarding information systems is the Self Determination Theory, which was related to the motivation variable by Gagné and Deci (2005). Psychological requirements set forth by the Self Determination Theory [autonomy, competence, relatedness], self-determination and concepts like internal or external motivation are psychological structures that have been widely studied recently . Self Determination Theory mainly focuses on two different types of motivation and their differences (Black & Deci, 2000). These are internal motivation and external motivation. Certain studies on the long-term usage of information technologies lack topics on motivation structure of the Expectation Confirmation, which is completed through this theory.

Technology Continuance Theory
In terms of long-term usage of technological innovations,  developed Technology Continuance Theory as joint theory in the form of a mixture of Technology Acceptance Model (TAM), Expectation Confirmation Model (ECM) and Cognitive Model (COG). The theory tested the other three theories in terms of their prediction power and put together the six variables contributing essentially to the general prediction percentage of these theory and models. Model is based on the following structures: confirmation, satisfaction, perceived usefulness, perceived ease of use and IS continuance intention. This theory is the most recent of all theories and models in the related literature. A prior variable such as attitude and a posterior variable such as satisfaction are brought together within the same model and a predicting relation in between is proved, which together differentiate the model form other models and theories. Although attitude is a type of perception which can be developed by individuals before the usage of a technology, it is required an adequate time to ensure the satisfaction.
Along with these theories and models, certain studies analyzed potential structures, which may affect the sustainability of the information systems. For instance, , in their research on voluntary Internet users included "habit" as a variable in their research model. Researchers defined "habit" as "the extent to which people tend to perform behaviors (use IS) automatically because of learning" (Limayem, Hirt, & Cheung, 2007, p. 709). Considering that continuing to use Internet is affected not only by intention but also by habit and considered habit as a moderator variable. As a result of the analysis on the model, this hypothesis was confirmed and it was determined that satisfaction past behavioral frequency and usage comprehensiveness had important effects on the structuring of the habit variable and that it had an indirect contribution to sustainability through habit.
Another study focused on the predicted variables as potential affecting factors on sustainable usage of e-learning . Researchers differentiated the Expectation Confirmation Model developed by Oliver (1980) (also known as Expectation Disconfirmation Model) and divided the perceived performance variable into three sub factors as usability, quality and value. Research conducted in Taiwan with 183 e-learning users concluded that the intention for sustainable usage could widely be predicted through the satisfaction variable. At the same time, perceived usability, perceived quality and perceived value variables were found to be important predictors of satisfaction.  attempted to predict the sustainable usage intention with variables such as satisfaction, concentration, subjective norm and perceived usefulness. The study was conducted with 363 students participating in the web-based learning program designed for sustainable education. As a result of the testing on the research model as joint study on four different theories and models, satisfaction was found to be the most powerful predictor while perceived usefulness, attitude, subjective norm and concentration were determined to be rather weak.
Another study by  analyzed factors affecting students' continuance intentions towards personal response systems. Expectation Confirmation Model, Information Systems Success Model, Motivation Theory and Agency Theory were structured as a mixture and 15 different hypotheses were tested. Perceived performance variable of Expectation Confirmation Model was analyzed in terms of education in the study and was transformed into learning performance. At the Information Management Department of Kaosiung State University in Taiwan, the study was conducted using the FingerClick personal response system and the data were collected in a hall of 120 participants with two management panels, each of which controls 60 students. Findings indicated that only the internal aim tendency out of the six variables deriving from Motivation Theory was effective on the intention for continuing to use. 23% of the variance observed in the confirmation variable, 48% of the variance observed in satisfaction variable and 50% of the variance observed in the continuous usage intention variable were predicted with the help of the independent variables of the study.
As the studies concluded, variables examined in different research activities conducted with different theories for different learning environments changed and this change would have different consequences in terms of continuous usage. Another clear-cut finding was that sustainability should be analyzed with variables other than adoption or acceptance and that each technology should be considered along with original variables with high levels of prediction powers. Therefore, based on this diversity seen in the studies, it can be said that there is a need to examine the studies with a holistic overview that exists in the literature. Consequently, the purpose of this study is to determine general tendencies through bringing a cumulative perspective to research on sustainability in terms of education as well as revealing their frequencies of analysis and making suggestions for the provision of sustainability in the usage of technology in education. In line with research findings to be obtained, it can be demonstrated by the terms of a systematic overview of the general trends and the similar or different ways of sustainability studies in educational context could be investigated. What percentage of acceptance-discontinuance anomaly is considered in the literature, on which groups and with what type of analyses were the educational sustainability studies carried out and which variables were mainly distinguished would be a source of inspiration for further studies. Because of this, this research seeks answers to the following sub-problems: 10. What are the predictive relations, the effects of which were found to be insignificant in articles on technological sustainability in education and what is their distribution like?

Method
Content analysis was selected as the method of this study. Content analysis, defined as an inductive and deductive method used with qualitative or quantitative data (Elo & Kyngäs, 2008), was indicated by Schreirer (2012) as a flexible and systematic method which reduces data. Fraenkel, Wallen and Hyun (2012) spoke of content analysis as a method with a wide scope of utilization in educational research providing researchers with a perspective on problems to be tested through more direct methods. It is suggested that content analysis did not display a certain difference when considered in terms of qualitative or quantitative; however, it had certain subjective characteristics (Schreirer, 2012). Additionally and more essentially, content analysis as a summarizing method draws attention to intersubjectivity or objectivity while setting a scientific method allowing for testing of hypothesis as well as reliability, validity, replicability and generalizability (Neuendorf, 2002). Meta-synthesis (thematic cocntent analysis) is used as a type of content analysis in this study. It is possible with the meta-synthesis (thematic content analysis) to synthesize and explicate the studies conducted in a specific content area via themes, templates and matrixes with a critical perspective (Çalık & Sözbilir, 2014). The reason of selecting the meta-synthesis (thematic content analysis) is to examine in depth with similar or different aspects of the current situation with identifying the similarities and differences in the specific subject area studies reached with previously determined criteria. Another reason is the lack of enough studies for descriptive content analysis because of the topicality of the subject. In addition, it can be said that the number of experimental research not yet carried out for a meta-analysis. Consequently, a metasynthesis (thematic content analysis) was performed through synthesis and interpretations in this study.
This study had access to original studies via ISI Web of Knowledge database for analyzing sustainability in terms of certain theoretical basis and these studies were examined within a holistic approach through meta-synthesis (thematic content analysis). The common point of all these research studies is their educational context. Some of the studies analyzed sustainability to be used in the usage of e-learning environments, social networks, web 2.0 technologies or teaching platforms enriched with innovative technologies. While new approaches to these technologies were introduced through these studies, some other studies were considered within a more theoretical perspective and useful suggestions were generated to improve the existing models. In the light of this variety observed in the studies, with the help of the keywords determined according to the problem situation of this research, electronic and published journals registered in the national and international databases were investigated regardless of year limitation. The reason for unlimited period of years was the recentness of the studies on sustainability and accordingly nor outdated studies were detected. Keywords essential to the problem statement are as follows: sustainability, intention to continue, continuance, continuance intention, behavioral intention, information systems continuance, real usage continuance, information systems continuance intention, sustainable usage, continuance usage. These keywords are decided as a result of the examination of variables of theories and models discussed in the literature. Keywords were made from the target variables tried to be explained in the models. These target variables provide addressing the sustainability of the theories or models they discussed.
The study made use of the ISI Web of Knowledge database and articles reviewed in Social Sciences Citation Index (SSCI) together with Conference Proceedings Citation Index -Social Science & Humanities (CPCI-SSH) citation indexes were chosen as primary resources. Out of the 315 articles, the ones outside the scope of education or instruction such as the ones in the field of banking and finance, environment and nature, political sciences, international affairs and tourism management were left aside. In the filtering process, titles of all articles and journals in which they were published is investigated by the researchers. Articles which are not in educational context excluded from the review process. When the article can not be decided according to the title and the journal published, their summaries were read and the final decision was given. Eighty-seven filtered and reviewed studies based on the models and theories in line with the research were included in the study. Selected studies were assigned codes by the researchers according to the theory or model they are based on, name of the publication they were published in, year of publication, number of citations, research groups, examined technology and the learning environment, data analysis technique, structures, the effects of which were found to be significant or insignificant, as well as the dependent variables to be predicted. An article investigation form was developed by the researchers for gathering the codes as holistic. In this article investigation form which also can be expressed as a data collection tool, the publication name, year, the journal published in, theory or model discussed in, total and average number of citations and the study groups (population / sample) of the articles are given. In the article investigation form, investigated media / technologies, the statistical methods used, dependent variables and their prediction percentages, independent variables and statistically insignificant relationships are also available. Before starting the coding process, article investigation form is analyzed by the researchers, reached a consensus on and it was decided that the article investigation form is suitable for the aims of the research.
Assignment of codes made by the researchers to the 87 articles was checked by 10 field specialist coders studying in the Department of Computer Education and Instructional Technology at the graduate level. The codes, which were found to be uncertain, were discussed with the researchers and the final decisions were made. 9 articles were given to 9 field specialist and 6 articles were given to 1 field specialist initially and an informative training session was held such as the coding education suggested by Neuendorf (2002). Neuendorf (2002) assigned researchers and coders with different tasks within the recoding process and suggested that all possibilities shall be revised. At the end of this process, which could be counted as an expert opinion, validity and reliability of the study were contributed and the coding table was finalized. In such a control process, each expert investigated the articles and checked the codes given to him/her and did not see the others articles and codes. Because of this, an agreement percentage is not calculated. However controls and feedbacks of all experts have been controlled by the researchers and finalized.

Limitations
Findings of this study are obtained through the thematic content analysis of articles in certain databases investigated in terms of predetermined indexes. More comprehensive databases could be reviewed to reach different research conclusions. Moreover, the number of citations reported in the study were obtained from the ISI Web of Knowledge database without any restrictions. Self-citations of researchers could be eliminated for further evaluation and analysis.

Findings
Findings of the study according to the sub-problems are listed hereby below: Findings related to the theoretical grounds Ten out of the research studies analyzed in this study set the 11,5% of the whole amount and was observed to be based on a single theory or model. Similarly, 77 (88,5%) of them were modeled as a combination of various theories or models. Table 1 displays the models and theories that the single theory or model research studies were based on in the literature.  (2001) was widely used in research studies that are based on single theories. In some studies, the model was enhanced with the introduction of different theories and models reaching up to higher prediction percentages. In this respect, Table 2 displays the theories or models along with their frequencies, on which the research using a combination of theories and models were based.  (Davis, 1989) 16 DeLone and McLean's Information Systems Success Model (DeLone & McLean, 1992 9 Self-Determination Theory (Gagné & Deci, 2005) 6 Hedonic and Utilitarian Values Literature 6 Habit Literature 5 Cognitive Model (COG) 4 Social Cognitive Theory (SCT) 4 Task Technology Fit (Goodhue & Thompson, 1995) 4 Theory of Planned Behaviour (TPB) 4 Flow Theory 3 UTAUT (Venkatesh, et al., 2003) 3 Theory of Reasoned Action (TRA) 3 Attribution Theory 2 Agency Theory 2 Van der Heijden's model ( Van der Heijden, 2004) 1 SAFE Model 1 Fairness Theory 1 Computer Based Assessment Acceptance Model (CBAAM) 1 Information Adoption Model (Sussman & Siegal, 2003) 1 Contingency Theory 1 Motivation Theory 1 Social Network Site Adoption model (Sledgianowski & Kulviwat, 2009) 1 Self-Efficacy Theory 1 As seen on Table 2, there is a large theoretical variety in research studies using combined models. Additionally, the Expectation Confirmation/Disconfirmation Theory developed by Oliver (1980) and the Information Systems Continuance Model developed by  were widely used in research studies to predict intentions of individuals to continue using information technologies (f=30 for Expectation Confirmation / Disconfirmation Theory and f=18 for Information Systems Continuance Model). These theories were followed by Technology Acceptance Model, which creates a contrast with the acceptance-discontinuance anomaly introduced by Bhattacherjee (2001) and . This study focused on 25 different theories and models among the 77 research studies analyzed in terms of combined theories within its scope, and this is a strong indicator that displays the enriched theoretical variety of research with combined theories and models. Table 3 shows the journals that published the articles within the scope of the study. As seen in Table 3, 12,6% of the articles were published in Computers and Education, while 10,3% of them were published in Computers in Human Behavior. Articles published in these two journals set the 22,9% of the whole articles within the scope of the study. A total of 87 articles in 34 different journals revealed a wide perspective in terms of journal richness. Another finding obtained was that certain journals were chosen depending on the technologies of concern and the relevant research studies were published in the chosen journals. A study by  analyzed factors affecting students' intentions for continuance towards e-learning systems in academic libraries. This research was published in the Library Management journal. Similarly, articles published in Information & Management, Information Systems and Informing Science: The International Journal of an Emerging Transdiscipline journals indicate that the journals hosting the articles resourcing this study were quite rich and all the relevant articles were published in these journals by their contexts.

Findings related to the article statistics
Distribution of the articles to years was analyzed and it was found that there was a significant increase in recent years. Figure 1 displays the yearly distribution of articles included in this study and published between 2001 and 2014. The black line in Figure 1 shows a logarithmic increase in the number of relevant articles, which is an indicator that relevant problems have been hypothesized, while the field has been updated and became more popular among the researchers.   Table 4 shows the study groups that the data were collected from for the articles analyzed in this study. According to Table 4, it is observed that the studies were often conducted with university students and students benefiting from e-learning programs. The limitedness of studies on younger students, teachers or academicians is also striking. Twelve out of the 87 articles analyzed in this study (13,8%), data were collected from heterogeneous groups.  Table 5. In studies on sustainability, it was observed that Web-based learning environments and learning management systems were highly preferred (total f=47; 55,3%). In the Webbased learning environments category seen in Table 5, not to be seen as a

Findings related to variables and statistical analyses of concern
Eighty-six (98,9%) of the articles within the research were analyzed through structural equation modeling, while one (1,1%) of them was analyzed through multiple regression analysis. All research studies are regression based and this indicates that researchers have tried to determine the independent variables predicting the dependent variable along with finding out to what extent the observed variance in the dependent variable could be predicted by the independent variables. Looking at the structures studied and predicted in the articles, it is observed that 77 articles focused on continuance intention, 9 of them analyzed real usage continuance and one of them examined satisfaction.
Articles analyzed within the study were observed to have prediction percentages of the dependent variable at a wide range between 23% and 92%. In the study by , where the negative critical incidents perspective was investigated on the continuance intention for elearning, a rather high prediction percentage as 92% was obtained and this value was found to be a lot higher than the prediction percentages obtained in previous research studies on e-learning (Chiu & Wang, 2008 (60%), Liao, Chen, & Yen, 2007 (70%) and Chiu, Sun, Sun, & Ju, 2007 (66%)). The Information Systems Continuance Model developed by  was extended to habit and prior behavior variables in another study and a rather low prediction percentage as 23% was obtained . Table 6 presents certain independent variables, which are analyzed for their significant effects on dependent variables, their frequencies, and the references of the articles they were contained in. The main reason for analysis on the effects of satisfaction and perceived usefulness on continuance intention is because these variables were the basic variables in the Information Systems Continuance Model. Additionally, the variety of independent variables in the studies could be related to the richness of theories or models made use of in the research studies with heterogeneous theories or models. Although many studies made use of the Information Systems Continuance Model as the theoretical basis, there are various studies enhanced with other theories or models in recent years for the prediction of observed variance on dependent variables at higher percentages.

Discussion, Conclusion and Suggestions
This research is designed to determine tendencies in studies in the field of technological sustainability in education and is prepared to display the existing situation in the form of a general total evaluation. Keywords expressed at the beginning of the research period, database to be considered (ISI Web of Knowledge) and the citation indexes (SSCI and CPCI-SSH) resulted with 87 articles, the thematic content analysis of which led to findings of the study. Therefore, findings of the study and the evaluations depend more on the findings of the articles analyzed rather than the subjective perspectives of the researchers.
Many of the articles analyzed were found to be modeled as a combination of various theories and models. This result may be due to the willingness of researchers to predict the continuance intention potentially emerging in the target audience towards technologies that researchers focused on. Another potential reason could be the intention to gather variables of different theories and models with strong prediction powers and to reveal the relations between them. Various studies  indicate theoretical basis, while this finding is supported by description of weak or strong aspects of targeted models or theories and inclusion of other variables found effective in the model together with their reasons.
Among all 87 articles analyzed, the most frequently used and referred model is observed to be the Information Systems Continuance Model developed by . On the other hand, various theories have been made use of in heterogeneous research studies. This theoretical diversity reflects on the variables and the research studies contain various independent variables. Accordingly, the most frequently used variables in the studies are satisfaction, perceived usefulness, attitude and perceived ease of use. Another finding deriving from this variable diversity is that the observed variance of the dependent variables could be predicted at a wide range. While quite high levels of prediction percentages could be observed in certain studies , some remained with low percentages . In the light of this finding, it is suggested that continuance intention or real usage continuance should be considered separately according to each type of technology and the predictive power indicated by the literature should also be considered in determination of appropriateness of the models. It would be beneficial for the researchers to analyze deeply the studies with strong theoretical basis but low predictive powers and to evaluate the links that the findings derive from.
Acceptance-discontinuance anomaly introduced by Bhattacherjee (2001) and  supports that certain sustainability may not necessarily be provided in the usage of an accepted technology; however, certain studies in the literature were observed to take Technology Acceptance Theory as basis . Some variables of Technology Acceptance Model (attitude, perceived usefulness and perceived ease of use) were able to produce significant findings in sustainability studies; nevertheless, it should never be taken as a theoretical basis by itself in research studies of this field. Although adoption and acceptance is the prior step for the sustainability of any technology, it would be more appropriate to consider that the two theories attempt to predict different behavioral intentions. While the intention variable in Technology Acceptance Model refers solely to the usage of technology, the intention in Information Systems Continuance Model refers to the continuance of the intention. Some studies suggest that both theories could predict the intention for continuance ; however, according to what is observed, this approach is not found to be appropriate.
Many (41,3%) of the 87 articles in this research study were found to be published in journals with strong impacts in SSCI such as Computers & Education, Computers in Human Behavior, Behaviour and Information Technology and Decision support Systems (Impact factors are 2.630, 2.273, 0.839 and 2,036 respectively in 2014). The reason for this finding lies in the willingness of researchers to produce more qualified publications in years and the coherence between the purposes of these journals and the sustainability in usage of technological innovations in education. The large impact sizes of the aforementioned journals indicate that the published articles have higher tendencies to receive citations and that they could reach to broader reader audiences due to their popularity. The significant quantitative increase observed in recent years reveals the increasing usage of technological innovations in education and indicates that new field-related research problems are produced, which contributed to the popularity of the field.
Looking at the articles establishing the data resource of this study, it was observed that the article, which presented the Information Systems Continuance Model developed by , received the highest number of citations both in general and on average yearly basis. This is because the article is the principle research study in the field of sustainability and has set a resource to various other research studies in terms of theoretical basis. Another reason may be that the article is the oldest article among all articles studies in this research. Although the average number of citations per year overtakes the historical effect on the number of citations, the aforementioned study is observed to have such a high number of citations that could not be compared to other articles.
Articles of concern were found to be conducted mainly with university or e-learning students. It is suggested that researchers preferred to work on groups that they could reach easily. On the other hand, working with other data collection groups and testing the effects of theories on different groups could contribute to the literature. Teachers as inseparable parts of learning and teaching process should be consulted for their opinions on sustainable usage of relevant technologies and existing studies could be enhanced through supporting the findings with qualitative studies. As the resarch studies are proceeded with technology, e-learning tools, learning management systems and mobile Internet, it is suggested that determining the variables affecting the sustainability of alternative technologies, which have recently been used in research studies, would contribute to learning processess to be realized with the help of aforementioned technologies.
In studies by Cheung dated 2008 and suggested that real usage continuance should be assessed directly and included in the models as the continuance intention is an internal variable; it is observed that the research studies still focus on continuance intention. The structure that researchers analyzed is an internal variable as intention and this is in coherence with the finding that the Information Systems Continuance Model was highly made use of. Hence, the target variable of this model developed by  is intention. This may be due to the assessment tools and easy access to the scale items in terms of the aforementioned variable. Items in the Information Systems Continuance Model developed by  could be revised and used in terms of the technology of concern. Additionally, continuance intention could be assessed in sections. The reason why real usage continuance is not focused on in the studies could be the difficulty in longitudinal approach. Analysis on whether independent variables that could be assessed in sections could turn into an actual sustainable usage is suggested to have contributions to the literature.
Another finding of this study is that certain predictive relations could produce significant results in some studies, while others could produce insignificant relations. For instance, the effect of perceived usefulness on continuance intention was found to be statistically insignificant in some studies , while some others concluded that it had a significant predictive characteristic . Similar findings were observed in different variables (see Table 7) and it is suggested that the reasons for this situation could be further analyzed. Certain predictive variables that can produce statistically significant results could produce insignificant results from time to time due to the relevant technology, study groups of data collection processes or other variables that were analyzed in studies in terms of their effects. The reason for inconsistent findings shall be deeply examined considering that the theories, models and data collection groups of the studies are similar to each other. Although researchers state reasons for such findings, researching of this inconsistency through qualitative studies and considering variables in multiple dimensions would have essential contributions to the literature.
Articles analyzed within the study had a wide range of prediction percentages for the variance observed in dependent variables. In the light of this finding, it is recommended that instead of holding the anxiety to reach higher prediction percentages, variables that do not produce significant results should be analyzed for their reasons, and the findings should be interpreted sufficiently in terms of the data collection groups. Rather than predicting more of the observed variance in the dependent variable, focusing on the previously suggested theoretical model and revealing the complex relation patterns between variables would be more appropriate.
Following suggestions are made in the light of the findings and conclusions of this study:  Predictive relations that are found insignificant in studies should be deeply analyzed for their reasons. This could help in understanding the nature of variables that produce different results in different studies.
 The new sustainability researches which will be held in the educational context, can be evaluated in the journals shown in the findings of this research. Because when considering the aims and scopes of these journals, specific to the technology or behavioral context journals have a potential for publication.
 Longitudinal research studies are required to talk about sustainability in real terms regarding the usage of education technologies. Assessments to be made in two different time periods would reach to more accurate conclusions.
 While predicting the intention for the sustainable usage of a certain technology, unique characteristics of the technology should also be considered. Hence, certain predictive variables may produce different results depending on the type of technology.
 It is suggested that the sampling of the study should be described in detail in terms of their characteristics. While discussing the significant or insignficant structures in the light of the findings in the literature, the characteristics of the sampling group should also be considered.
 Educational sustainability studies should be also carried out with teachers or university lecturers. Because this study's findings reveal that the studies are carried out with the students.
 Future research on sustainability to be conducted with teachers or academicians should be compared to those conducted with students and observed variables found significantly effective regarding certain technologies should be determined for consideration in further teaching and learning processes.
 Variables based on the perceptions such as satisfaction, attitudes, subjective norms, perceived usefulness and perceived ease of use generally show significant and predictive features. Thus, in future studies with these variables, the dependent variable can be explained by the large percentage of the variance and is likely to reach significant findings. However, the effects in the models are not considered to be a means of giving more weight to other variables not investigated thoroughly.
 In the event that the observed variance in dependent variables are found to be low, qualitative studies should be conducted to analyze this results deeply.