Quantifying student satisfaction with technology- enhanced textbooks in the Japanese university context

Thirty-three students used their personal mobile devices to access digital media in a paper-based textbook containing Quick Response (QR) codes. The students worked in small groups with minimal intervention from the course instructor to participate in activities and complete assignments. Information regarding student satisfaction with the course, as well as their perceptions of the benefits and disadvantages of the instructional method employed, were gathered through a questionnaire and a reflective writing activity. The results indicated that overall satisfaction was high (M = 3.38), especially regarding the role of the instructor (M = 3.35) and interaction (M = 3.35). However, participants indicated a lower level of satisfaction with the materials (M = 2.89). Further inquiry revealed that dissatisfaction with the course materials was due to the lack of a stable Wi-Fi connection at the university rather than the design or content of the textbooks.


Introduction
The relationship between teachers, students, and materials is the fundamental axis of the working classroom. In Japan, this axis has long been codified in a linear direction with teachers at the top, students at the bottom, and materials between (Humphries, 2014). Even prior to formalized public schooling the role of Students, with personal mobile deviceswhich have been shown to increase exposure to the target language (Demouy, Kan, Kukulska-Hulme & Eardly, 2015) are able to work with greater autonomy and pursue a depth of learning greater than that available in traditional class structures when using traditional textbooks. In turn, this greater autonomy and a greater depth of learning may lead to greater satisfaction for students. This study aims to quantify student satisfaction in regard to autonomous use of technology-enhanced paper-based textbooks

Autonomy in the EFL Classroom
Second language acquisition has been, until recently, an autonomous pursuit. In Japan, the teaching of English as a foreign language was first institutionalized in the 1860s at the Yokohama Academy and at Keio Gijuku (now Keio University) and has since developed into a multi-billion-dollar private industry (Ike, 1995). However, in the 21st century, even as the institutionalization of language learning increases, so does the autonomous pursuit of second language learning (Benson, 2013). While technology has increased the accessibility of informal language study, institutions are grappling with how to best facilitate autonomous learning in the classroom with the aid of technology (Lai, 2017).
Autonomy is the emphasis on the role of the learner rather than the role of the teacher (Wei, Siriyothin, & Lian, 2018), and, according to Dafei (2007, p. 2), "learner autonomy is one of the most important issues that determine whether an individual reaches his/her potential or falls short of that potential." However, in order to reach this potential a significant investment in developing the ability to identify learning needs, monitor progress, and reflect meaningfully on outcomes is required (Reinders, 2018). Thus, programs that demand the development of such skills can induce anxiety and frustration in students, which has often led to failure in the past (Knowles, 1975). Moreover, as Jiang (2014) has found, learning language in the target language context may interfere with the learners" identity, and can even disempower their autonomous learning behavior.

Technology Enhanced Materials
Certain preconditions, then, can determine the success or failure of autonomous learning in an institutional setting but, as Benson (2013, p. 152) has written, "technology-based approaches are potentially supportive of autonomy" by giving the learner direct control of their learning, allowing wider access to authentic target language content, as well as providing greater opportunities for authentic use of the target language. In fact, it has been found (Reinders, 2018) that to successfully implement technology into a formal learning environment a certain degree of autonomy is not only beneficial but necessary. Similarly, Wei, Siriyothin, and Lian (2018, p. 27) have stated that "the impact of new technologies in educational contexts has been mostly positive." The limitations to autonomous learning, then, can potentially be offset by the introduction of technology into the learning environment. Websites such as YouTube have been shown to provide a degree of second language immersion that was previously unavailable outside of a study-abroad program (Watkins & Wilkins, 2011) and, as Leung (2004) has found, the incorporation of technology into the classroom can increase learner autonomy.
While the impacts of using technology in the classroom have been found to be generally positive with regard to learner performance, it remains largely undocumented as to how incorporating multimedia technologies -such as videointo more traditional, paper-based, classroom materials can impact not only learner performance but also learner satisfaction.

Student Satisfaction
Students" satisfaction is defined as "a short-term attitude resulting from an evaluation of students" educational experience, services, and facilities" (Weerasinghe, Lalitha, & Fernando, 2017, p. 533). Universities and colleges are concerned with student satisfaction and often make course evaluations an important part of determining the success of a particular course and the effectiveness of faculty (Kember & Ginns, 2012). In recent years, a number of studies have shown the flaws in student evaluations including issues with gender bias (Boring, 2017) as well as their lack of validity in measuring instructor effectiveness (Stark & Freishtat, 2014). Yet, higher student satisfaction has been correlated with greater retention, persistence, and motivation (Keller, 1983;Koseke & Koseke, 1991). In studies concerning instructional technology, satisfaction was determined to be a contributing factor in students" intention to continue to use digital textbooks (Joo, Park, & Shin, 2017) and e-learning systems (Mohammadi, 2015).
A number of factors have been identified as important in measuring learner satisfaction in a variety of educational contexts. Two such constructs associated with learner satisfaction are instructors (Navarro, Iglesias, Torres, 2005;Chen & Yao, 2016, Hew, Hu, Qiao, & Tang, 2020, and the teaching method and curriculum (Navarro, Iglesias, Torres, 2005;Farahamandian, Minavand, & Afshardost, 2013). The learning environment and support (Yusoff, McLeay, & Woodruffe-Burton, 2015;Chen & Yao, 2016) have also been shown to influence satisfaction in education. Technology and design (Chen & Yao, 2016) are factors that are important for blended learning courses while quality is a determinant of student satisfaction with open resource materials (Smith Jaggers, Folk, & Mullins, 2018). A modified version of an online learning satisfaction model (Bolliger and Halupa, 2012) was utilized in the current study; the factors examined were instructor, technology, setup, interaction, outcomes, and overall satisfaction.

Research Purpose and Questions
The purpose of this preliminary study was to explore participants" satisfaction with a paper-based textbook that used quick response (QR) codes to provide access to digital media using personal mobile devices. In addition, the participants" perceptions of a teaching strategy that focused on autonomy and self-directedness was also investigated. The following research questions guided this study: 1. What is participants" satisfaction with a student-led class using technology enhanced materials? 2. What are participants" perceptions of the benefits and disadvantages of a student-led class using technology enhanced materials?

Setting and Sample
The research took place in the economics department of a private Japanese university in 2018. The university had an enrollment of 32,600 students in 2018, spread across three campuses. In the same year, 3,326 students majored in economics specializing in one of three tracks. Students in the dual language stream studied English as well Chinese, Korean, French, German or Spanish, while students in the English and global reach stream focused on only one language but to a different degree of intensity. The English stream prepares students to function in English for daily communication and in business environments. However, students in the global reach stream are expected to achieve a higher level of fluency in English through participation in additional EFL classes which focus on presentation and writing skills as well as communication and economics content classes taught in English. Furthermore, these students are required to spend at least once semester studying abroad at an English-language medium university. Thirty economics students from the global reach stream, who were participating in a required Englishlanguage course, were asked to take part in the study. One-hundred percent of the class chose to complete the quantitative survey, but written responses for the qualitative section were only received from seven students.

Participants
All participants were first year economics majors who self-identified as ethnically Japanese. The majority of participants were male (80%); females made up the remaining 20% of the class. Seven participants were 18 years old, 20 were 19 years old, and three were 20 years old. Participants were also queried about their access to mobile devices. One-hundred percent of the participants reported having access to a mobile phone, while fewer had access to portable gaming devices (39.4%), MP3 players (33.3%), e-books (27.3%), and tablets (12.1%).

Pedagogical Procedure and Materials
The procedure was implemented over a 15-week, 90 minute-class schedule. The primary function of the teacher was to facilitate an autonomous-study environment by organizing students into groups and providing the necessary materials for autonomous learning. Eight groups of four students were formed using a random card selection system and the classroom was organized as per Figure 1. These groups were changed in week 5, week 9 and week 13. The second requirement of the teacher was to provide the materials. In addition to the technology-enhanced textbook (discussed later), a Guide for Leaders, created by the teacher and distributed to everyone in the first class, and a Small Talk Guide, created collaboratively with the students in the first class, were provided. Both papers provided group leaders with an initial resource should they need assistance.
The Guide for Leaders included useful phrases for time management such as "we have ___ minutes to do this part, so let"s be quick," phrases for organization such as "OK, let"s stop there and change partners" and "it"s time to move onto the next topic," and phrases for asking for help from group members (first) and the teacher (second).
The Small Talk Guide included a list of greetings, opening questions, follow-up questions, and suggested topics for small talk. Every class would begin with a tenminute small talk period aimed at warming up the students to both using English and working as a group autonomously from the teacher.
The primary material used throughout the course was an in-house text called Thematic Development. The text was designed for use in a more standard, teacherled classroom but lent itself well to the approach taken for this research. This was primarily because the text was entirely self-contained, with links to outside resources -such as publicly hosted videos -embedded in the text using QR codes. Students could access these materials on their smartphones via the university Wi-Fi. (As all activities were group based, if one or two students didn"t have access to their own device or couldn"t connect to the Internet, they were able to share with a group member.) An example of a Thematic Development textbook activity utilizing QR codes can be seen in Figure 2. As the textbook was not designed specifically for an autonomous approach, another requirement of the teacher was to provide a class schedule (written on the blackboard) that the leaders could follow. Table 1 shows an example schedule from week 8 of the course. As can be seen, the instructions provide page numbers, times for each activity, and auxiliary information to supplement instructions in the textbook. Leaders were responsible for managing the time according to the schedule, organizing their group for each activity, and handing over to the teacher when required. The teacher"s final role was to provide feedback at the end of each class. Group leaders, for whom written feedback was provided, were assessed according to three criteria: 1. Speaking English 2. Involving all group members 3. Depth of discussion The leader was responsible for the entire group, meaning that if one group member spoke anything other than English the leader would be penalized. This, in practice, meant all group members worked very hard to speak only English in order to avoid negatively impacting another student"s grade. It was also the leader"s responsibility to involve all members of their group. The Guide for Leaders provided phrases such as "what do you think?" and "how about you?" to prompt them. As the textbook was based on developing themes such as pollution, poverty and food waste the final criteria against which the leaders were assessed was "depth of discussion." In practice, this asked the leaders to take their group discussion beyond the basic questions posed in the textbook and to get their group to express their opinions in detail.
The teacher also gave more general feedback verbally, at the end of the class. This feedback included pointing out common mistakes that had affected multiple groups as well as advice for future classes regarding leadership skills and how to positively support a leader as a regular group member.

Quantitative Instrument
The survey instrument consisted of 32 items, including the satisfaction scale and demographics. The first section of the survey was adapted from Bollinger and Halupa"s 2012 work on satisfaction. The researchers received permission to modify and translate the original instrument to fit the needs of this project. The modified instrument consisted of 27 items arranged into six constructs; instructor, technology enhanced materials, set-up, interaction, outcomes, and overall satisfaction. The survey items were translated by a native Japanese speaker with a high-level of competency in the English language and an advanced degree in educational technology. A second native Japanese speaker, a professor of EFL, checked the translation and offered further suggestions. A Cronbach"s Alpha was calculated of the satisfaction scale after the survey was administered and showed that it was highly reliable ( = .908).

Reflective Writing Activity
A follow-up qualitative instrument was utilized to collect more detailed responses from the participants. This consisted of six open-ended questions (Table 2) designed by the researchers. The reflective writing activity was conducted via email one week after the end of the course. Seven of 30 participants responded (23.3%). Table 2.
Open-ended questions used in the reflective writing activity.
Original Japanese English Translation

Data Collection and Analysis
The participants were asked to fill-out a paper-based version of the quantitative survey instrument in the final week of the semester. Before handing out the survey, one of the researchers provided the students with an explanation of the study and their rights, in Japanese. He made it clear that participation in the study was not mandatory and would not affect the participants" grades. The students were informed that returning the completed survey to the instructor constituted consent to participate in the study. The survey sheets were collected by one of the researchers and stored in a secure location. Then, the data was input into an SPSS worksheet in preparation of analysis. Frequencies were tallied for all items. Any missing values in the data set were replaced by the series mean. Finally, descriptive statistics were calculated for all sub-scales and the total scale.
An email was then sent to all students asking for their participation in a reflective writing activity. Seven students chose to participate. The responses were collected and organized in an Excel spreadsheet. The researchers used open-coding to develop themes and sort the responses based on their respective category (Bogdan & Biklen, 1998;Flick, 2006).

Research Question 1 (Satisfaction)
Responses to items in the sub-scale "instructor" were overwhelmingly positive. For example, 100% of participants agreed or strongly agreed with items 2, 4, and 5 showing that class assignments were clear, feedback was timely, and the instructor was seen as welcoming. The majority of students agreed or strongly agreed with items 1 and 3 and disagreed or strongly disagreed with item 6, which is understandable because it was reverse coded. These results are in line with expectations as the failure of the instructor in these areas would have likely meant the course would not have been able to operate as designed. Moreover, considering the course was successfully completed by 100% of the students, it could be assumed that the level of student satisfaction with regard to these areas would be high. Despite the focus of this study being student-centeredness and autonomy, it seems that the instructor still played a vital role in contributing to student satisfaction. In fact, even in research of massive online open courses, where one-toone interaction with the instructor is minimal, perceptions of the instructor were significant predictors of course satisfaction (Hew, Hu, Qiao, & Tang, 2020). However, instructor perceptions are just one of several factors that must be considered in determining student satisfaction towards instruction and may not be the most important predictor (Chen & Yao, 2016). ASIAN TEFL, Vol. 5 No.1, 2020, www.asian-tefl.com  3.36 .718 Note. Scale ranging from 1strongly disagree to 4strongly agree.
[R] = reversed item Technology enhanced materials. Responses to the "technology enhanced materials" items were also positive. More than 80% of participants agreed or strongly agreed with items related to their satisfaction with embedded videos and the topics which were covered. Over 90% stated that they agreed or strongly agreed that they were satisfied with the materials overall. In addition, most respondents were not dissatisfied with having to use their personal mobile devices in the class (83%). This result shows a change in the attitudes of Japanese university students. Earlier studies indicated that students were reluctant to use their personal mobile devices for formal study due to privacy and a desire to keep their private and school lives discrete (Kondo et al., 2012;Stockwell, 2008Stockwell, , 2010. However, 80% of students stated that they were dissatisfied with the university"s Wi-Fi service. This issue has been previously identified as a barrier to usage of mobile devices for educational purposes in the Japanese university context (Author, 2016). This sentiment is not limited to Japan. The ECAR Study of Undergraduate Students and Information Technology, 2019, which surveyed students from over 160 institutions of higher education around the world, found that Wi-Fi access and reliability were a concern, and that university dormitories/campus housing and open spaces were rated poorly in this regard (Gierdowski, 2019). Note. Scale ranging from 1strongly disagree to 4strongly agree.

Means and Standard Deviations of Technology Enhanced Materials.
[R] = reversed item Almost all students (96.7%) responded that they were satisfied with the independent learning that occurred in the class and 86.6% of students disagreed with the item asking if they were dissatisfied with the level of self-directedness required of them in the class. Finally, 86.7% of participants responded that they were satisfied with the independent class activities that did not contain explicit instruction from their teacher. As satisfaction has been shown to negatively correlate with anxiety (Bolliger & Halupa, 2012) these results suggest that students were comfortable with the level of autonomy required of them and did, therefore, not suffer from the anxiety and frustration often related to autonomous learning noted by Knowles (1975). The participants" responses were surprising considering that formal education in Japan is often teacher centered, especially in terms of assessment. In the words of Yamamoto and Kinoshita (2019, p. 299), "In Japan, it has been considered quite normal for learners to depend on teachers regarding assessment processes and is a probable cause for Japanese learners" tendency to value the result of tests and final grade rather than their own learning process." While the items in this construct did not focus on self-or peer-assessment, many of the class activities did. Therefore, the results of our study might indicate a change in attitudes among young adult Japanese learners regarding increased agency in learning for both tasks and assessment. Table 5 Means and Standard Deviations of Set Up. 14. I am satisfied with working on activities without explicit direction from the teacher 3.13 .628 Note. Scale ranging from 1strongly disagree to 4strongly agree.
[R] = reversed item Over 90% of participants agreed or strongly agreed that they were satisfied with the interaction (96.7%) and relationships (96.7%) between themselves and other students in the class. This is important because interaction between peers and with teachers have been shown to be predictive of engagement in educational settings (Ngyuen, Canata, & Miller, 2018). Over 90% also disagreed or strongly disagreed that they were dissatisfied with the collaborative activities (96.7%) and student-to-student interaction (96.7%) that occurred in the course. These findings were encouraging to the researchers who worried that the participants might have an aversion to the student-centered approach used in the class because formal Japanese education tends to utilize teacher-centered instruction, focusing on lectures and rote memorization (Littrell, 2006). However, responses to a statement related to the participants" comfort with taking the role of leader were mixed, with 30% of students disagreeing and 70% agreeing or strongly agreeing. These findings might be explained by the Japanese propensity to avoid taking on obligations, which in a collectivist culture, could lead to blame and loss of face (Gudykunst & Kim, 2003). All students who took part in the class agreed or strongly agreed that they would be able to apply what they learned in the course in future English-speaking environments. In addition, 97% of participants stated that they were satisfied with the level of effort the course required and 96.7% agreed or strongly agreed that they would be satisfied with their final grade. Only a small portion of participants (26.6%) indicated that they were dissatisfied with their performance in the class. These results could demonstrate a positive correlation between learner satisfaction and class environments that provide students with opportunities to take more responsibility in the learning process. Note. Scale ranging from 1strongly disagree to 4strongly agree.
[R] = reversed item Over 90% of participants indicated that they agreed or strongly agreed that they were satisfied with the course and would like it taught in the same manner in the future (96.6%). Furthermore, 93.3% of participants disagreed or strongly disagreed with item 25 regarding their dissatisfaction with the current course compared to others they had taken and 73.3% stated that their experience would encourage them to take further courses delivered using a similar student-centered approach. While the researchers were pleased with these results as they suggested a willingness among Japanese students to a engage in a student-centered approach to learning, it would require further research with a larger sample and with a wider range of English abilities before any conclusions about overall satisfaction with such a method could be drawn. Note. Scale ranging from 1strongly disagree to 4strongly agree.
[R] = reversed item Students seemed to be extremely satisfied with the course; the total scale and all but one of the subscales received a mean score above 3 on a Likert scale ranging from 1 (strongly disagree) to 4 (strongly agree). The highest mean score was associated with the overall subscale and the lowest with materials. However, it seems that dissatisfaction with materials was mostly directed at the university"s Wi-Fi service and not the technology-enhanced textbooks created for the course. Table 9 displays the mean values and standard deviations of the total scale and subscales.

Research Question 2 (Advantages and Disadvantages)
In a follow-up reflective writing activity, students were asked to express the advantages and disadvantages of the course, as they perceived them. The perceived advantages mentioned by the respondents can be divided into two categories: advantages of self-reliance and advantages of student interdependence. One student wrote that "opportunities for independent thought were much higher than in a teacher-centered class." While this was pleasing feedback for the researchers as developing independent thought was a goal of the class, and an indication that students are truly meeting their potential as learners (Daefi, 2007), it was not necessarily expected that students would themselves perceive it as a positive aspect of the class. It remains to be seen whether such a perception would be realized by students with a lower level of English ability.
The reliance on other students was reported in a positive light by a student who wrote, "as the students were the focus of the class we were able to stretch and push each other"s English ability," and another was happy with the utility of a course in which they "had to use real communicative English in order to work as a group." The requirement to work entirely in English was a concern for the researchers but such comments, coupled with the quantitative satisfaction results, are contrary to the notion purported by Jiang (2014) that language learning that takes place in the target language could pose a threat to the learner"s identity and might even cause students to lose motivation to study autonomously.
Disadvantages, similarly, were also related to self-reliance and student interdependence. One student, referring to their personal struggle with being a strong leader stated that "the group was dependent on the leader, so if the leader was weak, we would go off topic." Another student identified how a weak student in the group could negatively impact the whole, writing, "lazy students left everything to the more capable students." While these observations may reflect student dissatisfaction with the required autonomy in the class, the researchers believe they can be linked more closely to the Japanese cultural bias that favors avoiding roles of personal responsibility (Gudykunst & Kim, 2003). Another aspect of Japanese culture which might play a role in influencing these responses is that teachers are highly respected (Davies & Ikeno, 2002) and students might feel uneasy with the diminished role in the class setting. More research in other cultural settings would be required to verify these assumptions.

Conclusion
The purpose of this preliminary action research study was to explore participants" satisfaction with a paper-based textbook that used quick response (QR) codes to provide access to digital media using personal mobile devices. In addition, the perceptions of a teaching strategy that focused on autonomy and self-directedness were also investigated. The research findings showed that participants were satisfied with the course; the highest mean score was associated with the construct of overall satisfaction and the lowest with materials. While students reported to be satisfied with the content and the technology used for the materials, they were dissatisfied with the university"s Wi-Fi service. Students appreciated the opportunity to think independently and support each other while using English to accomplish a task. However, they were concerned that ineffectual leaders and students not willing to pull their own weight could ruin the experience.
Although the authors have identified the findings of this study as preliminary, it is important to identify the limitations that existed in the methodology. The research was conducted in only one class of high-level students at a private Japanese university. These students might have displayed a higher level of autonomy and selfdirection when compared to their counterparts in other sections of the course or at other universities. In addition, because the class was taught by one of the researchers, it is possible that the students" responses were swayed by this unequal power distance relationship.
In order to conduct a more valid and reliable study in the future, a quasiexperimental design will be utilized with at least two classes, including both experimental and control groups. Furthermore, it would be interesting to research how factors such as English-language proficiency, gender, and international posture and contact might influence responses. While the results presented here are preliminary and more research will need to be conducted in order to extrapolate these findings to Japanese university students as a whole, they do provide hope that student-centered models of classroom instruction and materials development can be effective in Japan. This may be surprising to many working in this context who are usually confronted with teachers and administrators mired in traditional teachercentered approaches and students who have become accustomed to being passive receivers of knowledge rather than creators. Yet, through the use of proper instructional scaffolding in the classroom and the development of engaging technology-enhanced materials, greater autonomy and self-directedness in the Japanese university classroom seem to be worthwhile and achievable goals.