Bibliographic record of an article published in Teaching and Teacher Education (Elsevier), Volume 119, article 103870. It is not a publication of this journal. Metadata retrieved from Crossref and OpenAlex (2026-11-17).
Abstract and full text reproduced from the published version under CC BY 4.0. © The authors.
Read the article on the publisher’s site ↗Teachers’ preferences for online professional development: Evidence from a discrete choice experiment
Show affiliations and correspondence
- aMaastricht University; Universitas Islam Negeri Sultan Syarif Kasim Riau
- bMaastricht University
- cMaastricht University; KU Leuven
DOI: https://doi.org/10.1016/j.tate.2022.103870
Outline
Abstract
This paper reports on teachers' preferences for online professional development programmes in Indonesia. Relying on data from 330 university English teachers, the results of our discrete choice experiment suggest a significant positive preference for synchronous interaction but a negative preference for costly programmes, programmes offering no certificates of participation or completion, conducted over a longer period and using digital reading materials on teachers' preferences. Subgroup analyses also show that age, gender and experience with online professional development programmes influence teachers’ preferences. Based on these findings, implications for the design of online professional development and gaps for future research are discussed.
Keywords
1. Introduction
Teacher professional development is traditionally delivered in the form of in-person teacher professional development (TPD). However, due to a variety of factors such as mobility restrictions (e.g., during the COVID-19 pandemic), or efficiency concerns, TPD programmes are increasingly provided online as technology has advanced and ease of use has improved. In response to the rise in online teacher professional development (OTPD), this article reports on Indonesian university English teachers' preferences for an OTDP programme. Many types of data, such as student achievement and teaching portfolios, are available within schools or higher education institutions. According to Schouten (2017), data are essential for improving the quality of education in an evidence-based manner.
For example, in higher education, both big data and traditional data can be used to analyse student needs, improve instruction, and provide advice on best practices. Furthermore, research reveals that when data are effectively used to improve instruction through a systematic inquiry process, it potentially increases student learning outcomes (Ansyari, Coelho, Hasibuan, Settiawan, & Kamallia, 2020; Carlson, Borman, & Robinson, 2011; Lai & McNaughton, 2016; Mandinach & Schildkamp, 2021). ‘Data use,’ or ‘instructional data use,’ is the term used in this study to describe the use of data to * Corresponding author. Maastricht Graduate School of Governance, Faculty of Business and Economics, Maastricht University, Boschstraat 24, 6211 AX, Maastricht, the Netherlands. suska.ac.id (M.F.
Ansyari). j o u r n a l h o m e p a g e : w w w . e l s e v i e r . c o m / l o c a t e / t a t e improve instructional practices and, ultimately, student learning outcomes. Data literacy is considered a critical component for effective data use (e.g., Schildkamp, Poortman, Luyten, & Ebbeler, 2017). It refers to the ability to transfer data (information) into actionable instructional knowledge and practices by collecting, analysing, and interpreting all types of data to help determine instructional steps (Mandinach & Gummer, 2016, p. 367). Given that teachers are not well prepared for data use during their pre-service teacher education or initial in-service teacher preparation (Mandinach, Friedman, & Gummer, 2015; Reeves, 2017; Reeves & Honig, 2015), policymakers and researchers have emphasized the importance of supporting teachers' data literacy through professional development (e.g., Mandinach & Schildkamp, 2021).
Professional development is a crucial opportunity for teacher learning and development. Literature suggests that professional development can change student learning outcomes (Desimone, 2009; Merchie et al., 2018). Desimone (2009), for instance, proposed a pathway model that explains how professional development can lead to student outcomes. This model illustrates that improved student learning outcomes are mediated by increased teacher quality and improved instructional practices. Translating the model into the context of data use, participation in a professional learning programme (professional development) can help teachers develop their data literacy (teacher quality). By having such literacy, teachers can use various kinds of data effectively to improve their instructional conditions (improved instruction) and this improved quality of instruction contributes to better student learning achievement (learning outcomes).
In-person TPD programmes to increase teachers' data literacy have been reported in numerous studies (e.g., Lai, McNaughton, Timperley, & Hsiao, 2009; Supovitz & Sirinides, 2017; van der Scheer & Visscher, 2018; van Geel, Keuning, Visscher, & Fox, 2017). Yet, in-person professional development is not always feasible due to e among others - mobility constraints and efficiency considerations. For example, van der Scheer and Visscher (2018) estimated that a one-year data use programme would cost around $2500 per teacher. Based on this, in-person TPD seems to be costly. Providing programmes online could make them more efficient in addition to other advantages of online provision of educational programmes.
As studies have shown, OTPD programmes have the potential to increase access for teachers to learn new knowledge and skills without geographical constraints, at more convenient times, thus making the programmes more cost-effective (Compen, De Witte, & Schelfhout, 2019; Dede, Ketelhut, Whitehouse, Breit, & McCloskey, 2009; Reeves & Pedulla, 2011). An online teacher professional development (OTPD) programme is defined as a TPD format that allows teachers, via information communication media, to provide continuous learning without having to meet their trainers/instructors and peers every time in person (Rogers, 2001). Concerning this TPD modality, studies generally found that there is no difference in teacher perceptions and learning effectiveness between in-person TPD and OTPD (Darling-Hammond, Hyler, & Gardner, 2017; Fishman et al., 2013; Tømte & Gjerustad, 2020; Yoon, Miller, & Richman, 2020).
According to Tømte and Gjerustad (2020), teachers in online and campus-based learning environments, for example, have relatively similar perceptions of learning outcomes. Similarly, Yoon et al. (2020) found similar positive teacher perceptions of both PD formats and equal student outcomes. In addition, the study by Fishman et al. (2013) demonstrates the impact of both PD modalities on teacher and student learning. The results of their study revealed that there is no significant difference in teacher or student learning gains between in-person and online professional development. The same effectiveness of in-person TPD and OTPD programs could be attributed to the programs' sound design rather than the modality or delivery methods.
Media contributes to learning only when it is integrated into the learning framework (See Clark, 1983; Kozma, 1994). Therefore, different outcomes cannot be expected when the difference is only the media used (Fishman et al., 2013). Darling-Hammond et al. (2017) found that effective TPD relies on the key features of teacher professional learning activities and technology can facilitate active learning and modelling during OTPD. For instance, teachers who participate in TPD either inperson or online and get feedback from professionally trained experts are associated with improved early literacy outcomes (Landry, Anthony, Swank, & Monseque-Bailey, 2009). Teacher engagement with the TPD content and control over time are also considered important factors.
According to Fishman et al. (2013), engaging teachers by reflecting on proximal practice, working in a short period, and reviewing individual TPD lessons can compensate for the affordances of the in-person TPD that has more opportunities for collaboration, information sharing, and discussions. The tremendous range of time that OTPD can offer is also a valuable affordance so that teachers can flexibly access and benefit from PD materials. Despite similar effectiveness, Kim, Morningstar, and Erikson (2011) warned that there is a concern about higher dropout rates among online participants. Therefore, OTPD should be more tailored to the needs of participants to reduce dropouts or to engage them better (Compen, De Witte, & Schelfhout, 2021; Community for Advancing Discovery Research in Education/CADRE, 2017).
To this end, however, designers of OTPD may face the challenge of what and how needs from different voices, interests or expectations can be incorporated. Without careful attention, this challenge might lead to design biases inherent in how designers see such a challenge (Mishra & Smith, 2020). For instance, the behavioural patterns taken from early users and adopters are used to inform the design of Massive Open Online Courses (MOOCs) and these data are not representative of the broader MOOC population (Mishra & Smith, 2020). Reflecting on this, OTPD should be designed based on the target participants' needs rather than on limited, unrepresentative available needs assessment data. The needs, for instance, can be obtained by identifying the participating teachers' preferences for the characteristics of OTPD.
So, OTPD can better cater to teachers’ needs, dropouts can be minimized, and the effectiveness can be optimised relative to in-person TPD programmes. Previous research has provided some insights into teacher preferences for OTPD. Barnes, Guin, and Allen (2018) assessed the TPD needs and OTPD preferences of North Carolina early childhood professionals (ECPs) in the US. They used a qualitative approach through focus group discussions and found that the ECPs' preferences for online course format vary, depending on teachers' experiences and learning preferences. Some prefer synchronous while others more like asynchronous or blended modes. Lin (2015) also investigated teachers' views and concerns about professional development in an online environment using mixed methods.
The findings showed that teachers prefer an OTPD programme using both synchronous interactions and asynchronous digitalised visual and written resources online. In a recent study, Sadeghi and Navaie (2021) investigated teachers' preferences for OTPD in the context of English language teaching. Through an online survey, they asked English teachers to rate the likelihood of their participation in OTPD based on its activities. The descriptive results show teachers’ preference for expert-teacher application followed by online video lesson study while real-time instruction feedback is the least preferred activity. The findings of the above studies provide insights into the M.F. Ansyari, W. Groot and K.
De Witte different characteristics of OTPD but the studies do not provide evidence on how teachers make trade-offs between the different characteristics (attributes) of an OTPD programme. Therefore, this study aims to examine teachers’ preferences for the uptake of an OTPD programme by using a quantitative approach to quantify the relative importance of and trade-offs between OTPD characteristics. To this end, this study is guided by this question: Which characteristics of an online teacher professional development programme for data use are most important in encouraging teachers to participate? This article is structured as follows. Section 2 presents the theoretical framework of the study.
Section 3 describes the methodology which consists of study design, sample, development of DCE questionnaire, data collection procedure and econometric analysis. Section 4 presents the findings of the study, followed by a discussion and conclusion in Section 5. Finally, the implications and limitations of the study complete the article in Section 6.
2. Theoretical framework
Teacher professional development is increasingly offered online in line with the development of technology (Dede et al., 2009), and the COVID-19 pandemic has accelerated the development of OTPD (Hartshorne et al., 2020). Given that OTPD may provide teachers with greater choice (flexibility), it may increase teachers' inclination to engage actively and productively with OTPD content (CADRE, 2017). To identify the components or characteristics of OTPD that potentially influence teachers’ willingness to participate, we discuss the relevant literature that helps us frame our study in the following paragraphs. Several studies have provided insights into the importance of interaction in OTPD (e.g., Barnes et al., 2018; Herrington et al., 2001; Lin, 2015; Rogers, 2001).
Interaction is described as the ways trainers interact and communicate with participating teachers in an online learning environment, such as synchronous or asynchronous interaction (Barnes et al., 2018; Lin, 2015; Rogers, 2001). Synchronous interaction allows for real-time meetings and discussions (e.g., video conferencing) while asynchronous interaction occurs in a time delay for the interaction to happen (e.g., email). Asynchronous OTPD provides participating teachers with the opportunity to self-pace and access materials at flexible hours (Yoon et al., 2020), but teachers can get direct responses or feedback from the trainer or other participating teachers through synchronous interaction (Yeo, 2021).
Despite the different advantages, both asynchronous and synchronous interaction formats are considered to have similar effects. As research shows, both synchronous and asynchronous interaction can facilitate reflection and evaluation (e.g., Denoyelles & Raider-Roth, 2016). In their meta-analysis, Bernard et al. (2009) found that synchronous, asynchronous, and mixed delivery contribute equally to learner achievement. Learning materials or resources are also considered a crucial component of OTPD (e.g., Bragg et al., 2021; Darling-Hammond et al., 2017; Herrington et al., 2001; Lin, 2015; Nunan, 2012). Learning materials are representations of OTPD content. In other words, content is usually transformed into what teachers are learning with (materials or resources).
Since technologies have advanced rapidly, learning content can be digitally represented or made into multimedia learning materials such as digital texts or audio-visuals. The use of such different representations of learning materials is intended to meet teachers' individual differences in learning (Bragg et al., 2021). In practice, learning materials can be videotaped or written to provide models of case studies, sample lessons, etc. (Darling-Hammond et al., 2017). Based on a literature review, Bragg et al. (2021) argued that current studies have not sufficiently provided evidence on the role of such material representations (e.g., videos) in fostering positive outcomes. However, the literature provides some insights into teachers’ preferences for learning material representations.
Some teachers may prefer viewing videos as a means of acquiring information while others may be more comfortable with text-based materials or the combination of both modalities (CADRE, 2017; Sadeghi & Navaie, 2021). Furthermore, common models of online learning to support teacher learning include individual or collaborative learning. Some teachers like to work individually at their own pace while others prefer collaborative work. There is substantial research that indicates collaborative learning is essential for teacher learning (e.g., Darling-Hammond et al., 2017; McElearney, Murphy, & Radcliffe, 2019). For example, McElearney et al. (2019 reported that most teachers acknowledge the importance of learning in groups or with other people such as the opportunities to discuss, share and engage in the PD content.
Nevertheless, this is not the case for teachers with different individual learning preferences. Collaborative work might be perceived by some teachers as a motivating factor for a sense of community but some teachers regard it as a demotivating course factor (Banegas & Busleiman, 2014). In addition to interaction, learning materials and approaches, we identify that OTPD can be an opportunity that teachers use to maintain or gain in their professional careers. Research indicates that teachers participate in OTPD because they want to obtain certifications for further requirements or needs in their teaching profession (Donavant, 2009; Taylor, 2011). Taylor (2011) found that teachers choose to enrol in an OTPD course because they want to maintain their certification requirements.
Some teachers indicate that their primary reason for participating in an OTPD course is to maintain their current teaching certification, some others to fulfil the need for continuing education credits, and still some others to earn credits for a higher level of teaching certification. Based on these studies, seemingly it is necessary that providing proof of OTPD participation or achievement may be worth it to teachers, depending on their personal needs and professional requirements.
3. Methodology
In this section, we discuss our study design, sample selection and the process of developing the DCE questionnaire. We also provide information about how we collect data and analyse them using an econometric technique.
3.1. Study design
We conducted a discrete choice experiment (DCE) to elicit university English teachers' preferences for OTPD programmes for instructional data use. DCE is a quantitative method that can be used to elicit individuals' preferences for product, service or programme characteristics (attributes) when revealed preference data (such as observed data) are not available or when eliciting preferences from actual behaviour is either invalid or not possible (Cleland, Porteous, & Skatun, 2018). As a stated preference method, DCE can also be used for quantifying the relative importance or strength of the characteristics of a product, service or programme and for determining potential uptake rates of the characteristics (Hensher, Rose, & Greene, 2005; Louviere, Hensher, & Swait, 2000).
Additionally, compared to most other valuation techniques, DCE mimics better real-life situations as the DCE attributes are meaningfully presented in realistic situations to respondents’ real life. The trade-offs between attributes are also more similar to real-life conditions where individuals have to consider multi-attributes by weighing up the pros and cons of the alternatives on a service/ product on offer (e.g., buying a mobile phone) or when making decisions (see Cleland et al., 2018; De Beckker, De Witte, & Van Campenhout, 2021). M.F. Ansyari, W. Groot and To implement DCE, surveys are normally used. A DCE survey consists of several choice sets and each choice set has two or more alternative hypothetical scenarios that describe a product, service or programme.
Each scenario (option/alternative) is described by several attributes (about five or six attributes) characterising a product, service or programme. The attributes differ only in the variations or levels allocated to the attributes. Then, participants are asked to state their preferred choice of the two or more alternative scenarios provided in each choice set. In this study, DCE is used as a quantitative method for valuing different characteristics of OTPD for data use. Data use here was the content focus of the OPDP. In so doing, we used a DCE survey that has seven choice sets. Each set had two alternatives to the OPDP programmes. Programme A and B had similar characteristics in terms of interaction, learning material, learning strategy, duration, certificate, and cost.
The difference between the two programmes was on the levels describing each of the characteristics. For example, the characteristic of interaction had two levels: synchronous and asynchronous. Programme A employed asynchronous interaction while Programme B used synchronous interaction. In general, there were some different and similar levels of the attributes for each programme in each choice set. The seven choice sets, one by one, were shown to teachers and then we asked them to choose one out of two scenarios/programmes offered to them. In case teachers did not have any preference over the two programme alternatives, they could choose an opt-out option. More details of this DCE procedure are given in the following sub-sections.
3.2. Study sample
The target population of this study consisted of about 600 university English teachers from 58 language centres of state Islamic higher education institutions in Indonesia. We selected the English teaching profession as massive professional development is important given that English is a language skill necessary in many fields of study and the workforce, and English communication skills can help students create lifelong learning opportunities to achieve decades of worthwhile employability in response to the synergistic digital economy. Providing professional development online can help to achieve the necessary scale. In addition, we chose Islamic higher education institutions because they are nationally distributed to different provinces and representative of various ethnicities as well as socioeconomic backgrounds.
The institutions also offer a variety of secular and religious study programmes. Although most English teachers at the institutions' language centres hold at least a master's degree or higher, there are still part-time teachers with a bachelor's degree. Finally, Indonesia became the study location for it is a big country that consists of five big islands and 16,771 small islands, inhabited by 272.229.372 people. With this geographic condition, mobility between one island to another takes time and is even more costly. Thus, there is a need for providing OTPD that is timeefficient and cost-effective, for instance. In this study, we used a total population sampling technique to recruit English teachers from the 58 language centres.
To do this, we first contacted the teachers to introduce our study and provided them with general information about the study. We then sent the study invitation and the survey link via email or WhatsApp message to the full target sample population to ascertain that all English teachers had an equal chance of receiving the survey invitation. This process reduced sampling selection bias in this study. We sent reminder messages two times to the teacher respondents after they got the first invitation. After this, the teachers were conveniently selected based on their willingness to participate in our study as indicated in their informed concern.
3.3. Development of the DCE questionnaire
In this sub-section, we describe the construction and validation process of our DCE questionnaire. The process starts with the identification of the potential attributes and levels to be used in the DCE questionnaire. It is then followed by the design of the DCE questionnaire and its pilot testing.
3.3.1. Identification of attributes and levels
The identification of suitable attributes and levels is crucial for the validity of the DCE results. Potential attributes and their levels were selected based on reviewing the relevant literature and by conducting focus group discussions (FGDs). In the first step, we synthesised the relevant literature to identify potential attributes that affect teachers’ willingness and attitudes when they make decisions on participating in an OTPD programme. As discussed in the previous section, four potentially important attributes emerged from the literature for inclusion in our DCE: the type (mode/format) of online interaction between a trainer and teacher-learners (in this case university English teachers) or between a teacher-learner and other teacher-learners (Bernard et al., 2009; Denoyelles & Raider-Roth, 2016; Herrington et al., 2001; Lin, 2015; Rogers, 2001; Yeo, 2021; Yoon et al., 2020), the type of learning materials representation (Bragg et al., 2021; CADRE, 2017; Darling-Hammond et al., 2017; Herrington et al., 2001; Lin, 2015; Sadeghi & Navaie, 2021), the learning strategy/ approach (Banegas & Busleiman, 2014; Darling-Hammond et al., 2017; McElearney et al., 2019), and recognition (Donavant, 2009; Taylor, 2011).
In the second step, we conducted FGDs to appraise the four attributes derived from the literature and to identify other potential attributes and levels that represent the target population's perspective and experience, as suggested by Hall, Viney, Haas, and Louviere (2004). We employed FGD because they have become an accepted method used in the initial stages of valuation research to generate potential DCE attributes (Kaplowitz & Hoehn, 2001). For this purpose, we organised two FGDs, each 2 h long. The first FGD was conducted with three English language teaching (ELT) professional development experts and professionals to appraise the relevance of the four attributes from the literature to our study context.
Two of them were researchers of teacher professional development in Indonesia and one was a professional development organiser who had more than 15 years of experience. The second FGD was conducted with ten prospective participants of interest to ensure the most important attributes and their proper levels for the decision-making process to participate in an OTPD programme. These participants were university English teachers from four different universities in Indonesia. They had an average of 10 years of teaching experience and had participated in at least two OTPD programmes. During the FGDs, experts and prospective participants were invited to appraise potential attributes from the literature, to ensure their relevance or consistency with current practice and then identify other potential attributes and levels in the Indonesian context, that the existing literature has not addressed.
New insights were obtained from the FGDs. An attribute, duration of OTPD, was proposed. This was considered important as the length of time spent for professional development could be converted into certain credit points for university teachers in Indonesian higher education. Moreover, it was argued that each attribute should have two levels to avoid complexities. For the attributes of interaction, learning materials and learning strategy, a blended model was proposed; so, the levels of the three attributes included the word “mostly or primarily”, for example, “mostly asynchronous interaction vs mostly synchronous interaction”. This also applied to the M.F. Ansyari, W. Groot and attributes of learning materials/resources and learning strategy.
Finally, since we were interested in identifying teachers' willingness to substitute, we also included “cost” as an attribute. In particular, the inclusion of a cost attribute allowed us to determine whether or not teachers were willing to substitute for the different levels of each attribute. The results from FGDs suggested two levels, namely free of charge (IDR 0; USD 0) and IDR 300,000 (USD 21.07). The proposed cost was considered a realistic fee based on the prospective teacher participants' input. This cost also represented about 2e4 per cent of the average certified teachers’ salaries in Indonesian higher education. In sum, based on the literature review and FGD results, the final set consisted of six attributes with two levels as presented in Table 1.
3.3.2. Construction of DCE questionnaire
To construct a DCE questionnaire, we employed a full-factorial design using all the six attributes where each attribute had two levels. This process produced 64 (62) profiles for selection. For a manageable survey, we developed a D-efficient design to create a minimum number of choice sets. Using the orthogonal fractional factorial design method (Dey, 1985) resulted in eight profiles with two alternatives each. The eight profiles were shown to be balanced with minimum overlap. We selected one out of eight profiles to be a baseline profile while the others were used as alternatives. Thus, seven choice sets were generated and each choice set contained the baseline and one alternative profile. Considering that teachers in their real lives can also choose not to participate in OTPD or to be indifferent towards an option, an opt-out option was included.
The opt-out option indicated that the teachers were indifferent between the two options. Every choice task had the following question: “Imagine that your employer advises that you participate in an online professional development programme (OPDP) for the use of data to inform instructional decision making. In which OPDP programme would you prefer to participate, OPDP 1 or OPDP 2?” An example of the choice task is shown in Fig. 1 below. The final questionnaire consisted of five parts (see Appendix A). The first part was the DCE. The participants were presented with a series of imaginary scenarios that described different OTPD programmes, and they were asked to choose which one they would prefer more.
The scenarios differed in terms of specific characteristics of the OTDP, which allowed the researchers to explore university teachers' preferences on different aspects of the OTDP models. The second part asked the participants to select the attribute levels of an OTDP programme (e.g., online training, workshops or courses) based on their preferences. The third part asked the teacher participants questions about their attitudes and opinions regarding data for instructional decision making. The fourth part included questions about the teacher participants’ demographics (e.g., age, gender, educational background, etc.). The last part asked the participants about their opinions and experiences with part one of the survey (DCE).
3.3.3. Pilot testing of DCE questionnaire
Before the formal data collection, the draft of the DCE questionnaire was pilot tested with a convenience sample of 20 (10 fulltime and 10 part-time teachers) university English teachers from the study population to determine face validity and to find out the teachers’ understanding of the DCE questionnaire in terms of concepts and wording. The questionnaire was offered fully online. The teachers in the pilot test were asked to highlight every question or answering category that they did not understand or found difficult to grasp. They were also requested to provide suggestions for improvement. There were no changes in the attributes and levels based on the results of this pilot study. All participants could access the online survey easily and understand the questions as well as the concepts used in the questionnaire.
As part one (the DCE scenarios) and part three (attitudes toward data) of the questionnaire are essential parts, we checked the reliability of the two parts. The results showed Omega (u) ¼ 0.819 for the DCE scenarios and 0.879 for the attitudes toward data use, which indicated sufficiently high reliability.
3.4. Data collection procedure
We used a survey to collect data. The survey was administered online for two months from September to October 2021. Initially, we got 356 responses (about 59% response rate) of which 26 respondents did not complete the survey or disagreed to participate in the survey, and these 26 data were taken out from the analyses, resulting in a 55% final response rate (330 responses). Table 2 describes the characteristics of all 330 respondents participating in this study. The respondent age ranged from 25 to 59 years with a mean age of 38.5 years. More than half were female Table 1 DCE attributes and levels. Attributes Levels Description Interaction Primarily synchronous (both the instructor/s and participants are present at the same time) Primarily asynchronous (both the instructor/s and learners are not present at the same time) It refers to modes of online learning interaction or communication between the instructor/s and participants or among participants themselves.
Learning materials/ resources Refers to Mostly digital reading materials (e.g., e-books, pdf articles) Mostly audio-visual materials (E.g., video clips) It means the types of learning materials or resources that participants get. Learning strategy Primarily individual (participants learn and do the assignment themselves without other peers) Primarily collaborative (participants learn and do the assignment with peers, such as through group work and discussions) It indicates the way how participants engage in learning during OTPD. Duration of course Short period: a course lasting for hours or days (e.g., a 6-h training, a two-day training) Long period: a course lasting for weeks or months (e.g., a threeweek course, a two-month course) This shows the length of the OTPD in which participants participate.
Certificate Yes (Participants will get a certificate upon completion) No (Participants will not get a certificate upon completion) This describes whether or not a certificate is provided upon completion of OTPD as proof of participation. Price IDR 0 (USD 0): Participants do not have to pay for the OTPD. IDR 300,000 (USD 20.07): Participants have to pay this amount for the OTPD. This is an out-of-pocket expense for an OTPD programme and determines whether or not participants may have to pay the program costs out-of-pocket. M.F. Ansyari, W. Groot and (52.7%). The average amount of teaching experience was 12.7 years (SD ¼ 6.4), ranging between 2 and 30 years.
The majority were fulltime English teachers (65.5%), held a master's degree (69.1%), and had online professional development experience at least one time (89.7%) during their career in the profession. These respondents were representative of 52 out of the 58 state Islamic higher education institutions from different provinces in Indonesia. Only six language centres did not participate in the study. This attrition may be due to several reasons, such as too many online activities because of COVID-19 measures, teachers' interest in the survey topic, etc. Fig. 1. Sample DCE choice task. Table 2 Characteristics of all respondents. Demographics No. (n ¼ 330) % Age
1. Mean (SD)
2. Min-Max
38.57 (7.53) years 25e59 years e Sex
1. Male
2. Female
47.3 52.7 Teaching Experience
1. Mean (SD)
2. Min-Max
12.70 (6.39) years 2e30 years e Employment status
1. Full-time employed
2. Part-time employed
65.5 34.5 Educational Level
1. Bachelor
2. Master
3. Doctorate
10.9 69.1 20.0 Frequency of online professional development experiences 1.0 times 2.1e2 times 3.3e6 times 4.7 or more times 10.3 37.6 30.3 21.8 M.F. Ansyari, W. Groot and
3.5. Econometric analysis
Descriptive statistics were used to analyse the data about teachers’ attitudes toward data and preferences for attribute levels while for the DCE data the statistical analysis relied on the random utility theory (McFadden, 1974). The model in this study assumes that each respondent faces a choice amongst J alternatives repeated under S scenarios or choice situations. The model assumes that the utility (U) associated with a particular OTPD programme is made up of the deterministic component (V) and the random component (ε). The utility (U) to individual i associated with OTPD j in choice task t can be specified as: Ui ¼ Vi þ εi ¼ b1 * syn þ b2 * digit þ b3 * ind þ b4 * long þ b5 * certn þ b6 * cost þ εi The DCE data were binary: 1 indicates the choice of the English teachers while 0 indicates that the English teachers did not choose the option.
All attributes were dummy coded but the attribute of “cost” was specified as a continuous variable for calculating the marginal rates of substitution (MRS). The responses to the DCE questions became the dependent variable coded as 0 if the reference (baseline) profile was chosen and coded as 1 if the alternative profile was selected. We coded all the attributes into 0 when an opt-out option was chosen because in this study an opt-out option also referred to the respondents’ current situation or reference point. The differences in attribute levels as well as the interaction between these differences and the selective demographic characteristics served as the independent variables. The coding for attributes and levels is presented in Table 3.
We analysed the DCE data through a mixed logit model as it allowed us to calculate the MRS for different levels and attributes, and provided better insights into the identification of the English teachers' trade-off between different attributes of the OTPD for data use. We employed a mixed logit model to look at the relative importance of OTPD attributes and MRS. In this study, ‘preference’ is used as a predictor variable while ‘willingness to participate’ as an outcome variable. In addition, we examined how factors related to OTPD characteristics affected the English teachers' preferences for the uptake of the OTPD for different subgroups based on age, sex and frequency of OTPD experiences.
4. Results
4.1. Descriptive result
The English teachers' attitudes toward data are generally positive (96.4%). Only 12 teachers (3.6%) have negative attitudes towards data; for example, they perceive that data are not useful resources for their instructional decision making. However, the teachers who have positive attitudes (316 (95.8%)) admit the importance of learning how to use data to inform their instructional decision making, for instance. In addition, the teachers were also asked to choose the attribute levels that they preferred. The teachers' choices on the attribute levels are shown in Table 4. When each attribute and its levels were presented one by one to the teachers, the results show that teachers prefer the OTPD programme that provides a certificate (97%), followed by the programme that is organised in a short period (75.2%), that provides audio-visual materials or resources (73.9%), that uses synchronous interaction (69.1%) and that employs a collaborative learning strategy (67.9%), respectively.
Although more than half of the teachers (59.4%) are willing to pay for an OTPD programme, this cost attribute seems to be the least important. These results allow us to compare teachers' preferences for each attribute and teachers’ preferences when they have to make trade-offs.
4.2. Main findings
Table 5 shows the relative importance of the attributes and the marginal rates of substitution (MRS). All attributes, except for the “learning strategy”, are statistically significant. However, only the attribute “interaction mode” has a statistically significant positive effect while the other attributes have significantly negative coefficients. The results indicate that the most important attribute is synchronous interaction with a coefficient of 0.457 (p < 0.001). The likelihood that teachers choose to participate in the OTPD programme for data use increases with synchronous interaction but decreases with higher cost, when no certificate is provided, the duration is long, and the learning materials or resources are in the form of digital reading materials, respectively.
We find that the attribute “cost’ appears to have a major negative influence on the teachers’ willingness to participate in the OTPD programme. When comparing the descriptive (Table 4) and mixed logit (Table 5) findings on the attribute level preferences, the two results are consistent. Both types of results similarly indicate the English teachers' preference for synchronous interaction in the OTPD programme for data use. Synchronous interaction is chosen by 69.1% of the English teachers and is proven to have a significant positive effect on the English teachers’ participation in an OTPD programme. Correspondingly, the mixed logit results reveal that the attributes of cost (premium), certificate (no certificate provided), duration (long duration), and learning material (digital reading) have significant negative effects.
Overall, English teachers prefer an OTPD programme that provides a certificate upon completion, conducted in a short period, using audio-visual learning materials/ resources and without charging any cost, rather than the OTPD programme with the other attribute levels (e.g., providing no certificate, long duration, etc.). The overview of the preferred and unpreferred OTPD for data use is illustrated in Fig. 2. Furthermore, we estimate the marginal rates of substitution (MRS), and the results support the comparison of the teachers' preferences from the monetary perspective as presented in Fig. 3. The results show that the English teachers are willing to pay more Table 3 Attributes and levels coding.
Attributes Baseline profile Alternative (alt) profile Alt 1 Alt 2 Alt 3 Alt 4 Alt 5 Alt 6 Alt 7 Mode of online interaction (synchronous ¼ 0; asynchronous ¼ 1) 1 0 0 1 0 0 1 1 Mode of learning materials (digital reading materials ¼ 0; audiovisual materials ¼ 1) 1 1 0 1 0 1 0 0 Learning strategy (individual ¼ 0; collaborative 1) 1 0 1 0 0 1 0 1 Duration (short ¼ 0; long ¼ 1) 0 1 0 0 0 1 1 1 Certificate (yes ¼ 0; no ¼ 1) 0 0 1 1 0 1 1 0 Cost (Rp.0 ¼ 0; Rp. 300,000 ¼ 300,000) 300,000 300,000 300,000 0 0 0 300,000 0 Note: 300,000 Indonesian Rupiah (IDR) equals to 21.07 USD at the time of study (1 USD ¼ 14,234.47 IDR). M.F. Ansyari, W. Groot and for the interaction mode than for the other attributes.
They are willing to pay about 269,211.1 IDR (18.91 USD) to participate in a synchronous OTPD programme for data use if compared with the asynchronous OTPD programme. On the other hand, they have lower preferences for the attributes of certificate, duration and learning material mode. The highest decrease in the English teachers’ willingness to pay is found in the OTPD programme without a certificate upon completion (401,835.4 IDR/-28.22 USD), followed by the OTPD programme with a long duration (227,854.3 IDR/-16.00 USD) and that with mostly digital learning Table 4 Teachers’ choices between attribute levels. Attribute Level Frequency Per cent Interaction mode/format Which type of interaction do you prefer when you participate in online training, workshop or course?
Synchronous 228 69.1 Asynchronous 102 30.0 Learning material mode/format Which type of learning materials do you prefer when you participate in online training, workshop or course? Digital reading 86 26.1 Audio-visual 244 73.9 Learning strategy Which learning strategy do you prefer when you participate in online training, workshop or course? Individual 106 32.1 Collaborative 224 67.9 Duration Which duration do you prefer when you participate in online training, workshop or course? Short 248 75.2 Long 82 24.8 Certificate Do you want to get a certificate after participating in online training, workshop or course? Yes, I want a certificate 320 97.0 No, I don't want a certificate 10 3.0 Willingness to pay (WTP) Are you willing to pay for online training, workshop or course?
Yes, I want to pay 196 59.4 No, I don't want to pay 134 40.6 N ¼ 330 Table 5 Results of the main effects model and MRS. Coefficient (SE) p-value 95% CI MRS (IDR) p-value 95% CI Lower Upper Lower Upper Interaction mode Primarily synchronous .457 (.091)*** 0.000 .277 .637 269,211.1 0.000 164,609.3 373,812.9 Primarily asynchronous Ref Learning material mode Mostly digital materials .329 (.064)*** 0.000 .455 .203 193,942.5 0.000 278,888.2 108,996.8 Mostly audio-visual materials Ref Learning strategy Mostly individual .176 (.100) 0.080 .373 .020 103,739.3 0.055 209,895.1 2416.522 Mostly collaborative Ref Duration Short Ref Long .387 (.066)*** 0.000 .517 .256 227,854.3 0.000 324,172.2 131,536.5 Certificate Yes Ref No .683 (.087)*** 0.000 .853 .512 401,835.4 0.000 517,180.9 286,489.9 Cost 1.700 (2.800)*** 0.000 2.250 1.150 Constant .226 (.073) 0.002 .370 .083 Number of respondents 330 Log likelihood 2144.0479 Wald x2 109.05 0.0000 Note: **p < 0.05; ***p < 001.
Fig. 2. An overview of the distinction between preferred and unpreferred OTPD for data use. M.F. Ansyari, W. Groot and materials (193,942.5 IDR/-13.62 USD). It seems that the English teachers are willing to pay for the OTPD programme that provides a certificate, is conducted in a short duration and provides mostly audio-visual learning materials. Additionally, since the majority of the English teachers (97%) seemingly prefer certificates as shown in Table 4 and this might create a bias in our results; thus, we then run another mixed logit to check the robustness of our model by excluding the attribute of the certificate. Overall, as shown in Appendix B, the results consistently show that the attributes of cost, duration, and learning material mode respectively have negative significant effects while the attribute of learning strategy also remains insignificant.
However, only the interaction mode attribute which previously has a positive significant effect becomes insignificant.
4.3. Heterogeneity
In this section, we examine how the impact of certain attributes differs between teacher respondents aged 40 years or younger and 41 years or older (separated based on median), male and female teacher respondents, as well as teacher respondents who have less experience with OTPD and those who have more experience with OTPD. Table 6 and Fig. 4 present the results of the selective subgroup analyses. Overall, the effect of the interaction mode as captured by synchronous interaction is more or less the same in all subgroups. The interaction model has a statistically significant positive effect in all subgroups (p < 0.05). The inclusion of this attribute increases the English teachers' likelihood to participate.
However, the effects of the learning material mode, duration, certificate and cost are statistically significant negative (p < 0.05). This indicates that teachers in these subgroups perceive those attributes as less preferred components to be included in the OTPD programme, so they decrease the likelihood that they participate in the OTPD programme. This finding is similar in all samples. Although the attribute ‘learning strategy’ is not significant for both male and female English teachers, it is statistically negative significant for English teachers aged 41 years or older and those who have less experience with OTPD. Older teachers and teachers with less OTPD experience consider the inclusion of ‘mostly individual learning’ attribute for not participating in the OTPD programme.
Although the cost has the highest negative effect in all subgroups, it is more pronounced for English teachers aged 41 years or older than those aged 40 years or younger. We assume that these teachers are more sensitive to cost as they have to use money more wisely for their professional development and living expenses. Furthermore, we provide an estimation of the subgroup's willingness to pay as shown in Fig. 4. In terms of interaction mode, 41 years or older teachers, female teachers and less OTPD experienced teachers are willing to pay more for the interaction mode (primarily synchronous). The negative effect of the certificate is stronger for 40 years or younger teachers, female teachers, and more OTPD experienced teachers.
In general, all teachers in the subgroup are willing to pay for interaction mode but not for the other attributes.
5. Discussion and conclusion
This study analyses how interaction mode/format, learning material mode, learning strategy, duration, certificate, and cost affect English teachers’ decision to participate in an OTPD programme for instructional data use. Drawing on a nationally representative sample of English teachers at the language centres of state Islamic higher education in Indonesia, the results show that the uptake of the OTPD programme for data use is positively influenced by using interaction mode. The English teachers are willing to participate in the OTPD programme that employs mostly synchronous online interaction but are reluctant to participate in the OTPD programme that is not free of charge, that does not provide a certificate upon completion, is longer in duration, and that uses mostly digital reading materials (e.g., pdf readings), respectively.
Moreover, the subgroup analysis indicates that teachers aged 41 years or older and those who have less OTPD experience are not willing to participate in the OTPD programme that employs mostly an individual learning strategy. We also find in the subgroup analysis that the negative effect of the OTPD programme without certificates is stronger for 40 years or younger teachers, female teachers as well as teachers with more OTPD experience. The findings are promising and provide new insights into the OTPD literature. Our study findings based on both the main effect model and subgroup analyses show a significantly positive influence on the teachers' willingness to participate in a primarily synchronous OTPD programme.
Yet, this result is not surprising, as a previous study also found that teachers in South Australia prefer an OTPD programme that includes synchronous interactions (Lin, 2015). Because our study was conducted when the outbreak of COVID-19 was still ongoing all over the world, including Indonesia, we assume that teacher preferences for synchronous interaction mode may be caused by the effect of social distancing where people mostly stayed at home and were away from others as much as possible to help prevent the spread of COVID-19. This condition presumably made the teachers feel isolated, so they looked for an OTPD programme using real-time communication to compensate for their limited in-person contact with others.
Based on this finding, policymakers or OTPD programme providers should take into account the characteristics of OTPD contents that can be synchronously delivered for effective outcomes. Despite the potential effect of an asynchronous intervention (Reeves & Chiang, 2019), the question of whether or not synchronous intervention influences teachers’ instructional data use is yet to be investigated. Our findings also reveal that cost, not obtaining a certificate, long duration and digital learning material mode reduce teachers' willingness to participate in the OTPD programme for data use. A negative effect of cost could be explained by the descriptive results (see Table 4) showing that almost 60 per cent of the teachers are willing to pay for the OTPD programme.
This seems to indicate that teachers might think that paying for an OTPD programme should not be their responsibility but their employers'. In Lin's study (2015), cost becomes a promoting or hindering factor in the uptake of a professional development programme but teachers do not care much about the cost being paid by their schools. Another study also indicated that one reason why non-permanent English teachers at Fig. 3. MRS for the full sample. M.F. Ansyari, W. Groot and an Indonesian Islamic higher education remain in the teaching profession is that they are triggered by professional development opportunities provided by the university without any cost (Ansyari, Coelho, et al., 2020).
As the English teachers prefer a certificate after an OTPD programme participation, it is no doubt that a certificate is important for university teachers, especially for full-time permanent teachers. This finding is supportive of previous studies, indicating that teachers participate in an OTPD programme as they want to obtain certificates for their career requirements or needs (Banegas & Busleiman, 2014; Donavant, 2009). However, it is not clear if teacher preference for a certificate is also stronger for self-chosen OTPD. Therefore, more research is necessary to examine the effect of certificates on such a type of OTPD. Although studies highlight the importance of a longer duration for in-person instructional data use interventions (Ansyari, Groot, & De Witte, 2020), the teachers prefer a short OTPD duration for instructional data use.
As reported by Reeves and Chiang (2018; 2019), an online data use intervention with a short duration can also have effects on teachers' beliefs, self-efficacy, and practices. However, this finding needs to be interpreted with caution. In other words, the tension between this teachers’ preference for a short duration and theory should be tackled appropriately. Designing a series of short OTPD programmes based on the topics or the steps required for instructional data use might be an option to balance the tension; so, teachers have a sense of belonging to the setup of the programme that accommodates their voices (Merchie, Tuytens, Devos, & Vanderlinde, 2018). The teachers' unwillingness to have mostly digital reading materials is understandable as such type of materials does not fully accommodate their different individual learning preferences (e.g., verbalisers versus visualisers).
The teachers seem to prefer a representation of combined multimodal learning materials or the integration of multiple modes for meaning-making, though the teachers' preferences do not automatically represent their multimodal literacy. From a communication perspective, according to Kress (2009), it is difficult to understand the meaning of a text without taking into account all modes that contribute to it. From a learning material representation lens, the teachers’ preference also supports multimedia learning theory which, according to Mayer (2009), people learn better from (spoken or written) words and (static or dynamic) pictures than from words alone” (p. 223). It is therefore suggested that learning materials be designed based on the principles of multimedia learning to maximise learning effectiveness (Mayer, 2009).
Overall, the findings of this study shed light on in-service TPD in general and TPD for data use in particular. The majority of research on TPD for data use has focused on in-person delivery (e.g., Ebbeler, Poortman, Schildkamp, & Pieters, 2017; Kippers, Poortman, Schildkamp, & Visscher, 2018; McNaughton, Lai, & Hsiao, 2012) but only a few studies have looked into how OTPD is delivered to support teachers' data use (Jimenez, Mims, & Baker, 2016; Reeves & Chiang, 2018, 2019). Given this knowledge gap, our findings contribute to the field by providing insights into teachers' preferences for OTPD for data use that have not been addressed in earlier studies. We argue that the preferences revealed in our study could be usefully incorporated into the design of the OTPD to meet the needs of teachers as adult learners.
Based on adult learning theory, learning is better when teachers can choose learning opportunities based on their interests and needs (Knowles, Holton, & Swanson, 2015; Trotter, 2006). In the context of OTPD, according to Quinn et al. (2019), the effectiveness and sustainability of OTPD depend not only on the technicalities of online course structures and tools but also on contextual sensitivity and needs. In other words, effective and sustainable OTPD is determined by the extent to which it meets teachers' own needs within their own contexts. In this regard, studying teachers' preferences for the OTPD is crucial for understanding teachers’ interests and needs, and incorporating them into the OTPD design may enhance its effectiveness and sustainability of the OTPD.
Table 6 Results of subgroup analyses. Age 40 or younger (n ¼ 208) Age 41 or older (n ¼ 122) Coefficient (SE) 95% CI Coefficient (SE) 95% CI Synchronous interaction 0.342 (0.115)** (0.117; 0.568) 0.646 (0.152)*** (0.346; 0.946) Digital materials 0.348 (0.082)*** (-0.510; 0.185) 0.301 (0.103)** (-0.503; 0.099) Individual 0.056 (0.123) (-0.299; 0.185) 0.368 (0.172)** (-0.706; 0.030) Long duration 0.320 (0.084)*** (-0.485; 0.155) 0.500 (0.109) *** (-0.715; 0.286) Certificate not provided 0.705 (0.100)*** (-0.901; 0.508) 0.642 (0.164) *** (-0.964; 0.320) Cost 1.470 (3.370)*** (-2.130; 8.060) 2.080 (4.880) *** (-3.040; 1.130) Male (n ¼ 156) Female (n ¼ 174) Coefficient (SE) 95% CI Coefficient (SE) 95% CI Synchronous interaction 0.424 (0.132)*** (0.164; 0.683) 0.489 (0.128)*** (0.237; 0.741) Digital materials 0.226 (0.086)** (-0.395; 0.057) 0.431 (0.095)*** (-0.619; 0.243) Individual 0.151 (0.171) (-0.488; 0.184) 0.195 (0.123) (-0.438; 0.046) Long duration 0.434 (0.101)*** (-0.633; 0.235) 0.348 (0.087)*** (-0.521; 0.176) Certificate not provided 0.531 (0.138)*** (-0.803; 0.259) 0.834 (0.118)*** (-1.066; 0.601) Cost 1.650 (5.150)*** (-2.660; 6.390) 1.750 (3.360)*** (-2.410; 1.100) Less experience with oTPD (n ¼ 158) More experience with oTPD (n ¼ 172) Coefficient 95% CI Coefficient 95% CI Synchronous interaction 0.446*** (0.177; 0.715) 0.479*** (0.233; 0.724) Digital materials 0.327*** (-0.507; 0.147) 0.334*** (-0.512; 0.156) Individual 0.284** (-0.559; 0.009) 0.101 (-0.389; 0.186) Long duration 0.487*** (-0.686; 0.288) 0.310*** (-0.483; 0.136) Certificate not provided 0.634*** (-0.899; 0.370) 0.741*** (-0.974; 0.509) Cost 1.620*** (-2.290; 9.470) 1.850*** (-2.740; 9.620) Note: **p < 0.05; ***p < 001.
M.F. Ansyari, W. Groot and Methodologically, our study also gives insights into how DCE addresses the shortcoming of traditional ranking methods. DCE can be used to uncover the trade-offs between the different attributes of the OTPD and provide quantifiable data on the relative importance of the different attributes. Similarly, by studying the tradeoffs between the different attributes of the OTPD, DCE helps us to identify the multi-faceted decisions that teachers face in their lives. DCE also allows for identifying the relative importance of the different OTPD characteristics and estimating the effect of incorporating a particular characteristic on teachers’ participation rates or (un)willingness to pay for the attributes.
So, the quantifiable results can be easily acted upon by policymakers, school leaders or providers of the OTPD to design a more tailored OTPD such that teachers can be better supported.
6. Implications and limitations of the study
This study gives new insights into several key attributes of an OTPD programme for data use based on the stated preferences of English teachers in Indonesian Islamic higher education. These findings have implications for the design of OTPD to support teachers' data use for instructional improvement purposes. A practical implication of the findings is that language centres and providers of OTPD may consider the promoting and hindering characteristics (attributes) when designing an OTPD for data use, particularly in the Indonesian context or other similar contexts. In other words, the positive characteristic of ‘mostly synchronous interaction’, for instance, can be incorporated while the negative characteristics should be minimized in an OTPD programme for data use so that the OTPD fits teachers' preferences or needs.
In doing so, the OTPD is expected to contribute to teacher willingness and active participation. Considering the negative effect of cost, it might be considered to provide financial support to teachers for continuing professional development that can practically help them improve their English instructional practices, such as data use for instructional improvement. In Indonesian higher education, English teachers still have very limited opportunities to access this kind of financial support. Additionally, given the potential benefits of OTPD, the language centres and the association of the language centres could include OTPD as a strategic tool for developing teacher professionalism. The association may develop an OTPD programme for data use that can be accessed nationally by teachers from different language centres.
Through this programme, the current organisational-level association could be expanded into an individual-level association. A theoretical implication is that this study provides insights into the six characteristics of OTPD for data use in the context of English language teaching in state Islamic higher education, although the findings may not be generalised to other subjects or contexts. Thus, future research may replicate this study in other contexts, such as non-faith-based higher education institutions or primary and secondary schools in Indonesia. Using an internationally representative sample will be useful for generalising the findings. Researchers might also focus on exploring why teachers prefer certain characteristics more than others, such qualitative findings will enrich the current study findings in terms of the nature of the characteristics.
Studies may conduct experiments on the effectiveness of the OTPD characteristics by answering the question: does a synchronous interaction influence teacher learning outcomes? Lastly, future research may use machine learning models to observe teacher preferences. Using such models will give insights into the relative importance of different OTPD attributes that can be presented in a decision tree, and the tree can visually illustrate the most to least preferred attributes. Despite the findings and implications, some limitations of this study have to be acknowledged. First, relative to the traditional models of ranking studies such as a qualitative method (Barnes et al., 2018; Lin, 2015; Sadeghi & Navaie, 2021), DCE usually puts a cognitive burden on respondents.
To minimise such burden in this study, we use only six attributes and two levels for each attribute so that we have only seven final choice sets. Second, this study does not employ a randomised sampling technique; therefore, the results might be subject to sample selection bias. However, the bias is reduced by sending out the study survey to all teachers at the language centres. Although not all teachers agreed to participate, Fig. 4. Willingness to substitute estimation for subgroup population. M.F. Ansyari, W. Groot and the teacher participants came from 52 out of 58 language centres. Therefore, we argue that our sample is representative enough to provide reliable evidence on the characteristics of OTPD for data use.
Finally, our study asks teachers to make a choice between two hypothetical profiles of OTPD for data use while in the real-word teachers might be presented with more alternatives. In other words, the profiles of the OTPD provided in the study might differ from the OTPD in a real-world situation. Therefore, the findings are limited to the models of OTPD offered in the study. Teachers also may not consider their personal and professional conditions when making decisions on the OTPD scenarios offered in this study. In a real-life situation, teachers may have to take into account their family and institutional support given that OTPD requires time and even financial matters. Despite this limitation, our findings provide evidence of teachers’ complex decision-making for the OTPD for data use by trading off the different attributes.
Ethical approval Ethical clearance for this study was obtained from the State Islamic University of Sultan Syarif Kasim Riau Indonesia No. KE/KEPFPP/September 09, 2021. Conflicts of interest All the authors declare no conflict of interest.
Declaration of competing interest
All the authors declare no conflict of interest.
Acknowledgements
This study was sponsored by (overseas) 5000 Doctoral Programme, Ministry of Religious Affairs, Indonesia. The views and opinions expressed herein are those of the authors and do not necessarily reflect those of the sponsor.
References
- 1Ansyari (2020). English instructors' motivation levels in Islamic higher education. Journal of Applied Research in Higher Education, 12, 649.
- 2Ansyari (2020). Tracking the process of data use professional development interventions for instructional improvement: A systematic literature review. Educational Research Review, 31.
- 3Banegas (2014). Motivating factors in online language teacher education in southern Argentina. Computers & Education, 76, 131. https://doi.org/10.1016/j.compedu.2014.03.014
- 4Barnes (2018). Training needs and online learning preferences of early childhood professionals. Journal of Early Childhood Teacher Education, 39, 114. https://doi.org/10.1080/10901027.2017.1347909
- 5Bernard (2009). A meta-analysis of three types of interaction treatments in distance education. Review of Educational Research, 79, 1243. https://doi.org/10.3102/0034654309333844
- 6Carlson (2011). A multistate district-level cluster randomized trial of the impact of data-driven reform on reading and mathematics achievement. Educational Evaluation and Policy Analysis, 33, 378. https://doi.org/10.3102/0162373711412765
- 7Clark (1983). Reconsidering research on learning from media. Review of Educational Research, 53, 445. https://doi.org/10.3102/00346543053004445
- 8Cleland (2018). What can discrete choice experiments do for you?. Medical Education, 52, 1113. https://doi.org/10.1111/medu.13657
- 9(2017).
- 10Compen (2019). The role of teacher professional development in financial literacy education: A systematic literature review. Educational Research Review, 26, 16. https://doi.org/10.1016/j.edurev.2018.12.001
- 11Compen (2021). The impact of teacher engagement in an interactive webinar series on the effectiveness of financial literacy education. British Journal of Educational Technology, 52, 411. https://doi.org/10.1111/bjet.13013
- 12Darling-Hammond (2017).
- 13De Beckker (2021). The effect of financial education on students' consumer choices: Evidence from a randomized experiment. Journal of Economic Behavior & Organization, 188, 962. https://doi.org/10.1016/j.jebo.2021.06.022
- 14Dede (2009). A research agenda for online teacher professional development. Journal of Teacher Education, 60, 8. https://doi.org/10.1177/0022487108327554
- 15Denoyelles (2016). Being an 'agent provocateur': Utilising online spaces for teacher professional development in virtual simulation games. Technology, Pedagogy and Education, 25, 337. https://doi.org/10.1080/1475939X.2015.1049652
- 16Dey (1985).
- 17Donavant (2009). The new, modern practice of adult education: Online instruction in a continuing professional education setting. Adult Education Quarterly, 59, 227. https://doi.org/10.1177/0741713609331546
- 18Ebbeler (2017). The effects of a data use intervention on educators' satisfaction and data literacy. Educational Assessment, Evaluation and Accountability, 29, 83. https://doi.org/10.1007/s11092-016-9251-z
- 19Fishman (2013). Comparing the impact of online and face-to-face professional development in the context of curriculum implementation. Journal of Teacher Education, 64, 426. https://doi.org/10.1177/0022487113494413
- 20van Geel (2017). Changes in educators' data literacy during a data-based decision making intervention. Teaching and Teacher Education, 64, 187. https://doi.org/10.1016/j.tate.2017.02.015
- 21Hall (2004). Using stated preference discrete choice modelling to evaluate health care programs. Journal of Business Research, 57, 1026. https://doi.org/10.1016/S0148-2963(02)00352-1
- 22Hensher (2005).
- 23Jimenez (2016). The effects of an online data-based decisions professional development for in-service teachers of students with significant disability. Rural Special Education Quarterly, 35, 30. https://doi.org/10.1177/875687051603500305
- 24Kaplowitz (2001). Do focus groups and individual interviews reveal the same information for natural resource valuation?. Ecological Economics, 36, 237. https://doi.org/10.1016/S0921-8009(00)00226-3
- 25Kim (2011). Strategies for successfully completing online professional development. International Journal of Knowledge Content Development & Technology, 1, 43. https://doi.org/10.5865/IJKCT.2011.1.2.043
- 26Kippers (2018). Data literacy: What do educators learn and struggle with during a data use intervention?. Studies In Educational Evaluation, 56, 21. https://doi.org/10.1016/j.stueduc.2017.11.001
- 27Knowles (2015).
- 28Kozma (1994). Will media influence learning? Reframing the debate. Educational Technology Research & Development, 42, 7. https://doi.org/10.1007/BF02299087
- 29Kress (2009).
- 30Lai (2016). The impact of data use professional development on student achievement. Teaching and Teacher Education, 60, 434. https://doi.org/10.1016/j.tate.2016.07.005
- 31Lai (2009). Sustaining continued acceleration in reading comprehension achievement following an intervention. Educational Assessment, Evaluation and Accountability, 21, 81. https://doi.org/10.1007/s11092-009-9071-5
- 32Landry (2009). Effectiveness of comprehensive professional development for teachers of at-risk preschoolers. Journal of Educational Psychology, 101, 448. https://doi.org/10.1037/a0013842
- 33Lin (2015). In-service professional development in an online environment: What are South Australian English as an additional language or dialect teachers' views?. Professional Development in Education, 41, 527. https://doi.org/10.1080/19415257.2014.902860
- 34Louviere (2000).
- 35Mandinach (2015). How can schools of education help to build educators' capacity to use data: A systemic view of the issue. Teachers College Record, 117, 1.
- 36Mandinach (2016). What does it mean for teachers to be data literate: Laying out the skills, knowledge, and dispositions. Teaching and Teacher Education, 60, 366. https://doi.org/10.1016/j.tate.2016.07.011
- 37Mandinach (2021). Misconceptions about data-based decision making in education: An exploration of the literature. Studies In Educational Evaluation, 69.
- 38Mayer (2009).
- 39McElearney (2019). Identifying teacher needs and preferences in accessing professional learning and support. Professional Development in Education, 45, 433. https://doi.org/10.1080/19415257.2018.1557241
- 40McFadden (1974). Conditional logit analysis of qualitative choice behaviour.
- 41McNaughton (2012). Testing the effectiveness of an intervention model based on data use: A replication series across clusters of schools. School Effectiveness and School Improvement, 23, 203. https://doi.org/10.1080/09243453.2011.652126
- 42Merchie (2016). Evaluating teachers' professional development initiatives: Towards an extended evaluative framework. Research Papers in Education, 33, 143. https://doi.org/10.1080/02671522.2016.1271003
- 43Nunan (2012). Foreword.
- 44Quinn (2019). Developing, situating and evaluating effective online professional learning and development: A review of some theoretical and policy frameworks. Australian Educational Researcher, 46, 405. https://doi.org/10.1007/s13384-018-00297-w
- 45Reeves (2017). Pre-service teachers' data use opportunities during student teaching. Teaching and Teacher Education, 63, 263. https://doi.org/10.1016/j.tate.2017.01.003
- 46Reeves (2011). Predictors of teacher satisfaction with online professional development: Evidence from the USA’s e-learning for educators initiative. Professional Development in Education, 37, 591. https://doi.org/10.1080/19415257.2011.553824
- 47Reeves (2018). Online interventions to promote teacher data-driven decision making: Optimizing design to maximize impact. Studies In Educational Evaluation, 59, 256. https://doi.org/10.1016/j.stueduc.2018.09.006
- 48Reeves (2019). Effects of an asynchronous online data literacy intervention on pre-service and in-service educators' beliefs, self-efficacy, and practices. Computers & Education, 136, 13. https://doi.org/10.1016/j.compedu.2019.03.004
- 49Reeves (2015). A classroom data literacy intervention for pre-service teachers. Teaching and Teacher Education, 50, 90. https://doi.org/10.1016/j.tate.2015.05.007
- 50Rogers (2001). Traditions to transformations: The forced evolution of higher education. AACE Review, 9, 47.
- 51Sadeghi (2021). Iranian EFL teachers' experiences with online professional development: Perceptions and preferences. Iranian Journal of Language Teaching Research, 9, 9.
- 52Schildkamp (2017). Factors promoting and hindering data-based decision making in schools. School Effectiveness and School Improvement, 28, 242. https://doi.org/10.1080/09243453.2016.1256901
- 53Schouten (2017). On meeting students where they are: Teacher judgment and the use of data in higher education. Theory and Research in Education, 15, 321. https://doi.org/10.1177/1477878517734452
- 54Supovitz (2017). The linking study: An experiment to strengthen teachers' engagement with data on teaching and learning. American Journal of Education, 124, 161. https://doi.org/10.1086/695610
- 55Taylor (2011).
- 56Tømte (2020). Online or offline – does it matter? A study of in-service teachers' perceptions of learning outcomes in Norway. Nordic Journal of Digital Literacy, 15, 259. https://doi.org/10.18261/issn.1891-943x-2020-04-04
- 57Trotter (2006). Adult learning theories: Impacting professional development programs. The Delta Kappa Gamma Bulletin, 72, 8.
- 58Yeo (2021). Experiencing theory first-hand was delightful and informative”: Bridging the theory-practice gap in online language assessment training. Iranian Journal of Language Teaching Research, 9, 93.
- 59Yoon (2020). Comparative study of high-quality professional development for high school Biology in a face-to-face versus online delivery mode. Educational Technology & Society, 23, 68.
There are no recommended articles yet.
Only verified metrics entered by the editorial office are shown. Values marked “Not available” have not been measured yet.
- Views
- Not available
- Downloads
- Not available
- Citations
- 19
Source: Crossref cited-by count, 2026-11-17

