Abstract

Improving the educational process through evaluation requires adopting new concepts and approaches. In addition to the traditional methodologies applied in this field, such as observation or surveys, business methods are increasingly being implemented, including the Deming cycle or selected elements of project management methodologies. Their application allows for a holistic and unbiased view of the quality of teaching and learning. This article aims to illustrate and evaluate the usefulness of project management methodologies in identifying Critical Success Factors (CSFs) and risks in creating and evaluating academic courses. The subject of the research was the course “Creativity and Decision-making”, and the main method of the research, which was conducted in four cycles from 2020 to 2023, was the Deming cycle (PDCA). Data collection was based on data triangulation, including unstructured interviews with students and experts, as well as participant observation. Content analysis with a categorisation key was used to analyse the data, with the results presenting lists of extracted critical success and risk factors determining the quality of the course. The implemented procedure represents an innovative approach to the evaluation of academic courses, at the core of which is the creation and updating of two substantively opposing lists: the determinants of learning successes and failures constituting of a map of strategic control points, and a basis for the improvement of the educational process.

Keywords: critical success factors (CSF), risk management, project-based learning (PBL), project management methodology (PMM), Deming cycle (PDCA)

Introduction

The priority of education has to be providing high-quality teaching and learning, which refers to the extent of valuable and efficient experience delivered to learners throughout the educational process (McLean & Ashwin, 2016; Zajda, 2021), enabling effective education that contributes to knowledge acquisition and develops students' skills and attitudes (Netshifhefhe et al., 2016). The quality of teaching commences with concept and curriculum planning (Richards, 2013), whereby lecturers select teaching and learning methods appropriate to the needs of the programme delivery and the requirements of the students (Toufaili, 2018). A further significant determinant of quality directly related to planning is the monitoring of task completion, the progress of students, the assessment of their work, and the evaluation of the whole process (Raza et al., 2015). Accomplishing that range of activities can improve the teaching and learning process (Ifeoma, 2022; Yambi, 2018), while the essence of evaluation is to provide feedback (Hounsell, 2003) that would assist in identifying the strengths and weaknesses of the educational process. Using this as a basis, lecturers may undertake measures to optimise their syllabuses, methods, and teaching and learning techniques to comply responsively with the needs of the students and enhance their learning efficiency (El-Hassan & Al-Hroub, 2013). In addition to traditional teaching and learning evaluation forms, which include observation, surveys, performance analysis and teacher self-evaluation (Anh, 2018), there is an increasing tendency to implement business methods, which, in particular, include PDCA, benchmarking, Six Sigma, etc. (Chen, 2012). Applying business methods in evaluating the teaching and learning process leads to a more holistic and unbiased approach to assessing the quality of teaching (Zuluaga-Ortiz et al., 2022).

Adopting business methods in the evaluation process is also prompted by the popularisation of the project approach (Kokotsaki et al., 2016). Treating the teaching course as an academic project increases the effectiveness of both the teaching and learning process by enriching the students' experience and thereby enhancing their engagement with the tasks and challenges assigned to them (Eckardt et al., 2020). With the project-based approach, students can learn by acting, experimenting and solving real-life problems, thus allowing the practical application of knowledge and the development of practical skills (Sumarni, 2015). Educational projects stimulate greater creativity and innovation in teachers and students (Johnsen et al., 2023), providing an environment to experiment with different methodologies and explore innovative solutions (Krajcik & Shin, 2014; van Rooij, 2007). However, the teaching and learning methodology mentioned above is not explicitly related to the project management methodology and its constant vital elements that determine the quality and efficiency of a project (Crawford & Pollack, 2007), even though there is a commonality between them that justifies their integration.

Project management methodology (PMM) in education is being applied at the level of the "approach" (van Rooij, 2009), using general assumptions and principles, without strictly following a systematic set of rules, regulations, procedures, and tools resulting from the methodology (Liegel, 2004). Among the few works illustrating the use of PMM in higher education is the described learning project for postgraduate students in the digitisation of solar energy building design (Gunarathna et al., 2023) and the results of research on the design of virtual team communication in undergraduate psychology courses (Chiocchio, 2007). An exception is eduScum, which methodology follows closely from PMM (Fernandes et al., 2021). However, it is mainly used to learn software development (Neumann & Baumann, 2021).

A key element of project management methodology are critical success factors, which define the conditions necessary to achieve project goals and ensure high-quality results (Melton, 2011) in order to identify areas that require special supervision and management during the project. Risk plays a significant role among the critical success factors (Ruzic-Dimitrijevic & Dakic, 2014). Their identification, assessment, mitigation strategies, and threat action plans enable better preparation for potential problems and challenges, allowing for more effective project management and increasing the likelihood of success. The introduction of an innovation consisting of developing a list (with a description) of CSFs and risks in the course description enables the effective implementation of teaching objectives and effects. Illustrating the proposed solution with a case study may encourage other lecturers to implement it, while investigating its effectiveness and sharing the results would help fill the current research gap in this area, and integration of project-based learning and PMM can improve the teaching process. Educational institutions could develop guidelines and frameworks for integrating CSFs and risk management into course evaluation processes, emphasising the importance of these elements in ensuring the quality and success of educational projects.

Background

The application of project management methodologies improves the management of the learning process, attains the intended educational goals, optimises the use of resources, minimises risks, and ensures effective evaluation (van Rooij, 2009). The project approach, when developing a course, enables focusing on the aim and the context in which it is set, enabling a complete understanding of the problem (Peraza & Furumura, 2022). Students' work is regularly monitored, which allows for ongoing control and reaction, correcting actions and performing the retrospective analysis (Kokotsaki et al., 2016). The domain of the project approach is to gather the knowledge needed to improve future projects.

Successful project management, including educational projects, depends on several factors that affect the execution process and the final results (Project Management Institute, 2004), including activities such as identifying objectives, planning, allocating resources, setting a budget, involving participants, etc. These, in turn, are also areas for defining critical success factors, which are an essential element in project management (Alias et al., 2014), representing factors that must be fulfilled or adhered to for a project to succeed (Zwikael & Globerson, 2006). On the other hand, insufficient consideration or disregarding them could lead to failure.

Critical success factors (CSFs) significantly impact the evaluation process in assessing activities' effectiveness and efficiency (Charvat, 2003), contributing to an overall understanding of whether objectives have been achieved, to what extent, and what steps should be undertaken to improve performance.

CSFs may vary following the context of the project/organisation. In education, they are vital to providing a valuable and high-quality teaching and learning experience (Nadim & Al-Hinai, 2016). CSFs in education comprise:

• Students' engagement and motivation are crucial to leading to effective learning experiences. Students are more likely to participate actively in classes and assimilate knowledge more efficiently when motivated.
• Qualification level of lecturers - proficient, well-qualified lecturers have the requisite skills, knowledge and experience to teach the material to students comprehensibly and engagingly.
• Individualised approaches in education allow for adapting the methodology and content to the individual needs of each student of their learning patterns, pace of knowledge acquisition and performance level.
• Utilisation of modern technology can augment the learning process and make it more gripping and interactive for students.
• Content and curriculum quality guaranteed by thoroughly designed and revised materials and curriculum are crucial in delivering a complex education.
• Effective assessment methods are used to verify students' progress and obtain insight into whether they are accomplishing their intended goals and learning objectives.
• Continuous improvement of the didactic process to alter needs and demands.

A prominent role amongst CSFs belongs to risk management (Dandage et al., 2017), with risk in a project pertaining to the possibility of unexpected events that can negatively affect the project (Schieg, 2006). Nevertheless, risk is an intrinsic part of any project that may originate from factors such as market uncertainty, ambient fluctuations, technological challenges, financial concerns, lack of resources, team issues, etc. (Lavanya & Malarvizhi, 2008). Furthermore, risk management is key to successful project design and implementation (Wideman, 1992).

Effective risk management covers the identification, analysis, assessment, preventative and reactive action planning, and project tracking and management of risks (Cagliano et al., 2015). Adopting appropriate risk management strategies can diminish the adverse potential effects of risk, reinforcing the chances of project success (Crispin, 2020).

Risk in an educational project pertains to potential threats or opportunities that may affect the achievement of the project's learning objectives (Kirk et al., 2022). The following are some examples of risks posed by an educational project:

• Engagement and motivation of students: Getting students involved in the learning process is key to an effective learning experience. Students are more likely to participate actively in classes and assimilate knowledge more efficiently when motivated.
• Qualification level of lecturers: Proficient, well-qualified lecturers have the requisite skills, knowledge, and experience to teach the material to students understandably and engagingly.
• Individualised approach in education: Students differ in learning patterns, the pace of knowledge acquisition, and performance level. The tailor-made teaching approach has the advantage of adapting the methodology and content to the individual needs of each student.
• Utilisation of modern technology: Implementing modern technological tools can augment the learning process and make it more gripping and interactive for students.
• Content and curriculum quality: Thoroughly designed and revised materials and curriculum are crucial in delivering a complex education.
• Effective assessment methods: Effective assessment methods verify students' progress and obtain insight into whether they are accomplishing their intended goals and learning objectives.
• Continuous improvement: Professional lecturers constantly strive to improve and adapt their practices to altering needs and demands.

To sum up, it is vital to incorporate risk identification and its analysis throughout the planning and execution of an academic project, as employing suitable risk management strategies can mitigate the negative effects of risk and enhance the chances of a successful educational project (Jones & Fevre, 2021). Monitoring progress regularly and adjusting strategies in response to new challenges are the core components of effective risk management in an educational project (Helsloot & Jong, 2006; Marchewka, 2010).

The identification of CSFs has been popularised in education, including higher education, although it is most often used at the organisational level. It solves problems, e.g. changing the university's operating model (Saleh et al., 2015). In the didactic sphere, it is used, among others, in the implementation of a new form of education (Cheawjindakarn et al., 2012; Min & Yu, 2023), in the analysis of online course resources (Soong et al., 2001), or in assessing the effectiveness of e-learning (Alhabeeb & Rowley, 2018; Puri, 2012).

However, there is a gap in applying the CSFs in academic course evaluation. Among the advantages of its application in this respect are setting clear and measurable learning objectives; actively involving participants in the learning process; providing relevant resources, such as learning materials, technology, etc.; evaluating the project's progress regularly; collecting reviews and feedback from project participants. These arguments are valid for implementing the CSFs method in academic projects concordant with the PMM and the business approach.

In contrast, identifying educational risks has yet to find widespread application in teaching and learning practice (Kirk et al., 2022). When building the quality of education, attention is focused on issues that predict quality improvement (O'Mahony & Garavan, 2012). Issues enabling the identification of potential failures and problems related to changes in educational processes, which would allow for adequate preparation for changes and minimising adverse effects, should be analysed in detail (Ruzic-Dimitrijevic and Dakic, 2014). On the contrary, within business project management, it is a top priority affecting each task in detail (Berg, 2010). Still, there is a gap in the way of thinking, a pragmatic business mindset is preoccupied with the need, if not necessity, to anticipate anything that could go wrong and to prepare a contingency plan accordingly. The difference may be attributable to the low emphasis in the educational systems on ensuring the golden triad of the project: quality, time (work schedule), and cost (budget).

Research design

Purpose

The research aimed to apply project management methodology (PMM) to identify critical success factors (CSFs) and educational risks - key elements in the process of designing and evaluating an academic course.

Problem

How can the quality of education be improved when implementing practical academic courses aimed at generating innovative solutions?

Research question

How do the elements of project management methodology: critical success and risk factors apply to the design and improvement of an academic course?

Setting

The research was conducted based on a quality cycle (Plan-Do-Check-Act (PDCA)), a quality management method that is being successfully applied in education (Pietrzak & Paliszkiewicz, 2015). Alternatively, it may be considered an evaluation instrument (Mergen et al., 2014). The evaluation within the PDCA cycle provides a continuous flow of feedback and data supporting evidence-based decision management. This facilitates the improvement of processes, the elimination of errors, and the continuous refinement of operations. The procedure pattern to achieve better results consists of four steps, iterated systematically: 1. to plan, 2. to do, 3. to check, 4. to act (Divjak & Redjep, 2015).

The application of the PDCA framework in the following research project is presented in the list below.

Plan

• Determining: the purpose of the research, the method of achieving the goal, the method of collecting and analysing data, and the method of measuring results.

Do

• Conducting research among students, invited experts, and observations by the lecturer.

Check

• Data analysis according to the categorisation key.
• Measuring the results.

Act

• Improving the course framework and introducing it to the next edition.

The starting point for preparing the plan was to diagnose the problem: students fail to understand the proposed solution fully, missing critical details that determine the solution's usefulness, as well as the whole range of factors that impact its implementation. It was observed during the realisation, in 2018 and 2019, of a one-semester course called "Creativity and Decision-Making", an elective course for management students with a major in project management (Faculty of Management, University of Lodz, Poland).

A concise description of the course is presented on the course card below.

Table 1

Course card

Source: author's own work.

Content analysis was used to analyse the data, which aims to systematically understand and interpret the content of a text (White & Marsh, 2006). The starting point is coding, i.e. identifying key thematic categories or issues that will be analysed in the content, which should result from the research objectives. The following study used closed coding with a categorisation key, assigning previously defined categories or codes to specific text fragments (Zhang & Wildemuth, 2005). A list of potential critical success factors (CSF) and threats was used to create a categorization key, developed based on assumptions regarding the organisation and implementation of the course (Table 2).

Table 2

Data categories model / categorisation key

Source: author's own work.

Data was collected based on data triangulation using various data sources (Flick, 2018). The research below used a method of participant observation and free-form interviewing undertaken:

• with the students who expressed a wish to take part in the survey both during and after the course,
• with invited experts/judges.

Participant observation involves the active participation of the researcher in the group or community under study to comprehend and analyse its behaviour, values, norms, social interactions and other aspects of life (DeWalt & DeWalt, 2002). This method of collecting information lets the researcher gain insight into the perspective and viewpoint of the respondents (Kawulich, 2005).

A non-structured (free-form) interview, in contrast to a structured interview where questions are pre-prepared and imposed, therefore allows for a spontaneous response from the respondent (Arksey & Knight, 1999). If necessary, the researcher may ask for further clarification or elaboration of the respondent's answers (Zhang & Wildemuth, 2009).

The data collection period spanned from 2020 to 2023. The table below shows the number of respondents participating in unstructured interviews each year.

Table 3

The number of respondents in each year

Source: author's own work.

Results

As a result of the content analysis using a categorisation key, CSFs and risks determining the quality of education during the implementation of the course "Creativity and Decision Making" were identified. The information obtained during the evaluation was assigned to a specific thematic category and synthesised, and this process was repeated after each edition in 2020-2023. The presented results are the result of recent research.

Critical Success Factors

The figure below shows the list of four CSFs established for the course under research with their synthetic description.

Figure 1

List of critical success factors (CSF)

Source: author's own work.

Identification of educational risks

The research identified four didactic risks. Their categories, descriptions, and ways of dealing with them are presented in the following two figures below. Knowing and monitoring them is aimed at effectively dealing with problems that may occur during the implementation of an educational project.

Figure 2

Lists of educational risks

Source: author's own work.

Improved framework

Analysis of the identified CSFs, including risks, has prompted a change in course management from a traditional, linear approach to an agile one, which was associated with the addition of meetings with experts/practitioners throughout the semester. As a result, elements of the eduScrum methodology were adopted. The programme was divided into sprints culminating in a meeting with the experts and a presentation to them of the output achieved during the sprint (review), where students receive expert feedback, further serving as a basis for determining potential problems or areas for improvement. Students receive feedback on the product from potential solution users, thus helping the project team recognise possible problems or areas for improvement. The acquired feedback, reviews and analysis of the sprint results are used to improve measures and plan upcoming sprints.

The evaluation is based on a threefold oral presentation of the project, supplemented by a written (descriptive) presentation. Presentations are graded on an incremental scale: 6-point idea presentation, 10-point prototype presentation, and 20-point final presentation.

Improved course schedule:

1. Sprint
1. Start of the project: Introduction
2. Creativity - originality vs. usefulness
2. Sprint
1. Empathising with the customer (interviews with invited persons)
2. Idea generation
3. Idea presentation
3. Sprint
1. Prototyping
2. Presentation of prototypes/expert evaluation (sprint review)
4. Sprint
1. Testing
2. Presentation of projects/expert evaluation (sprint review)
5. Conclusion of the project: summary of classes

Impact of programme improvement on solving the problems

The description was adopted as a method for measuring the results of implementing research results into the next edition of the course. This research project is based on a qualitative descriptive methodology, which implies and justifies this choice (Zhang & Wildemuth, 2005).

Based on the results by analysing the observations of the students' current work, the final results of the innovative propositions and the content of the students' and experts' oral contributions, the following conclusions were drawn:

• students delve into the details of the suggested solution,
• after the first sprint review, the teams that have received negative feedback declare their willingness to modify the direction of the project,
• the level of self-reflection and the desire for self-development intensify,
• the heightened interest in understanding the proposed solution and the way it works,
• the growing need to understand the context of implementation, its conditions and barriers,
• students are more open to communication with the lecturer.

Discussion

The practical value of this research is thus to propose an approach to designing and evaluating an academic course with a particular focus on CSFs, including risks, an approach that involves creating and updating two substantively opposing lists, representing a map of strategic control points and a basis for improving the educational process. These are expected to vary according to the specific characteristics of each course as well as its objectives and expected outcomes.

Since the above study proposes an innovative approach to using PMM elements, such as CSF and the risks for the development and evaluation of academic courses, it is one of the few available in the literature that highlights the benefits of implementing PMM in education. They are consistent with those emphasised by other researchers: strengthening creativity, critical thinking, and problem-solving (Gunarathna et al., 2023), improving work organisation by dividing the programme into stages (Chiocchio, 2007), improving the efficiency of information flow, including obtaining information necessary to improve performance (Fernandes et al., 2021). These findings provide insight into the complexities and nuances of designing and implementing innovative solutions in academic contexts. They can be a reference point for innovative design and evaluation of academic activities, research, and sharing results with other lecturers. The value of the above research is therefore new knowledge in developing academic courses designed with an emphasis on acquiring professional competences.

Limitations of the research include the lack of reference to potential challenges or limitations in implementing the proposed strategies across different academic institutions, course types, or cultural contexts. Additionally, no quantitative measures of student performance were used in this research project.

Further, the implementation of the recommended approach to the didactic project is limited by the knowledge about PMM and other business methods that can be used in the teaching and learning process, which is not widely disseminated among lecturers outside the fields of business, management, and IT, and which can be used in the teaching and learning process (Ahtee & Poranen, 2009; Boehm et al., 2002).

Conversely, the research/evaluation method employed in the above-described research project, the Deming cycle, is widely applied in education at every level (Miller, 1991), and is implemented directly to refine teaching and learning processes, and as a tool to facilitate the subject/course evaluation process (Aggarwal, 2020). The limitation of this method is the duration of the standard academic cycle, impacting the prolonging of the research process, and providing the material for an extensive analysis of the issue and an effective solution for dealing with it.

The limitations of using PDCA in education include the difficulty of defining clear and measurable improvement goals due to the complexity of educational outcomes and, therefore, the pursuit of simplification and excessive standardisation. However, qualitative indicators can be used in PDCA, such as: subjective assessments, descriptions, quality of relationships, etc. (Dam et al., 2020). A single PDCA cycle is linear, but the method is designed to proceed iteratively. Once one cycle is completed, it can be repeated for continuous improvement. Thus, iterativeness is a key element of PDCA, allowing for experimentation and continuous improvement (Morgan & Stewart, 2017). Additionally, while PDCA is often associated with improving and optimising existing processes, it can also be used to suggest and implement innovative solutions. A key aspect is a flexible approach to each stage of the PDCA cycle, which allows the method to be adapted to various needs, including innovative projects (Hakim et al., 2020).

Further research on utilising PMM in developing academic courses should focus on the degree and scope of implementation and effectiveness evaluation, including developing quantitative metrics for student outcomes.

The continued investigation into the application of project management methodologies in the course development process should specifically examine the extent to which these methodologies have been implemented. It is crucial to analyse the degree and areas in which specific project management elements have been incorporated into the course development process. Research will help identify areas where these methodologies are most effective and allow their adaptation to specific educational needs.

Additionally, research should assess the effectiveness of applying PMM in creating academic courses, covering various aspects such as achieving educational goals, student satisfaction, resource utilisation efficiency, and the overall quality of the teaching and learning process. Developing quantitative metrics for student outcomes is also essential to objectively assess the educational achievements resulting from applying project management methodologies.

Conclusion

The above article presents the importance of critical success factors and didactic risks that influence the design and evaluation of a project-oriented academic course. In the "Creativity and Decision Making" course under research, lists of these elements were distinguished. Aspects related to course introduction, schedule, role of the lecturer, and participation of experts were classified as CSFs. The risks were categorised according to the key elements of the course: idea creation, client emaptisation, prototyping and testing, and preparation of the final presentation.

They were introduced into the teaching and learning process, which allowed for optimising learning outcomes. Therefore, the research results presented above can serve as an example of the effective integration of business methods into academic education. They also provide new course design and assessment knowledge, proposing innovative approaches.

References

• Aggarwal, A. K. (2020). Using Deming's Cycle for improvement in a course: A case study. International Journal of Web-Based Learning and Teaching Technologies, 15(3), 31-45. https://doi.org/10.4018/IJWLTT.2020070103
• Ahtee, T., & Poranen, T. (2009). Risks in students' software projects. 22nd Conference on Software Engineering Education and Training (pp. 154-157). https://doi.org/10.1109/CSEET.2009.31
• Alhabeeb, A., & Rowley, J. (2018). E-learning critical success factors: Comparing perspectives from academic staff and students. Computers & Education, 127, 1-12. https://doi.org/10.1016/j.compedu.2018.08.007
• Alias, Z., Zawawi, E. M. A., Yusof, K., & Aris, N. M. (2014). Determining critical success factors of project management practice: A conceptual framework. Procedia - Social and Behavioral Sciences, 153, 61-69. https://doi.org/10.1016/j.sbspro.2014.10.041
• Anh, V. T. K. (2018). Evaluation models in educational program: strengths and weaknesses. Journal of Foreign Studies, 34(2), 140-150. https://doi.org/doi:10.25073/2525-2445/vnufs.4252
• Arksey, H., & Knight, P. (1999). Interviewing for social scientists: An introductory resource with examples. Sage.
• Berg, H. P. (2010). Risk management: Procedures, methods and experiences. Risk Management, 17(1), 79-95.
• Boehm, B. W., Port, D., & Brown, A.W. (2002). Balancing Plan-Driven and Agile Methods in Software Engineering Project Courses. Computer Science Education, 12(3), 187-195. https://doi.org/10.1076/csed.12.3.187.8617
• Cagliano, A. C., Grimaldi, S., & Rafele, C. (2015). Choosing project risk management techniques. A theoretical framework. Journal of Risk Research, 18(2), 232-248. https://doi.org/10.1080/13669877.2014.896398
• Charvat, J. (2003). Project management methodologies: Selecting, implementing, and supporting methodologies and processes for projects. John Wiley and Son, Inc.
• Cheawjindakarn, B. , Suwannatthachote, P., & Theeraroungchaisri, A. (2012). Critical success factors for online distance learning in higher education: A review of the literature. Creative Education, 3(8B), 61-66. https://doi.org/10.4236/ce.2012.38B014
• Chen, S. H. (2012). The establishment of a quality management system for the higher education industry. Quality and Quantity, 46, 1279-1296. https://doi.org/10.1007/s11135-011-9441-1
• Chiocchio, F. (2007). Project team performance: A study of electronic task and coordination communication. Project Management Journal, 38(1), 97-109. https://doi.org/10.1177/875697280703800110
• Crawford, L., & Pollack, J. (2007). How generic are project management knowledge and practice? Project Management Journal, 38(1), 87-96. https://doi.org/10.1177/875697280703800109
• Crispin, G. (2020). The essence of risk identification in project risk management: An overview. International Journal of Science and Research, 9(2), 1553-1557.
• Dam, M., Janssen, F. J. J. M., & van Driel, J. H. (2020). Making sense of student data in teacher professional development. Professional Development in Education, 46(2), 256-273. https://doi.org/10.1080/19415257.2018.1550104
• Dandage, V. D., Mantha, S. S., & Rane, S. B. (2017). Exploring the critical success factors for project management and project risk management. Journal of Advances in Science and Technology, 13(1), 348-352.
• DeWalt, K. M., & DeWalt, B. R. (2002). Participant observation: a guide for fieldworkers. AltaMira Press.
• Divjak, B., & Redjep, N. (2015). Strategic decision making cycle in higher education: Case study of e-learning. Proceedings of the International Conference on E-learning (pp. 19-26). https://files.eric.ed.gov/fulltext/ED562492.pdf
• Eckardt, P. N., Craig, M., & Kraemer, L. (2020). The impact of project-based learning on student content knowledge in an undergraduate, teacher preparation, foundations of education course. Journal for Leadership and Instruction, 19(1), 38-42.
• El-Hassan, K., & Al-Hroub, A. (2013). Strengths and Weaknesses of the Evaluation System of Education in Libya. UNESCO Regional Bureau for Education in the Arab States. https://doi.org/10.13140/RG.2.1.3997.4645
• Fernandes, S., Dinis-Carvalho, J., & Ferreira-Oliveira, A. T. (2021). Improving the performance of student teams in project-based learning with Scrum. Education Sciences, 11(8), 444. https://doi.org/10.3390/educsci11080444
• Flick, U. (2018). The SAGE handbook of qualitative data collection. SAGE Publications Ltd. https://doi.org/10.4135/9781526416070
• Gunarathna, C., Yang, R., Wijeratne Mudiyanselage, P., Amarasinghe, G., Samarasinghalage, T., Weerasinghe, R. P. N., Zhao, H., Zhang, C., Liu, C., Wang, K., & Dev Sureshkumar Jayakumari, S. (2023). Project-based learning for proactive skills development of postgraduate students in solar energy building design digitalization. Smart and Sustainable Built Environment (ahead-of-print). https://doi.org/10.1108/SASBE-08-2022-0173
• Hakim, L., Rosadi, K. I., Widdah, M. E., Us, K. A., Shalahudin, & My, M. (2020.) Did PDCA Cycle, service quality and innovation capability influence private universities performance? Systematic Reviews in Pharmacy, 11(10), 874-883.
• Helsloot, I., & Jong, W. (2006). Risk management in higher education and research in the Netherlands. Journal of Contingencies and Crisis Management, 14(3), 142-159. https://doi.org/10.1111/j.1468-5973.2006.00490.x
• Hounsell, D. (2002). The evaluation of teaching. In: S. Marshall, H. Fry, & S. Ketteridge (Eds.), A handbook for teaching and learning in higher education: Enhancing academic practice (2nd revised ed.) (pp. 200-212). Kogan Page.
• Ifeoma, E. F. (2022). The role of evaluation in teaching and learning process in education. International Journal of Advanced Academic and Educational Research, 13(5), 120-129. https://arcnjournals.org/images/2726145223713511.pdf
• Johnsen, M. M. W., Sjolie, E., & Johansen, V. (2023). Learning to collaborate in a project-based graduate course: A multilevel study of student outcomes. Research in Higher Education, 1-24. https://doi.org/10.1007/s11162-023-09754-7
• Jones, T. K., & Le Fevre, D. M. (2021). Increasing teacher engagement in innovative learning environments: Understanding the effects of perceptions of risk. In: W. Imms, & T. Kvan, (Eds.), Teacher transition into innovative learning environments (pp. 73-84). Springer. https://doi.org/10.1007/978-981-15-7497-9_7
• Kawulich, B. B. (2005). Participant observation as a data collection method. Forum: Qualitative Social Research, 6(2). https://doi.org/10.17169/fqs-6.2.466
• Kirk, D, Luxton-Reilly, A., & Tempero, E. (2022). Refining a risk framework for student group projects. Proceedings of the 22nd Koli Calling International Conference on Computing Education Research (pp. 1-11). https://doi.org/10.1145/3564721.3564730
• Kokotsaki, D., Menzies, V., & Wiggins, A. (2016). Project-based learning: A review of the literature. Improving Schools, 19(3), 267-277. https://doi.org/10.1177/1365480216659733
• Krajcik, J., & Shin, N. (2014). Project-based learning. In: R. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences (2nd ed.) (pp. 275-297). Cambridge University Press.
• Lavanya, N., & Malarvizhi, T. (2008). Risk analysis and management: a vital key to effective project management. Paper presented at PMIR Global Congress 2008. Project Management Institute. https://www.pmi.org/learning/library/risk-analysis-project-management-7070
• Liegel, K. M. (2004). Project-based learning and the future of project management. Paper presented at PMIR Global Congress 2004. Project Management Institute. https://www.pmi.org/learning/library/project-based-learning-future-project-management-8273
• Marchewka, J. T. (2010). A framework for identifying and understanding risks in information technology projects. Journal of International Technology and Information Management, 19(1), 61-74. https://doi.org/10.58729/1941-6679.1057
• McLean, M., & Ashwin, P. (2016). The quality of learning, teaching and the curriculum. In P. Scott, J. Gallacher, & G. Parry (Eds.), New languages and landscapes of higher education (pp. 84-102). Oxford University Press. https://doi.org/doi: 10.1093/ACPROF:OSO/9780198787082.003.0004
• Melton, T. (2011). Project management toolkit: The basics for project success. Elsevier.
• Mergen, S., Kepler, F., Silva da Silva J., & Cera, M. (2014). Using PDCA as a General Framework for Teaching and Evaluating the Learning of Software Engineering Disciplines. iSys - Brazilian Journal of Information Systems, 7(2), 5-24.
• Miller, R. I. (Ed.) (1991). Applying the Deming method to higher education, for more effective human resource management. College and Personnel Association.
• Min, W., & Yu, Z. (2023). A systematic review of critical success factors in blended learning. Education Sciences, 13(5), 469. https://doi.org/10.3390/educsci13050469
• Morgan, S., & Stewart, A. (2017). Continuous improvement of team assignments: Using a web-based tool and the plan-do-check-act cycle in design and redesign: Continuous Improvement of team assignments. Decision Sciences Journal of Innovative Education, 15(3), 303-324. https://doi.org/10.1111/dsji.12132
• Nadim, Z. S., & Al-Hinai, A. H. (2016). Critical success factors of TQM in higher education institutions context. International Journal of Applied Sciences and Management, 1(2), 147-156.
• Netshifhefhe, L., Nobongoza, V., & Maphosa, C. (2016). Quality assuring teaching and learning processes in higher education: A critical appraisal. Journal of Communication, 7, 65-78.
• Neumann, M., & Baumann, L. (2021). Agile methods in higher education: Adapting and using eduScrum with real world projects. 2021 IEEE Frontiers in Education Conference (FIE) (pp. 1-8). https://doi.org/10.1109/FIE49875.2021.9637344
• O'Mahony, K., & Garavan, T. N. (2012). Implementing a quality management framework in a higher education organization: A case study. Quality Assurance in Education, 20(2), 184-200. https://doi.org/10.1108/09684881211219767
• Peraza, A. V., & Furumura, Y. (2022). Project-based learning to develop intercultural communicative competence in virtual exchange contexts. International Journal of Computer-Assisted Language Learning and Teaching, 12(3), 1-17. http://doi.org/10.4018/IJCALLT.307059
• Pietrzak, M., & Paliszkiewicz, J. (2015). Framework of strategic learning: The PDCA Cycle. Management, 10(2), 149-161.
• Project Management Institute. (2004). A guide to the project management body of knowledge (PMBOK Guide) (3rd ed.). https://skat.ihmc.us/rid=1240963222395_1554076313_2451/PMBOK-%203rd%20edition.pdf
• Puri, G. (2012). Critical success Factors in e-Learning - An empirical study. International Journal of Multidisciplinary Research, 2(1), 149-161.
• Raza, M. A., Bilal, M., Rasheed, M. R., Chandio, B. A., Ahmad, N., & Sawand, F. A. (2015). Quality assessment in higher education. International Letters of Social and Humanistic Sciences, 50, 162-171.
• Richards, J. C. (2013). Curriculum approaches in language teaching: Forward, central, and backward design. RELC Journal, 44(1), 5-33. https://doi.org/10.1177/0033688212473293
• Ruzic-Dimitrijevic, L., & Dakic, J. (2014). The risk management in higher education institutions. Online Journal of Applied Knowledge Management, 2(1), 137-152.
• Saleh, A. A., Mohammed, A. H., & Abdullah, M. N. (2015). Critical success factors for sustainable university: A framework from the energy management view. Procedia - Social and Behavioral Sciences, 172, 503-510. https://doi.org/10.1016/j.sbspro.2015.01.392
• Schieg, M. (2006). Risk management in construction project management. Journal of Business Economics and Management, 7(2), 77-83.
• Soong, M. H. B, Chan, H. Ch., Chua, B. Ch., & Loh, K. F. (2001). Critical success factors for online course resources. Computers & Education, 36(2), 101-120. https://doi.org/10.1016/S0360-1315(00)00044-0
• Sumarni, W. (2015). The strengths and weaknesses of the implementation of project based learning: A review. International Journal of Science and Research, 4(3), 478-484.
• Toufaili, B. (2018). Factors influencing quality teaching in higher education. International Journal of Research, 5(21), 553-557.
• van Rooij, S. W. (2009). Scaffolding project-based learning with the project management body of knowledge (PMBOKR). Computers and Education, 52(1), 210-219. https://doi.org/10.1016/j.compedu.2008.07.012
• van Rooij, W. S. (2007). WebMail versus WebApp: Comparing problem-based learning methods in a business research course. Journal of Interactive Learning Research, 18(4), 555-569.
• White, M. D., & Marsh, E. E. (2006). Content analysis: a flexible methodology. Library Trends, 55(1), 22-45. https://doi.org/10.1353/lib.2006.0053
• Wideman, R. M. (1992). Project and program risk management: A guide to managing project risks and opportunities. Project Management Institute.
• Yambi, T. (2018). Assessment and evaluation in education. https://www.academia.edu/35685843/ASSESSMENT_AND_EVALUATION_IN_EDUCATION
• Zajda, J. (2021). Globalisation and education reforms. Creating effective learning environments. Springer.
• Zhang, Y., & Wildemuth, B. M. (2005). Qualitative analysis of content. Human Brain Mapping, 30(7), 2197-2206.
• Zhang, Y., & Wildemuth, B. M. (2009). Unstructured interviews. In: B. Wildemuth (Ed.), Applications of social research methods to questions in information and library science (pp. 222-231). Libraries Unlimited.
• Zuluaga-Ortiz, R., Delahoz-Dominguez, E., Perinan-Luna, A., Escorcia, J., Moreira-Villegas, F., & Arteta, A. (2022). Evaluation of Educational Quality Under a Six Sigma Approach to Engineering Degrees in Colombia. In: K. Saeed, J. Dvorský (Eds.), Computer Information Systems and Industrial Management. CISIM 2022. Lecture Notes in Computer Science, 13293 (pp. 172-188). Springer. https://doi.org/10.1007/978-3-031-10539-5_13
• Zwikael, O., & Globerson, S. (2006). From critical success factors to critical success processes. International Journal of Production Research, 44(17), 3433-3449. https://doi.org/10.1080/00207540500536921