Highlight, copy & paste to cite:
Altarawneh, I. (2009). Training and Development Evaluation in Jordanian Banking Organisations, Research and Practice in Human Resource Management, 17(1), 1-23.
Training and Development Evaluation in Jordanian Banking Organisations
Although training evaluation is a very important stage for a successful Training and Development (T&D) programme, this activity is the most neglected and problematic. As a result, T&D has been considered in many organisations as a waste of money and time, a cost which needs to be minimised and is often perceived as an ineffective process. This study sought to explore the current practices and challenges of the T&D evaluation stage within Jordanian banking organisations through a combination of semi structured interviews and a survey questionnaire, which was administered to the persons responsible for the T&D within the organisations. The study findings reveal, that although the majority of the organisations evaluate their training programme, usually there is an absence of systematic and effective procedures for evaluation, and most of the organisations rely on external providers to evaluate their T&D programmes. The most popular evaluation tools and technique used are questionnaires, and the most common model used is the Kirkpatrick model, however, the most common level of evaluation is the reaction level. Implications for the findings of the current study and directions for future research are discussed in the paper.
With the huge investment in developing training strategies and programmes, the question is no longer whether organisations should train or not, but rather it is about whether or not training is worthwhile and effective (Mann 1996). Redshaw (2000) argues, that although T&D plays a critical role in organisational performance, it is viewed differently in different organisations. It may be considered as an unnecessary, underused and unrecognised function. Some organisations find that T&D does not contribute to employees’ commitment, and sometimes it is considered as a waste of money and time. Unfortunately, in many other organisations the management does not believe in T&D as a function to contribute in improving the overall organisational performance. They believe that T&D does not work since it is too difficult to prove its effectiveness in terms of calculating the Return on Investment (ROI) of T&D programmes.
A review of the T&D literature shows in many Arab organisations, including Jordanian organisations, training is still not considered as an important organisational function, which contributes to the organisation’s success. Instead, it is viewed as a vacation activity or leisure time pursuit, which is given to some people, normally to the managers’ relatives and friends. Moreover, the literature suggests that in some Arab organisations evaluation is rarely undertaken, because training is being seen as an overhead and not an investment to be evaluated (Al-Athari & Zairi 2002). In many other Arab organisations T&D evaluation is not conducted in a professional manner. In this regard, Abdalla and Al-Homoud (1995), Bahar, Peterson and Taylor (1996), and Abdalla, Maghrabi & Raggad (1998) argue that many Arab organisational practices, in terms of their T&D management, are deficient because of the lack of systematic planning, implementation and the evaluation phases. Moreover, Atiyyah (1993) argues that evaluation methods in Arab organisations are highly subjective and their results have limited impact on improving the on going programmes or even the designing of new programmes. Atiyyah (1993) also contends that T&D programmes were usually evaluated at the reaction level via questionnaires distributed to the trainees to give their opinions on the quality of instruction, programmes materials, and suitability of the training techniques.
From a broader landscape the international T&D literature reveals that the T&D evaluation stage faces many vexations and challenges. In this regard Shandler (1996), Redshaw (2000), as well as Burrow and Berardinelli (2003) argue that proving T&D effectiveness in achieving organisational performance criteria, such as profit, quality, customer satisfaction, ROI and market share, is not an easy task. Consequently, the evaluation stage in the training process is often ignored. There are many factors that underpin the reluctance to engage evaluation initiatives. For instance, T&D benefits accrue over a long period of time, there are many other intervening variables, and a great deal of factors can influence an employee’s performance, while most T&D outcomes are subjective, complex and difficult to measure. Support for these contentions can be gleaned from the work of Abdalla and Al-Homoud (1995), and Al-Athari and Zairi (2002), who found that the most common evaluation challenges in the surveyed Arab organisations are the cost of conducting this process, difficulty in finding evaluation methods, difficulty in finding evaluation quantitative financially criteria or language, time required to accomplish this process and the lack of information needed for evaluation.
The purpose of this study is two fold. First, is to attenuate the gap in the T&D literature in Arab financial entities and in particular Jordanian organisations by exploring the current management T&D evaluation practices and challenges in the Jordanian banking industry. Second, to suggest practical solutions for T&D evaluation problems in these organisations. A review of T&D literature shows that there is a lack of relevant studies in the field of T&D practices and management in Jordanian organisations and in Arab organisations, generally. Moreover, most of the previous studies conducted in Arab organisation did not target the banking industry in particular, which is considered as a people oriented industry. Indeed, it can be claimed that a firm’s success comes from the people who work for these institutions. Jordanian banking organisations spend a lot of money in T&D programmes, but these companies do not appear to know the extent of effectiveness of these programmes (Altarawneh 2005). Arguably, this study is different from other previous studies in terms of being conducted in the context of Jordanian organisations, which have not been previously targeted. In addition, it is different in terms of employing the banking industry, the study sample, as this sector has been neglected in the previous Arab research, despite claims of extensive investment in T&D programmes.
This paper has three main parts. In the first part is provided an overview of the most relevant T&D evaluation literature presenting the definition and the importance of T&D evaluation; main T&D evaluation challenges and problems; T&D evaluation models; and T&D evaluation in Arab organisations. The second part of the paper describes the findings of the study related to current T&D evaluation practice and challenges in Jordanian banks. Thus, much of the second part is also summarised on informative figures. The concluding section, the third part of the paper, succinctly integrates the two preceding parts, and in addition suggests practical solutions to the most common T&D evaluation problems found in Jordanian banks.
Theoretical Background of the Study
An Overview of T&D
T&D is the most important subsystem or element of human resource development. It concerns increasing, improving, enhancing and modifying employees’, and managers’ skills, abilities, capabilities and knowledge to enable current and future jobs to be more effectively conducted. These desirable achievements are likely to increase an individual’s as well as an organisation’s growth and performance. Generally, there are two main ways for conducting T&D programmes. First, in house T&D designed initiatives that are conducted within the organisation, which could be directly on the job experience. A second way for conducting T&D programmes is off the job or out of house T&D designed that is conducted outside the organisations through external providers who will be responsible for evaluating T&D effectiveness through many objectives. These outcomes are likely to embrace such topics as developing the skills and abilities of employees and managers to improve their performance; familiarising employees or/ and managers with new systems, procedures and methods of working; helping employees and new starters to become familiar with the requirements of particular jobs in the organisation; and guiding the organisation to improve customer service, satisfaction and total quality in the organisation.
A fundamental enigma for Jordanian organisations is the determination of adequate resources as well as a willingness to deal with the requirements for comprehensive examination of prominent issues associated with T&D. Clearly, T&D is an important and crucial activity for any organisation intending to improve the productivity and competitiveness of operations. In fact, the need for T&D has increased dramatically due to the rapidly expanding use of technology within industries and businesses, and the continuous threats of knowledge and technology obsolescence. In addition, the growing emphasis on quality and customer satisfaction has compelled organisations to recognise the importance of T&D in terms of job satisfaction, productivity and overall profitability. Whatever the type of T&D, and which ever are the dominant objectives, evaluating T&D effectiveness to achieve these primary outcomes within a mechanism that is based on systematic criteria is a very important stage in the T&D process. The next section will show why and how.
What is Training Evaluation and Why?
As organisations are spending more and more on training, evaluating the effectiveness of training has become critical. Hesseling (1966) suggests that one of the main tasks of the trainer is to test for training effectiveness and to confirm that the selected training methods have achieved the desired results. It could be argued that evaluating training effectiveness is the starting point when talking about T&D benefits and contributions in any organisation. Training is a tool used to change people’s behaviour, while evaluating training effectiveness is centred on measuring that change.
The Manpower Services Commission (MSC) (1981) defines training evaluation as the assessment of a total value and benefits of the training process, course or programme in social as well as financial terms. MSC distinguishes evaluation from validation in that it attempts to measure the overall cost benefit of the course or programme and not just the achievement of its laid down objectives. The term is also used in a general judgmental sense of the continuous monitoring of a programme or of the training function as a whole.
Training evaluation helps to collect all the descriptive and judgmental information required to make effective training decisions. Burrow and Berardinelli (2003) suggest that the evaluation stage should be viewed as part of an effective training process and a base to improve organisational decision making about human performance improvement. The evaluation activity identifies where a T&D department could demonstrate that it adds valuable service or adds value to the organisation’s performance. From training evaluation an organisation can make a judgment of what employees gain from training and what is transferred and implemented to their work. In this context Bee and Bee (1994) state that it is necessary to evaluate T&D programmes, since the evaluation process improves the efficiency and effectiveness of training programmes, the results display the organisation’s impression about trainers’ performance in the programme and helps them to improve their training skills as well as justifying the expenditure on the programmes. Therefore, it could be said, that as long as an organisation pays attention and emphasises training evaluation to identify training contributions, training is viewed by this organisation as an important organisational function that improves or helps to improve the organisational performance.
Training Evaluation Challenges and Problems
Although training evaluation is a very important phase for a successful T&D programme, this phase is the most neglected one. The caution in endorsing evaluation methods can be due to a lack of time and resources to complete the evaluation process, or may be a lack of top management support, ineffective design of the evaluation process and unclear evaluation criteria. Thus, the data is likely to be misunderstood, misused and misinterpreted (Beardwell & Holden 1994, Houlton 1996, Combs & Falletta 2000). In this regard, Hale (2003) has included two practical examples that demonstrate an insufficient evaluation process. A recent survey undertaken in the USA has shown that 57 per cent of the surveyed American organisations consider evaluation as important, but only 27 per cent use an action plan after training; just 16 per cent use ‘follow up’ from T&D and only seven per cent of the surveyed organisations evaluated the ROI in training. These findings imply that the T&D evaluation stage was not used in many organisations or in fact, after engaging the rigors of a T&D initiative the investment is not wisely used.
Measuring the gains of organisational effectiveness that result from training is probably the most difficult and problematic task of all of the training stages. It is described by Foxon (1986) as “the art of the impossible”. Also, Grove and Ostroff (1990), Redshaw (2000), and Shandler (1996) argue that evaluating training gains or influences on organisational effectiveness and performance is a very hard task for at least the following three main reasons.
- There are many other factors which can have an influence, positively or negatively, on the organisation’s performance. Such as market forces, competitive activity, new technology, legislation, and the economy (Shandler 1996, Redshaw 2000, Burrow & Berardinelli 2003), as well as luck, industry, power and size.
- The impact of training may take a considerable time to show up in overall results (Redshaw 2000).
- Training results and outcomes are usually complex and difficult to be measured or quantified, such as employee satisfaction, commitment, motivation, and changes of behaviour and attitudes. A plethora of outcomes can cause confusion between accountants and line managers who view training evaluation differently from the way trainers view the process. For instance, outputs can be compared with inputs, while the fact is that the relationship between training intervention and organisational performance cannot be easily identified.
Another important training evaluation challenge is that it is difficult to measure the relationships between training and organisational performance. A primary acknowledged difficulty is the cause and effect relationships are not necessarily straightforward and hardly ever clearly defined (Yeo 2003). For instance, learning occurs in all organisational levels thus, there are many intervening variables. And there are other organisational factors which may influence the organisational outcomes, such as leadership, workplace arrangements, and organisation structure.
To overcome most of the evaluation difficulties, Burrow and Berardinelli (2003), and Redshaw (2000) suggest that there are two important dimensions to be taken into account. First, line managers should be involved. Indeed, effective T&D programmes require top and line management support, commitment and leadership. Thus, line managers should participate in determining the training objectives and evaluation criteria, and is more likely to occur when trainers are consulting with line managers, who may ask them questions such as, what they want to improve, how they will recognise success and how they will measure this success. During this process all the other factors that influence the organisation’s performance would be identified for agreement. Second, because it is hard to connect training interventions to the organisation’s performance measurements directly, these measurements need to be revised, refined and resized to document the impact of planned learning (T&D) on organisational performance.
Training Evaluation Models
In order to understand how the T&D evaluation stage should be conducted successfully, it needs to be based on a particular T&D evaluation model. There are many training evaluation approaches and techniques. However, there are four main complementary evaluation frameworks: (a) Kirkpatrick, (b) the Bell system, (c) Parker, and (d) CIRO. However, Kirkpatrick’s evaluation model is the one most commonly used by many organisations (Hale 2003).
According to Kirkpatrick (1996), there are four stages to be considered when evaluating training effectiveness: reaction, learning, behaviour and results level. The reaction level answers whether people are happy with the training inputs (Hall 2003). It evaluates participants’ reactions, opinions, impressions and attitudes toward the programme. The learning level answers, “what do people remember from the training session?” (Hale 2003). To what extent have the participants learned the material and the particular skills or ‘know how’ that are contained in the programme? This measurement is made through special standardised tests at the end of the programme, such as pre, post, paper and pencil tests, skill practice, workshops and job simulation. The behaviour level addresses the issue of “whether people use what they know at work?” (Hale 2003), and the changes in the participants’ behaviour, skills, patterns of work, relationships and abilities that are necessary to undertake the task at hand. These indications could be collected through observations, survey, interview and comments of supervisors, and colleagues and from performance appraisal reviews. Finally, the results level determines “what are the outcomes of applications on the job over a period of time?” (Hale 2003). This level of evaluation focuses on the impact of behaviour change on the organisation’s performance. Because changing an employee’s behaviour and attitudes is not the final objective of T&D, the end results should include important elements such as, improved productivity, better quality, lower costs, more speed, fewer accidents, improved morale, lower turnover, and ultimately, more profit and better service. To determine the cost efficiency of training, recent ROI is used to establish whether the monetary values of the results exceed the cost of the programme. In this regard, Mann (1996) cited two empirical studies that reviewed the most popular evaluation levels used by USA organisations when evaluating their training programmes. Both of the studies found that most of the surveyed organisations (over half) use assessment of their participants’ reaction and satisfaction with the programme as the key methods of evaluating training.
All of the evaluation models focus on the following important criteria for determining training effectiveness.
- Stakeholder perceptions (level one: reaction)
- Learning gain (level two: learning)
- On the job performance improvement (level three: behaviour)
- Organisational improvement and ROI (level four: results).
Stakeholder reaction provides insight into trainees’ motivation and satisfaction, but does not directly measure training results. Historically, training was conducted through measuring the number of trainees, as well as their perceptions and attitude towards the programme. Nevertheless, while trainees’ attendances, perceptions, motivation and attitude towards the programme are important outcomes of any training programme, there has yet to be reported a significant relationship between these outcomes and achieving the programme objectives, including improving an organisation’s performance (Combs & Falletta 2000, Green 2001). Assessing training effectiveness based on trainees’ perceptions is described by James and Roffe (2000), and Acton and Gloden (2003) as ad hoc, unsystematic, informal and unstructured evaluations of training programmes, which tends to be post training appraisals rather than approaching the evaluation of training programmes from their design stages.
Training effectiveness no longer focuses on trainees’ perceptions, but is directed at other important factors. These factors incorporate such questions as: Did employee performance improve as a result of training? How did training contribute to achieving the organisation’s critical goals? (Burrow & Berardinelli 2003). Now there is a stronger acknowledgement for T&D results to be connected to the organisational performance measurements, such as ROI, changes in productivity, quality improvement, customer satisfaction and market share (Miller 2002). Alternative outcomes are less valued and valuable. Therefore, of the identified evaluation criteria there are two critical and essential levels that affect other evaluation levels, and they are (a) learning gain (acquired skills, competencies, attitudes and capabilities); and (b) on the job performance improvement achieved through changing trainees’ behaviour and work patterns, which is often called training transfer. Bramley and Kitson (1994) contend that the appropriate training evaluation model is the model that facilitates evaluating the third and fourth levels of the Kirkpatrick evaluation model (behaviour change and results). These training needs and objectives are defined in terms of the changed behaviour and increased effectiveness, rather than just increasing knowledge, skills and abilities. Also, top management and line managers are involved in the whole training process and are able to evaluate the changes in behaviour and effectiveness, which are occurring as a result of training.
Training Evaluation and Organisational Performance (Level Four)
The foremost strengths of a particular T&D evaluation model are especially those that emphasise the role of T&D on organisation performance and show a ROI from T&D. These perceptions are found in the Kirkpatrick evaluation model, which introduces the relationship between T&D evaluation and organisation performance. Assessing level four of the Kirkpatrick evaluation model demonstrates these important features.
Valued results are connected to the organisational performance measurements, such as ROI, changes in productivity, quality improvement, customer satisfaction and market share (Miller 2002). Continuous training programmes are essential for achieving higher productivity, better on the job performance and improved quality (Tennant, Boonkrong & Roberts 2002). These social scientists also pointed out that Japanese organisations emphasise the roles of training more than USA and European manufacturing organisations. This is revealed in the automotive sector with 380.3 training hours per year for Japanese workers, compared with 46.4 for USA workers and 173.3 for European workers. This heavy investment in T&D has led to dramatic performance improvement, in terms of cost, quality and delivery in Japanese organisations. Horwitz (1999) contends that T&D contributes to the organisation’s performance, success and competitiveness through providing the capacity to enhance individual abilities and competencies as necessary requirements for effective organisational performance. Armstrong (1989) also refers to performance related training designed to develop competencies that impact directly on the bottom line by promoting flexibility and supporting innovation. He also argues, that if the learning that is derived from training is not associated with enhanced job performance or raised capability, then the credibility of both HRD specialist and the process are at risk.
The effectiveness of T&D programmes on organisational performance improvement is a difficult, vexatious and problematic task. Thus, training professionals have been challenged to document or demonstrate the results of training on organisational performance improvement (level four) by using organisational performance impact measures, such as ROI, changes in productivity, customer satisfaction, and quality and market share. In this regard, Muhlemeyer and Clarke (1997) argue that it is difficult to measure the correlation between the implementation of T&D and the overall success of the organisation, especially in the case where the organisation’s success is measured in financial terms, because there is a little evidence to suggest that training per se can improve the financial performance.
Arguably, evaluation should focus on all organisational stakeholders, comprehensively, and on the long-term results, and should also use the most suitable information to make judgments about training effectiveness (Russ & Preskill 2001, Burrow & Berardinelli 2003). It is also suggested that evaluation could be very focused and short term or quite comprehensive with a focus on the long term results depending on the type of training and its objectives (Russ & Preskill 2001, Collins 2002, Burrow & Berardinelli 2003). Evaluation should be based on a clear identification of the purpose and results expected from the programme. Thus, if the programme is designed to respond to a particular problem, or meet a specific requirement, evaluation should be used to determine if that specific goal was successfully achieved and not focus on broader expectations. According to Brinkerhoff (1988), good evaluation is likely to provide convincing evidence that the programme is aimed at important organisational benefits. Specifically, robust evaluations are likely to lead to participants reporting greater job satisfaction, the achievement of important job reverent skills, a gaining of knowledge and improved work attitude as well as effective transference to the job of learned skills acquired from the T&D programme. However, much of the training efforts in organisations are not specifically related to organisational final outcomes.
Training Evaluation Stage in Arab Organisations
Despite widespread acknowledgement of the importance of the evaluated stage of T&D programmes this element is regularly neglected and seldom undertaken professionally. Indeed, some Arab organisations rarely conduct T&D evaluation. Often training is seen as an overhead and not an investment. When Al-Athari and Zairi (2002) conducted an empirical study in Kuwait organisations, they found that a minority of managers believed that evaluation is an important task. Moreover, a majority of organisations occasionally evaluate their training programmes through questionnaires, observation and performance records. And regarding the evaluation models, these researchers observed that most of the surveyed organisations rely on level one (trainee’s reaction) of the Kirkpatrick’s (1996) evaluation model when evaluating their T&D programmes. Al-Athari and Zairi (2002) also reported that evaluation is not considered to be the most important stage in the training cycle for most Kuwaiti organisations, which rely on evaluating the level of trainees’ reaction towards the programme. Furthermore, Abdalla and Al-Homoud (1995) argue that there are no specific ‘follow up’ procedures for evaluating the effectiveness of T&D programmes. In most Arab organisations the effectiveness of training programmes is evaluated based on the reaction level, instead of focusing on the results level and on the transferred knowledge to the workplace. In spite of these findings generally, managers endorse T&D programmes.
Atiyyah (1993) argues that evaluation methods in Arab organisations are highly subjective and their results have limited impacts on improving the on going programmes or even designing new programmes. He also argues that the programmes were usually evaluated at the reaction level via questionnaires distributed to the trainees to give their opinions on the quality of instruction, programmes materials, and suitability of the training techniques. Moreover, Abdalla and Al-Homoud (1995) contend that there are no specific ‘follow up’ procedures for evaluating the effectiveness of T&D programmes, as in most Arab organisations the effectiveness of training programmes are evaluated based on the reaction level, instead of focusing on the results of training and the transferred knowledge to the workplace, which are considered, according to Kirkpatrick, as the best evaluation systems, and focus on the effects of the application of information and learned concepts on the organisation performance. The most common evaluation challenges in Arab organisations are the cost of conducting this process, difficulty in finding evaluation methods, difficulty in finding evaluation quantitative financially criteria or language, time required to accomplish this process and lack of information needed for evaluation.
Participants and Site
The population of the present study is defined as all the banking organisations operating in Jordan that were listed and licensed as banks at the Association of Banks in Jordan, Central Bank of Jordan according to their reports for the year 2004 and listed also in Amman Stock Exchange (ASE) as banking organisations. Therefore, the targeted organisations reflect the whole population rather than a sample of the population. The main reason for choosing the entire population is to ensure that the sample is representative and not biased. Appendix 1 shows the name, groups and the number of the banking organisations. In total 22 banks were listed as banks in 2003/2004 and all were in this study. There were 14 commercial banks (of which five are foreign bank branches), six investment banks and two Islamic banks.
The rationales behind targeting these organisations as the subject of this study refer to the important role they play in enhancing Jordanian economic performance. The banking sector, which is a very active sector in Jordan, is one of the main pillars of the national wealth and it is ‘pushing’ the whole economy forward. Moreover, banking organisations in Jordan are known for their commitment to develop their employees to offer better quality customer services, these financial institutions pay more attention to T&D and expend more effort and money in T&D than any other Jordanian organisations. Due to the importance of human resources in economic growth and social development, Jordanian banking organisations pay more attention to improving the performance of their employees, as well as their abilities, education and professional skills, through continuous T&D and updating them with changes and development, so that they can react properly to the process of economic growth to enable the banking sector to better face the challenges of the coming decade.
A purposive or judgmental sampling technique was applied in determining the participants that will best enable the researcher to answer the research questions and to meet the research objectives. Thus, the study targeted the banking organisations T&D, HRM and top managers. These people were chosen to be the respondents of this study because they are the people who have the required information about T&D issues, management, problems and its importance for organisational success, and also for their critical roles in their banks.
The rational behind choosing top corporate managers in addition to HRM and T&D managers as participants in this study is related to the fact that top managers are considered by many researchers in the field of management T&D as being the most important group in the organisation to secure the success for T&D functions/programmes. Their support, commitment, opinions and attitudes towards the importance of T&D functions/programmes in the organisation decide the success of these functions/programmes. In addition, these managers are the key influential decision makers in the organisation. They are the key strategy makers. Moreover, their appreciation and admiration of the benefits of T&D, which are very important indications that T&D really is working, will guarantee their commitment and support to this important function. The term ‘top managers’ used in this study reflects those who are on the board of directors or who report to the managing director (e.g., Chairman, CEO, GM, MD, Deputy MD/GM, Director, and Executive Manager).
This study was conducted in the headquarters of the banking organisations with T&D, HRM and top managers rather than targeting the banks branches because all Jordanian banking organisations have headquarters in Amman and many other branches in Amman and other cities in Jordan. However, in most of these branches there is no T&D department/section or unit and in many branches there is no section or unit for HRM. Therefore, the banks’ branches do not have sufficient information about T&D. All T&D decisions and issues are formulated in, or centralised to, the banks’ headquarters in Amman. Thus, the headquarters are the main source of the desired data and information about T&D in each bank; in these headquarters T&D managers and/or HRM/HR managers are responsible for T&D issues, rather than any other managers.
In this study, primary data as well as secondary sources of data are used. For the primary data this study employed the multi methods approach (method triangulation) conducted through survey self administered questionnaires and semi structured interviews in a complementary, and supplementary way, rather than in competition with each other, that was an appropriate and flexible way to conduct this research.
Employing triangulation in one study has a number of advantages. For instance, the technique provides a kind of convergence of results that complements findings reached from analysing various observations and enhances the scope and breadth of a study. Triangulation increases validity because it ensures that the variable variance is attributed to the trait of the subject examined rather than to the method used for investigation. In short, triangulation consists of crosschecking data for internal consistency and external validity, which are matters of concern for any study. The small study sample encourages the use of triangulation.
The logic behind using the multi method approach as a research strategy for the current study refers to two main reasons. First, the small size of the study sample and population necessitated collecting as much rich and deep data as feasible by both qualitative and quantitative data collection methods. The qualitative responses in this study elucidated the meaning of the quantitative results (the sample size of N = 38 is too small to make confidence predictions). The second reason is that the research includes social behavioural factors, beliefs and attitudes that need to be explored and explained in detail. Therefore, applying the chosen research design helps the researcher to address all qualitative and quantitative data that provide more flexibility to meet multiple research interests and needs. Using different quantitative and qualitative data collection methods in one study facilitates the generation of richer and deeper facts and data that have potential to enrich the study findings.
The self administered questionnaire, has a number of benefits. These benefits have been identified as a prompt and relatively low cost strategy for obtaining information in a context that is likely to establish a good rapport with respondents. The rationale for using semi structured interviews in this study is very successful in Arab organisations, where managers prefer to talk rather than to complete a questionnaire. Thus, it was decided that top managers would be interviewed, rather than asked to complete a questionnaire on which they may be reluctant to spend time providing written explanatory answers. In addition, from the outset the researcher was convinced of the value of using face-to-face interviews because of their advantages in exploring a wide range of issues related to the research questions. In addition, they complement the questionnaires and can explore or explain, indepth, any further details, information, themes and facts under investigation behind the questionnaire’s responses.
A total of 44 questionnaires were distributed by the researcher in late March 2004 to the HRM and T&D managers. An initial stage of the study procedures was a personal visit to Amman Financial Market and the Association of Banks in Jordan to get the organisations’ addresses. This action was followed by personal visits to each organisation’s headquarter by the researcher to introduce herself, to outline the boundary of the study, and to build a strong trust with the executive decision makers. This kind of visit is very important to build trust with participants, to encourage them to participate and to determine a deadline for completing the questionnaires. In addition, through these visits the researcher was able to determine the name of the persons to be interviewed. An informal or an unplanned discussion relating to T&D practices and problems in Jordan accrued during these visits. Moreover, appointments to distribute the questionnaires and to conduct the interviews were decided during these visits.
A number of actions were undertaken to optimise the collection of questionnaire data. At each visit the researcher delivered the questionnaire to the participants’ offices to make sure that the questionnaire would be completed by them. A deadline for completing the questionnaire was identified by asking the participant when s/he would be able to complete the document. The researcher left a phone number to advise when the questionnaire was ready to be collected, and during the completion period the researcher phoned the participants to ask whether or not they had finished. When collecting the questionnaire the researcher made sure that she saw the participants who had completed the questionnaire, and usually, she had the opportunities to have another conversation (chat) about the questionnaire contents or any other comments about the questionnaire. Some participants, after completing the questionnaire, offered more documentary data which meant that they were interested and understood the questionnaire.
The personal letter (covering letter) from the researcher urging the participants to cooperate with the researcher was very useful to get the participants’ assistance in completing the questionnaire. In this letter the researcher stressed that the information provided by the participants would be treated with the utmost confidentiality and no personal or organisations’ names would be mentioned in the report of the study. The researcher also stated in her letter that a brief report of the final analysis would be given to the participant if requested.
Appointments with top managers to conduct semi structured interviews were set up by the researcher’s effort through visiting the secretaries’ offices and building trust with them to set appointments with their bosses, or by questionnaire participants in each organisation. However, ‘follow up’ telephone calls were made to the secretaries’ offices to secure the appointments with the top managers. All the interviews in this study were conducted in the respondents’ offices either in the morning or in the afternoon. Nevertheless, the interviewees chose the place and the time of interviews, since they were so busy it was a good decision to conduct the interviews at times convenient to them. The schedule allowed the researcher to spend between one and two hours maximum with each respondent. The questions were mainly open ended with some close ended questions. In conducting the semi structured interviews, the researcher took notes in addition to recording the whole interview in order to remember all the details. In each interview the researcher thanked the respondents for their time, gave a brief overview of the research provided and exchanged business cards. The researcher assured utmost confidentially of the provided data. Finally, it is worth mentioning that close questions were written on cards to be provided to the interviewees giving them the opportunity to read these questions to avoid any kind of bias.
A total of 38 questionnaires were returned which was about 86 per cent response rate and 15 interviews were conducted by the end of July 2004. The reason for the realtively high response rate may be due to the fact that the questionnaire were distributed and collected by the researcher in addition to her ‘follow up’ letter and visits to the banks. A total of four questionnaires were returned from two foreign banks with a letter explaining that their bank policy does not allow them respond to surveys and two questionnaires were unusable because of missing data.
In sum, 22 banks were the population of the current study, 44 questionnaires targeted T&D and HRM managers in each bank and 15 semi structured interviews were done with 15 top managers. Despite the intention to interview 22 top managers (of whom 15 responded). The inputs from the executive management can be considered as favourable. The other managers declined to take part in this study because they were too busy, or because their banks’ policies do not allow them to give any information about T&D policies and management.
This study employed two primary data collection procedures (a) self administered questionnaires distributed to T&D and HRM managers, and (b) semi structured interviews conducted with top managers. The items included in the questionnaire were adapted from previous research studies (Garavan 1991, Albahussain 2000, Al-Atheri 2000, Al-Athari & Zairi 2002, Tennant, et al. 2002) that were related to T&D and HRM. The semi structured interviews assessed four dimensions of T&D evaluation. These elements were (a) the position of T&D evaluation in the organisations, (b) the significance of T&D evaluation stage, (c) T&D evaluation models and criteria, and (d) T&D evaluation problems and obstacles. Appendix 2 shows the items that were employed in the questionnaire to obtain the self administered survey data and the questions of the semi structured interviews.
The self administered questionnaire has six main sections. The first section contains one item measuring the importance of T&D evaluation stage in Jordanian banks using a five point scale (that ranged from 1 = Not at all important, to 5 = very important). The second section contains an item that measures the frequency of conducting T&D evaluation using a five point scale (that ranged from 1 = never to 5 = always). The third section of the questionnaire has five items that measure evaluation methods and techniques using a five point scale (that ranged from 1 = never to 5 = always). The fourth and fifth sections were about T&D evaluation models and criteria that were assessed with a five point scale (that ranged from 1 = to a very small extent to 5 = to a very great extent). Finally, the last section contains 12 items asking for T&D evaluation challenges and problems in Jordanian banks using a five point Likert scale (that ranged from 1 = strongly disagree to 5 = strongly agree).
The qualitative data of the interviews was evaluated by content analysis. Initially, the data are partitioned. This step is followed by recognising relationships and developing categories by designing or reorganising the data in a suitable matrix and placing the information within cells.
Descriptive statistics include frequencies, measure of central tendency (mean, median and mode) and measure of dispersion (range, Standard deviation) was the most appropriate statistic for analysing questionnaire questions. In addition, Kruskal-Wallis H test and Chi-Square were used to establish the difference among bank groups.
This section presents the results in two parts. First, is given findings from the questionnaire responses (quantitative data). Second, the results of the responses from the interviews (qualitative data) are delineated.
In this study the four Jordanian banking groups participated in this study as shown in Table 1. Commercial banks represent the majority of the participating banks (47.4 per cent). All of the investment and the Islamic banks participated in this study; they represented 26.3 per cent and 10.5 per cent of the study participants, respectively. Three of the foreign banks agreed to participate, which represented 16 per cent of the overall participants.
|Bank category||Job titles|
|Commercial||47.4||Manager of T&D department||50.0|
|Foreign||15.8||Manager of HRM||50.0|
|Participant status||Work experiences (years)|
Table 1 also shows that all the questionnaire participants were the managers of T&D or those who were in charge of T&D activities in each bank, in addition to the managers of HRM and HR department in each bank. A total of 66 per cent of them categorised their job status in the middle management levels, as functional managers or senior non board managers in their organisations. However, the Chi-Square test showed the following result: χ2 = 21.235, df = 9, sig = .012, p < 0.05, which indicates that there was a significant difference among bank categories in terms of participants’ status in the banks, ranking foreign and commercial banks in the first and second levels, which means that people responsible for T&D were in the higher status within foreign and commercial banks. A further feature of the data is that 68 per cent of the participants had work experience in the field of T&D of between one and seven years. The overall average work experience in the field of T&D in the Jordanian banks was just 8.4 years. The Kruskal Wallis H test result was χ2 = 3.504, df = 3, sig = .320, p < 0.05, which implies that there was a significant difference among bank groups in terms of their participants’ work experience in the field of T&D, ranking the commercial banks and the Islamic banks T&D work experience in the first and second levels. Indicating that the people who were in charge of T&D issues in the commercial and Islamic banks were more experienced in the field of T&D than in other bank groups.
Table 2 lists mean scores and standard deviations for the other items used in this study described in the measure sub section and shown in Appendix 2. Regarding to the importance of T&D evaluation stage in the participated organisations, Table 2 shows that most of the participants asserted that their organisations viewed training evaluation as a very important stage, (mean of 4.52). The Kruskal-Wallis H test shows that there was a non significant difference among bank groups in terms of whether they view training evaluation as important or not (χ2 = 6.553, df = 3, sig = .088, p > 0.05). In addition, Table 2 shows that most of the participants believed that their organisations evaluate their T&D programmes on a regular basis, with a non significant difference among bank groups in terms of the frequencies of conducting training evaluation (χ2 = 6.553, df = 3, sig = .088, p > 0.05).
|How important is measurement training evaluation in your bank||4.52||.60|
|How frequently does your bank evaluate training and development||4.23||.88|
|Filling in a questionnaire||4.47||.95|
|Interviewing the trainees at the end of programmes||2.61||.75|
|Testing the trainees before and after the programme||2.37||.85|
|Asking trainees‘ managers or supervisors for their assessment of the trainees’ learning||3.82||.98|
|Employees’ performance records||3.71||.96|
|Evaluation difficulties and challenges|
|Difficulties in measuring improvement in a certain jobs||3.05||1.06|
|Difficulties in measuring the change of trainees’ behaviours over short period of time||3.71||.73|
|Lack of quantitative measures||4.03||.49|
|High cost evaluation process||2.66||.94|
|Training outcomes are subjective||3.11||1.01|
|Lack of knowledge about evaluation process||2.71||1.25|
|Absence of job description||2.56||1.18|
|Time required to do evaluation||2.84||1.08|
|Difficulties to separate training impact on the final results from other actives impacts||3.50||.80|
|No specific body in charge of evaluation||2.90||1.20|
|Difficulties in getting managers participate in evaluation process||3.29||1.04|
|T&D objectives are not clear||2.68||1.37|
Note: Std. = Standard deviation of the mean.
In terms of the T&D evaluation methods and models in Jordanian banks Table 2 shows that most of the participants agreed that completing a questionnaire at the end of any training programmes was the most frequent used evaluation method (mean 4.47). This result compares favourably with a study that was conducted by Al-Ali (1999) who also found that the most commonly used evaluation method by Kuwaiti organisations was the questionnaire. In addition, about 40 per cent of the participants declared that they almost rarely tested the trainees before and after any training programmes, while 16 per cent said they never did. Table 2 also shows a mean score of 3.47 for the Kirkpatrick evaluation model, which is used to a considerable extent in the organisations as revealed by the non significant difference among bank groups in the mean ranking frequencies of using the Kirkpatrick evaluation model (χ2 = . 545, df = 3, sig = . 959, p < 0.05).
Table 2 shows that most of the participants stated, that when evaluating the effectiveness of T&D programmes, they depend firstly and greatly on trainees’ reaction level given an average of 4.50. This observation suggests that trainees reaction is the level which is focused on to a very great extent when evaluating T&D programmes. In addition, 63 per cent of the respondents affirmed that they depend greatly on evaluating learning outcomes level, while 50 per cent of the participants agreed that their organisations depend to some extent on behavioural level when evaluating T&D effectiveness, yet the Results had a value of 2.39, which indicated that this level is neglected when evaluating T&D programmes.
Table 2 also shows that most of the participants agreed T&D evaluation and ‘follow up’ stages face many difficulties and challenges. The prominent items were (a) lack of quantitative measures (4.03), (b) difficulties in measuring the change of trainees’ behaviours over a short period of time (3.71), (c) difficulties in separating the impact of training on the final results from other activities impacts (3.50), and (d) difficulties in getting managers to participate in the evaluation process (3.29). Alternatively, participants disagreed that the absence of job description and the lack of knowledge about evaluation were difficulties facing evaluation process in the Jordanian banks. These findings regarding to T&D difficulties in the Jordanian banks are consistent with other research findings (Atiyyah 1993, Abdalla & Al-Homoud 1995, Al-Athari & Zairi 2002).
A total of 15 executive managers provided qualitative data. Six top managers were from six commercial banks, one manager was from a foreign bank, and a further six top managers were from investment banks, and two top managers were from the two Jordanian Islamic banks. All of the interviewed managers believed that T&D evaluation is a very important stage. For instance, two of them stated,
“Training is a long term investment. We spend a lot of money and time on it, thus we need to evaluate its results …” (Commercial bank)
“We actually spend a lot of money on T&D every year. We offer all facilities for T&D activities, but it seems that these programmes do not work and we do not know why?” (Islamic bank)
All of the interviewed top managers declared that they evaluate their T&D regularly. However, 13 top managers stated that external providers conduct most of the T&D programmes outside the banks. And furthermore, these banks depend on those external providers to do the evaluation for them. In this context two of the interviewed managers said,
“T&D evaluation is conducted by the external centre which conducts T&D programmes and I think they do it through questionnaire.” (Commercial bank)
“Thus, after attending T&D programmes, our employees came back to their workplace trained and evaluated. Actually, we indeed, trust those external providers because they have long work experiences, they are specialists or professionals.” (Investment bank)
The top managers held different perspectives about how T&D programmes were evaluated. Some managers stated that in most times external providers evaluated their T&D programmes, but most of the managers were not sure whether those providers depend on the Kirkpatrick model or not. However, when they were asked about T&D evaluation methods, criteria or levels, most of them knew that external providers evaluate T&D programmes through distribution of a particular questionnaire for each trainee at the end of training session asking for their opinions of the programmes. Also, the evaluations of the T&D programmes ask trainers to fill in a particular questionnaire and to evaluate the course as well as the trainees. Some top managers said trainers observe the trainees rather than filling in a questionnaire. In respect to this matter one comment was,
“In evaluating the trainees we depend on their reaction toward the T&D course in addition to asking the trainers about the trainees, what is more we sometimes ask those trainees to do a presentation for what he/she learnt from the course, but of course that is not always.” (Commercial bank)
Finally, managers were asked about what were T&D evaluation challenges and problems. The responses from these managers showed a belief there were two important challenges facing evaluation process, which were 1) lack of qualified internal T&D staff; and 2) difficulties in measuring T&D outcomes. But the managers did not consider leaving external providers to evaluate their T&D programmes was a substantive issue.
The results of this study provide some disparate findings in terms of evaluated dimensions of T&D programmes in Jordanian banks. Despite a considerable extent of projection by a great number of commentators (Garavan 1991, Bee & Bee 1994, Taylor 1996, Harrison 1997, Redshaw 2000, Russ & Preskill 2001, Burrow & Berardinelli 2003, Hale 2003), who have evaluated procedures that show there are responsible elements of T&D initiatives, the study findings disclose, that many managers (of these banks) have an ambivalence about this part of the T&D programmes. Indeed, often bank policy is to outsource the activity and then to defer responsibility for the evaluation of the proceedings to the same body that is administering the T&D programmes.
In order to understand how the evaluation stage is considered and conducted in Jordanian banks, the study targeted all Jordanian banks. Within these banks those people who are in charge of the T&D programmes were the study respondents. These respondents were asked about the level of importance of the evaluation process in their organisations, the regularity of conducting evaluation, the evaluation methods and techniques used in their organisations, the evaluation model and levels of outcomes they depend on when evaluating T&D programmes and the challenges and difficulties that face of evaluating the T&D programmes, conducted by their banks. Also, executive managers were asked about the evaluation stage in many contexts during their interviews.
Interestingly, it was found, that all participating banks represented by (T&D and HRM managers in each bank, the questionnaire participants) believed the T&D evaluation is an important stage. Therefore, the majority of Jordanian banks conducted T&D evaluation on a regular basis. These findings, together with earlier literature suggest that in Jordanian banks T&D is conducted more frequently than in other Arab organisations. In fact, earlier Al-Athari (2000) found that although the majority of the government and private Kuwaiti organisations usually conducted T&D evaluation, 35 per cent of government and 19 per cent of private organisations rarely evaluate their T&D programmes. In addition, Albahussain (2000) found that just nine per cent of the medium sized and 31 per cent of the large sized surveyed Saudi organisations conducted evaluation on a regular basis, whereas the majority of organisations conduct evaluative procedures only sometime. Consequently, it might be inferred that Jordanian banks believed in the importance of T&D evaluation and conducted these procedures regularly.
The study revealed the most commonly used evaluation method by Jordanian banks is the questionnaire. According to T&D and HRM managers, questionnaires are used to measure perceptual responses from trainees’ managers or supervisors for their observation and assessment of the trainees’ learning as a result of attending training programmes. Interviewing the trainees was occasionally used as an evaluation method, while testing the trainees before and after the programme was the most rarely used evaluation technique. Interestingly, it was found from the interviews that some commercial banks had their own ways of evaluating T&D programmes. One way was by sending particular customers to particular branches to observe how employees conduct their jobs and see how they serve the customers. Another manager said they ask the trainees to do presentations of what they get from T&D programmes. Yet another manager refused to talk about their own ways of evaluating T&D outcomes since they consider them highly confidential issues. However, all of these methods of evaluation are subjective in nature and their assessment value for T&D effectiveness is subject to personal aspects and preferences.
The interviewed top managers declared that most external T&D programmes are evaluated by the external providers. Wilkins (2001) argues, that in Arab countries, educational institutions are not generally in the position to assess workplace performance so must find other ways to measure the effectiveness of their programmes. In Jordanian banks they know that external providers evaluate T&D programmes through distributing a particular questionnaire for each trainee asking for their opinions of the programmes. Another way to evaluate external T&D programmes, in addition to asking the external providers to evaluate them, is to ask the trainees’ direct managers about the trainees’ performance after they have finished the programmes. In addition, asking external providers to evaluate T&D programmes reflects that Jordanian banks do not know exactly what they want from a T&D programme.
The choice of an evaluation model to determine the effectiveness of a T&D programme is critical. An evaluation model is a systematic way to benchmark the organisation’s training activity to find out how much the organisation benefited from the T&D programme, what the employees’ reaction is, what they learned and how much they learned and if they improved their performance, work related behaviours and attitudes. In Jordanian banks this study found that most of the participants (the people who are in charge of T&D) were not familiar with the international evaluation models that were provided. However, when the researcher defined each model and identified the criteria for each evaluation model, the majority of them thought they used the Kirkpatrick model criteria when evaluating their in house T&D programmes. The unfamiliarity of the evaluation models reflects how much they know about how to conduct an effective evaluation process by using a systematic evaluation model. The shortage of knowledge of the evaluation models could be explained by many reasons. The over dependency on the external providers to evaluate a T&D course, lack of well qualified T&D staff who are supposed to be familiar with, and knowledge about evaluation issues, the lack of a T&D department or section in most of the banks, and finally, because some banks found their own way of evaluating their T&D programmes.
Results of the study showed the most common level of evaluation for Jordanian banks was the trainee’s reaction level. However, some of the commercial banks concentrate on the behavioural changes and the Results outcomes. All of the interviewed top managers disagreed that they focused on cost effectiveness training evaluation, because investing in training is a long term investment with fewer tangible and complex benefits. Nevertheless, it is likely that some successful large commercial banks have recognised the importance of measuring the benefits of T&D.
The study findings have a degree of concurrence with previous research findings. The most commonly used evaluation method was found to be the questionnaire, which included trainee’s evaluation of T&D programmes, which is often assessed on a ‘smile sheet’. In this context, Robinson and Robinson (1989) state that almost all HRD professionals provide end of course questionnaires that are completed by participants and given to the trainer. However, these reaction evaluations, frequently, are poorly designed and yield minimally useful information. Finally, in terms of evaluation problems and challenges in Jordanian banks it was found that in this study and according to the people who were in charge of T&D, the evaluation stage faced some difficulties and challenges. These perplexing issues can manifest as the lack of quantitative measures, difficulties in measuring the change of trainees’ behaviours over a short period of time, difficulties in separating the impacts of training on organisational final results from other the impacts of other organisational activities, and difficulties in getting managers to participate in the evaluation process. Interestingly, participants disagreed that the absence of job description and lack of knowledge in the evaluation process, were difficulties for the evaluation stage in Jordanian banks. Moreover, the interviewed banks’ top managers revealed some challenges and problems related to the evaluation process. They said that frequently there were no specific qualified people to evaluate T&D effectiveness. Thus, the responsibility for training evaluation was formally left to the training coordinators or external providers, who usually evaluate the programmes just for reporting to the banks.
This study examined the current management T&D evaluation practices in Jordanian banks; and identified the most important concerns, problems, and challenges that are facing the T&D evaluation stage in these organisations. Investigating the Jordanian banking industry, which has not been targeted before, within a caveat that T&D practices are assumed to be managed better than in any other Jordanian industry has potential to contribute relevant knowledge to the existing literature. The research findings were strengthened by employing interviews to elucidate the questionnaire data. Arguably, this study shows that the T&D evaluation stage in the Jordanian banking industry is not in a better situation than in other Arab organisations. Also, there is a lack of sustainable evidence that the T&D evaluation stage in the Jordanian banking industry is likely to be conducted differently than when the activity is undertaken in other Arab organisations. Furthermore, there is an absence of argument that T&D in the Jordanian banking industry (as services organisations) should be managed differently from other Arab organisations.
The study revealed that although all Jordanian banks believe in the importance of the evaluation process they conduct somewhat limited evaluation processes for their in house T&D programmes. Despite regular evaluations of the T&D programmes these assessments are constrained to the use of questionnaires, asking trainees’ managers or supervisors about their observation and assessments of trainees’ learning. These methods of evaluating trainees are subjective in nature, and indeed, supervisors’ assessment for T&D effectiveness can be highly personalised.
In terms of the range of evaluation frameworks the Kirkpatrick evaluation model is mainly used by Jordanian banks. Nevertheless, they depend to a very large extent on the reaction level, which is about the trainees’ reaction, opinions, attitudes and satisfaction about the T&D programmes, followed by learning outcomes which they evaluate through asking the trainees’ direct managers for their assessment of the trainees’ learning. Thus, evaluation is mainly based on subjective assessments. These evaluations range across an individual’s perceptions, opinions and attitudes toward T&D outcomes, rather than focusing on behavioural changes, organisational relevant work practices and operational improvements. For practitioners the revealed facts and findings have implications for contemporary banking organisations. For instance, coordination between Jordanian banks and external providers in managing their T&D programmes, finding specific qualified people to be in charge of evaluation and/or educating, and developing the people who are currently in charge of T&D evaluation process. In addition, T&D evaluation should not just be based on trainees’ reactions, but rather it should be based on learning and behavioural outcomes, which could be gained from observing the trainees’ performance after they complete T&D programmes. Two important behavioural outcomes that might be observed are whether they transfer what they learn from T&D into their workplaces; and the extent to which on the job performance was improved.
Training evaluation faces many difficulties. A lack of quantitative measures, difficulty in measuring the change of trainees’ behaviour over a short period of time, difficulty in separating training impact on the final results from the impacts of other activities, difficulty in getting managers to participate in the evaluation process, lack of knowledge about evaluation, lack of job description, and time required to do evaluation are some other difficulties. These issues are exacerbated by a lack of well qualified people who are in charge of T&D evaluation, lack of knowledge about the evaluation process and over dependency on external providers to evaluate the T&D programmes.
From a practical point of view, this research has provided several major managerial contributions. As many organisations have suffered from a low success rate with their T&D activities and most of them feel that T&D does not work, this study has provided useful guidelines in the form of the critical elements and factors that can enhance success in T&D. The study results provide assessments for the effectiveness of the current practices relating to the evaluation process of T&D and reflect many of the related problems and challenges. From this study T&D professionals have an opportunity to derive a better understanding of the role of their T&D activities in the development of the organisations, especially in today’s business world, which is characterised by ever increasing competition, globalisation and change. The findings of this study may guide T&D professionals to understand how they can better manage their T&D activities, what they need to focus on, and what strategies may be employed to improve T&D effectiveness.
The conclusions and implications derived from this study remain tentative until such time as they are confirmed by similar studies using identical or alternative research methodologies. A refinement of this work would consist of verifying that all relevant and important aspects of T&D have been covered thus, providing other researchers with an opportunity to offer a more complete picture of the situation as it stands. Arguably, the boundary conditions of this study could be viewed as opportunities for future research.
Based on the results of this study suggestions are given for organisations seeking to improve their T&D evaluation stage. For instance, more attention and effort is required when evaluating T&D programmes. Importantly, there is a need to evaluate the outcomes of external T&D programmes rather than asking the external provider to perform the examination phase. Organisations are encouraged to use a systematic evaluation model. For more effective benefits an emphasis might be to consider the Kirkpatrick evaluation model outcome levels instead of an overemphasis on the reaction level, which do not provide or reflect much about T&D effectiveness. Finally, evaluation needs to be conducted based on the objectives of the planned T&D programmes. The paradox for senior executives of the Jordanian banking sector is how to become enlightened about the evaluative parameters of T&D programmes that provide credible underpinning for generating effectual T&D initiatives and valued outcomes.
, PhD, is an assistant professor in Al Hussien Bin Talal University, department of business, business and economics school, Jordan. She did her Master and PhD degrees in UK. She has particular research interests in assessment and planning human resource management, training and development, women leadership, job satisfaction and performance appraisal.
Emails : email@example.com; firstname.lastname@example.org
Abdalla, I., & Al-Homoud, M. (1995). A Survey of management training and development practices in the state of Kuwait. Journal of Management Development, 14(3), 14-25.
Abdalla, H., Maghrabi, A., & Raggad, B. (1998). Assessing the perceptions of human resource managers toward nepotism: A cross-cultural study. International Journal of Manpower, 19(8), 554-570.
Acton, T., & Golden, W. (2003). Training the knowledge worker: a descriptive study of training practices in Irish software companies. Journal of European Industrial Training, 27(4), 137-146.
Al-Athari, A. (2000). The impact of multimedia based training on employee training effectiveness and organisational performance. Unpublished PhD Thesis, Bradford University.
Al-Athari, A., & Zairi, M. (2002). Training evaluation: An empirical study in Kuwait. Journal of European Industrial Training, 26(5), 241-251.
Al-Bahussain, A. (2000). Human resource development: An investigation into the nature and extent of training and development in the Saudi private manufacturing. Unpublished PhD thesis, Bradford University.
Altarawneh, I. (2005). Training and development effectiveness: Practices, roles and impacts on performance in Jordanian banking organisations. Unpublished PhD thesis, Huddersfield University, UK.
Atiyyah, H. (1993). Management development in Arab countries: The challenges of the 1990s. Journal of Management Development, 12(1), 3-12.
Armstrong, M. (1989). Personnel and the bottom line. London: Institute of Personnel Management.
Bahar, A., Peterson, S., & Taylor, D. (1996). Managing training and development in Bahrain: The influence of culture. Journal of Managerial Psychology, 11(5), 26-32.
Beardwell, I., & Holden, L. (1994). Human resource management: A contemporary perspective. London: Pitman Publishing.
Bee, F., & Bee, R. (1994). Training needs analysis and evaluation. London: Institute of Personnel Management.
Bramley, P., & Kitson, B. (1994). Evaluating training against business criteria. Journal of European Industrial Training, 18(1), 10-14.
Brinkerhoff, R. (1988). Achieving results from training. San Francisco: Jossey-Bass.
Burrow J., & Berardinelli, P. (2003). Systematic performance improvement-refining the space between learning and results. Journal of Workplace Learning, 15(1), 6-13.
Combs, W., & Falletta, S. (2000). The targeted evaluation process. Alexandria, VA: American Society for Training and Development.
Collins, D. (2002). Performance-level evaluation methods used in management development studies from 1986-2000. Human Resource Development Review, 1(1), 91-110.
Foxon, M. (1986). Evaluation of training: The art of the impossible. Training Officer, May, 133-137.
Garavan, N. (1991). Strategic human resource development. Journal of European Industrial Training, 15(1), 17-30.
Green, E. (2001). The influence of individual and work environment characteristics on trainee motivation and training effectiveness measures. Unpublished Doctoral Dissertation, North Carolina State University, Raleigh, NC.
Grove, A., & Ostroff, C. (1990). Training programme evaluation. In Wexley, N., & Hinrichs, R., Developing human resources, ASPA/BNA series, Washington, DC: Bureau of National Affairs.
Hale, R. (2003). How training can add real value to the business. Industrial and Commercial Training, 35(2), 49-52.
Harrison, R. (1997). Employee development. London: Institute of Personnel and Development.
Horwitz, M. (1999). The emergence of strategic training and development: The current state of play. Journal of European Industrial Training, 23(4)180-190.
Houlton, E. (1996). The flawed four-level evaluation model. Human Resource Development Quarterly, 7(1), 5-22.
James, C., & Roffe, I. (2000). The evaluation of goal and goal-free training innovation. Journal of European Industrial Training, 24(1), 12-20.
Kirkpatrick, D. (1996). Four levels of evaluation. Journal of American Society for Training and Development, 33(2), 78-92.
Mann, S. (1996). What should training evaluations evaluate? Journal of European Industrial Training, 20(9), 14-20.
Manpower Service Commission (1981). Glossary of training term. London: HMSO.
Miller, L., (2002). Perception of training and non-training managers of organisational impact measures based on design intent. Unpublished Doctoral Dissertation, North Carolina State University.
Muhlemeyer, P., & Clarke, M.(1997). The competitive factor: Training and development as a strategic management task. Journal of Workplace Learning, 9(1), 4-11.
Redshaw, B. (2000). Evaluating organisational effectiveness. Industrial and Commercial Training, 32(7), 245-248.
Robinson, G., & Robinson, C. (1989). Training for impact. California: Jossey-Bass.
Russ, D., & Preskill, H. (2001). Evaluation in organisation. Cambridge, MA: Pereseus Publishing.
Shandler, D. (1996). Reengineering the training function. Delray Beach: St Lucie Press.
Tennant, C., Boonkrong, M., & Roberts, P. (2002). The design of a training programme measurement model. Journal of European Industrial Training, 26(5), 230-240.
Taylor, D. (1996). Training and developing people. In Molander C. Human resource at work (26-32). London: Chartwell Bratt.
Wilkins, S. (2001). Management development in the Arab Gulf States: The influence of language and culture. Industrial and Commercial Training, 33(7), 260-266.
Yeo, R. (2003). Linking organisational learning to organisational performance and success: Singapore case studies. Leadership and Organisation Development Journal, 24(2), 70-83.
|A. Commercial banks
Arab Bank PLC
Arab Banking Corporation (Jordan)
Bank of Jordan PLC
Cairo Amman Bank
Export & Finance Bank
Jordan Gulf Bank
Jordan Kuwait Bank
Jordan National Bank PLC
The Housing Bank for Trade & Finance
|Commercial banks perform general financial activities, such as: financing working capital, term finance, import and export activities, and deposit services.
They also allocate and invest financial resources in a variety of economic sectors.
Most of the operating banks are commercial banks.
|B. Foreign banks
Standard Chartered Grindlays Bank
Egyptian Arab Land Bank
HSBC Bank Middle East
|Foreign banks are branches for foreign international banks investing in Jordan, but still directed by foreign management.
Two of them are British, one American, one Egyptian and one Iraqi.
All of them are commercial in nature.
|C. Islamic banks
Islamic International Arab Bank PLC
Jordan Islamic Bank for Finance and Investment
|Islamic banks are characterised by an approach that avoids the payment and receipt of interest.|
|D. Investment banks
Arab Jordan Investment Bank
Jordan Investment and Finance Bank
Philadelphia Investment Bank
Union Bank for Saving & Investment
Industrial Development Bank
Middle East Investment Bank
Societe Generale de Banque au Liban
|Investment banks offer long term lending to a wide variety of economic sectors.
They finance equity for start-ups, and thus, contribute to enlarging business activities in Jordan.
Source: The Association of Banks in Jordan, Central Bank of Jordan, 2004.
Study Questionnaire and Semi Structured Interview Schedule with Top Managers
Section A: Evaluation and follow up
A1. How important is measurement training evaluation in your bank?
|[ ] Not at all important||[ ] Relatively unimportant||[ ] Somewhat important||[ ] Relatively important||[ ] Very important|
A2. Does your bank evaluate training and development programmes?
|[ ] Never||[ ] Rarely||[ ] Sometimes||[ ] Mostly||[ ] Always|
A3. If your bank evaluates its training and development programmes, what methods are usually used? Please tick (√) in the box that best reflects your answer where:
|1 = Never||2 = Rarely||3 = Sometimes||4 = Mostly||5 = Always|
|Evaluation tools and techniques||1||2||3||4||5|
|Asking employees to fill a questionnaire at the end of the programme|
|Interviewing the trainees at the end of each training programme|
|Testing the trainees before and after the training programmes (pre and post test)|
|Asking the trainees’ managers or supervisors for their assessment of the trainees’ learning|
|Performance appraisal reports|
|Others, please specify............................................................|
A4. Please indicate which of the following training evaluation models does your bank use when evaluating training and development programmes? Please tick (√) in the box that best reflects your answer where:
|1 = To a very small extent||2 = To a small extent||3 = To a considerable extent||4 = To a great extent||5 = To a very great extent|
|Kirkpatrick (trainees’ reaction; learning outcomes; behaviour outcomes, results the impact on profits, productivity, quality|
|CIRO ( training context, training input, trainees’ reaction; and training outcomes)|
|CIPP (training context, training input, training process, training products or outcomes)|
|Other, please specify....................................................................|
A5. To what extent do you evaluate the following levels of outcomes? Please tick (√) in the box that best reflects your answer where;
|1 = To a very small extent||2 = To a small extent||3 = To a considerable extent||4 = To a great extent||5 = To a very great extent|
|Levels of outcomes||1||2||3||4||5|
|Trainees’ reactions: Feeling and opinion of the trainees about the programmes’ material, facilities, methods, contents, trainers, durations and relevance of the programmes.|
|Learning outcomes: The skills, knowledge and attitudes acquired during the programme.|
|Behaviour change: The change in on the job performance, which can be attached to the programme.|
|Results: The effect in the organisation’s performance resulting from the change of behaviour such as: cost saving, quality improvement, customer satisfaction.|
A6. To what extent do you agree or disagree with the following possible difficulties and challenges that might face your bank when evaluating training and development programmes? Please tick (√) in the box that best reflects your answer where:
|1 = Strongly disagree||2 = Disagree||3 = Not sure||4 = Agree||5 = Strongly agree|
|Evaluation challenges and difficulties||1||2||3||4||5|
|Difficulties in measuring the performance’s improvement in certain jobs|
|Difficulties in measuring the change of trainees’ behaviour over a short period of time|
|Difficulties in identifying the appropriate quantitative measures|
|High cost evaluation process|
|Most of training outcomes are subjective in nature|
|Lack of knowledge about evaluation process|
|Absence of job description|
|Time required to do evaluation well|
|Difficulties to separate training influence on the final results from other organisation’s functions’ influences|
|No specific body is in charge of evaluating the training programmes|
|Difficulties in getting managers to participate in evaluation process|
|Training and development objectives are not clear|
|Other, please specify...................................................|
Section B: (General information). The purpose of this section is to obtain general information related to your bank and yourself as a participant in this research
B1. What is your bank category? [ ] Commercial [ ] Foreign [ ] Investment [ ] Islamic
B2. How long has your bank been in business for? [ ] Years
B3. Please indicate approximately, how many employees does your bank employ in total? [ ] Employees
B4. Please indicate your present job title? (Please tick one)
|1 Manager of training department/unit|
|2 Manager of human resource department|
|3 Senior manager|
|4 Other, please specify..........................|
B5. Please indicate your status within the bank? (please tick one)
|1 Top management (e.g. CEO, MD, or other Board member)|
|2 Middle management (e.g. functional manager, or other senior non board member)|
|3 Lower management/ supervisory|
|4 Professional /technical|
|5 Other, please specify.............................................................|
B6. How many years of work experience do you have in total? [ ] Years
B7. How many years of work experience do you have in the field of training and development? [ ] Years
B8. Please indicate your highest education level: (please tick one)
|1 High school|
|2 Community/ technical college|
|3 University Bachelor’s degree in (please specify)......................................|
|4 Master degree or equivalent in (please specify).......................................|
|5 PhD or equivalent in (please specify)................................................|
|6 Other, please specify ............................................................|
B9. What is your age? [ ]
B10. What is your sex? [ ] Male [ ] Female
Semi structured interview schedule with top managers
1. The position of T&D evaluation in the organisation
Q1. In general, are you satisfied with overall T&D evaluation stage in your bank? Why?
Q2. Do you have a T&D department in the bank? If no, why? Do you think T&D function and programmes are important for the bank’s success? If yes why? Alternatively, do you believe in the importance of T&D evaluation?
Q3. Do you conduct T&D evaluation? If yes, how and who conduct it?
Q4. Could you please give me your opinion in the following statement: T&D is a waste of money and time?
2. The significance of T&D evaluation stage
Q5. Do you conduct T&D evaluation to measure T&D effectiveness function? How often do you conduct T&D evaluation? How and who conduct it?
Q6. Do you use particular model when evaluating T&D programme? What are the criteria that you focus on when evaluating T&D programme?
Q7. Do you think T&D influences the bank performance in any way? If yes how. If no why?
Q8. How significant is the impact of T&D in relation to the following areas: 1 = high impact 2 = low impact 3 = do not know 4 = no impact
3. T&D evaluation main problems and obstacles
Q9. What are the main problems of T&D evaluation in your bank?
Q10. To what extent do you agree or disagree with the following reasons that might be challenges for T&D evaluation stage within your organisation. 1 = strongly disagree, 2 = disagree, 3 = undecided, 4 = agree, 5 = strongly agree.
1. [ ] High cost of T&D evaluation process
2. [ ] Lack of training centre
3. [ ] Lack of separate T&D department
4. [ ] Lack of experienced training staff
5. [ ] Difficulty in measuring T&D outcomes
6. [ ] Training programmes introduced from external sources off the house