Read EST_Associate_Manual.pdf text version

Employee Selection Tool - Associate

ADMINISTRATION MANUAL

Performance-Based Selection, Ltd., 2004; National Association of Convenience Stores, 2004. All rights reserved.

Table of Contents

Introduction.......................................................................................................................... NACS Employee Selection Tool ­ Associate: An Overview........................................... Test Administration ........................................................................................................ Using the System ........................................................................................................... General Test Administration Instructions ........................................................................ Informing Applicants....................................................................................................... Score Reports ................................................................................................................ Understanding Scores.................................................................................................... Commonly Asked Questions .......................................................................................... Technical Summary and Overview ................................................................................. Tables Table 1: NACS CD System Requirements ..................................................................... 3 1 2 3 5 6 7 8 11 14 17

Performance-Based Selection, Ltd., 2004; National Association of Convenience Stores, 2004. All rights reserved.

i

Introduction

The NACS Employee Selection Tool for Associate was developed to help convenience-store operators identify top quality, dependable, and service-oriented employees. It assesses six competencies identified as critical for successful performance as a Sales Associate (or frontline, customer-contact position) in the c-store industry: · · · · · · Basic Abilities, such as reading comprehension and the ability to work with numbers; Dependability, which includes reliability and conscientiousness ­ the extent to which the applicant can be counted on to do the job right; Work Ethic, includes integrity and the willingness to work hard; Commitment and Caring, refers to the applicant's sense of pride in a job well done and their commitment to the job and their career; Respect for Attendance Policies, is a measure of the extent to which the applicant values showing up for work on time and as scheduled; and Focus on the Needs of Others, which includes cooperation with team members, helpfulness, responsiveness to customer requests, and customer service orientation.

The Employee Selection Tool for Associates has proven to be job related and predictive of performance effectiveness, attendance, and customer service orientation. It is intended for employee selection and used to support hiring and placement decisions.

Performance-Based Selection, Ltd., 2004; National Association of Convenience Stores, 2004. All rights reserved.

1

NACS Employee Selection Tool ­ Associate: An Overview

The NACS Employee Selection Tool ­ Associate (EST-A) was developed specifically for the convenience store industry and is tailored to its job requirements, context, and applicant pool. Other primary characteristics are as follows: · The EST-A is to be used when making selection decisions. Although it provides scores by competency, it is not intended for applicant feedback or career guidance and counseling. The EST-A may be administered to one applicant at a time or in a group administration format. The CD-based administration format allows you to administer the test on a computer or to print and administer a paper and pencil version that is then scored via the CD. If the paper and pencil version is used for administration, data entry is required for test scoring. Most applicants will complete the test in 30-40 minutes. For administrative convenience, applicants should be stopped if they have not completed the test within 45 minutes. The test must be administered on-site. Applicants are NOT allowed to take the test or any test materials with them. Because score reports and results are available within minutes, the applicant may be interviewed immediately following test scoring. Or, if preferred, the applicant may be dismissed upon completing the test, with an interview scheduled at a later (and mutually convenient) date and time. It is recommended that applicants not be given detailed information about test scores or outcomes. The score report and details are best used to identify areas on which to focus in the subsequent steps of the screening process. Suggested follow-up interview questions are given by competency, in a separate Interview Guide. The questions are designed to gather additional information on important competencies that current information would indicate may be a concern and/or may negatively impact performance. The questions are designed to gather the kind of information needed to increase the accuracy of your hiring decision. The EST-A is a job-related, valid, and cost effective tool to help you make sound hiring decisions. All available information, not just the test scores, should be considered when making a hiring decision.

· ·

·

· ·

·

·

·

Performance-Based Selection, Ltd., 2004; National Association of Convenience Stores, 2004. All rights reserved.

2

Test Administration

The NACS Employee Selection Tool ­ Associate (EST-A) has a proven track record of effectiveness in identifying top quality employees. Its validity and utility are, however, dependent upon consistency in use and administration. Regardless of how the test is to be administered (paper and pencil or via computer), the CD must be loaded to support test use. The CD is used to access the test, print test copies, and score applicant responses. Before beginning ... 1. Verify that your local system meets the minimum requirements given in Table 1 below. These specifications must be met to ensure that the system operates as designed. Table 1 NACS CD System Requirements

Operating Systems · · · · · · · · · · · · Microsoft® Windows® 98 Microsoft® Windows® 98 Second Edition Microsoft® Windows® Millennium Edition Microsoft® Windows NT® 4.0 Workstation with Service Pack 6.0a or later Microsoft® Windows NT® 4.0 Server with Service Pack 6.0a or later Microsoft® Windows® 2000 Professional with Service Pack 2 or later Microsoft® Windows® 2000 Server with Service Pack 2 or later Microsoft® Windows® 2000 Advanced Server with Service Pack 2 or later Microsoft® Windows® 2000 Datacenter Server with Service Pack 2 or later Microsoft® Windows® XP Home Edition Microsoft® Windows® XP Professional Microsoft® Windows® Server 2003 Family

Software Requirements · · · Microsoft Internet Explorer 6 or later Windows Installer 2.0 (will come with our program and be installed, if necessary) Microsoft Data Access Components (MDAC) 2.6 or later (will come with our program and be installed, if necessary)

Hardware Requirements · · Pentium 266 MHz* 64 MB RAM*

*Or the minimum required by the operating system, whichever is higher. Graphical Requirements · · Screen Resolution: 800x600 pixels Color Depth: 256

Performance-Based Selection, Ltd., 2004; National Association of Convenience Stores, 2004. All rights reserved.

3

2.

Install the CD, as follows: · · Insert the CD to install the system. InstallWizard will automatically install the system on your computer. As it installs, you will be asked to read licensing agreements and to check whether or not you agree to the terms specified. Click "I accept the terms in the license agreement" to continue to install the system. Instructions are given periodically throughout the install. Note that installation may take several minutes to complete. During the installation, you will be asked to enter a User Name and Organization. This information "registers" the system. We recommend that you enter the store number as the "User Name" and your company name as the "Organization." Finally, when given the choice, "Install for anyone who uses this computer" or "Only for me," we recommend that you select "Install for anyone who uses this computer." Upon completion, an icon for the NACS Sales Assessment will be added to your desktop. Double-click the icon to open the system and begin.

· ·

·

· 3.

Decide which administration modality you are going to use ­ paper and pencil or computer. For paper and pencil administrations, you will need to access the Management section to print the test and then enter the applicant's responses into the system for scoring through "Start a new assessment."

Performance-Based Selection, Ltd., 2004; National Association of Convenience Stores, 2004. All rights reserved.

4

Using the System

The system has three primary functions: 1. Start a new assessment. 2. Continue an assessment. 3. Management. Click to select any one of the three alternatives. The capabilities of each function are outlined below: 1. Start a new assessment - Use this to administer a test on the computer or to enter an applicant's responses for scoring. · · For each new administration, create a new user. Be sure to tell the applicant to remember the email address and password entered. The email address can be fake and we recommend using last name, first [email protected] (e.g, [email protected]). The applicants will never receive anything via email associated with this test administration platform. The email address and password are solely used to identify the applicant in the system. That is, the system tracks users by their email addresses. Email addresses were selected for use due to security/confidentiality concerns often associated with the use of social security numbers. (In the event of a power outage, applicants will need to know their email address and password to continue the assessment.) 2. Continue an assessment - Use this to complete an administration or data entry session that was interrupted or paused. The applicant MUST enter the same email address and password used to begin the assessment. Management - Use this to: ! ! ! Print a paper copy of the test for administration. Change passwords for system access. Renew your license for test administration. (The number of tests available for administration is limited by license number. When you submit a request to NACS for additional tests, you will be given a license number that will allow you to access additional test forms.) View individual score reports. View composite score reports (a report summarizing the test scores of all applicants to date). Upload the data to PBS, to support the ongoing maintenance of the assessment system, to guide improvements, and to ensure continued legal defensibility of the product.

3.

! ! !

You must use the email address and password assigned to you by NACS and distributed with the CD to access the Management section. Pre-assigned passwords and emails are used to limit access and maintain test and score security.

Performance-Based Selection, Ltd., 2004; National Association of Convenience Stores, 2004. All rights reserved.

5

General Test Administration Instructions

Prior to test administration, ask applicants if an accommodation is required. Note that the test will require the ability to read from a computer screen (or, if paper and pencil, the ability to read regular size typeface) and to respond by clicking a mouse (or marking responses on the test booklet). Applicants requesting modification should be rescheduled and accommodation should be provided in compliance with your corporate policy. Seat the applicant in a quiet and private location to take the test, with adequate lighting and space. Ensure that scratch paper and pencils are available. Introduce the testing process to the applicant. A sample introduction is given below: Welcome. We are pleased that you are interested in working for (insert company name). As a part of our selection process, we are asking you to complete an assessment that will take approximately 30 minutes. The assessment is designed to measure skills, abilities, and work attitudes that are important for successful job performance as a cstore sales associate. It will be used to help us determine how well your skills, abilities, and interests match our job opportunities. Any questions? Hand out the test, or display the test on the computer screen. Answer questions about the administration process, but not about any particular test item. Collect all test materials before the applicant leaves, including scratch paper. Keep any and all paper copies of the test as well as the Interview Guide in locked cabinet. Limit access to the administration system.

Performance-Based Selection, Ltd., 2004; National Association of Convenience Stores, 2004. All rights reserved.

6

Informing Applicants

CD administration/scoring provides the convenience of being able to notify an applicant of your interest while he/she is still on-site. Applicants are not, however, to be given specific information about test scores or outcomes. If the applicant "passed," continue the screening process. For applicants who have "failed" the test, tactfully explain that the screening process is now complete. A sample script is given below. Thank you for your interest in (company name). We greatly appreciate your cooperation and time in completing our job application and participating in our screening process. Our next step is to interview those applicants whose skills and attitudes best match those required for success at (company name). We will be in contact within (insert timeframe--e.g., the next few days) to let you know the status of your application. Once again, thank you for your interest in (company name). If necessary, more specific information can be given in follow-up correspondence or communication. For example: Our screening process is designed to give us information about skills and attitudes that are important for success at (company name). It helps us to identify the applicants who best fit the jobs that we have available. Although your qualifications may indeed be impressive, they are not the best fit with our current job requirements. We sincerely appreciate your interest in (company name) and we wish you all the best.

Performance-Based Selection, Ltd., 2004; National Association of Convenience Stores, 2004. All rights reserved.

7

Score Reports

Score reports, or the Individual Assessment Reports, are available for each applicant via the CD test administration platform. Each section of the score report is described below. A sample score report has been provided on the following page. Section 1 ­ Personal Information On the left side is the applicant's name, address, and Social Security number. On the right side is the test date, overall score (score), pass/fail determination, percentile norms, and minutes taken to complete the test. Section 2 - Scores Presents test scores graphically as well as numerically. The Overall Score is given first, followed by the more detailed or specific Score by Competency. Use the competency scores to determine an applicant's relative strengths and weaknesses. Areas of concern can be targeted for follow-up in an interview. A separate Interview Guide has been provided which lists possible follow-up questions for each competency area. Warning Notation A "Warning" may appear at the bottom of the Individual Assessment Report. The warning is triggered by the pattern of the applicant's answers. A warning is printed when the applicant's responses are considered to be extremely positive or socially desirable. When the warning has printed, it indicates that the applicant may be more focused on "putting his/her best foot forward" or describing the "ideal" employee than he/she is on realistically describing his/her own interests, behaviors, and preferences. The warning does not indicate that the applicant is lying or intentionally misleading you ­ only that his/her responses are very consistent with the responses that generally would be considered to be "good." If the warning is printed, be sure to pay particularly close attention to other indicators of the applicant's qualifications. Verify that the high scores obtained for each competency are justified through the review and compilation of information obtained through the job application and/or a structured interview. See the Interview Guide for possible interview questions. The interview questions were developed to be applicable to a variety of jobs and organizations. They may be rephrased to better target your company, job, or position requirements. A Composite score report is also available which summarizes and compiles the scores for all applicants tested via a single CD and license. That is, if the test is being administered locally at a store, the Composite Report combines all scores for all applicants tested to date. It does not include any scores from other stores within the company ­ it is a locally-based report and summary. The scores show you how your applicant pool, on average, compares to the national norms. The Composite Report will continue to compile test results and scores of all completed assessments at your location and across all license updates (that is, the Composite Report will include results for all tests administered at a single location). A sample Composite Report follows the sample Individual Assessment Report. Remember that the Composite Report shows you how your typical applicant compares, on average, to the national norm.

Performance-Based Selection, Ltd., 2004; National Association of Convenience Stores, 2004. All rights reserved.

8

Page 1 of 1

Individual Assessment Report

Assessment: Employee Selection Tool - Associate

Personal Information First Name: Last Name: SSN: Address: Address2: City: State: Zip: Phone: Scores

0 Overall

Carol Robinson 999-99-9993 28915 Clemens Road, Suite 216 Westlake Ohio 44145 4408081514

Date: 5/6/2004 11:05:36 AM Pass/Fail: Local Percentile: National Percentile Minutes To Complete: Pass 76.43 95.88 18

PERCENTILE

..................................................................................................... |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

100

95.88

SCORE BY COMPETENCY

Basic Abilities Commitment and Caring Dependability Focus on the Needs of Others Respect for Attendance Policies Work Ethic

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

66.65 79.50 99.65

|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

76.14 82.91 94.53

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

file://C:\Program%20Files\NACS%20Sales%20Assessment\NACS%20Sales%20Assessment\c3a3cad301...

5/6/2004

Page 1 of 1

Composite Assessment Report

Assessment: Employee Selection Tool - Associate

Start Date: 4/6/2004 End Date: 5/6/2004 Statistics Completed Tests: 4 Passed: 3 Failed: 1 Score by Competency

0 Basic Abilities Commitment and Caring Dependability Focus on the Needs of Others Respect for Attendance Policies Work Ethic

..................................................................................................... ||||||||||||||||||||||||||||||||||||||||

100

39.49 90.49 93.42

|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

80.01 91.95 93.06

|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

file://C:\Program%20Files\NACS%20Sales%20Assessment\NACS%20Sales%20Assessment\.htm

5/6/2004

Understanding Scores

The overall and competency scores included on the reports are presented as percentiles. The percentile for each competency indicates how well the applicant scored relative to others. Specifically and for example, a score of 80 indicates that the candidate scored above 80% of the c-store incumbents who participated in the test development and validation project. 1. Overall Score An overall score is provided for the applicant. The higher the score, the greater the likelihood that the applicant will be successful if hired. The Overall Score is used to evaluate/determine Pass/Fail status which is also given on the score report. Note that the Overall Score is not an arithmetic average of the scores on the six competencies. The competency scores have been weighted to reflect their importance, number of items, and validity. The relative weights are given below: Overall Score = Basic Abilities 1/3rd Work Approach Competencies 2/3rd with each of the following receiving equal weight in the 2/3 composite · · · · · Commitment and Caring Dependability Focus on the Needs of Others Respect for Attendance Policies Work Ethic

Achieving a high Overall Score on the test is a strong indication of subsequent success. You should, however, consider all the information you have available about an applicant, not just the test score. For example, also consider the applicant's work history, references, and interview responses. 2. Pass/Fail Decision The applicant must receive an overall percentile score of 25 or higher to "pass" the test. The use of this decision criterion is expected to eliminate only about 25% of the job applicants ­ approximately 75% will pass. The pass criterion is set relatively low to accommodate additional steps in the process (e.g., background checks, interviews) and to reflect differences between local labor markets and our nationwide norms. Your local applicant pool may be significantly more (or less) qualified. The pass/fail criterion should be interpreted as only screening out those people who have minimal chance of success, if hired.

Performance-Based Selection, Ltd., 2004; National Association of Convenience Stores, 2004. All rights reserved.

11

3.

Percentiles: The Selection Criterion Immediately below the pass/fail decision, are two percentile scores ­ both representing overall performance. Remember that percentiles represent how well the applicant performed in comparison to other applicants who have taken the test; specifically, the percentage of other applicants who scored below the applicant in question. For example, a percentile score of 80 is well above average; it indicates that the applicant scored better than 80% of the other people who have taken the test. The percentiles indicate how well the applicant ranks in relation to others in the c-store industry. The "Local Percentile" is based on only the applicants tested at your location. The Local Percentile will not begin to stabilize or become meaningful until at least 50 applicants have taken the test. In the interim, we recommend that you use the "National Percentile" which is based on our nationwide sample of job incumbents.

4.

Specific Scores ­ also presented in terms of percentiles and always with reference to National Norms. Basic Skills ­ measures basic reasoning abilities like reading comprehension and the ability to work with numbers. High scores indicate that the applicant is likely to be able to: · · · · · accurately understand written policies; successfully complete all training requirements; learn new tasks quickly; rotate through new assignments with ease; and handle job-related situations where math is required, such as scheduling, managing time, monitoring inventory, and completing sales transactions.

Dependability ­ measures an applicant's attitudes toward work commitments and obligations ­ the extent to which the applicant can be counted on to do the job right. High scores on the Dependability scale indicate that the applicant is likely to: · · · · be a productive and contributing team member; support the company and its mission; commit to long-term employment; and follow through on job assignments.

Work Ethic ­ measures an applicant's attitudes toward integrity and hard work. Applicants who score high on Work Ethic are likely to: · · · · value personal integrity and professionalism in the workplace; avoid theft and alcohol/drug use; comply with policies and procedures; and use safe work practices.

Performance-Based Selection, Ltd., 2004; National Association of Convenience Stores, 2004. All rights reserved.

12

Commitment and Caring ­ measures an applicant's likelihood of personally investing in work quality and/or a career. High scores indicate that the applicant is likely to: · · · · · remain in this or a related industry; value a job well-done; believe that he/she is good at what the job requires and will enjoy doing it; understand personal career needs and have a sense of direction their work history; and have high job satisfaction

Respect for Attendance Policies ­ is a measure of an applicant's attitude toward work schedules, time commitments, and absenteeism. High scoring applicants are likely to: · · · show up for work consistently and on time; report to work when and as scheduled; and ensure that personal commitments are scheduled outside, and with recognition of, work schedules/commitments.

Focus on the Needs of Others ­ measures service-related attitudes, and attitudes toward team-based environments. High scores on this scale indicate that the applicant is likely to be: · · · · · polite and courteous to customers and coworkers; responsive to customers' needs; helpful; interested in building relationships with customers and coworkers; and committed to providing top-notch customer service.

Performance-Based Selection, Ltd., 2004; National Association of Convenience Stores, 2004. All rights reserved.

13

Commonly Asked Questions

Following are several commonly asked questions and standard responses about the test administration process. These questions and responses should be reviewed prior to test administration. Consistent administrative procedures help to ensure reliability and fairness across administrations, and to maintain the integrity of the test and screening process. Q. If an applicant seems confused about the test instructions, can I reread them to him/her? Yes. It is important that the applicant understands the directions before testing begins. Understanding the instructions is NOT part of the test. Q. What if an applicant objects to completing the ethnicity/gender section? Explain that federal law requires that all assessments be fair and appropriate for all groups of people. The demographic information will not be used for any purpose other than to ensure that the test is fair. The demographics will NOT appear on his/her score report, and will NOT be used to make a hiring decision. If the applicant still objects, indicate that providing demographic information is voluntary and omitting it will not affect the hiring decision. Q. What if I notice that an applicant is having difficulty using the mouse? Quietly approach the applicant and demonstrate mouse use. Suggest that the applicant practice and open a word document (other than the test) or access the Internet and allow the applicant to practice before continuing with the test. Q. How much time should an applicant be given to complete the test? The test is not timed. Most applicants will complete the test in 30 minutes. If the applicant is not done at the end of 30 minutes, suggest that he/she try to finish up soon as the administration session will end in another 15 minutes. At the end of 45 minutes, thank the applicant for his/her time, interest, and cooperation, and end the administration session. Q. If an applicant requests more time at the end of 45 minutes, is it okay to allow him/her more time? No. Administration procedures and policies should be consistent. If paper and pencil test is implemented, all materials should be gathered at the end of 45 minutes. Q. If an applicant asks a question specific to one of the test items, can I answer it? No. You may not answer questions specific to test items. Simply tell the applicant to do the best that he/she can do.

Performance-Based Selection, Ltd., 2004; National Association of Convenience Stores, 2004. All rights reserved.

14

Q.

Should the applicant be told to guess if he/she does not know the answer to an item? Yes. There is no penalty for guessing, so it is to the applicant's advantage to answer every item.

Q.

What should I do with the applicant's test materials once he/she has completed the test? Be sure to collect all materials, including scratch paper. The scratch paper should be disposed of in such a manner that it can not be read.

Q.

How is the score reported to the applicant? Scores are not reported to applicants. Score reports are to be used to help you to make hiring decisions.

Q.

If the applicant receives high scores on the test, should I hire him/her? Scoring well on the test is a strong indication of subsequent success. You should, however, consider all the information you have available about an applicant, not just the test score. For example, also consider the applicant's work history, references, and interview responses.

Q.

What should I do if a "Warning" appears? The warning is triggered by the pattern of the applicant's answers. A warning is printed when the applicant's responses are considered to be extremely positive or socially desirable. When the warning has printed, it indicates that the applicant may be more focused on "putting his/her best foot forward" or describing the "ideal" employee than he/she is on realistically describing his/her own interests, behaviors, and preferences. The warning does not indicate that the applicant is lying or intentionally misleading you ­ only that his/her responses are very consistent with the responses that generally would be considered to be "good." If the warning is printed, be sure to pay particularly close attention to other indicators of the applicant's qualifications. Verify that the high scores obtained for each competency are justified through the review and compilation of information obtained through the job application and/or a structured interview (see the Interview Guide for possible interview questions). The warning is perhaps best used to focus the remaining steps of the screening process. For example, a warning may be used to generate follow-up questions for the applicant interview or for the reference check. In fact, suggested interview questions, for each competency, are given in a separate Interview Guide. The interview questions were developed to be applicable to a variety of jobs and organizations. They may be rephrased to better target your company, job, or position requirements.

Performance-Based Selection, Ltd., 2004; National Association of Convenience Stores, 2004. All rights reserved.

15

Q.

Can an applicant retake the Employee Selection Tool for Associate? Letting an applicant retake the test is not recommended, unless there were problems encountered during the initial test administration. Under standard administration conditions, it is doubtful that an applicant's score will change much from one administration to another, and it is unlikely that any changes noted are due to real differences in skill or attitude. Most companies do, however, establish a formal retest policy and allow applicants to retest three, six, or 12 months following their initial attempt.

Performance-Based Selection, Ltd., 2004; National Association of Convenience Stores, 2004. All rights reserved.

16

Technical Summary and Overview

Intended Test Use The NACS Employee Selection Tool for Associate was designed to measure critical competencies required for successful job performance for front-line, sales associates. These competencies include: (1) Basic Abilities, (2) Commitment and Caring, (3) Dependability, (4) Focus on the Needs of Others, (5) Respect for Attendance Policies, and (6) Work Ethic. The EST-A was designed specifically for employee selection. It measures job applicants' ability to perform; their willingness to succeed; and their key attitudes toward customers, co-workers, and supervisors. Inappropriate Applications The EST-A was neither intended nor designed to be administered to incumbents for individualized feedback and development. In addition, although the EST-A may prove useful as an initial screen for all retail personnel, a local validation study would be necessary to determine and document its relevance and effectiveness for managerial selection or for support/administrative/office positions. Pilot Testing, Validating, and Norming The experimental item types were pretested and validated using job incumbents across several different companies. A total of 289 people participated in the validation phase of the project. Four different experimental forms were administered to support the development of an item bank and multiple test forms, resulting in a total of 72 people who took all items selected for the final, operational form. The non-cognitive items (Commitment and Caring, Dependability, Focus on the Needs of Others, Respect for Attendance Policies, and Work Ethic) were selected in a crossvalidation design. That is, the 289 employees were randomly divided into two groups; the data from the first group were used to select items and cast the final form of the test, and the data from the second group were used for cross-validation. Validation results are based solely on the cross-validation sample; they represent an independent assessment of validity. Current test norms are based on 72 c-store job incumbents, a subset of the total validation sample. Current norms were collected in 2001 through 2003. Reliability Reliability was estimated using Cronbach's alpha, which represents the internal consistency of the items. The overall reliability is .85, which compares favorably with similar measures especially given the brevity of the test.

Performance-Based Selection, Ltd., 2004; National Association of Convenience Stores, 2004. All rights reserved.

17

Criterion-Related Validity EST-A was designed to be an integrated, pre-employment selection tool that would evaluate the key competencies required for entry-level c-store sales associate. Each of EST-A's six scales (Basic Abilities, Commitment and Caring, Dependability, Focus on the Needs of Others, Respect for Attendance Policies, and Work Ethic) demonstrated evidence of validity and was found to be predictive of at least one important dimension of job performance (e.g., customer service, teamwork, integrity, knowledge, attendance, punctuality). Composite validity (for the combined and overall score and corrected for criterion unreliability) is estimated at .27 for overall performance and .35 for ratings of Customer Service, generally thought to be the single most important dimension of performance. Content Validity Content-related evidence of validity is gathered during the test construction process by representatively sampling a defined content domain. Specifically, EST-A was developed following a comprehensive job analysis of the c-store sales associate position. PBS staff members observed job performance; evaluated critical incidents of job success and failure; conducted focus groups with c-store managers; and developed, administered, and analyzed a detailed job analysis survey. Each test item is posed in an employment context and targets specific work behaviors or work activities. All experimental items were developed from actual job materials and intended to model typical job activities. For example, store manuals, corporate newsletters, and memoranda were used to develop the reading passages. Math questions focused on common work activities such as making change, balancing the cash drawer, and calculating expiration dates. Construct Validity Evidence of the test's construct validity is given by considering the correlation of EST-A's scale scores with the various ratings of the dimensions of performance. In general, the correlations are as would be expected given the nature of the "constructs" being measured. For example, the Basic Abilities scale, measuring cognitive ability, correlates significantly with the knowledge rating (.34) while the non-cognitive or attitudinal scales do not. Readability The content of EST-A should be easily read and understood by most people. The test falls into the "easy" range (approximately a sixth grade level) of readability, as measured by the Flesch Reading Ease index. If the job requires reading of at least a sixth-grade level, EST-A is appropriately administered to any and all applicants.

Performance-Based Selection, Ltd., 2004; National Association of Convenience Stores, 2004. All rights reserved.

18

Test Administration Procedures Test administration procedures must be standardized and consistent to ensure that the resulting scores are fair, reliable, and accurate. The testing process is designed to allow administration by staff with no formal training in employee evaluation or selection. The Administrator's Guide details complete and standardized administration instructions. Test administration instructions replicate those used in test validation, except that all tests were administered via paper and pencil during the validation study. Reporting Results to Applicants We do not recommend disclosing test results to applicants, unless required by state law. EST-A score reports are designed for hiring officials, HR representatives, and/or management team members. Test Interpretations Scores are reported in terms of percentiles. Percentile scores indicate how well an applicant performed on the test in comparison to others who have taken the test. Percentiles range from 1 to 99, with a percentile of 50 being average. A percentile score of 70, for example, is well above average; it indicates the applicant scored better than 70% of those who have taken the test. A norm table (national) has been provided but the most meaningful norm or comparison group is your company's applicant pool for the targeted job. It is to your advantage to also consider the local norms whenever possible in interpreting the overall score. Local norms will NOT, however, be useful until a sufficient number of applicants (more than 50) have taken the test. The National Norms should be used until then.

Performance-Based Selection, Ltd., 2004; National Association of Convenience Stores, 2004. All rights reserved.

19

Information

21 pages

Find more like this

Report File (DMCA)

Our content is added by our users. We aim to remove reported files within 1 working day. Please use this link to notify us:

Report this file as copyright or inappropriate

662352