Read Microsoft PowerPoint - pres1112.ppt [Read-Only] text version

Balanced Performance Measures: From The Bottom Up

CAPT Gib Kerr PMS 435, PEO Submarines (202) 781-1556 [email protected]

Discussion Topics

· · · · · Performance measures, a definition and tenets Tools The Process The Results The Takeaway

CAPT G. Kerr- 2

Performance Measures

· Definition (one version)

­ Objective indicators that are both necessary and sufficient to assess an organization's processes such that intelligent and informed decisions can be made to improve efficacy


Basic Tenets

­ Few in number ­ Balanced ­ Must:

Link output to input Evolve from what exists (or have an ROI for the change) Be measurable and objective Evaluate rate and account for rework

­ No measurement for measurement sake ­ No success oriented measures

­ They should be useful for making decisions

CAPT G. Kerr- 3

Some Useful Tools

· "The Balanced Scorecard" by Kaplan and Norton (1996):

­ Four perspectives - customer; financial; internal business; innovation and learning

· · · ·

System Mapping Workshop and Government Performance Measures Workshop (Quality Alert Institute) (1998) U. S. Air Force IDEF Model for SYSCOM Description (late 1980s) "The World According to CAPT Kerr": Strategic goals; work process output; client impact/benefits; innovation and learning YOUR EXPERIENCE

CAPT G. Kerr- 4

The Process

1. Define Organization's Mission 2. Map your system

· · · · · · Use a consistent protocol Be honest and be prepared to laugh Decide what to keep and what to throw away Keep tenets in mind Think in terms of desired results first, measures second Balance them

3. Describe existing measures 4. Assess against your system 5. Develop performance measures

6. Apply measures 7. Assess and monitor

CAPT G. Kerr- 5

The Results

PMS 4352 (Towed Systems) Systems Map

Mission: Delivering the world's best submarine towed acoustic systems to the Fleet


Requirements Funding Parts Lists CDRLs PECPs/ECPs Info/Status Plans/Metrics SBIRs Services HW Guidance Ideas Policy ILDPs/Timesheets

Key (Direct) Processes

Acquisition Financial Execution Budgeting Systems Engineering Configuration Control Proc. Inventory Forecasting Logistics Support


Budget Exhibits HW/Specs/Plans PRs/FADs Decisions/Policy Correspondence Processes/Tasking/Guidance MOAs/MOUs/ECIs Brief Sheets Perf App./Time Sheets/Awards Leadership Contract Awards

Support Processes

Training Admin/HR Program Management Modeling/SIM Engineering Weapons Supt. BCAs IT Collision Avoidance Red Teams/IPTs/IPPD Design Reviews (CDR/PDR, etc.) Quality Assurance


OEMs Field Activities N77 CSS Contractors Sub Contractors Fleet BSC PMs/SEA 02, 05, 00C, 04 PEO Subs Subordinates Congress


· Fleet/Shipyards · N77/N4/SEA 02 · Congress/Tax Payers · PEO-Sub/ BSC/HR · PARMs/SHAPMs · OEMs/Industry · CSS Contractors · FMS/Academia · Field Activities ·CAPT G.Command Chain of Kerr- 7 · ONR/CTO

Existing Measures

CAPT G. Kerr-7

PMS 4351 (I & EW) Systems Map

Mission: Acquire and plan I&EW solutions by developing and maintaining the eyes and ears to transform the Fleet's/today's needs


Equipment/ Hardware Tech Reports/Doc Logistics/CM Software Requirements/Specs People Contract Actions Financial Actions/Money Short falls/needs ITMPs Standardized policies MIL STDs/Guidance/Law Program mgmt/planning CASREPs Actions/Decisions/ ?s

Key Processes

Tech doc/ECP review and approval TMP COR PPBS Modernization/Installations Acquisition process/doc/contract award ILS process/Training R&D/SUBTECH process CASREP monitoring/Fleet support RM&A/Maintenance planning Performance measurements Process Improvement Information processes Human resources Depot est. Policy/guidance Security/ Program protection


Briefings/Issue papers Capability/Technology Spend Plans/budget Acquisition/Financial Plan Program Plan/requirements Information/Lessons learned Systems/Equipment Documentation/Specs/Drawings Logistics Design Reviews/requirements Staffing Plan/Jobs/ LOE CM/ECPs Direction/Guidance/vision Intellectual property ITMPs Letters Contracting/Tasking (TIs)


OEMs Warfare Centers/Ship Yards ONI/ONR/DARPA Academia NAVSEA/SPAWAR PEO Subs TYCOMS/Fleet/PARMs SHAPMs Matrix/CSS Vendors Congress/ American People DoD/DoD Labs/Other svcs.

Support Processes

Travel Admin/Email Scheduling Action Item tracking SLCADDA Tops BSC Livelink MILCON Cell phone Awards Personnel mgmt Navy MSGs


Warfare centers/labs Fleet/TYCOMs/sailors SHAPMs/PARMs Trainers ONI/CNSG OPNAV/OSD Industry/Public Shipyards/Other Services Exec branch/Congress Public affairs PEO/SYSCOMs CAPT G. Kerr- 8 Academia/Other countries Waterfront support DEVRON 12/COTF

Existing Measures

Spend Curves CAPT G. Kerr-8

Knowledge IPT Systems Map

Mission: Provide Capability Solution Recommendations to FCIPT (via SUBTECH)


Technology Vectors (OPNAV) 2020 Vision/Road Ahead Programs of Record Requirements CCI's FCWG Priority List Technology Descriptions/Data ?

Key Processes

Current: Develop Technology Data Sheets Conduct Technology Assessments Make Technology Recommendations Develop Capability Road Maps Develop Technology Quad Charts Handle Data Calls Future: Objectively Rank Technologies Conduct Gap Analyses Conduct Technology Searches


Capability Road Maps Technology Data Sheets Technology Quad Charts Prioritized list of Tech. Unmet Needs Technology Recommendations Data Call Responses ?



Support Processes

Update POC Lists Set up/Monitor Working Groups Interface Functions Meeting Support Web Site/Data Base Updates Conference Attendance



Existing Measures

Current: None. Future: (M&TA ?) Internal EV/Perf Measures?

CAPT G. Kerr- 9

S/CC/A Delivery Systems Map

Mission: Provide and Support Reliable and Fully Integrated S/CC/A Hardware and Software to PMS450 to Support the Shipbuilders Schedule.


PTRs (Finds & Fixes) TI Requests CDRL/Contract Rqm'ts Hardware Sys Code (for functionality) BCRs/DIRs Tech Issues/Questions Data/Eng. Svcs Requests Test Reports Delivery of GFE "Good Ideas"

Key Processes

PTR FFR System/Subsystem Test TI Dev/Exec & Comp SW Production/Gen Hardware production Integration (Sys/Subsys Test)


PTRs PTR fixes Code/Functionality System HW Test/JTG Reports BCRs/DIRs ECPs/Contract Mods Tech Iss/Quest Resolution TIs CDRLs (TDP, ILS) "Good Ideas"

Support Processes

Bus./Fin. Analysis & Reporting Proposal Development Contract Change ECP Implem./Sub-CSS Mgt Cost Acc, Time & Attendance Training (Currency) Management Facilities Mgt (COATS) Personnel Support Systems Support/Maintenance Estimating and Projecting


Dev & Test Engineers PMS450/401 NPES/CWITT Individual Cost Account PMS440 (Q70) Fleet (CON OPS) 3rd parties



Existing Measures

PTRs (Open, Closed, Plan, Tadstand, Productivity, SLOC Metrics) Test Cases (Dry Run, Cert, Plan, Pass, Fail) Req'ts Complete (also Test Steps, Verifies) CPR Data against Contract (ACWP, BCWP, BCWS, Manning) Spend rate

CAPT G. Kerr- 10

Organization's Processes Desired Results

· · · · · · · · · · · Reduced political pressure Minimize impact to ship construction Timely and accurate________________. Get cost effective compliant system Improve processes/lower costs Get operationally effective system Provide complete and compliant hardware/software Provide framework for on-going tech insertion Reduce Total Ownership Cost Develop business and industrial base Technological leadership

CAPT G. Kerr- 11

"Balanced" Desired Results

· Work Process Outputs

­ Timely and accurate ______________ ­ Provide complete and compliant hardware/software


Client/Customer Benefit/Impacts

­ Get cost effective compliant system ­ Minimize impact to ship construction ­ Get operationally effective system


Strategic Outcomes

­ ­ ­ ­ Reduced political pressure Provide framework for on-going tech insertion Develop business and industrial base Reduce Total Ownership Cost


Innovation and Learning

­ Improve processes/lower costs ­ Technological leadership

CAPT G. Kerr- 12

Performance Measures

· Work Process Output

­ Items/month (or interactions/month, week, etc.) ­ Elapsed Time from Request to Approval (ETRA) (include Admin. Delay Time, ADT) ­ Time from Request to Approval vs. Plan (approval efficiency) (app) ­ Mean Time to Approve (MTRA) ­ Elapsed Time to Execute (ETE) ­ Time to execute vs. Plan (execution efficiency)(exec) ­ Mean Time to Execute (MTE) ­ Number and time of process re-cycles (TR) ­ Mean Time of Process Recycle (MTR)

CAPT G. Kerr- 13

Performance Measures (cont)

· Client/Customer Benefits/Impacts

­ Minimize impacts to ship's construction

· Percent of GFE deliveries late to contract · Unsatisfactory material report (UMR) measures (see work process measures)


Strategic Outcomes

­ Reduce political exposure (grouped by hierarchy)

· Number of higher level requests or reports per unit time with degree of urgency · Elapsed time to respond · Number of issue recycles (i.e., clarifications)

­ Develop business and industrial base

· CPARS grade trends and analysis · Business case analyses · Third party involvement metrics (subcontractor performance, etc)


Innovation and Learning

­ Process improvements

· · · · · Percent processes reviewed Percent processes requiring re-engineering Resource cost for review or re-engineering ratio Resource cost to implement ratio ROI CAPT G. Kerr- 14

Global Process for Performance Measurement

Initial Development

Develop System Map Define Key Processes Define Desired Results

Increase Scope for New Contractor

Reduce Scope this Contractor



Is Scope Reduction Possible?

Define all Remaining Work Define Performance Measures & Thresholds Assign System Tasks [Remaining Work] Apply Performance Measures Analyze "Like Tasks" @ Possible Supplemental Contractors Evaluate all Work Evaluate Measures [Analyze & Measure] Establish Viable Supplemental Contractors

Yes No

Performance Threshold Exceeded?


Performance Measure Eval. Sat.? No

Modify Performance Measures [Establish UCL, LCL, etc. ]

CAPT G. Kerr- 15

The Process World through CAPT Kerr's Rose Colored Glasses

Performance Measures Must:

- Link Output with Input

Constraints - Evolve from what exists - Organizational (or there's an ROI for the change) - Physical - Be measurable - Others - Be objective wrt Cost, Sched, Technical, & Risk Processes: (Direct, Indirect) - Evaluate Rate - Account for Rework - We have this problem (define) - We found this solution Inputs (solution/approval) - Suppliers - We'll do this work (execution) - Data - We'll remember what we did - Others (archive) Balanced Perspectives

- Strategic Goals - Work Process Outputs - Client/Customer Impact/Benefits - Innovation and Learning

It's all about: - Analysis

- Relationships - Communication


- Customers

- Financial - Internal - Growth


- People

- Money - Expertise

CAPT G. Kerr- 16

Last Zingers

· It's about management....the technical problems will solve themselves if managed well. Resist using what has been used before...doing stupid for 20 years doesn't make it any less stupid.

­ More importantly, resist success oriented measures...they only mask the truth.


· ·

Provide incentives for desired results or you may get results you don't want. Remember to capture what it is you did...otherwise you'll never learn. (Most lessons learned aren't, they're merely observed). Don't measure for measurement sake (or to impress anyone). If it's not useful to you, it won't add much value to those who work for you. LISTEN to what you're being told about your processes, and don't stop there...ask why, over and over and over....

CAPT G. Kerr- 17



One Last Time:

It's all about communication, relationships and analysis. If you don't communicate well, you won't be able to build the relationships necessary to get buy-in and commitment. Without good relationships (and understanding), you can't do the analysis to know what your problems really are (or know where you are.)

It's the PEOPLE, silly, not the Technology that will resolve your challenges.

CAPT G. Kerr- 18

List of Acronyms

List of Acronyms

· · · · · · · · · · · · · · · · ACWP ­ Actual Cost of Work Performed ADT ­ Administrative Delay Time ASN ­ Assistant Secretary of the Navy BCA ­ Baseline Change Agreement BCRs ­ Baseline Change Request BCWP ­ Budgeted Cost of Work Performed BSWS ­ Budgeted Cost of Work Scheduled BSC ­ Business Service Center CASREPs ­ Casualty Reports CCI ­ Critical Capabilities Items CDR ­ Critical Design Review CDRLs ­ Contract Data Requirements List CFFC ­ Commander, U.S. Fleet Forces Command CM ­ Configuration Management CNSG ­ Commander, Naval Security Group CNWC ­ Commander, Net Warfare Command

CAPT G. Kerr- 20

List of Acronyms (Cont)

· · · · · · · · · · · · · · · CON OPS ­ Concept of Operations COATS ­ Command, Control, Communications and Intelligence (C3I) System Off-Hull Assembly and Test Site COR ­ Contracting Officer's Representative COTF ­ Commander, Operational Test and Evaluation Force CPARS ­ Contractor Performance Assessment Report CPR ­ Cost Performance Report CTO ­ Chief Technology Officer CSS ­ Contractor Support Services CWITT ­ C3I System Weapons Integration Test Team DARPA ­ Defense Advanced Research Projects Office DEVRON 12 ­ Development Squadron 12 DIRs ­ Design Investigation Request DoD ­ Department of Defense ECI ­ Engineering Change Instruction ECP ­ Engineering Change Proposal

CAPT G. Kerr- 21

List of Acronyms (Cont)

· · · · · · · · · · · · · · · ETE ­ Elapsed Time to Execute ETRA ­ Elapse Time from Request to Approval EV ­ Earned Value FADs ­ Financial Accounting Data Sheets FCWG ­ Future Capabilities Working Group FFR ­ Find, Fix and Repair FMS ­ Foreign Military Sales FSG ­ Future Studies Group HR ­ Human Resources HW ­ Hardware I&EW- Imaging and Electronic Warfare IDEF ­ Integrated Definition ILS ­ Integrated Logistics Support IPPD ­ Integrated Product and Process Development IPT ­ Integrated Product Team

CAPT G. Kerr- 22

List of Acronyms (Cont)

· · · · · · · · · · · · · · · IT ­ Information Technology ITMPs ­ Individual Task Management Plans JTG ­ Joint Test Group LM ­ Lockheed Martin LCL ­ Lower Control Limit LOE ­ Level of Effort M&TA MIL CON ­ Military Construction MIL STDS ­ Military Standards MOA ­ Memorandum of Agreement MOU ­ Memorandum of Understanding MSGs ­ Messages MTE ­ Mean Time to Execute MTR ­ Mean Time of Process Recycle MTRA ­ Mean Time to Approve

CAPT G. Kerr- 23

List of Acronyms (Cont)

· · · · · · · · · · · · · NAVAIR ­ Naval Air Systems Command NAVSEA ­ Naval Sea Systems Command NPES ­ Non-Propulsion Electronics System OEMs ­ Original Equipment Manufacturers ONI ­ Office of Naval Intelligence ONR ­ Office of Naval Research OPNAV ­ Office of the Chief of Naval Operation OPTEVFOR ­ Commander, Operational Test and Evaluation Force OSD ­ Office of the Secretary of Defense PARMs ­ Participating Managers PDR ­ Preliminary Design Review PECPs ­ Preliminary ECPs PEO SUBS ­ Program Executive Officer, Submarines

CAPT G. Kerr- 24

List of Acronyms (Cont)

· · · · · · · · · · · · · · · PMs ­ Program Managers PPBS ­ Planning, Programming and Budgeting System PRs ­ Procurement Requests PTRs ­ Problem Trouble Reports R&D ­ Research and Development RES ­ Raytheon Services RM&A ­ Reliability, Maintainability and Availability ROI- Return on Investment SIM - Simulation SBIRs ­ Small Business Innovative Research SHAPMS ­ Ship Acquisition Managers SLCADDA ­ SLOC ­ Software Lines of Code SPAWAR ­ Space and Naval Warfare Center STRG ­ Submarine Tactical Requirements Group

CAPT G. Kerr- 25

List of Acronyms (Cont)

· · · · · · · · · · SUBTECH ­ Submarine Technology SW - Software SYSCOM ­ Systems Command TIs ­ Technical Instructions TMP ­ Test Master Plan TOPS- Travel Order Processing System TR ­ Time to Recycle (Recycle Time) TYCOMS ­ Type Commanders UCL ­ Upper Control Limit UMS ­ Unsatisfactory Material Report

CAPT G. Kerr- 26


Microsoft PowerPoint - pres1112.ppt [Read-Only]

26 pages

Report File (DMCA)

Our content is added by our users. We aim to remove reported files within 1 working day. Please use this link to notify us:

Report this file as copyright or inappropriate