SPM Testing Template - Part 5

Incentive Compensation Management test results can be recorded in several ways. One of the approach I like to use groups all plan names, rule names, conditions, expected results and testing status on the same spreadsheet. I find that by keeping all this information together, it is easier to quickly get the picture of the overall testing progress. It also allows to keep all the information on the same spreadsheet instead of having to maintain 2 spreadsheets with identical information.

Here is an example to get started:

[download the spreadsheet]

Other benefits of using an Excel spreadsheet to record test results instead of a Word document include:

• Ability to quickly highlight failing tests in red
• Ability to filter information displayed (e.g. display only failing tests)
• Test results can be printed on fewer pages
• Ability to create macros to perform more "fancy" features such as displaying the number of days a certain issue has been opened.
  

Other columns could be added to add additional information such as the date at which the test was performed, the name of the tester, how critical the test is, the actual result when different from the expected result, and comments.

PlanIt: Another On-Demand Sales Performance Solution

I received an e-mail some time ago from Matt Tyre, a Sales Compensation Consultant working at Sales Resource Group (SRG).

Sales Resource Group Inc. is an industry leading sales compensation professional services organization that provides world class software, consulting and outsourcing services, with expertise in sales incentive programs and sales force effectiveness.

I had heard about their president David Johnston before, from his articles in the Canadian Professional Sales Association (CPSA) Contact magazine. David has over 20 years experience consulting for various organizations and teaches sales compensation for WorldatWork.

What distinguishes Sales Resource Group from many other small sales compensation management companies is that they offer their own sales compensation solution called "PlanIt".

PlanIt is an On-Demand Sales Performance Management solution offering the same functionalities as other applications in its category (incentive calculations, reporting, analytics, etc). In addition, PlanIt also has a module providing guidance to optimize compensation programs and align these programs to business needs, and allows to route relevant documentation to its users.

User Interface
Similar to other applications, PlanIt displays some high level compensation information to the payees when they login. From this screen they can easily review their current revenues, the top 10 performers and their ranking compared to their peers. What I particularly like about the main screen is the News /Announcements which occupies an important position at the top center of the screen. This ensures payees don't miss out on important news.


Reporting
The reporting section is also very important for the payees. Most of the reports are available "out-of-the-box" and illustrate all the information in different level of details. Payees can drill down to the order level, or see high level summary of their compensation and ranking. Custom reports can also be created easily.


Implementation
Overall the user interface was very nice, but the big question remaining was, how easy is it to implement and maintain solutions built with PlanIt. Not surprisingly the answer was, the implementation is user friendly. It is all done through a user interface and does not require any deep technical skills.

What I particularly liked about the Plan Design interface is its ability have approved plans and plans awaiting approval. I have not seen this functionality in other applications I am familiar with.
Mobility Feature
Only a week ago I discussed Varicent's mobility feature and I said I hadn't seen this anywhere else; PlanIt is actually available for use on PDA's including Blackberry handsets an Palm devices since April 18th. I haven't seen PlanIt mobile or Varicent mobile, but I wouldn't be surprised if most vendors followed with a similar feature.

Stevie Award for Sales Compensation Program of the Year
In 2006 and 2007, major clients of SRG won the prestigious Stevie Award for Sales Compensation Program of the Year.

Conclusion
In conclusion, I was a bit surprised I hadn't heard about Sales Resource Group before being contacted by Matt. However, with the level of maturity I noticed from their application, I would not be surprised if we started to hear more about them.

ICM Testing - Reviewing Results - Part 4

When test scenarios for an SPM application are defined properly, reviewing the results should be straightforward. However because of the volume of results typically generated, finding the correct result can be a very time consuming process without the proper skills (finding output of rule A in the 1 million results generated can be like looking for a needle in a hay stack, except that during testing, there are several needles that need to be found!).

Every ICM application I have implemented offer an interface to browse through the results. This is usually a GREAT tool to troubleshoot and to resolve issues, but it's rarely the best and quickest tool to analyze results in bulk. Fortunately, every ICM application I have seen also have a functionality to export the results. Exporting the results will create a file locally which can then be opened by a program such as Excel or Access for further analysis. I will discuss how to use Excel to review the results, and later I will create another quick tutorial on using Access.

Step 1) Export the Results from your ICM System
Every system has its own way of doing this, but generally this should be doable from the interface and not requiring a database administrator to pull the data from the database (although this can also be done if required).

Step 2) Import the Data in Excel
First, open Excel and click on File->Open. Click on the dropdown menu beside "Files of type" and click on "All Files".
Find the location where you saved the exported file of step 1 and click on "Open".

If the file is a .CSV file, Excel will automatically place the data in columns. If the file is a .TXT file, more steps are required (see step 3).

Step 3: Opening a Delimited Text File in Excel
In the first Wizard step, choose "Delimited" and click on "Next". In Step 2, select the character used to delimit each column (Xactly uses semi-colons). Click "Next" twice and the data will be imported in columns.


Step 4: Filtering the Result Data
At this point we have a worksheet with all of our result set displayed. Each of the column can be sorted and the search feature can be used to quickly find a specific order. However there is another trick which allows to quickly display data for a specific rule, result name, person, date, etc.

Click on Date->Filter->AutoFilter

This will make the first row (usually column names) dropdown menus which will contain all the values contained within the column. In the example below, the drop-down for column "Currency" allows to choose "USD" and "YEN". Choosing USD will filter out all the other currency types.

Filtering can be done in this manner for multiple columns at the same time. For example you could filter a certain position (Julien Dionne), and further filter to display only the USD currency.

Quick Summation
Summing credit and commission amounts is one of the most frequent activity I perform when testing.

Excel has a built in feature which allows to quickly find out the sum of selected cells. By simply highlighting the cells containing a number or a dollar amount, the total value of these cells will be displayed in the bottom-right corner of the window.

Additional Notes

Excel offers more advanced functionalities such as Pivot Tables. They can be useful when reviewing a large volume of results. I will discuss this in more details in the future.

Excel Limitations

Excel cannot have more than 65536 rows. If the result set exceeds this, it will not get imported. It is possible to "workaround" this issue by only importing a subset of data (maybe by restricting the incentive period). MS Access could also be used instead of Excel and would not be subject to this limit. I will discuss using Access to review results in the near future.

ICM News This Week

Centive Named as Finalist in 2008 American Business Awards
On-Demand Sales Compensation Management Leader Recognized for Providing Exceptional Customer Service

Xactly Analytics: Look at Sales Compensation On-Demand
Xactly, a sales performance management company, recently released Xactly Analytics, an on-demand application designed to provide analysis, metrics, ad-hoc reports and dashboards based on aggregated post-sales data.

Skin in the Game
Centive smoothes away the calluses for a dermatology-product vendor. Thanks to implementing Centive Compel, Obagi Medical:
- increased net sales by 32 percent;
- increased operating income by 56 percent;
- increased net income by 149 percent; and
- added 200 new active physician accounts, an 18 percent increase.

ICM Test Planning, Scenarios and Templates - Part 3

In my last post about testing I said that ICM / SPM systems should be tested in phases; the reason for this is that discovering issues late in the development life-cycle could add unexpected delays and ultimately make the budget run over-budget.

It is surprising how often I have seen the business users and stakeholders not agree with the results and the development team exclaim, "Oh, that's how it's supposed to work!" or "That's what you meant!". Without proper planning, it is possible that during UAT the users will try to perform various activities and "break" the system - to which the development team usually answers "You're not supposed to do that", or "That's not how it's supposed to work". Often, shortly before the payroll date, business users often ask questions such as "What about the draws?" or "What if orders have negative amounts" (good requirement documentation is also important!). Proper planning should eliminate all those nasty surprises.

There is an excellent article on Wikipedia regarding creating a test plan based on the IEEE 829 format. The test plan describes what will be tested, how it will be tested, what will be the deliverables, who will be responsible for what, etc.

Creating Test Scenarios for a Sales Performance Management System

I believe the most important aspect of testing is the test scenario preparation. I briefly mentioned how creating good test scenarios was particularly difficult with an ICM application because of the volume of test cases it will typically generate. This is unavoidable, but proper planning is required to ensure that tests are not testing the same conditions twice (wasting time) and that all conditions are being tested (not cutting corners).

A test scenario should have a name, a scenario ID and a description. This will help quickly refer to them during meetings. The test scenario should include the initial conditions, input (such as the order type), and the outcome or expected results. Finally, it is a good practice to list the business requirement ID that the scenario is testing. It is important for the test scenarios cover each plan, each rule, and each formula used within the rules. To test an ICM system, I like to group the scenarios, by Plan and by Rule:

1. Plan A

1.1 Rule A

1.1.1 Scenario 1

1.1.2 Scenario 2

1.1.3 Scenario 3

1.2 Rule B

...

Creating Test Data
The test data is the data that will be "staged" to test the scenarios. Typically, an order or a combination of orders will be required to test different scenarios. These orders should be created in the appropriate format to be staged and it should be documented which orders test which scenarios. Because of the different testing phases and because test data is often altered or corrupted during testing, it is important for the test data files to be kept together and be readily available to be re-staged when required.

After processing all the test data, all the test scenarios should be tested. That should cover the entire system, and in theory, after completing these steps, the system should have no outstanding defect.

The Sales Manager and the Bear

Once in a while I review the statistics for this blog to find out which keywords people use to find me. Surprisingly, people often find this blog when looking for funny sales compensation jokes. Here is a good one for your enjoyment.

The Sales Manager and the Bear
A sales manager and an operation manager went bear hunting. While the operation manager stayed in the cabin, the sales manager went out looking for a bear. He soon found a huge bear, shot at it but only wounded it.

The enraged bear charged toward the sales manager, who started running for the cabin as fast as he could. He ran pretty fast but the bear was just a little faster and gained on him with every step. Just as he reached the open cabin door, he tripped and fell flat.

Too close behind to stop, the bear jumped over him and went rolling into the cabin. The sales manager jumped up, closed the cabin door and yelled to his friend inside, "You skin this one while I go and get another!"

Source: http://www.workjoke.com/projoke60.htm (Not all jokes here are politically correct)

Common Pitfalls in Sales Compensation Design

Today I attended the "Common Pitfalls in Sales Compensation Design" webinar, hosted by Makana Solutions, featuring guest speaker Donya Rose, Founding Partner of the Cygnal Group, a sales compensation consulting company.

I did not manage to get the audio working (the toll-free number was only for Americans and the International number was out-of-service). However I will quickly recap the major pitfalls identified, based on the presentation deck.

Pitfall 1: Sales Credit Wars

Symptom: Time is spent fighting over who is supposed to get credit

Cause: Lack of documentation, rules not formalized

Cost: Lost sales, management distraction, potentially double crediting, morale issues

Solution: Document the policies and credit-sharing criteria

My comment: Another cost which must be considered is the waste of time for the comp team trying to resolve issues and conflicts. In large organizations this can be a huge time burden. However it is generally fairly easy to minimize this situation by having well established rules.

Pitfall 2: Too many Measures

Symptom: Sales people ignore some of the required results and only focus on what makes them earn the biggest commission

Cause: Too many measures...

Cost: Lack of focus, compensation hard relate to actual results

Solution: Only use a few measures.

My Comment: This is a topic I addressed a few times on this blog. Consultants generally agree that there should be no more than 3 independent measures.

Pitfall 3: Commissions Rates only go up

Symptom: Sales people can earn too much money compared to the value they bring

Cause: Commission rates are related to the level of sales even if those sales are attributable to windfalls.

Cost: Comp cost is not in line with sales contribution

Solution: The commission rate should diminish passed a certain performance level

My Comment: A "regressive" commission can protect against an unexpected windfall, but can also avoid an excessive payout caused by a quota set too low.

I often see different rules, formulas and quotas used for orders exceeding a certain mount to avoid a windfall scenario.

Pitfall 4: Extraordinary Performance is Over-Rewarded

Symptom: Dependence on over-achiever sales people

Cause: Over-performance is too attractive to sales people

Cost: Sales people developed entitlement and demanding attitude, more risks

Solution: Use appropriate deceleration in commission rates

My Comment: Deceleration does not necessarily needs to be applied as soon as the initial target is reached. I have often seen cases where the rate increased once the target was reached, and decelerated after another performance level was attained.

Pitfall 5: Unattainable Goals

Symptom: Sales people give-up because goals are too high

Cause: Goal setting issue

Cost: Lack of motivation and engagement, results below expectations

Solution: Set goals appropriately

My Comment: Goal setting should be based on historical data if possible to be "just right". Making goals too easy to attain can lead to other problems such as a lack of motivation to exceed goals if rate decreases after, or an excessive commission payout.

Pitfall 6: "Phantom Base"

Symptom: Sales People whose salary largely depends on commissions act like they are salaried and under-achieve.

Cause: Compensation plans that pay too much for prior-year sales

Cost: Sub-optimal level of performance, losing account acquisition and penetration skills

Solution: Pay more for new business and less for prior-year sales

Pitfall 7: First Dollar Commission + Base

Symptom: Sales people are too comfortable with below-target earnings

Cause: Sales people are paid a significant base salary and earn commission on sales from first dollar

Cost: Income+Commission too high for actual productivity

Solution: Only pay commission after a threshold level of sales is achieved

My Comment: Other alternatives are possible to fix this situation. The entire compensation mix could be re-evaluated and the base salary could be lowered. It would also be possible to adjust the commission rate before a threshold to minimize the impact of removing commission completely before a certain threshold.