Okay, let’s get ‘real world’ with Demand Rating™.  What can you do with it?  More importantly, how does it help publishers and information providers monetize opportunity?  Let’s first take a look a very common subscription service concern, account sharing.

Account sharing for subscription services is often a given—for some industries it’s rampant and understood, in others it’s less common, but still done.  Let’s assume for now, that with every subscription service, some account sharing is occurring.  As a business leader in that organization, do you care?  Do you chase each and every case of account sharing?  Can all be successfully monetized? The answer is:  It depends.

If the subscriber is sharing one account but not using the other 99 of their 100 paid accounts, do you care?  Probably not.  The real problem might be risk of losing the subscriber, not sharing.  If a long term, large subscriber had significant sharing in a single office for a 3 week period, do you care?  Probably not.  The sharing may have been due to a specific project or to a logistical issue—nothing worth damaging the long-term relationship.  What if you discover that over a half of all of your accounts are being shared?  Which accounts do you chase first?

Demand Rating helps publishers and information providers determine whether account sharing can or should be monetized.  It helps qualify opportunities that are ripe, and helps sift out those that aren’t. Subscriber usage, cost and value are intrinsic to the formula—normalized—allowing customers with identifiable sharing to be ranked, compared and rated.  Our work with providers shows when sharing is occurring, about half of lost revenue can be monetized successfully.  Demand Rating lets you know which half.

• How good can these tools be?
• What is the difference in performance from Scout Analytics?

Investigation
Using real world data, we looked at different techniques to analyze the IP addresses such as: total count, average interval, frequency, smallest interval, and others.  All the techniques computed an IP-related metric for an account (such as the total count of distinct IP addresses) and compared that measure to a threshold.  Next we compared the percentile of shared accounts above the threshold to the percentile of non-shared accounts above the threshold. This gives us an idea of how effective a measure is.

The best results for detecting unlicensed use from IP address was measuring the smallest interval between logins originating from different IP addresses.  Unfortunately, the technique has a high false positive rate associated with it.  To detect 25% of the accounts with unlicensed use, the technique would have a false positive rate of 10%!  Dropping the false positive rate to 1% would only allow 4% of the violators to be identified.  Overall the performance of IP address techniques is poor.

Thoughts
Why do IP address related measures perform so poorly?  IP address counts have a big range in values for a single-user account (e.g., mobile workforce). Some accounts used a single address, others used over 80! This range overshadows the vast majority of differences that might be caused by having multiple users in an account vs. only having a single user.

Scout Analytics has been analyzing access patterns for one online service that has approximately 6,000 active users.  A recent regression analysis was performed where the percentage of shared accounts was correlated to the frequency of access as measured in standard deviations from the median.  The graph below shows that frequency of access was not a good indicator until more than 9 standard deviations above the median which represented < 1% of the shared accounts.