ACA International Publishes Dispute of CFPB Consumer Survey

ACA international has released a new white paper, “An Overview of the Analytical Flaws and Methodological Shortcomings of the CFPB’s Survey of Consumer Experiences with Debt Collection,” written in response to the CFPB’s recent survey report on consumer experiences with debt collection.

In the paper, ACA argues that although the CFPB touts its consumer experience survey data as the “first comprehensive and nationally representative data,” the report, related press release, and remarks from Director Cordray highlight key findings that are primarily focused on the most negative results of the consumer survey, unfairly represent debt collection as a predatory industry, and are generally presented without critical explanatory context.
 
Some key findings include:

• The overall sample of individuals with experience with the debt collection industry represented in the CFPB survey is remarkably small. Of the 2,132 survey respondents, only 682 individuals (32%) report being contacted by a debt collector. Despite this, the CFPB continually couches its findings in relation to all American consumers with debt collection experience.
   
• Rather than report its findings with any degree of statistical certainty, the CFPB describes the survey report as a “descriptive” exercise to “highlight patterns that may be of policy interest” and “to sketch, from consumers’ perspectives, the broad experience of debt collection.” The CFPB further cautions that this descriptive sketch “does not present standard errors or statements about the statistical significance of the differences” across groups.
   
• The presentation of data lacks clarity and lends itself to overestimating the prevalence of certain findings. By focusing almost entirely on percentages throughout the report, coupled with a near-total absence of raw numbers or sample sizes for individual questions, the CFPB offers only limited context for interpreting responses or situating them within the larger sample.
   
• For areas of particular importance to the debt collection industry, the CFPB survey asked consumers about their experience without defining the legal regulations that govern some interactions. For example, the report notes that disputes are not specifically defined and “consumers’ perspectives on whether they had disputed a debt may differ from the definition of dispute used by a given creditor or collector or what may constitute disputes pursuant to the FCRA and FDCPA.”
   
• Many of the findings highlighted in the CFPB’s press release rely on the presentation of a percentage that obscures the total number of responses for a given question. For example, the CFPB reports that “three-in-four consumers report that debt collectors did not honor a request to cease contact.” A more accurate description of this finding would note that the 75% of consumers who reported continued contact after a request to cease communication are a subset of the 42% who requested contact to cease; this 42% is itself a subset of the 32% of the total sample that have been contacted about a debt in collection. Thus, the “three-in-four consumers” actually represents roughly 215 of the 2,132 consumers surveyed, or only 10% overall.
   
• In differentiating the experience of consumers when interacting with creditors versus third-party debt collectors, the CFPB relies on a faulty measurement to support its claims. While the CFPB notes that “consumers reported more favorable experiences with creditors than debt collectors along many of the dimensions surveyed,” the CFPB acknowledges the possibility of consumer confusion, pointing out that “it may also be that consumers do not perceive regular billing statements sent by creditors as collection efforts even if the statement includes a delinquent balance.” As such, the CFPB appears to be making a fundamentally flawed distinction between creditors and third-party debt collectors.

The white paper also notes that the survey was not designed to distinguish between the different types of participants in the varied debt collection market,

Although each type of market participant is distinguishable from the others and unique, the survey ultimately used by the CFPB did not allow for responses that separate the different types of market participants. As such, the conclusions drawn from the responses cannot reasonably support effective and nuanced rulemaking that is needed to properly regulate the debt collection market. Furthermore, the survey is devoid of any mention of, or specific questions regarding, several areas that would be informative to the CFPB as it considers rulemaking for the debt collection industry.

For example, there are no questions that mention or relate to the proposed survey participant’s use of information, advice or services of credit repair organizations or high-volume consumer attorneys. Likewise, there is no mention of for-profit or not-for-profit credit counseling agencies. Importantly, in the “Disputing a debt in collection” section of the survey, there are no questions relating to the level of specificity provided by the survey participant when disputing the debt, whether the survey participant provided information or documentation relevant to the dispute to the debt collector, or whether a third-party submitted the dispute on the survey participants behalf. As such, the survey results fail to include important data that is necessary to support evidence-based rulemaking for the debt collection market.

The paper concludes that the data obtained by the CFPB through the consumer survey is "insufficient at best and fundamentally flawed at worst -- therefore the survey data cannot be used as the basis to properly inform the Bureau’s debt collection rulemaking efforts."

insideARM Perspective

ACA International did a nice job analyzing the CFPB's survey report without the benefit of the raw data, which the Bureau has not made available. The results of the survey as reported were certainly not a surprise to those in the industry. 

The CFPB FinEx group, which provides resources for finanical educators, held a webinar last week regarding its consumer survey, as well as other self-help tools related to debt collection that it provides to consumers. During this webinar the group played two video testimonials of consumers expressing appreciation for the help they received from the CFPB. Like the survey, these videos left the impression that all interactions with debt collectors are negative ones. 

Debt collectors often say they receive many letters and comments of thanks from consumers for providing financial education or in some way assisting them in working their way out of difficult financial situations. During the Q&A period of the webinar one person asked whether any positive stories had been submitted by consumers, or whether the Bureau was only soliciting negative encounters. The response was that they did receive some positive stories. However, in order for those stories to make it to the public stage, they reach out to get permission from the consumers, and the ones they showed were the only ones for which permission was granted.

One of the webinar leaders, Leslie Parish (from the CFPB's Research & Markets team), acknowledged that there was indeed a "flip side" to data reported from the Bureaus's survey. For instance, she said that while one in four respondents said they felt threatened; that means that 75% said the collector gave accurate information and treated them fairly.