A Defense of Ratings
(The following is an exposition by a Christian ministries rating and ranking group called Ministry Watch and Wall Watchers which expresses the ideology of ARK AID in an excellent way)
The name 'Ministry Watch' in this revised article has for that purpose been replaced with 'ARK AID'.
Introduction
One of the more controversial issues often explored in any broad discussion of nonprofit accountability is the issue of ratings. Academics, accountability groups, charity watchdogs, and individuals express a broad spectrum of opinions on this issue. ARK AID is a nonprofit research organization that produces ratings on nonprofit organizations, specifically Christian ministries, and more specifically those that are involved in poverty alleviation. This article is being written primarily to address the most common arguments against producing ratings on Christian ministries. In order to provide background and context to this defense, a brief introduction is offered for ARK AID and its 5 Star Ratings system. The remainder of the article discusses the issue of ratings by summarizing the basic
reasons for producing them, and then addressing the most common arguments against producing them for Christian ministries.
Background
Who is ARK AID?
ARK AID is a Christ-centered, nonprofit organization committed to promoting biblical principles of stewardship and helping people apply those principles in their lives. The ministry currently operates three programs - Appraisal (a-AAA.com), Awareness (a-AAA.net) and Assistance (a-AAA.org). The activities of ARK AID Appraisal revolve around researching, evaluating, and rating Christian ministries in order to make useful information about those organizations available to the public. ARK AID does not charge for its services.
What are the 5 Star Ratings?
Incorporated within the AAA research program, the 5 Star Ratings are summary measures of a ministry’s results. The purpose of the ratings is to provide a useful tool that researchers, donors, and nonprofit managers can use to assess a ministry’s value.
The Case For Ratings
What are ratings?
Before engaging in a debate about ratings, it is important to define exactly what ratings are. According to one dictionary, a rating is simply a “classification according to a standard.” Two things are necessary toproduce ratings. The first is a classification system. This system could use numbers, letters, names, or symbols to describe different classifications. The second is a standard for assigning the chosen classifications. A standard could take many forms. It could be simple or complex, objective or subjective, absolute or relative, quantitative or qualitative. The standard determines what traits or conditions will yield each classification. Ratings, then, are nothing more than the results of assigning things to classifications using a specified standard. Understanding that ratings are nothing more than what is communicated in this simple definition helps one to see that many of the arguments against ratings are really just arguments against a particular ratings system and not arguments against ratings themselves.
What is the purpose of ratings?
The purpose of ratings is to summarize a large amount of information and to convey the result in an easy to
understand form (a rating). Ratings are popular because many people lack the time, resources, skill, or
desire to process a large amount of information themselves. Individuals use ratings to increase the
efficiency of the decision-making process. Despite this obvious benefit, people will only use ratings
systems that process information that is relevant to them, and which classify things according to a standard
that is consistent with their own values and assumptions. In many cases, individuals use multiple rating
systems to compare alternatives and make decisions because no single system meets their preferences
exactly. Ratings, then, are simply tools of decision-making used in conjunction with other information
relevant to an individual. They are valuable because they provide consumers with information that they
may not have easily obtained on their own in a form that is easily understood.
Why is ARK AID producing ratings on Christian ministries?
ARK AID has created the 5 Star Ratings system in order to improve the quality of information
available about Christian ministries. This improvement benefits donors and ministries alike.
Donors benefit from the introduction of ratings primarily because they allow them to more efficiently
process a large amount of information. When donors are able to incorporate more information into their
decision-making process, they are able to make better-informed giving decisions. Better decision-making
translates into better stewardship of the time and resources with which individuals are entrusted. (It is
important to note here that although introducing ratings is an important step in the effort to improve the
quality of information available, it would be foolish to suggest that it is the only step, or even the most
important step. Increasing the availability of financial statements, annual reports, and other information
about the activities of Christian ministries is just as important to improving the quality of information as
providing new tools like ratings which allow this information to be processed more quickly and easily.)
Ministries benefit from the introduction of ratings for two important reasons. First, ratings can help
ministries to evaluate their own practices and possibly identify areas for improvement. It is not often
practical for a ministry to gather extensive information about other ministries for use in self-evaluation.
Ratings can provide some of this information for them. Second, ministries benefit when donors feel more
confident about their giving decisions. This is true because donors who are more confident are likely to
give more. This is a benefit that can have a far-reaching positive effect on Christian ministry as a whole.
Because they increase the quality of information available about Christian ministries, ratings can empower
donors, help ministries, and increase giving. It is for these reasons that Wall Watchers has created the 5
Star Ratings system.
The Arguments Against Ratings
Introduction to the Four Main Arguments
Many arguments have been put forth as to why ratings of Christian ministries should not be produced.
While these arguments are often viewed as separate from each other, each one can usually be classified into
one of four main arguments. These four main arguments can be stated as follows:
1. Ratings of Christian ministries are inconsistent with Christian principles. (They are bad by
their very nature.)
2. Ratings of Christian ministries have a negative impact on the cause of Christ. (They produce
bad consequences.)
3. Ratings are useless for the evaluation of Christian ministries. (They have no value.)
4. Ratings of Christian ministries cannot be produced fairly and objectively. (They are not
possible.)
Throughout the remainder of this article, each of the four main arguments against producing ratings of
Christian ministries will be addressed. For each main argument, examples of how the argument is usually
made will be discussed.
Main Argument # 1
Ratings of Christian ministries are inconsistent with Christian principles.
(They are bad by their very nature.)
This argument is probably the most important to address because if it could be shown that ratings are in fact
inconsistent with Christian principles then no further argument would be necessary. However, this is also
an argument that should not be made lightly since it requires one to make a judgment about what is right or
wrong, not just about what is best or practical. This argument is essentially that ratings, due to their very
nature, contradict the basic principles of the Christian faith. Two of the forms this argument takes will be
addressed.
Argument: Ratings require us to assess ministries in a way that God does not want us to do.
ARK AID's Response
The argument is sometimes made that anyone attempting to rate a ministry is somehow exercising
judgment upon it in a way that is inappropriate. This argument is based on a misunderstanding of what
ratings are. According to the definition given previously, ratings are nothing more than classifications
according to a standard. They are not, by definition, judgments of a ministry’s eternal significance.
Ratings simply establish a standard and communicate how a ministry is classified using that standard.
In an effort to encourage accountability within the nonprofit sector, several respected watchdog and
membership organizations have established standards by which to assess nonprofits. These standards are
often conveyed as operating principles that would be desirable in any nonprofit. Although these
organizations would not consider their standards to be a rating system, they are nevertheless providing a
way for organizations to be classified. Either an organization meets their standards, or it does not. Wall
Watchers agrees that assessing an organization according to whether or not it conforms with a set of fixed
standards is a positive step toward encouraging accountability, but also wonders how classifying
organizations by another standard (i.e. producing financial efficiency ratings) is any different? Many
donors use standards as an important criterion for determining whether or not to support an organization.
Are we to believe that the final word has been spoken about organizational effectiveness? Or is the
increased availability of information and the tools to process that information a tremendous opportunity to
improve the quality of information available to donors interested in being good stewards with their
resources? Wall Watchers believes that donors should be able to decide for themselves what information
they will use to make their giving decisions.
Producing ratings does not contradict God’s directives. It recognizes that God desires us to be accountable
to one another and to be good stewards with our resources. To this end, ratings can increase the
opportunity for accountability and stewardship to occur.
Argument: Christians should rely on God alone for help in deciding to whom they will give their time and
money.
ARK AID’s Response
Clothed in spiritual language, this is essentially an argument in favor of ignorant donors. The assumption
that using ratings to help make a giving decision subverts or replaces a donor’s ability to assess an
organization within the context of his or her own values and vision for the ministry is simply false. When
making other important choices in life, is it a responsible course of action to ignore any information that is
obtained in a way other than prayer and reflection? Or are we encouraged to seek advice from those we
trust, search the Word, and use our God-given gifts as well? Ratings are just another piece of information
that can potentially help donors with their giving decisions. Donors always have the ability to choose
which information they will take into account and what information they will disregard. This argument
against ratings is akin to saying that people should not be encouraged to think when giving, but instead
should simply rely on the Holy Spirit’s inspiration to lead them. If this is the case, why do we bother with
any attempt at accountability? Perhaps we should simply rely on the Holy Spirit to keep us from giving to
organizations that should not receive our contributions. Wall Watchers believes that donors should be
trusted with not only more information, but with different types of information, such as financial efficiency
ratings. A better-informed giving population will only benefit the entire Christian community.
Main Argument #2
Ratings of Christian ministries have a negative impact on the cause of Christ.
(They produce bad consequences.)
The first argument focused on the nature of ratings. This argument focuses on the consequences of
producing ratings. Even if ratings are not bad by nature, the argument goes, they produce consequences
that negatively impact the Christian cause and so should not be produced. Two forms of this argument will
be addressed.
Argument: Ratings create division in the Christian community because they diminish each ministry’s
uniqueness. This discourages cooperation and mutual respect among ministries.
Wall Watchers’ Response
It is difficult to understand how providing more information about Christian ministries cultivates division
in this way. The argument often made here is that ratings cause division because they diminish the
uniqueness of ministries. First, how do ratings affect a ministry’s uniqueness? It is not difficult to see that
each ministry can be considered “unique,” just as it is not difficult to see that each person, or place, or
animal, or company is also unique. Few things are exactly alike, so ministries are hardly unique in their
uniqueness. Although ministries are unique, it is difficult to see how classifying them according to a
common standard (i.e. rating them) diminishes their uniqueness in any way. The act of classification (or
rating) involves an examination of a set of traits about something (determined by the standard) and then a
description of how the thing being examined compares to the standard (resulting in classifications). This
process of examination is very much like looking at the same thing that you have looked at many times
before, but through a slightly different lens. This lens highlights aspects of the thing that you may not have
noticed before. This kind of examination cannot change the nature of something in any way. In fact,
examination of a ministry in different ways will more than likely uncover new aspects of the ministry that
make it seem even more unique. Though it may appear similar to one organization when examined using
one lens, it may appear to be completely different when examined using another.
Second, if ratings do nothing to diminish a ministry’s uniqueness, then how do they cause divisions among
ministries? The answer is that they don’t do this at all. Divisions between organizations are not caused by
the attempts of others to examine them in new ways. True “division” can only occur between organizations
when the organizations themselves allow it to occur. Furthermore, if cooperation and mutual respect are
predicated upon a refusal to examine one another more closely, then this cooperation does little to
encourage true accountability.
Although this response was primarily intended to address the argument that ratings cause a certain kind of
harm in the Christian community, it is important to point out that ratings can in fact have a positive effect
on the Christian community. It is when we choose to limit the amount of information available (not when
we make more information available), or when we choose to look the other way when information is not
flattering that we are truly doing the Kingdom a disservice. When information raises a question in a
donor’s mind, it is often much easier to argue that the information being examined is not relevant than it is
to really try to understand the reason for the results. Ratings can benefit the Christian community because
they provide an opportunity for donors to become more informed about a particular ministry or ministries.
True respect and accountability is not achieved through mutual praise, but through asking the tough
questions out of Christian love and desire for improvement. Are Christian organizations so incapable of
organizational improvement that there is no need to compare themselves to one another and seek
improvement in the ways they operate? Do we really believe that the principles of effective management
and the tools of analysis cease to apply when we attach the name of Christ to an organization? It seems
more likely that organizations are sharpened by increased analysis of their activities and that donors can
serve an important role in providing feedback to the ministries they support when they are empowered with
more information and better tools for analysis. Ratings are simply one such tool.
Argument: Ratings will cause competition in the Christian community.
ARK AID’s Response
To adequately address this argument, it is important to clarify what is meant by the term “competition.” In
the ministry world, the scarce resource over which ministries are most likely to compete is financial
support. A limited amount of financial support is provided each year to ministries. In order to ensure that
they have the resources necessary to carry on their work, ministries must continually demonstrate to current
and potential donors how the work they are doing is deserving of continued support. To say that ratings
cause competition between ministries in this area is ridiculous because it implies that this competition
would not occur if ratings were not there. Competition for a limited amount of resources has existed and
will continue to exist regardless of whether anyone decides to produce ratings on Christian ministries.
Perhaps the important question to ask at this point is not whether ratings are the cause of competition
between ministries, but whether or not this competition is necessarily a bad thing.
Competition is not a word that Christians should associate with evil and selfishness. Quite to the contrary,
competition is what facilitates efficiency and innovation in our economy. Many would acknowledge the
positive effects competition has had on the general economic growth of this country. Why shouldn’t it
produce similar results in the nonprofit sector? For example, let’s assume that there exists a Christian
ministry that has successfully accomplished its mission of feeding hungry children around the world for
decades. The ministry is well respected, but as it has grown in size, it has lost some of its edge. Donors to
this ministry hear about a new Christian ministry that uses innovative cost-saving strategies to feed more
hungry children with fewer resources. This ministry also provides its donors with more opportunities to
become directly involved in the process. As a result, some of the donors to the established relief
organization begin giving to this new ministry instead. Should the new ministry be criticized for taking
support away from the more established ministry? Should we prevent competition by mandating that only
one organization be formed to feed hungry children? Clearly, the answer to both of these questions is no.
The point of this example is that competition for limited resources is not only unavoidable, but is a force
for positive change in the nonprofit sector just as it is in the for-profit sector. Efficiency and innovation
occur when donors allocate their contributions to those organizations that accomplish their mission in a
way that is most consistent with donor expectations. Ratings are merely an information tool that donors
can use to help them identify those organizations. Limiting the information and tools available to donors
only limits their ability to make responsible choices.
Main Argument # 3
Ratings are useless for the evaluation of Christian ministries. (They have no
value.)
The first two arguments against ratings focused on reasons why ratings are inherently bad or produce bad
consequences. The third argument takes a more passive approach. It argues that even if ratings are not
inherently bad and don’t produce bad consequences, they are basically useless. If they are useless, the
argument goes, they should not be produced because producing something that is useless will only waste
resources and cause confusion. Two forms of this argument will be addressed.
Argument: Quantitative analysis is not a relevant basis on which to evaluate nonprofits.
ARK AID’s Response
This argument is similar to the argument that ratings place us in a position of judgment over ministries
inasmuch as it is based on a fundamental misunderstanding of what ratings are and what they are not. If
you will recall from our definition, a rating is merely an assessment of an organization according to a
certain criteria (the standard of the rating system). A rating cannot give any indication of how an
organization fares in any other criteria. It is concerned only with its own criteria. For example, the 5 Star
Ratings system used by Wall Watchers attempts to assess an organization’s “financial efficiency” as
defined in that rating system. It tells nothing about changed lives, transformed minds, or saved souls. It
was not designed to measure such things. This may beg the question about whether financial efficiency
measures are relevant. To some, they may be relevant and to others they may not be relevant. The point is
that it is up to the donor to decide if the rating is relevant. To say that it would never be relevant is to claim
knowledge about the preferences of every donor. If a rating system is developed, and no one finds its
results relevant, then that rating system will not be around for long. To claim that no ratings system based
on quantitative analysis would ever contain information that is relevant to donors is quite presumptuous.
Wall Watchers believes that empowering donors with as much information and as many tools as possible is
preferable to deciding for donors what they should consider important. In fact, Wall Watchers
acknowledges that its rating system may prove to be irrelevant to donors. Ratings systems themselves are
subject to competition. To use an example from the for-profit world, you can probably find as many ways
to rate and evaluate stocks as you can find analysts to use those systems. It is one thing to argue that a
particular ratings system does not examine the appropriate information. It is quite another thing to argue
that the process of rating itself is inappropriate. As long as the act of examination has the potential to add
value, the act of rating has the potential to add value as well.
Argument: Quantitative analysis may be useful to an organization’s internal management, but is not useful
to anyone outside of the organization.
Wall Watchers’ Response
This argument demonstrates very little faith in the intelligence and common sense of donors. If it is
acknowledged that organizations can benefit from financial analysis, why can’t donors benefit as well?
The only answer is that somehow donors are incapable of drawing accurate conclusions from the same
information. Although there is always a need for increased education, keeping donors in the dark because
they are not to be trusted does not seem to be a good way to improve accountability.
Main Argument # 4
Ratings of Christian ministries cannot be produced fairly and objectively.
(They are not possible.)
The fourth argument against ratings relies the least on any theory or philosophy about ratings or ministries.
It looks at the task of producing ratings and concludes that this task simply cannot be done properly. Even
if ratings could be a useful tool for decision-making, the argument goes, it is simply not possible to produce
them fairly and objectively for ministries. Therefore, ratings should not be produced. Three forms of this
argument will be addressed.
Argument: Since ministries are unique, it is unfair to compare one to another.
ARK AID’s Response
This argument relies on the same faulty premises used in the argument that producing ratings diminishes a
ministry’s uniqueness. Ministries are not unique in their uniqueness. Moreover, the fact that something is
unique does not preclude it from being compared to something else. Two unique ministries often face
similar challenges and deal with those challenges in different ways. Examining different ministries using
the same standard provides insight into how successful each ministry has been at meeting similar
challenges. Despite their uniqueness, ministries have a great deal in common with each other. Most
ministries must seek financial support to carry out their mission. Most ministries must use assets such as
buildings and equipment to conduct their activities. All ministries use people in some way to accomplish
their mission. Comparing the ways that ministries perform these fundamental activities is one of the
insights that ratings have the potential to provide. Ratings are not precluded by uniqueness. On the
contrary, uniqueness increases the potential value of ratings.
Argument: Truly objective standards cannot be created for use in a rating system.
ARK AID’s Response
The definition of ratings used throughout this article does not specify what types of standards must be used
when producing ratings. The definition only requires that a standard exist. The fact that there is no single
perfect standard to use in a ratings system does not render the act of rating fruitless. If the preferences of
every donor were the same, then it could possibly be argued that one set of standards would suffice.
However, every donor is unique and so any one ratings system will not be equally relevant to every donor.
Wall Watchers believes that a fully developed nonprofit information marketplace will eventually offer
donors a myriad of tools and ratings systems to help them with their giving decisions. It will be up to each
donor to decide what factors are relevant to them and seek out the tools that allow them to assess those
factors adequately. In fact, the standards employed do not even have to be objective. The 5 Star Ratings
system used by Wall Watchers employs standards that are objective in determining financial efficiency, but
this does not have to be the case. The point made many times throughout this article is that the purpose of
ratings is to empower donors with useful information, not to determine for them what factors they should
use when making their giving decisions.
Argument: The data available on nonprofits is not suitable for producing ratings.
ARK AID’s Response
We live in a world of imperfect information. This is a fact of life that will always exist to some degree.
However, to ignore information because it is incomplete or imperfect will only leave us in a state of
ignorance and paralysis. If we are not using the information that is currently available to us, imperfect as it
may be, what impetus will exist for improving the quality of information in the future? Unfortunately,
information (especially financial information) about nonprofit organizations is much more difficult to
obtain and much less reliable than the information available from for-profit organizations. Because
investors in for-profit companies place such a high premium on good information, improvements are
continually being demanded and realized in the availability and quality of information about companies. If
donors wishing to improve their decision-making demand better information with equal fervor, then the
availability and quality of information about nonprofits will improve as well. Creating a well-informed
giving population is a long-term goal. However, this goal will not be accomplished if we allow imperfect
information or a lack of faith in individual donors to freeze us into a state of developmental paralysis.
Summary Comments
This article has attempted to articulate a fundamental belief in the ability of information to empower
individuals and facilitate positive change. Because ratings are essentially just another piece of information,
they are just as capable of doing this as any other piece of information. It was noted at the outset that
opponents of ratings often make their arguments based on a false understanding of what ratings really are,
so a clear definition of ratings was offered. The reasons for producing the Wall Watchers 5 Star Ratings
system were offered to provide a glimpse into how one organization believes ratings can have a positive
impact upon the nonprofit environment. The rest of the article addressed each of the main arguments
against producing ratings on Christian ministries. Although many reasons are often given, there are four
fundamental arguments typically offered. In the course of addressing each of these arguments, an attempt
was made to keep the focus of the debate on a big picture view of ratings and how nonprofit accountability
and stewardship are achieved in the long run. This article was not intended as a defense of one specific
ratings system, but as a defense of the very act of producing ratings. This article was not intended as a
defense of ratings as the only legitimate basis upon which donors should make giving decisions, but as a
defense of ratings as one tool that can be used when doing this. At the end of the day, AAA is in
favor of any tool that places more information in the hands of donors, because it is only when donors are
truly empowered that real accountability and real stewardship can occur.