UNDERSTANDING THE RESEARCH RESULTS
The major quantitative and qualitative research results were distilled into four criteria (“Reliability,” “Inclusiveness,” “Timeliness” and “Key Info”), each representing several variables. The results were ranked on a four-point scale, and stars were used to represent the scores.
The four criteria were defined as follows:
- Reliability: measured the disconnect between the results returned from the search engine on the website and those returned by a search through the RSS feeds. “Reliability” evaluates what percentage of stories (on average) appeared through the website search but not the RSS search. The more stories that appeared through both searches, the better the score.
- Inclusiveness: evaluated whether the RSS feeds returned archived stories and non-staff-written stories (i.e. wire stories), if the website did. The more those kinds of stories appeared through the RSS feeds, the higher the composite score for Depth.
- Key info: checked if the RSS feeds included key factual information (headline/summary, dateline, byline, time posted). If all the key information was supplied, the RSS feeds would receive four stars. Otherwise, they would lose one point for each missing key information.
- Timeliness: rated how timely were the stories returned by the RSS feeds relative to those on their corresponding website. The closer the dates were between identical stories posted on the website and those received through the RSS feeds, the higher the score.
The “Overall Score,” an average of those four criteria, was reported as a number to two decimal points so a clearer comparison among the news outlets could be made.
Narrative evaluations, accessible by clicking on each news outlet on the “Main Results” page, reveal further information about the four criteria used and also comment on any additional irregularities discovered in the research.
A Caution about Evaluating RSS and Archived Stories
Creators of RSS feeds make many choices that can result in very different user experiences. If RSS feed creators (such as newspapers) decide to provide the full text of their articles in their feeds - they basically have decided to permit the Feed Readers to act as syndicators of their content. As mentioned in the 'What is RSS?' section, Feed Readers make available to new subscribers all the content already provided to existing readers in the past. It is most common for the mainstream news agencies to only provide either a headline or teaser paragraph in their feeds, requiring readers to click through to the news agency's website to read the full article. As links in RSS feeds grow old, the articles to which they link may move into the archives of the news website. What happens when an RSS feed subscriber clicks on an old archived news story's link? Well - that depends entirely on how access to old news stories is handled in general.
One final thing to keep in mind - just because a Feed Reader has a copy of an old article in its archive does not mean that the content creator can't modify the article and remove some mistake or otherwise change the links or content. Content creators can alert Feed Readers to the fact that an updated version of an article exists - at which point the prior version is overwritten.
|
