Forum:Peer Review

So, I've been working on the concept of Sourced Fan Fic, and I'll be ready with that in just a bit, but first I was also pondering a Peer review system. There is an awful lot of fan fiction here, and I, for one, would love to be pointed to the "good stuff", instead of having to wade through Ms. Captain Carter's First Sexual Voodoo Experience... So I've been pondering a review system (provided, of course, that this isn't a "Aabh, shut up" sort of thing :) ).

A little note: Things with a ">" are things I'm simply not sure about... Either these were problems I saw and couldn:t find a cute way to fix, or they are alternative rules that I wasn:t sure were good ideas or not...

Nomination, this is rather simple: To be reviewed, a series must be nominated for review.

>Should the nomination be seconded before it can be reviewed?

>Can an individual nominate their own works?

Once Nominated, a group of no less than 3 members of this community must read/watch/listen to and review it for content. Each member would then give it a number value from 0-10, 0 being absolute Voodoo, and 10 being Cannonical. During this time, the original article would bear the "This Article has been nominated and is currently being reviewed for content" label (Or something similar).

>Members who review the work must be registered users.

>Members who review should not be a close friend of the reviewee, though I:m sure in a pinch this rule could be lifted, we all sort of become friends over time...

After the nomination is approved The Sourced Article is then put on the main page as open for members to review under its own section with a plea for reviewers. Because of the nature of this, only Sourced Fan Fiction is allowed to be reviewed.

>I don't know what to do if a mess of Trolls decide that Starship Farragut is complete crap and all give it a unified "0" just to be mean... Should we have a review for reviewers? Should we give more weight to people in "good status"? These are things that keep me awake at night...

>What do we do if only 2 people are available to review it? Should it stay on the "To be reviewed" table until it gets the third? I don't see why not... I guess there wouldn't be a "Nominated Timeout" or anything like that... It is something to think about, though.

>I guess it might be a good idea to put some sort of time limit on reviewing... but this has its own sets of goods and bads: On the good, reviews would actually be completed... in theory... by the folks whom are notorious procrastinators and just need a little push, or they would be weeded out... on the bad, it might discourage folks from reviewing things as they don't think they have enough time to do it...

Once a review is given by the three or more members, the average score is compiled (Rounded to the nearest whole number) and the master article is awarded that number as an STEU Peer Review Rank. It is put in Catergory "Fan Fiction Reviewed by the STEU community" and the article will have its own little mark for being a Reviewed article, complete with its rank, good or bad. (This way we know what articles we have reviewed and the Voodoo articles don't keep coming back up to haunt us for review over and over).

>I think that if you have 5 or more reviewers, then the highest and the lowest scores should be tossed out and the remainder averaged. It's controversial when it happens in sports, but really, it does keep Joe Voodoo from logging in as Voodoo2 and reviewing his own piece and giving it a 10... it also curbs JoeTroll from panning everything he reviews that doesn't fit his exacting idea of FanFiction.

Reviews need not be more than a number from 1 to 10, but Super bad or super good reviews would be very suspect, and it might be better to write a brief "Why I liked/hated it" just to defend your point should the review become suspect... It would keep any subsequent reviewers from throwing out your opinion out of hand during a re-review.

Ranking and awarding If the article has a rank of greater than 5, it appears in the Catergory "Ranked as among the best of STEX Fan Fiction", if it is a 8 or 9, it also appears in the Catergory "Must Read", things ranked as a "10" are considered, "The Best Fan Fiction has to offer" and, I might add, that I will faint dead-away if anything ever achieves a perfect 10... That is almost as suspicious as a Complete "0"...

>Note: This is a very slippery slope. We have to be prepared for people to completely pan Tamerlane or Pendragon or even New Voyages... And we'd have to be willing to put our pride aside and stand behind that review... though I have to admit, if any of the biggies here popped up as a "0" or a "2", I'd call for a re-review and review it myself.

>Here's yet another problem I just thought of; Let's say I read Pendragon and I think it is absolute Voodoo, so I give it a "2"... then Tim comes along and sees that I gave it a "2"... what does he then think of me? Would I, as a user, be a little worried about giving his beloved child a "2" (Tim, thank you for being an unwilling guinea pig for my example :D)? Perhaps I'll give every series a "6" all thie time because I don't want to make anyone mad... it's like a "Reverse Troll"...

Re-review: It's bound to happen, the poor series was reviewed by Troll1701, IHateEverything and FanFicSucks23. Dispite its eloquent writing, and extremely well defined characters, the three reviewers agreed that it is a "2" (And that's only because Troll1701 felt pity for the series and gave it a "4")... The series can then be called up for a re-review, but this would only be something the Admins and Moderators can do (Otherwise, Voodoo would pop up all the time for re-review and it would drive us crazy constantly re-reviewing Voodoo).

Lastly: Guidelines. On the "I want to review something" page, there is a series of helpful guidelines that could help the reviewer think about critical functions of the review. For example: Are the characters original and interesting? Is the story new and different? Does the story fit, in your opinion, the vision of Gene Roddenbery? If not, does it take Star Trek in a new and different direction that is engaging? Of course, these are simply sugestions, but it might help the reviewers look beyond "Oh my GOD I hate the way they portray Shelby!!!!! Her hair was MUCH blonder than that!!! This show is a "0" definitely!"

So this is simply a suggestion, of course, But I figured I'd start with a framework that we can all work from, tear apart, rebuild, throw out completely, etc... but at least we will be chatting about this topic! I think it's a great idea, and I support the concept! :D Thanks Aabh 03:14, 21 June 2007 (UTC)

---