Best Brokers - The Reviews Data exposed?

[dbrawner]

I'm having a challenge trying to interpret the broker evaluation data.

Since there is no way to sort the data on ratings AND reviews, one can not get a clear picture of which broker is really on top.

If we sort by rating, a dozen brokers are rated 5 star BUT... the these have too few reviews to be statistically significant by reasonable standards. In other words, having only 7 reviews rating a broker at 5 stars does not give me a warm and fuzzy feeling.

On the other hand, it seems the most reviewed sites (432 to mid 100's reviews) are mostly sub to average at 2-3 stars.

So, statistically, a high rating from a few reviewers is probably simple bias, while a statistically significant number of reviews plummets the ratings to, well, just average or slightly below. (actually this could be expected but it doesn't help a broker selection process).

Am I missing something or does the industry have a real problem with retail brokerages being perceived as sub-par?

Is there a statistician out there who can make sense of the broker review data?

Frankly, I have two issues with the broker reviews;

First, the reviews themselves and the ratings given. I'll wager that fewer than half are meaningful from the standpoint of the rating system (https://www.forexpeacearmy.com/fore...riptions-forex-peace-army-review-ratings.html) or the qualifications of the reviewers. Certainly, with the exception of some very specific examples, most reviews were too generic in content to support the rating provided. Also it is difficult to determine who, or how many, reviewers weighed in more than once, or who may have had a personal agenda (i.e. bias) for rating a broker one way or another.

My Second Challenge; the evaluation and presentation of the broker ratings. The top reviewed broker, with 432 reviews, received a 2.5 star rating, it's next competitor, with 341 reviews was rated 3 stars... on down to a rating of 3 stars from 215 reviews. In fact, of the top 6 (those having more than 200 reviews) the average rating was only 2.75 (the median being 3.0).

Further analysis revealed;

12 brokers with 5-star ratings received an median of only 4 reviews (high 7, Low 3).
17 brokers with 4.5-star ratings received an median of 5 reviews (high 34, low 3).
18 brokers with 4-star ratings received an median of 5 reviews (high 137, low 3).

35 brokers with 3.5-star ratings received an median of 13 reviews (high 197, low 3).
18 brokers with 3-star ratings received an median of 8 reviews (high 341, low 3).

and 368 brokers received a rating less than 3-stars with the best of the lot receiving 2.5 stars on 341 reviews.

Based on these data, statistically speaking I wouldn't risk my money with any of the 607 brokers reviewed.

Those having a significant review count are sub to average at best, and those with commendable ratings have an insufficient review count for the rating to be considered reliable.

Am I missing something???

 

[Pharaoh]

I think the flaws you've pointed out are common to all review systems. At least the FPA tries to filter out the ones where what's written isn't even close to the given rating. Exactly where to draw that line isn't going to be an easy task. Not all reviewers are very skilled writers.

I covered this in my article about how to select a forex broker. You can't just look at the overall rating. You have to read all the reviews and assign your own weighting to the scores they give based on the content of the review itself.

Checking the total number of reviews and how long the broker has been listed is also a very good idea. Cross-checking some of those 5 reviews=5 stars new brokers in google to see if there are complaints elsewhere is a basic part of due diligence that every trader should do before funding an account.

The FPA's reviews are a tool. Like any tool, they are much more useful if you use them with skill.

 

[AsstModerator]

I'm in charge of the FPA's Review Moderation Team.

Are the FPA's reviews perfect. No.

Are they about as good as I and my team can make them. Yes.

I wish I could get everyone posting about a broker to give more information. I'd like to know how long each reviewer has had a live account with the broker. I'd like to know if each reviewer has tried other brokers or not. If one complains about a withdrawal problem, I'd like to know the date that the withdrawal was first requested. I wish all of the reviewers had good English writing skills. I don't want a review to take up pages of space. I do want to see at least a couple of sentences.

If someone complains that a withdrawal is 1 day late and leaves 1 star, that rating usually gets removed. If someone leaves a 5 star review about how easy it was to open an account, that review usually gets deleted.

You complain that not all the approved reviews are detailed enough. Go here and see a small sample of what doesn't get approved...

https://www.forexpeacearmy.com/forex-forum/basic-training/2228-worst-forex-reviews.html

On the other end of the spectrum from your complaint, I get people emailing in accusing the FPA of taking bribes to help some companies with the reviews. One recent one wrote several emails ripping the FPA. His review got deleted because he wrote two different 1 star reviews for the same company under 2 different names in less than 6 hours. He seemed to think the rules against review spamming only applied to people who didn't agree with his opinion. Another one with the same accusation was submitting reviews and giving wrong information about what country he was in. Neither one of them checked the Commonly Emailed Questions. The information there would have told them exactly why their reviews were deleted. You can read it here...

https://www.forexpeacearmy.com/fore...ost-commonly-emailed-questions.html#post78728

The truth is that the FPA does get bribe attempts. All of them are turned down. The record bribe attempt so far is $250,000. Many new advertisers are very surprised that their money only buys ads and does not come with preferential treatment from review moderators.

The tools I and my team use to catch people trying to cheat in the reviews keep getting better. There's no way to catch all fake reviews. A few real reviews will get deleted. If someone submits a review that doesn't get approved within 5 business days, there's a "Contact Us" link on every review page. Tell what company you left the review for, the name you used and about what date was submitted on. If you see a review that looks like it shouldn't have been approved, you can also use "Contact Us" and talk about it. Give the company name and name and date on the review. I or one of my staff will look at the situation.

If someone has a great idea to improve the reviews. the "Contact Us" link is sitting on the bottom of this page.

The FPA's reviews are not perfect. I think that the long hours I and my staff put into them make them the most helpful and honest forex reviews on the web.

 

[dbrawner]

To Pharaoh, AsstModerator, and anyone else who has dedicated time and effort to the Broker Review Ratings process,

My sincere apology for coming off critical or contemptuous. This was not my intention and I am sorry.

I was trying to gain feedback regarding the analysis and use of the data made available and not attempting to cast any disrespect to those who have made it possible.

Pharaoh, you ended with; "The FPA's reviews are a tool. Like any tool, they are much more useful if you use them with skill. " I wholeheartedly agree with this statement. It was my desire to begin a discussion thread that would help members gain those very skills, with myself being of particular interest.

Mr. AsstModerator, you have labelled my post a complaint, for which I can only blame myself if I was not clear. It was not meant as a complaint but rather a plea to help me understand. I used the references as clarification of my point of view, not as a criticism of your efforts or methodology.

To that end, you have indeed added clarity as to the methodology of the review screening and the limitations of the result set provided.

This all started in response to an email that challenged us to spend 10 minutes a week to improve the industry. I was taking up that challenge and shall continue to do so.

Perhaps, given the additional challenge to "put up, or shut up" (paraphrasing "If someone has a great idea to improve the reviews. the "Contact Us" link is sitting on the bottom of this page."), I shall endeavor to come up with a more objective scoring and analysis model that I will pass on to you in the next few months.

All this being said, the resources, forums, and intentions of FPA are not now, never where, and will not in the foreseeable future be in question. You team effort is applauded and recognized.

Again, my hand is extended in peace and goodwill from a grateful junior member.

Dave

 

[Pharaoh]

Hi Dave,

I think our AsstModerator is open to ideas on how to improve the reviews. The hardest part is getting reviewers to sit down and think for a moment before posting something based on their most recent trade. So many of them either just closed a profitable trade or just had an experience with slippage (legit or not) and leap at the chance to post a 5 star or 1 star review as quickly as possible.

There are some great reviewers. They write detailed, yet concise (unlike me, I'm never concise) reports and also come back and leave followups later on. Those are the ones I pay the most attention to.

 

[RahmanSL]

Dave...wow...you do take the time to count all those reviews

For me, I only read reviews on those brokers which I am interested in or think are "reasonably" trustworthy. If there are outstanding issues from their clients left dangling, sometimes I would prod the broker representative for answers to that or those problem(s). If no reply(ies) is/are forthcoming from that broker'e rep, I simply move on....but I do check back once in a while to see if there is a response.

It's impossible to decide with any degree of accuracies on the reviews given not only here at the FPA, but elsewhere in other review forums. Ultimately, its up to us traders to make our own judgement and decision on the broker. For me I simply try out the broker with a small amount of capital to test them out. If they are "reasonably" good, I keep them, if I am not happy with them, I simply dump them....sort of like dating


AsstModerator Lieutenant Sir!...I have been meaning to ask this for a while.
How come you are not a "General" or some higher rank?
And that "Son-Of-Ra'a, Ruler of Upper & Lower Egypt" has been a Master Sargent" for like forever even though he has almost 4k posts under his belt....and a Pharaoh to boot!
Not that it counts much in the way of....ehhem..hem...salary, but the title does add a certain flair to ones status


"The truth is that the FPA does get bribe attempts. All of them are turned down. The record bribe attempt so far is $250,000...."

By the way, all offers will be duly considered and much appreciated by my office!