PDA

View Full Version : A critique of Calibre's reviews



GunsNotPuns
01-04-2015, 12:39 PM
At the outset, let me say that my criticism of Calibre's reviews should not be reviewed as an attack or a signal that I hate the magazine. For the record, I paid for a subscription as a sign of my support for a relatively new entrant into firearms journalism and barring some unforeseen event I intend on renewing at the end of this year.

That said, I've noticed a bit of a tendency in the magazine's reviews that do irk me -- and its the tendency to sweep aside problems on the path of giving a generally very positive review. When I read the magazine's first year issues on the web site I looked at this as the growing process of a staff that was transitioning from enthusiasts to journalists. The two aren't mutually exclusive but there is a higher standard required for the later then the former.

I might be cherry picking but Vol . 2 Issue 6's (incidentally, a magazine with no date anywhere?) review of the Dagger SAP6 tactical shotgun perfectly illustrated what I'm talking about. Described on the cover as a "working class hero" there was no MSRP so I will assume that it retails for approximately $600. That's not a lot considering how much you can spend on a tactical shotgun but it's still a good hunk of money for most people.

Going through the review the first issue that Geordie Pickard noted was that after just 200 rounds the screws holding down the rail began to loosen up. Pickard describes this as "not a huge concern" for an owner which I must respectively disagree with. Pickard writes that an owner could resolve this themselves with Loctite or a tool but I would argue that if the mounting system for a rail becomes loose after 200 rounds, even a shotgun, it's a problem.

The next issue that Pickard identifies is the very stiff safety on the SAP6. Even 700 cycles of the safety failed to loosen it up -- and Pickard apparently has a lot of experience working with difficult materials. Pickard states that his is obviously an early production model and it can be forgiven if an "oversized part" is responsible. I would say that is unacceptable. I have long operated under the maxim that if you make something difficult for people to do, they eventually will stop doing it. A safety that apparently requires considerable effort to engage falls into that problem. It is remarkable that whatever testing Tactical Imports conducted on their early/prototype models, ensuring a smooth functioning safety wasn't a consideration.

Another "quirk" that Pickard describes -- and he seems to love the word "quirk" -- is the absolute inability to field strip the shotgun for cleaning. The only way to remove the bolt is to remove the trigger assembly which itself requires the removal of both the pistol grip and the stock. So either you completely disassemble the SAP6 or you give it a very cursory clean. I do not have the experience with firearms that some people do but I have cleaned more than my fair share of firearms and I cannot remember in several decades of cleaning of a weapon that features an inability to field strip and clean. You can add to this problem Pickard's reporting that the manual that comes with the SAP6 to be brief at best and there exist no detailed instructions (at the writing of his review) anywhere on the internet.

Do note, Pickard identified some other issues -- sorry, quirks -- with the SAP6 but these are three big ones for me. He wraps up his review endorsing the shotgun and notes "if you want a mag fed pump action shotgun and you would rather deal with a couple of quirks than the build quality issues we've seen with some of the competitors, the SAP6 is your gun."

Woah, really? Of the three issues I've noted, two of them are in fact build quality issues -- a safety that is difficult to engage and a rail that begins to become loose after 200 rounds -- while one of them is the product of a poorly thought out design process. Forget the odd rail system that precludes centre mounted additions, difficulty side loading and a lack of an action lock -- things that Pickard himself noted.

I'm not sure how a positive endorsement could have resulted. I realize the Canadian civilian firearms industry -- though rapidly expanding -- remains small. Offering poor reviews of productions could result in a company deciding not to offer Calibre review units in the future. That's not a problem in the U.S. obviously -- diss an AR builder and you have a thousand others you can review. That, however, cannot be a consideration. Reviewing a product, be it a firearm or gear, has to be done clinically.

Don't get me wrong -- the SAP6 isn't the Remington R51. The issues that Pickard described in his review don't rise to the level of that troubled handgun's short history. That said, build quality issues and a ludicrous design choice can't lead to a positive review. Mixed, perhaps with a cautious endorsement, but not one where issues are described as "quirks". If I'm a new SAP6 owner and my rail becomes loose after 200 rounds, I don't go to the store to buy Loctite, I go to the store to return the firearm.

Again, I'm not picking on Pickard or Calibre. Publishing a magazine isn't easy. I was a professional writer for many years and have had hundreds of pieces published in magazines and newspapers across the world and did my fair share of editing as well. Writing ain't easy, yo. However, we need a higher standard from our journalists in the firearms world, not simply enthusiasts.

lone-wolf
01-04-2015, 01:52 PM
If I'm a new SAP6 owner and my rail becomes loose after 200 rounds, I don't go to the store to buy Loctite, I go to the store to return the firearm.

Wow, that's a bit excessive.

GunsNotPuns
01-04-2015, 02:09 PM
Wow, that's a bit excessive.

Okay, I wouldn't but someone off the street who doesn't have the same experience would be forgiven if they wondered if they had just bought a problem. We live in a culture where usually Things Just Work out of the box.

Canuck
01-04-2015, 02:20 PM
I liked most of what you said, GNP, in your original post. But I disagree about your description of our culture. For me, I am pleasantly surprised when something works right out of the box. I'm a lot more picky about filling in warranty cards, reading the fine print, etc. than I used to be because so many things don't work out of the box. My measure of a product these days is how quickly they fix it. Case in point: I just bought a UTS shotgun, new. Right out of the box, the rail was not to milspec and you couldn't attach sights or red dots to it. I called the supplier and they immediately replaced the rail-no question, with apologies. That brings me back to them because I know they will stand by what they sell. I can think of a lot of other things that didn't work when new, not just in the gun world. The standard, for me, has become will they fix it and how fast?

Calibre
02-02-2015, 02:38 PM
Thanks for the feedback! It's interesting to read your perspective; I actually thought it was one of the best reviews I'd bought (Publisher Dan here). Having shot the SAP6 we had in for testing (which was sent back by the way, just to assuage any fears of impropriety!), I can say that in my opinion, it was a good gun with two problems: the safety on ours was way too stiff and the design did not make me want to clean it much. Needing to Loctite a screw... meh, that's just one of those "life with guns" sort of moments that you should expect when you buy a product that needs to contain tens of thousands of pounds per square inch but remain entirely user-serviceable with minimal tools but more on that later.

All we can do, as reviewers and publishers, is report on the design and quality of the gun... and then follow that with an opinion. And opinions vary. Some folks will think needing to loctite a screw is unacceptable. Geordie himself didn't think the difficulty we had taking the gun down was a problem because as he pointed out, it's a pump-action shotgun. Hose it out with CLP twice a year and chances are it'll run until kingdom come. And he has a point! And again, as writers and reviewers all we can do is report on the issues we face, and close that out with an all-encompassing opinion. And in his review, Geordie did indeed mention every single issue he faced, and closed it out with the opinion that it's a good gun that's worth the $600 asking price. I don't think played down any of the issues, nor did he hide any; he simply said "Here's a SAP6. Here's what it does well, here's what it doesn't do well, and this is what I think about it in summation."

Another thing to consider when reading any review is that they are always written with some attention paid to the product's market. Compact handguns are reviewed from the perspective a potential CCW gun. A Remington 783 is reviewed as a potential gun for a newer or budget-conscious hunter. And the SAP6 was reviewed as a budget-friendly quasi-competition, quasi-fun gun. To bring forth an idiom from the motorsports world, you can have it "fast, cheap, and reliable; but you can only pick two." And guns are no different if you replace "fast" with "high quality." We pay more for Benelli M4s because they are quite simply better-made, better-designed guns that are also reliable... but they are not cheap. At $600, the SAP6 is relatively cheap, and in our experience, very reliable, so you need to be prepared to accept some issues such as a safety that'll need massaging and a design that doesn't lend itself well to take-down. So while it's entirely fair for you to question the use of the term "quirk" when describing something like the design of the SAP6 with regards to its disassembly, remember that there are others out there for whom the price tag of a better-designed gun like an M4 is "prohibitive," and from that perspective the problems with the SAP6 seem much less insurmountable. In short, it's all a matter of perspective!

As for the screws... well, I used to wonder about the QC with firearms because, having come from the automotive industry, I saw a lot more "QC issues" with guns in one year than I did in 10 years of cars. But I've since realized it's due to their design. Your car has a great warranty because you're never supposed to work on it. Guns, on the other hand, must contain explosions 3" from your face, function through all manner of adverse conditions, and then be taken apart by everyone from trained gunsmiths to complete numpty backwoods bubbas with as few tools as possible. And I think that poses a very difficult challenge from both a design and manufacture perspective, and is what results in situations where factory guns need to have small jobs like loctite applied or screws backed off (in the case of the early Remington 597s) to function. Like I said, I've just taken to calling them "life with guns" events.

In any case, thanks for the feedback, and I do genuinely appreciate you taking the time to let us know your feelings and perspectives. It's only through feedback that we'll get better and I hope you continue to enjoy Calibre! Speaking of, the latest issue hit the mail system for CSSA members on Friday and bulk shipments go out tomorrow!

Foxer
02-02-2015, 10:01 PM
Yeah, i've gotta say that while you raise some fair points I think you're a little TOO critical of the review. Honestly I thought it was one of the better ones. It highlighted design elements that are different than normal and may be of some concern (such as no bolt lock up) and dealt with whether there'd be challenge overcoming it. Seriously - a screw coming loose is a pretty minor thing. Not ideal, but it's not a 2500 dollar M4 either. The safety thing is very annoying and you could call it a build issue - it should be fixed. But it really is a 'quirk' for most people - it doesn't make the gun unusable by any stretch.

As to not being able to field strip - it's a shotgun. Most shotgunners i know have never actually cleaned their shotguns like EVER. My grandad ran hundreds of rounds thru his pump for decades, never cleaned it and that damn thing cycled so fast and reliably you had to make sure you didn't double tap the birds if you got too excited. If you run a few thousand rounds down the tube a year you might feel the need once in a while but you'll do it at home. They don't really get dirty enough in the field to require a cleaning to function properly unless you drop it in a cement mixer.

Compare that to the problems we've seen with other mag feds - complete failure to feed, ejection issues out the ying-yang, absolutely useless stock bead systems due to shortened barrels not being compensated for, charging bars being weak, etc etc all of which require major screwing around to resolve and even then they don't always work right - i have to say he's right overall - the problems with this gun sound like 'quirks' and 'idiosyncrasies' rather than actual 'problems'. For me, the fact that you can't just open it and leave it open is a bigger problem but that's obviously not a 'flaw' - it's a design element that'll appeal to some and not others.

Here's the bottom line - did you walk away from that review knowing the ins and outs and pro's and cons of that gun? Enough to form an opinion of your own on the gun? If yes - then regardless of his 'general' opinions it was a good review. That's how I personally rate reviews anyway.

Just my two cents of course. YMMV

Petamocto
02-02-2015, 10:22 PM
My respect for this magazine just went up 1000%, knowing that they monitor these forums enough to care about what people think.

If the NFA did the same thing we'd all have unlimited mags and no ATTs already, but they don't give a sh!t about their base.

Good job, Calibre, you just got a subscription from me. Keep it up and I'll stay loyal. Stop doing things like monitoring these forums and I'm gone.

Gunter
02-03-2015, 06:55 AM
My respect for this magazine just went up 1000%, knowing that they monitor these forums enough to care about what people think.

If the NFA did the same thing we'd all have unlimited mags and no ATTs already, but they don't give a sh!t about their base.

Good job, Calibre, you just got a subscription from me. Keep it up and I'll stay loyal. Stop doing things like monitoring these forums and I'm gone.

Simply knowing that comments (positive or negative) are read by the publisher is a plus for me.
A professional to-the-point answer in a public forum, becomes a subscription for me.
Good work Calibre and staff.

Cheers
Gunter

lone-wolf
02-03-2015, 09:36 AM
My respect for this magazine just went up 1000%, knowing that they monitor these forums enough to care about what people think.

If the NFA did the same thing we'd all have unlimited mags and no ATTs already, but they don't give a sh!t about their base.

So the only thing preventing unregulated magazines is the NFA not being on GOC?

Petamocto
02-03-2015, 09:54 AM
So the only thing preventing unregulated magazines is the NFA not being on GOC?

If the NFA leadership posted on here, it would be a symptom of being an effective organization that cared about the needs and desires of their base. Another symptom of being effective would be accomplishing things in government.

Thank you for continuing to defend them in every thread though, even though they continue to accomplish nothing except for making non-gun owners think we're all crazy.

kennymo
02-03-2015, 10:28 AM
If the NFA leadership posted on here, it would be a symptom of being an effective organization that cared about the needs and desires of their base. Another symptom of being effective would be accomplishing things in government.

Thank you for continuing to defend them in every thread though, even though they continue to accomplish nothing except for making non-gun owners think we're all crazy.

They do post on CGN, but it seems like most criticisms and policy questions promptly disappear....

lone-wolf
02-03-2015, 10:55 AM
If the NFA leadership posted on here, it would be a symptom of being an effective organization that cared about the needs and desires of their base. Another symptom of being effective would be accomplishing things in government.

Thank you for continuing to defend them in every thread though, even though they continue to accomplish nothing except for making non-gun owners think we're all crazy.

I can see the things they accomplish. I feel a need to defend the only group attempting to change things, and not just bitch on the internet cause a picture hurt their feelings.
So, I guess you'll ignore your own foolishness when confronted with it, and just throw more nonsense out.

Doug_M
02-03-2015, 11:33 AM
I can see the things they accomplish.

List them.


I feel a need to defend the only group attempting to change things,

CSSA has changed things. NFA has bitched about it.


and not just bitch on the internet cause a picture hurt their feelings.

My feelings certainly weren't hurt by any of their meme pics. But some of them sure hurt our cause. And I haven't "just" bitched. I've re-joined the other org getting things done, I've joined my local EDA, I've organized letter writing campaigns, I've participated in letter writing campaigns, and I actively counter anti propaganda on social media with counter point and fact without coming off like a neanderthal and so turning off the true audience of my counter points.

lone-wolf
02-03-2015, 12:52 PM
List them.

PEI's first gun show, petitions, actually communicating with me & guiding me on what to do, conference calls, being public, & calling BS when it is.
Being in the CSSA was like joining nothing at all. They did nothing to show they were doing anything. I seen no results from what they were doing. If anyone needs better PR it's them.


CSSA has changed things.
List em


And I haven't "just" bitched. I've re-joined the other org getting things done, I've joined my local EDA, I've organized letter writing campaigns, I've participated in letter writing campaigns, and I actively counter anti propaganda on social media with counter point and fact without coming off like a neanderthal and so turning off the true audience of my counter points.
Those dang neanderthals running around everywhere.
Thanks for being active though, since our goals are the same.

Foxer
02-03-2015, 01:08 PM
PEI's first gun show, petitions, actually communicating with me & guiding me on what to do, conference calls, being public, & calling BS when it is.
So... not much at all to actually change anything. The gun show is nice, but really has nothing to do with the nfa's stated purpose of getting firearms rights back. That's just organizing a show. That's great... but not really actually 'doing' anything for our rights.

Petitions are pretty much useless unless they're done in such a way that they actually pressure for change. The current petition is largely useless, it's far too disorganized to be effective. Someone took 15 minutes to whip up a form and they've asked people to download it if possible and hopefully get some signatures. Doug actually did more by himself with his 'one clear voice' project. In Ottawa, a petition and 3.50 will get you a cup of coffee.

Heck - i'll do you one better, I believe they still do intervene on behalf of gun owners who run afoul of the law in a stupid way. I know they called me not THAT long ago to ask information about an area I have expertise in, so they're obviously stepping up for some people when the cops or other authority cause them unnecessary grief. That's actually doing SOMETHING for gun owners rights, unless they've stopped since.

But - that's a pretty rare thing, not an everyday thing, and we have legal aid to help with that now. Their main job is to push for and make changes to gun law and policy to actually help our cause. And are they doing that? No. In fact they may be hurting.

If i go to a lawyer because i'm getting sued, I expect him to fight and hopefully win my lawsuit. I don't care if he serves me tea while i'm there or validates my parking. That's not what I"m paying for. It's nice - but only if they still do their job.

Same here - the tiny little things they do are great. But - pointless unless they're focused on the job we need them to do. They are not the 'national gunshow association', they are not the 'national firearms owners legal defense association', they are the National FIREARMS assoication - and they are supposed to be working for our rights and making positive changes (rather critical while we have a gov't that's at least SYMPATHETIC, and not overtly hostile).

IN any case, this doesn't have much to do with the original post.

Like i said - I thought the review was actually great because I walked away knowing a lot more about the gun, it's strengths and weaknesses, and I'd seriously considered buying one and may yet, and If i do this review will help make sure I get what I thought I was going to get with fewer surprises. I'd read more like that personally, tho I get that everyone's different. I like that the editor came and checked out the feedback.

The NFA is really a whole different kettle of fish and we've kind of done that.

lone-wolf
02-03-2015, 01:17 PM
Petitions are pretty much useless unless they're done in such a way that they actually pressure for change. The current petition is largely useless, it's far too disorganized to be effective. Someone took 15 minutes to whip up a form and they've asked people to download it if possible and hopefully get some signatures. Doug actually did more by himself with his 'one clear voice' project. In Ottawa, a petition and 3.50 will get you a cup of coffee.

Petitions are largely useless but caused Switzerland to have a referendum on gun laws which saved them from further restrictions, with the CPC/Liberals petitions are likely useless, but not trying is definitely useless.
Your pretty quick to call things you know nothing about disorganized and ineffective.
What did oneclearvoice do/does?


So... not much at all to actually change anything. The gun show is nice, but really has nothing to do with the nfa's stated purpose of getting firearms rights back. That's just organizing a show. That's great... but not really actually 'doing' anything for our rights.

They're too vocal and if they do anything on the grass roots level, it's not enough. You guys are hard to please.

Foxer
02-03-2015, 01:30 PM
Petitions are largely useless but caused Switzerland to have a referendum on gun laws which saved them from further restrictions.

You appear to be exercising selective reading their bud :) I said: "unless they're done in such a way that they actually pressure for change."

Now - lets put that in perspective. One petition in one country was part of a successful campaign to pressure the gov't to look at change. However - literally thousands of petitions go to the gov'ts of north america every year and almost none actually make ANYTHING happen.

Petitions, if done correctly, CAN be a useful part of an OVERALL campaign for change IF DONE PROPERLY. This one is not and took next to no time for them to do. It does make them LOOK like they're doing something to those who don't know better tho :) This petition is more effective as advertising to them than as an actual vehicle for change.



They're too vocal and if they do anything on the grass roots level, it's not enough. You guys are hard to please.

Again - a little selective reading there. I don't think anyone is saying that the problem is they're 'vocal'. The problem is that their methods and what they're being 'vocal' about (if we can call a facebook page 'vocal') is not helpful. If a man is drowning - yelling for help is a good idea if there are people nearby. Yelling how ugly the people's swimwear is would be LESS helpful, and may even hurt. It's not that he's yelling - it's WHAT he's yelling.

And I don't think anyone's saying 'don't do other things". IF they are doing their core job AND organizing gun shows - hey, great! But - if they're not doing their core job then the gun show stuff is just fluff. Again - it promotes the brand but it doesn't actually do what they're supposed to.

Here's the deal - if what they are doing is achieving real results, then people aren't going to care what it is they're doing they're just going to be happy. If not - then not. Right now we can see they're doing next to nothing on the gun rights front even tho this is definitely the time (lead up to an election, friendly gov't instead of hostile, etc etc). They've created a hostile relationship with the gov't, they are NOT organizing the community in any meaningful way, they are not achieving their goals and what very small actions they HAVE taken are in fact so questionable that they might actually hurt the cause.

There was a time when things were different, and there may be a future time when things are different, but that's where we are right now. Clearly - you feel that whatever you ARE getting out of them is helping you out and you like them, and that's great. Nobody's saying you should turn on them or firebomb their local office or anything. But - the fact is that most of us want them to be a voice for gun owners about gun laws and that's not happening - or at least when it does happen it's a pretty crappy voice.

lone-wolf
02-03-2015, 01:36 PM
The government turns rifles prohib overnight, people into criminals overnight, but the NFA is the ones who turned things hostile?


Nobody's saying you should turn on them or firebomb their local office or anything.
I tried to find a funny meme for that to lighten things up but failed.



IF they are doing their core job
What is there core job and how are they not doing it?
If their core job is to get firearm ownership more mainstream, gun shows in places that never had gun shows is doing their core job.

awndray
02-03-2015, 01:45 PM
So, how'bout that Calibre magazine, eh?

Foxer
02-03-2015, 02:05 PM
The government turns rifles prohib overnight, people into criminals overnight, but the NFA is the ones who turned things hostile?


Ahh, no. That didn't happen. The RCMP 'reinterpreted' rifles into prohib, the gov't didn't do anything. And this is what I mean - the NFA suggested the gov't 'signed' something to make this happen and that just didn't occur. So is spreading misinformation a 'positive' thing? I doubt it. Is turning gun owners on a gov't that was just as surprised as we were and is actually willing to do something about it positive? No.


I tried to find a funny meme for that to lighten things up but failed.

LOL - well there's only so many ways to make firebombing amusing :)


What is there core job and how are they not doing it?
If their core job is to get firearm ownership more mainstream, gun shows in places that never had gun shows is doing their core job.

Well no and no - Their core job is to advocate for the rights of gun owners, and that's not happening. The OPPOSITE is happening, they're organizing people to ATTACK the gov't over something the gov't didn't do, then hope that a petition they organize will be listened to by that gov't which is pretty unlikely.

And a gun show does nothing to make guns 'more mainstream'. Who goes to gun shows? Gun owners and their families, people who have already accepted firearms ownership. It's still a good thing, more gun shows are better, but it's not even close to doing it's job.

In an effort to keep awndray happy, i'll bring that back to the orginal post :) What is the job of the review of a gun? Surely - the review is intended to equip the reader with a realistic understanding of the pro's and con's of a specific gun, and details such as fit and finish, unique features etc etc. In addition, they may be said to give the reader an educated over all opinion or impression of the gun based on their experience with other guns etc. We might get a better feel for how it is to actually shoot the gun and what it's like.

I think overall the review did that. I get what the op said about 'downplaying' various things which are negatives. Fair play. Some people may find that safety a deal breaker - a gun which you can't go from 'safe' to 'fire' with very fast is a liability in some cases. But - you STILL knew about it and that's what you're looking for.

In the end - having read the review, do I get the feeling that I know enough about the gun to make a far more educated buying decision, or at least know what to research in order to feel comfortable buying it? I would probably say I do - if a smooth 'safety' is important to me, i'm going somewhere else unless I find out they've addressed that. If having the bolt stay open is important, I'll have to consider that. If I want a reliable feed - well this seems to fit the bill, etc.

So while we can nit pick it apart (and should, that's always fun :) ) i think we can say that on the whole the article had a job to do and it did do that job.

Which we can't really say for the NFA at the moment :)

lone-wolf
02-03-2015, 03:43 PM
gov't didn't do anything.
That's the problem


NFA suggested the gov't 'signed' something to make this happen and that just didn't occur.
I believe they're called laws/bills.


And a gun show does nothing to make guns 'more mainstream'.
People who aren't gun owners listen to advertisements too you know. And the news likes to mention such things as well.


Their core job is to advocate for the rights of gun owners, and that's not happening.
Cause, lobbying isn't effective? lol


then hope that a petition they organize will be listened to by that gov't which is pretty unlikely.
Unlikely why? Because our government is totalitarian? Or because people are putting it down before it's even finished?


they're organizing people to ATTACK the gov't over something the gov't didn't do
I swear, the NFA never asked me to firebomb the gov't

awndray
02-03-2015, 04:00 PM
http://thedoctormommy.com/wp-content/uploads/2013/02/Time-out-chair.jpg