• WELCOME Grumblers
    Backup is now done at 3PM EDT. You may find the server down for up to two minutes at that time.

Peer Reviewing

Jim Miller

SPFG, Supreme Picture Framing God
In a conversation about The Grumble's future in a Facebook framing group, one member said that online framing forums (presumably all of them) offer some information that is questionable or out-of-date, which has never been challenged and remains available, because there is no "peer reviewing".

My opinion is that every framing forum is subject to peer reviewing by its members, through open discussion and debate of questionable advice, information, or opposing opinions. His response was that if bad information were peer reviewed, it would be corrected and not allowed to remain available. He says that opinions and debate do not amount to peer reviewing.

Here is the definition of "peer review" according to Dictionary.com:
noun; evaluation of a person's work or performance by a group of people in the same occupation, profession, or industry.
One purpose of our professional trade associations, PPFA and FATG, has been to provide peer reviewed information, but these organizations seem to be overshadowed by online informational resources.

Questions come to mind...
Do inquisitive framers expect the answers to their questions to be peer reviewed, or are they asking for opinions? Does open conversation and debate serve the purpose of peer reviewing? If not, then should information, advice, and opinions be allowed without any supporting research or evidence? Does a formal peer review process exist within our industry?

What are your thoughts on the peer review topic?
 
Sponsor Wanted

Rick Granick

SPFG, Supreme Picture Framing God
Interesting question, for sure. I would say that our industry (its small size taken into consideration) is probably about on par with the kind of peer review that takes place in other industries- at least in terms of practices that are not goverened by specific laws or regulations. The scientific and medical communities are different in that they have highly respected journals and peer-review processes that are an inherent part of what they do. While there are certainly plenty of exceptions I think that, for the most part, our discussions, our trade publication, and the classes available to us in our conventions do a pretty good job of advancing the state of the art over time. Of course, there is no "framing police" who will come and make sure we do things according to current best practices. But it seems that if outdated or simply bad information is offered here or on the FB groups, it gets challenged pretty quickly. Ultimately, it is up to individual framers and business owners to properly represent the work we do to our customers so that they can make informed decisions, and to do what we say we will do. Our reputations (which we have to earn) are a big part of the value proposition of custom framing.

:kaffeetrinker-2: Rick
 

stcstc

SGF, Supreme Grumble Framer
I think the peer review tends to be more formal than debate and discussion on forums.

I also don't agree that all bad information gets challenged quickly rick. both on on forums and even in classes.

I went to WCAF 2 years ago and 2 of the classes had misleading information and stuff that was not correct.

I also believe that some of the "leaders" of our industry don't take debate and discussion very well, and prefer being almost pandered to
 

wpfay

Angry Badger
Jim, Peer review is more like what went on when we were working on FACTS than pretty much anything else readily available via Internet.

The on-line discussion groups are OK as far as they go but there is nothing publishable as a result, even if it is a "best we know just now" scenario. I would guess that a majority of what comes from such discussions has kernels of truth in it, but it's up to the individual to decide just what those are.

This is also true of the organizations that test our trade for certification. The tests available for certification are not current (8 years since the last review for the CPF?), and with the technology driving so much discovery and change, they would have a hard time ever getting current.

It's a resource issue as well. The industry is short on both financing and volunteers to undertake such studies.

With a real Peer Review, a single issue could be changed or obsolete by the time the review is complete. This conversation, for example, could conceivably go on for months, but the conclusion might not really define the subject.
 

Jim Miller

SPFG, Supreme Picture Framing God
Jim, Peer review is more like what went on when we were working on FACTS than pretty much anything else readily available via Internet.
You're referring to the Generally Accepted Framing Practice (GAFP) committee we worked on back in the 1990s, aren't you? Yes, that program was a peer reviewing process to establish (in Don Pierce's words) "consensus standards" for publication. Likewise, most of the PPFA work in developing guideline publications involved a similar research and peer reviewing process.

But does peer reviewing have to involve publication? Dictionary definitions are not so limited. But if they were, then wouldn't it be unreasonable to expect such high quality peer reviewing in online discussions? They're generally not intended as gospel, nor for publication, but to converse on random topics and answer specific queries; share expertise, personal advice, information, facts and opinions.

The on-line discussion groups are OK as far as they go but there is nothing publishable as a result, even if it is a "best we know just now" scenario. I would guess that a majority of what comes from such discussions has kernels of truth in it, but it's up to the individual to decide just what those are.
Yes Wally, "... as far as they go..." is a salient point. You are quite right, but just the same, can we agree that "...the process of someone reading, checking, and giving his or her opinion about something that has been written by another scientist or expert working in the same subject area..." is also a useful form of peer reviewing? Admittedly, there are few scientists among us and the term "expert" is used loosely in our industry, but this Cambridge Dictionary definition essentially describes our online framing discussions.

Thanks for a fun and informative conversation.
 

shayla

WOW Framer
The way I see it, there's tremendous worth in establishing a set of consensus standards, as has been done before. Formal peer review, as well. It sounds like those need some updating to include more recent changes, and with a shrinking industry, that's proving to be a challenge. There's also tremendous worth in free discussion among all levels of framing knowledge and ability. Framers who wish to improve their knowledge have found the latter to be a very useful springboard for engaging with the former. It's also a helpful balance against harmful inflexibility among those who have so much sunk cost in past methods (i.e. their careers/egos hinge on supporting whatever it was), that they resist beneficial change. It should be noted that some kinds of inflexibility are valuable (i.e. the fact that certain materials in close proximity to certain media/substrates will cause harm, etc...), but there's vitality in creating an atmosphere of free dialogue.

One thing I love about the Grumble is that it allows for both to be expressed. And the archives are a treasure of helpful guidance. I hope that the future sees both formal and informal approaches toward improved framing standards thriving. Part of what allows this is to have an honest dialogue.
 

Jim Miller

SPFG, Supreme Picture Framing God
The way I see it, there's tremendous worth in establishing a set of consensus standards, as has been done before. Formal peer review, as well. It sounds like those need some updating to include more recent changes, and with a shrinking industry, that's proving to be a challenge.
Yes, Shayla, most of the previous work on "consensus standards" has been around for two or three decades, and that very tedious and time-consuming process was all accomplished by volunteers.

The word "standards" has been perceived negatively by some framers as an attempt to tell them how to do their work. That has never been true, so the more framer-friendly term "guidelines" has been used on later publications, and the term "recommendations" could be applicable, too. In any case, it's all for informational/educational purposes and compliance is purely voluntary. The "Framing Police" never did or ever could exist.

The body of framing knowledge continues to grow and our methods and materials continue to improve. That seems to be especially evident among a small cadre, most of whom are in the United States. But your points are well taken; reviewing & updating the existing publications or making new standards/guidelines/recommendations/suggestions may not happen again in the foreseeable future. There are few volunteers to do the work and, besides shrinkage of the retail custom framing market, many framing practitioners - especially the newcomers - believe that everything they need to know is available online.

Without the benefit of high-quality, organized research and peer-reviewed publications desired by some, classroom education and moderated roundtables probably offer the most useful information, followed by properly-vetted, professionally produced books and videos, and then followed by online discussions, which can be good or bad.
 

FramerCat

SGF, Supreme Grumble Framer
Was the objecting person referring to the archives on forums like this? I can see how an old entry that has not been referenced in a decade or two might be found to have some outdated information that was never debated because of lack of searching as opposed to a Facebook group where searching past discussions on a topic is nearly impossible so the debate just starts anew, updated with some of the more current techniques and materials being referenced when someone posts a previously debated question.

Personally I am fine with active debate being the closest thing that we have to peer review. There are so many highly educated and skilled practitioners of our trade that are so willing to discuss the same topics over and over with such detailed and complete references and logic that anyone serious about finding the closest thing to truth in a subject should not have a very hard time. Those who can back up their stance with hard evidence will and do, and those who cannot generally drop out of the discussion. The problem comes when some just don't want to hear the arguments against their preferred technique or material and just double down on their position continuing to give bad information to their customers and employees. This just makes it harder for the rest of the industry to advance beyond the folklore of the olden days. So, just everybody be more open minded and listen to what makes logical sense even if it is different than what you've been doing for the last thirty years.

Ed
 

Jim Miller

SPFG, Supreme Picture Framing God
Was the objecting person referring to the archives on forums like this?
Well, you be the judge. Here's a direct quote:
"There is some very valuable information at the Grumble, however there is also some very questionable and out of date framing information that was never challenged which is still quoted as possible framing fact or solutions....It is a difficulty with many of the framing forums that there is no peer reviewing which allows for opinions to be stated as fact without any supporting research or evidence."
"Opinions and debate is not peer reviewing."


He did not offer any plans or suggestions about how to accomplish the sort of peer reviewing he seems to believe is needed, or how to improve the quality of communication/opinions/advice/information among framers.

Anyway, in my personal opinion "peer reviewing" could be a topic worthy of discussion. And since his commentary was in the context of The Grumble, it seems proper to have it in this forum.
 

David Hewitt

CGF II, Certified Grumble Framer Level 2
Yes to peer reviewing, facts or standards, and credentials.
F.W.I.W.
I just retired after 36 years of being in this business.
(I earned the credentials, attended most of the shows, and educational seminars, and followed forums with all of you, which allowed me confidence in my work and ethics.)
For what it's worth, PPFA NEEDS a major come back, (Or something similar)
and the Grumble needs to continue. I have tried Facebook Forums. and for me they are not a option.

I would like to thank Bill for the Grumble, its been a life saver for all of us.
Wally Fay for the trade shows FrameFest, and his contribution to the survival of PPFA and as a moderator for many of the framing sites.
Jim Miller, Rob Markoff, Wally Fay, and all of the other educators that have educated themselves only share that knowledge with others.
And To All Of You that have shared your talents and expertise only to better our industry.
 

wpfay

Angry Badger
David, Thanks for the props, but I was only part of a team. You and Darlene, Barry Shapiro, Sondra and Harry Quatraro , and our secret weapon, Cathy (Chickie) Coggins, from our chapter, and Mike Labbe and Andy Langlois from New England were all instrumental in making FrameFest what it was. The educators, vendors, and attendees all contributed as well.

It's ironic, and sad, that just this morning the owner of The Grumble announced that without a good offer to take over the forum it will go dark at the end of the month.

PPFA's Framer's Corner has 4-5 active members and at least two of them are on the payroll. I can't see it surviving much longer.

The prospect of any valuable peer review at any significant level on FB is depressing.
 

Cliff Wilson

SGF, Supreme Grumble Framer
I am constantly frustrated by bad or outdated information that is passed around as good and sound. (on pretty much every platform.) Every time I read the words "I've been doing it like <this> for N years" I cringe. That said, I have also taken a number of classes where respected "experts" are passing out clearly bad information. (It happens more in business classes than technical, but happens everywhere.)

I thought the PPFA guidelines and the work we were doing on certification was working toward reasonably reviewed information, but alas, that has become quiescent.

I don't know how to do it, but I really wish we had some way to accomplish something close to "peer review" because many customers are being provided improper work expecting it to be a "professional" job. <sigh>
 

MnSue

SGF, Supreme Grumble Framer
I used to say "Picture framers are a lot like farmers. The concept to get a crop is the same, but the methods are different from each other to get to the result." And independent framers are as ornery as independent farmers too! Note: I am a farmers daughter...and an indy framer. LOL
 

Ylva

SPFG, Supreme Picture Framing God
It is not a protected profession, so anyone can open a frame shop.
Through this forum, I started to learn and I continue to learn. Although I am a member of most Facebook groups, it is too fleeting. I can never find a post back, it is fast, come and gone.

There is a ton of information out of there, True and false. I believe in continuing education, I’m not sure how it will and can be be accomplished.
At least when I post here, I will get comments I can sift through. And see what is newly posted.
 

wpfay

Angry Badger
I used to say "Picture framers are a lot like farmers. The concept to get a crop is the same, but the methods are different from each other to get to the result." And independent framers are as ornery as independent farmers too! Note: I am a farmers daughter...and an indy framer. LOL
Having lived on a farm I can relate, and have often compared the attitude of framers and farmers. Here's a line from a song that I remember from my youth that pops up frequently in my head when reading these opinions.

And we're so by God stubborn
We could stand touchin' noses
For a week at a time
And never see eye-to-eye.

Extra credit if you can tell me where it is from.
 

Larry Peterson

PFG, Picture Framing God
And we're so by God stubborn
We could stand touchin' noses
For a week at a time
And never see eye-to-eye.

Extra credit if you can tell me where it is from.
Without Googling it.

I did the Google so no extra credit for me.
 
Sponsor Wanted
Top