Peer Review

Higher risk than who? Well, certainly at higher risk than pilots flying for a large airline. The airlines and other collections of professional pilots in large groups have several advantages over those of us in small flight departments. (I am including myself; we have four pilots, two mechanics, a dispatcher, and a line technician.) These larger organizations have dedicated training and standardization departments that dedicate themselves to making sure you are an SOP maven. These pilots do not normally fly with you on the line, so they don't have a vested interest in keeping on good terms with you. You either follow SOP, or your career will be at risk.

If you think you are not at risk, it could be that you have already succumbed to this normalization of deviance. It goes like this: one pilot finds a shortcut that violates SOPs but is convinced his or her way is better. Nobody else objects, because they want to get along. Pretty soon, everyone is taking the same short cut. After a while, SOPs start to mean very little.

This sounds bad, but there is an easy solution. As much as we tend to hate checkrides, orals, and written exams, there is something else we hate that doesn't require a ruthless check airman: looking bad in front of our professional pilot peers. And that is the key to avoiding the three hazards I've listed here: subject yourself to regular peer reviews.

1 — What is a peer review?

2 — The hazards of being "peerless" and how a peer review can help

3 — How to do a peer review

4 — Taking it to the next level

5 — Real world results

6 — A Peer Review will make you a better pilot

What is a peer review? First, what it isn't. It isn't a check ride. The observer has no legal authority and your license is not at risk.

Some management companies conduct Line Oriented Observations, emphasizing these are "how are you doing" rides and they are certainly not check rides. I did many of these for a few management companies and can tell you they were perceived to be check rides because every now and then pilots were fired. For good reason, of course. A peer review does not carry that risk, since it is being done by a peer with no standing in the company.

A peer review is nothing more than the opportunity to have a peer — another professional pilot — to observe your crew in action. The reviewer is a guest of yours whom you've asked to watch and provide feedback that will help you and your team to become better. You may think you already have this opportunity during regular recurrents in a simulator. But those are artificial environments where the observer has a tightly structured "to do list," and no time for anything else. Besides, the simulator observer is probably not a peer, not someone who does what you do for a living.

I've already mentioned that small flight departments have a special vulnerability to three hazards because there is so little oversight. I think we can mitigate those hazards with peer reviews. Let's take a look at these hazards, an example accident case study, and how a peer review could have saved the day.

Hazard 1: Lacking the required skills and knowledge

We've all seen pilots out of their depth at regular recurrents and had to wonder how they got to where they are without some kind of quality control step along the way. They didn't know what they needed to know and didn't have the skills needed.

Case Study: Learjet 35 N452DA. On May 15, 2017, a pair of poorly qualified Learjet 35 pilots so badly scared their passengers flying into Philadelphia, the passengers decided to rent a car and drive instead to their next stop, Teterboro. The pilots continued on empty and crashed a mile short of their intended runway. It was a challenging day at Teterboro, with gusty winds and the need to circle from an ILS to an adjacent runway. These pilots didn't know the difference between an IFR circling approach within circling radii at minimum descent altitudes and the need to visually maneuver from one runway to another. This just brushes the surfaces of the knowledge and skills these pilots lacked, and yet they somehow became qualified to fly their jet with passengers.

Of course these pilots didn't set out in their Learjet with the objective of doing something they were ill prepared to do. I think had they undergone a peer review at some point before finding themselves having to negotiate challenging winds at an airport that is a challenge even on a day with calm winds, they might have been given a wakeup call. At the very least, the peer could have sat down with them and told them they were at risk and talked some sense into them.

Keep in mind that this hazard isn't limited to poorly qualified pilots, it also happens to good pilots who fail to keep up with changes to their aircraft or the industry. Think of how lost an international procedures expert from just ten years ago will be today in the North Atlantic Track System. If you don't keep up, you fall behind.

I've seen this many times over the years. I once gave a line observation to a pair of Falcon pilots who didn't understand their takeoff minimums are different at most Canadian airports and politely pulled out the appropriate minimums chart before they fired up their aircraft's APU, hours before the visibility would be sufficient for takeoff. I might have saved them a violation or perhaps just some embarrassment. But I hope I gave them cause to understand the need to better prepare themselves for flights outside their home country.

Hazard 2: Becoming complacent by taking shortcuts around Standard Operating Procedures (SOPs)

Even very good pilots can be tempted by shortcuts to SOPs that seem to get the job done more quickly, with less fuss, and are just easier. The problem with taking that shortcut "just this one time," is that success with the shortcut encourages further deviation. Pretty soon the shortcut becomes the new SOP.

Case Study: Bombardier BD-700 C-GXPR. On November 11, 2007, a pair of highly experienced Challenger 604 pilots flew their brand new Global 5000 into a small airport with a short, 4,885' runway. On paper the runway was more than long enough and the captain had flown there 75 times in the Challenger 604. He had apparently gotten into the habit of aiming to touchdown in the first 500 feet. He did this once in the Global, a month before the accident flight, and his techniques resulted in a touchdown 200 feet beyond the threshold. On the day of the accident, the wheels touched 7 feet prior to the runway, causing the gear to collapse and damage the aircraft beyond repair.

These pilots apparently did not understand the geometry of their new airplane meant the main landing gear would touch hundreds of feet short of their aim point. See The Landing Flare for an explanation about how this is. A peer review from a more experienced Global or Gulfstream pilot could have saved the day. Their peer may have picked up their tendency to "duck under" and then explained how this is harder to get away with in a larger jet.

I've seen this very tendency in another Global a few years before this accident. Like the accident crew, these pilots had upgraded from the Challenger 604 and just assumed that even without a flare, the wheels hit just 50 feet behind the cockpit. After I drew them a few stick figure diagrams, they realized just how important a 1,000' aim point is.

Hazard 3: Becoming intentionally noncompliant with SOPs

The natural progression of complacency, if left unchecked, is willful and intentional noncompliance with SOPs. While simple complacency is insidious and its victims can be thought of as unfortunate innocents, I think intentionally noncompliant pilots can be criminal. Case in point . . .

Case Study: Gulfstream GIV N121JM. On May 31, 2014, two highly experienced Gulfstream GIV pilots killed themselves and five others by attempting to takeoff with their gust lock engaged, and then attempting to disengage that gust lock at a speed that made such an action impossible. Subsequent investigations revealed that these pilots had a pattern of not using checklists or callouts. They managed to fool their evaluators at regular recurrents by flying one way for those charged with checking them, and another way when left alone. Contract pilots who flew with them noticed many of their transgressions, but as contract pilots had a vested interest in keeping quiet.

A peer review could have saved the day had someone they knew and respected noticed the procedural noncompliance and let them know just how reckless their behavior was. "You guys are courting disaster," the talk could have begun. "You might be good, but nobody is immune from making mistakes. Your behavior might kill someone." And, of course, it did.

Behavior this flagrant is more common than we might think. Since this incident, I've watched several Gulfstream operators with similar gust lock systems start up and takeoff without a flight control check. Over the years of giving check rides, line observations, and peer reviews, the most frequent example of willful noncompliance deals with oxygen use, followed by sterile cockpit rules, headphone usage, and inadvisable automation procedures. Of course all of these are fairly common occurrences and we usually get away with them. As the reviewer I try to diplomatically point out the advantages of compliance and hope my words are taken to heart.

1. Find like-minded pilots nearby

Look for pilots flying a similar operation in similar aircraft. They don't have to be type rated in the same equipment, just conversant in what you do. One of the best comments I got while flying Gulfstreams came from a Learjet pilot. You can use this article as a script to convince them that a peer review program will be mutually beneficial.

There is a divergence of opinion about compensation. Some will argue that for any professional endeavor the professionals need to be paid and that compensating the review pilot at the standard day rate is fully justified. Others will argue that will turn the review into a paid audit and can color the reviewer's opinions and that it is better handled as a professional courtesy paid by your offer to conduct a review in return. I will leave that determination to you and your situation.

2. Get company buy-in

Your company takes for granted that there are many steps along the way needed to keep you trained, proficient, and legal. You can sell the peer review as a step above and beyond the bare minimum legal that takes you to the level of best practices. It may be best to sell this peer review as a flight evaluation, even though it isn't. You can make the same case to your company as you did to your like-minded peer.

Your company will probably be worried about their business being exposed to an outsider and will want this outsider to sign a Non-Disclosure Agreement (NDA). If they don't have such a document, I have one for you and their lawyers to consider. You can download it in Microsoft Word format here: Non-Disclosure Agreement, Generic. It is fairly verbose but I think it includes just about everything you may need. You will have to enter your company name, etc.

Your company may also be concerned about your peer becoming injured (or worse) during the flight and may want legal protection in the form of a Hold Harmless Agreement (HHA). If they don't have such a document, I have one for you and their lawyers to consider. You can download it in Microsoft Word format here: Hold Harmless Agreement, Generic. It is fairly verbose but I think it includes just about everything you may need. You will have to enter your company name, etc.

3. Find the right trip

I find the best peer review happens on an out-and-back trip that gives you a chance to operate both as the Pilot Flying and the Pilot Monitoring. These types of trips also reduce the time spent to one calendar day and reduce associated travel and meal expenses.

Ideally, the trip will be with passengers, if your company will permit this. This shows your peer observer how you handle the pressures of dealing with the passengers and their requirements.

4. Train your peer and crews

Observing pilots from a jump seat takes a certain amount of tact and willingness to watch and listen without automatically offering opinions on anything worth noting. Peers with previous check airman or standards pilot experience will have a leg up in this process, but that experience is not required. You should emphasize the purpose of the peer review:

1. Observe crew performance under normal operating conditions.
2. Assess the effectiveness of training programs.
3. Determine awareness of industry best practices and regulatory requirements.
4. Provide a feedback opportunity for crews.

The peer will have an opportunity to watch the crew in action and to compare that with his or her own experiences. Any discrepancies are likely to be deficiencies in the training programs and are therefore a good way to assess the effectiveness of those programs. Discrepancies can also be traced to awareness; not everyone gets the word on the latest and greatest practices in the aviation world as a whole. A chance to provide feedback to the reviewer can also provide crews with an opportunity to talk through practices they may have adopted unknowingly.

Establish a few "rules of conduct"

1. The review pilot will dress according to company standards, will introduce him or herself to the passengers (if desired) as an "observation pilot."
2. The review pilot will not impede any activity and will attempt to remain "in the background" for the entire flight.
3. The review pilot will monitor cockpit and radio communications and will not interrupt on going communications.
4. The review pilot will be another "set of eyes," and will let the pilots know if he or she spots a hazardous situation, such as conflicting traffic or a missed radio call.

5. Conduct it

1. The review pilot should attend all crew events, everything from the crew briefing to post flight debriefing and everything in between.
2. Pilots should conduct themselves as they normally would without the additional set of eyes, feeling free to collaborate with each other and even asking the review pilot for opinions.
3. Pilots should give the review pilot an emergency egress briefing as well as instructions on the use of the jump seat.
4. The review pilot will have to be adept with deploying and stowing the jumpseat so as to "get out of the way" as the pilots will need to get by for normal duties.
5. The review pilot should take notes and save non-critical remarks for slow periods of the flight or for after the flight has been completed. The review pilot can offer comments during flight if there are slow periods during cruise flight and the pilots invite discussion.

6. Debrief it

1. The review pilot should reconstruct the flight starting with the initial meeting and covering each phase of the flight(s).
2. The review pilot should emphasize the positive while covering areas that could be improved. The review pilot should realize that he or she has a different experience base and these ideas may help improve the subject pilots in the future.
3. The reviewed pilots should listen with the aim of learning from another professional pilot, should not become defensive, and should provide their own comments with the view that the review pilot can learn from them as well.

7. Document it

The reviewed pilots may find it useful to document each review as a way of tracking progress towards continuous improvement. I have a form that may be used or modified for this purpose: Peer Review Debrief.

You can learn a lot from a Peer Review but you can also use it to prepare yourself for the next time an FAA Inspector comes visiting. You should take advantage of having another licensed pilot look at you with a more critical eye. This is optional, of course. But asking your peer to look a bit deeper can help you discover things that might have otherwise gone unnoticed. Kent Jackson, my colleague at Business & Commercial Aviation Magazine has some advice.

1. Training Records — The flight department should have quick access to everyone's training records to prove everyone, including mechanics, are trained to do what they are doing.

"What will the FAA look for when inspecting a Part 91 flight department? The agency recognizes that the record-keeping rules of Part 135 do not apply. But the inspectors are advised: “Even though record-keeping is not required of an executive/ corporate operator, many do maintain training records. The inspector should encourage all operators to keep and maintain records to verify compliance with 14 CFR §§ 61.55 and 61.58.” The guidance goes on to instruct the inspectors to examine such records if they are maintained."

Source: Jackson

2. Aircraft Lease

"If you are operating an aircraft subject to a lease under Part 91, make sure that you have a copy of the lease and that you, and everyone in the flight department, understands the lease. If the aircraft has a max gross takeoff weight over 12,500 lb., then a copy of the lease must be kept in the aircraft."

Source: Jackson

3. Minimum Equipment List

"If you have a Minimum Equipment List, the inspectors will check to see if the Master Minimum Equipment List has been subsequently revised. If you operate a large or turbine-powered multiengine airplane, you are required to have an emergency checklist, one-engine inop climb performance data and a two D-cell flashlight (Part 91.503). Expect to show where each of these items can be found in the cockpit, and make sure that the flashlight works. Passenger briefing cards are not required, but if they are used to supplement an oral briefing, then they must be available to all passengers and must refer to the specific type and model of airplane (Part 91.519)."

Source: Jackson

4. ARROW

"The FAA is also ramp-checking business aircraft operators as part of the current effort. Are you ready? At some point in your flying career, you probably memorized “ARROW” so that you would be ready for an inspector visit: Airworthiness Certificate, Registration Certificate, Radio Station License, Operator Handbook, Weight & Balance. These are the aircraft’s required documents, and this is still a pretty good acronym to jog your memory, but the ARROW requirements have evolved. Your aircraft needs an FCC Radio Station License if you will fly internationally. And if you do, you also need to carry a Restricted Radiotelephone Operator Permit. The FAA will also be checking to see if each pilot has “a photo identification.” (Part 61.3 lists all of the acceptable forms of ID. A state-issued driver’s license will do nicely.)"

Source: Jackson

5. License and Medical & This is part of an FAA ramp inspection and it might not be a bad idea to present both to your peer. Over the years I've found pilots unaware their medicals had expired or that they needed to document English proficiency on their licenses. Better to have a peer detect these problems than someone carrying an FAA badge.

Things I've learned from Peer Reviews that have changed the way I fly

One of my earliest "ah ha!" moments was given to me by a younger Learjet pilot who noticed I used the words "for" and "to" in clearance readbacks that could have been confused with the words "four" and "two." For more about this technique: Clearance Readback.

One of my more recent "ah ha!" moments was learning that my before takeoff and approach briefings had become rituals that might as well have been scripted. I was giving them without really considering what I was saying and the other pilot had heard them so often he wasn't really listening. We've since adopted a more threat-oriented approach to these briefings: Threat Error Management.

Things I've taught that were received as "game changers"

I've often noticed pilots briefing their approaches without looking at the FMS, a few times the FMS had the wrong approach and once the missed approach was in error. Comparing the FMS to the approach plate is not only a check on the pilot who did the programming, but the FMS itself.

Every now and then I see two pilots independently program their onside FMS, undoing the work of the other pilot and sometimes creating confusion when the FMS didn't react as planned. I explain the first two tenets of crew FMS usage: flight plan changes should only be made by the pilot monitoring, and no changes should be made without the other pilot's knowledge.

I quite often see pilots confused by the top of descent cue given by their cockpit avionics, or unsure of when to begin their descents without the cue. There are several good techniques out there: Descent.

I once witnessed a Gulfstream pilot struggle with a crosswind landing, attempting a wing low landing with about 20 knots on the wing. It ended with the airplane in one piece but it wasn't pretty. The other pilot agreed that Gulfstreams are not easy in a crosswind. Of course this isn't true. Both pilots were unaware of the correct landing procedure in their aircraft. More about this: Crosswind Landings.

At my last peer review, conducted by a friend who is a check airman at a major airline, I learned that I am not as good at observing the sterile cockpit rule as I think I am, but that my crew leadership style has rubbed off on the rest of my pilots and that that is a good thing. The peer review has made me a better pilot. I think it can do the same for you.

Jackson, Kent S., Part 91 Department Inspections, Business & Commercial Aviation Magazine, April 2020, p. 56

NTSB Accident Docket, ERA14MA271

NTSB Aircraft Accident Report, AAR-15/03, Runway Overrun During Rejected Takeoff Gulfstream Aerospace Corporation GIV, N121JM, Bedford, Massachusetts, May 31, 2014

Transportation Safety Board of Canada, Aviation Investigation Report A07A0134, Touchdown Short of Runway, Jetport In., Bombardier BD-700-1A11 (Global 5000) C-GXPR, Fox Harbour Aerodrome, Nova Scotia, 11 November 2007