Line Operation Observation

• If the only person you have to answer that question is you, or perhaps one or two other pilots who you fly with day in and day out, you are at extreme risk for complacency. As pilots we tend to look for the easiest way to get things done and that is usually the right answer. But after a while doing it easy means doing it without the necessary safety precautions. More about that: Complacency.
• Your simulator recurrent training is one step up from nothing, but how often do you hear of a busted checkride at the sim? Even the feared 14 CFR 135.293 and 297 rides aren't all that difficult. Besides, without the pressure of your passengers on board is it really indicative of how you do your job?
• What about pilots who annually fly a 14 CFR 135.299 evaluation? First off you aren't flying a normal trip and the only required items are a takeoff, an en route segment, and a landing. I've given a ton of these and received more than a few. For one of them, the Fed didn't even bother coming to the cockpit.

The essence of the issue here is nobody but you really knows how you perform on a real, live mission with your every day passengers on board, the normal pressures of weather, ATC, and other variables mucking everything up. Pilots flying for major airlines get their annual 121.440 line check which, on paper, is just as anemic as the 135.299. But in a large pool of pilots with a professional cadre of check airmen things can be tougher. The check airman is indoctrinated into the idea that the solvency of the company relies on them ensuring the line pilots are doing their jobs. We business aviation pilots don't have that.

You can fix all this by simply inviting an outsider to observe you on a trip with passengers. This outsider should have an airline transport rating flying a similar mission, but need not be type rated in the same aircraft. He or she is simply there to see how you do your job and if you comply with industry best practices. You can formalize this process, in fact you should. But even if you don't, the experience will make you a better pilot. When I was a check airman at TAG Aviation US, we called this the Line Operation Observation (LOO) program. Here's how it is done.

1 — Line observation philosophy

2 — Getting started

3 — Line observation pilots

4 — Observation pilot training

5 — Observation pre-brief

6 — Conduct of the observation flight

7 — Assessment

8 — Debrief

The purpose of the Line Operation Observation is to:

1. Observe crew performance under normal operating conditions.
2. Assess the effectiveness of training programs.
3. Determine awareness of company policies and regulatory requirements.
4. Provide a feedback opportunity for crews.

The LOO is not a checkride and the observed pilots' careers are not at risk. The LOO is an opportunity to see how well the pilots are doing in the eyes of another pilot. The better the qualifications of the observing pilot, the better the feedback is likely to be. The primary purpose of the LOO is to catch complacency before it becomes a problem.

The LOO also provides an excellent means to upgrade pilots from SIC to PIC status, from domestic to international captain, and for any other step along the way in the company pilot hierarchy.

The outcome of the LOO should be documented with a form (an example is provided below) that gives the observed pilot a tangible "pat on the back" or reminder of areas of possible improvement.

If you are in a large organization you can select highly qualified and respected pilots for the role of line observation pilots. These aren't necessarily the most experienced pilots, but those with the right "mind-set" for the job, more about that below. They should be trained, more about that below.

If you are in a smaller organization you can still institute an LOO program using existing pilots, but you should also look for an opportunity for an outsider's opinion as well. In either case, select a pilot you respect, someone who satisfies the requirements of a line observation pilot, given below. Invite their participation, something along these lines:

Our flight department is always looking for ways to ensure we are at the top of our game and would welcome an outside look from someone we respect. We are flying _____ (date) and would be pleased if you could join us in the jump seat for a look at how we do our jobs on a typical line operation. This isn't a checkride for us; it is simply a way for us to ensure we are observing best practices, flying safely, and operating to the level we aspire. We would welcome your honest feedback and know we can profit from your professional observations. If this is a new role for you, we have included a few guidelines about how these Line Operation Observations (LOOs) are normally conducted. Thank you very much for your assistance.

Even before you select your line observation pilots, you need to prepare your line pilots for the experience. Most pilots approach any evaluation as a checkride. Even after being told this is a "non punitive observation," they cannot help but think their jobs are at risk. You need to set their minds at ease.

1. The purpose of the LOO is two-fold. First, it is a check to ensure we as an organization are adhering to industry best practices and our own company procedures. Second, it is a way to see if we as an organization are meeting the needs of our crews.
2. The LOO pilot does not have an official crew position and does not have the authority to take any enforcement action. The LOO pilot is simply an observer.
3. If at any time you believe the LOO pilot is interfering with your duties as pilots, you have the authority to ask the LOO pilot to leave the cockpit.

If you are selecting an in-house pilot or inviting an outsider's opinion, select your line observation pilots with care.

1. LOO pilots should be knowledgeable in the applicable requirements of 14 CFR 61, 63, 91, 121, 125, and 135; other applicable FAA policies, and safe operating practices. A working knowledge of company policies and procedures would be very helpful.
2. LOO pilots should be skilled in observation, interpretation and analysis.
3. LOO pilots should be patient and better listeners than speakers.
4. LOO pilots need not be typed in the same aircraft but they should have a similar background. If, for example, your flight department is in the business of carrying business executives on long trips, the LOO pilot should have first-hand experience doing just that.
5. LOO pilots should not have domineering personalities that may look upon any observation as an opportunity to exert power over other pilots. In that spirit, it may be advantageous to select a pilot with former civilian, airline, or military experience as a flight examiner, check airman, or instructor pilot.

If you are selecting an in-house LOO pilot, it is critically important this pilot have a reputation for technical expertise, integrity, and operational capability.

If you are inviting an outside pilot to perform a line observation, perhaps all you really need to do is ensure the pilot meets the qualifications given above and ask them to review the material on this page.

If, on the other hand, you are starting an in-house standards organization for your flight department, something more formal may be called for. You may find a good "check airman" or "flight examiner" program at many major airlines, the military, and in EASA countries. FlightSafety International used to have a very good course, but it is no longer of the same caliber. What follows is a track of self study. (If you have something better, please press the "Contact" button below.

Line Observation Pilots should have a firm grasp of the following fundamental concepts of flight instruction. (Each link contains an in-depth discussion of the topic.)

• Checklist Philosophy
• Complacency
• Crew Resource Management
• "Look Out" Doctrine (Clearing)
• Pilot Psychology
• Situational Awareness
• Stabilized Approaches

The performance of any pilot under even these "relaxed" conditions will often be adversely affected by some degree of nervous tension. The LOO pilot can do much to alleviate this by adopting a friendly and sympathetic attitude. Any suggestion of haste during briefing should be avoided and the applicant should be encouraged to ask as many questions as he or she wishes. Clear and unhurried instructions at this stage will not only serve to put the applicant at his ease, but will ensure when airborne that the flight proceeds smoothly and without unnecessary delay.

Here is the pre-brief I used at TAG Aviation. You would obviously tailor this to your organization.

Line Observation Pre-brief

• _____ and _____, thanks for the opportunity to observe your flight operations
• I’d like to cover the purpose of this line observation briefly, how we’re going to accomplish it, and answer any questions you may have. Then I’ll get out of your hair and let you do your jobs.
• I’m here representing TAG to see how we’re doing when it comes to flying by our company SOPs, (safety, comfort, schedule, economy, client relations). Note I said “we." That means how you’re observing SOPs and how we’re doing when it comes to developing those SOPS, publicizing them, and training our crews.
• This is a line trip from _____ to _____.
• ____, you are the PIC and you are in charge. ____, as the SIC you also have specific responsibilities for the success of the trip.
• Me, I’m just a fly on the wall. As a line observer, there are certain things I can and cannot do. I am here to observe. I will not do anything to get in your way or in anyway make the execution of your duties more complicated – if I do, it is unintentional and please let me know so I can blend back into the woodwork.
• If, on the other hand, ____, you decide you are experiencing an abnormal condition where a third cockpit crewmember would come in handy, just let me know.
• Do you have any questions about my role here?
• Once we are done for the day I’ll need just 30 minutes of your time so we can discuss the flight, go over some paperwork, and answer any questions you might have. Speaking of paperwork, are you both familiar with the standard TAG Line Observation form? It is just a way of documenting the line observation, providing you with feedback, and letting the company know what a great job you are doing. 99.9% of our crews are doing great, so the TAG “average” is satisfactory and that is what I expect to see today.
• One last thing, I am not qualified in the _____ so I’ll need a short briefing on the jump seat, the exit door and hatches, and what you want me to do in the event of a ground evacuation. You can do that at anytime prior to engine start, whenever it is convenient for you.
• Okay, I’ve already heard a lot about the great job you both do each and every day and I know that’s what I’m going to see. Please don’t treat this as a we versus they situation – I am truly here to help.
• If you don’t have any questions, let’s go fly!

Before meeting the crew the LOO pilot must be properly prepared for the flight, reviewing the schedule requirements, the departure and arrival airports, the route of flight, weather, and possible contingency procedures.

The LOO pilot should encourage a friendly and relaxed atmosphere both before and during an LOO. A negative or hostile approach should never be used. Avoid negative comments or criticisms; all assessments should be reserved for the de-briefing.

The LOO pilot will follow the pilots from the initial crew briefing and preflight activities, through all flight activities, shutdown, and post-flight briefings.

The LOO pilot should never occupy a pilot's seat and must take care not to distract the crew. The LOO pilot should help in the effort to clear for other traffic and should politely point out traffic that the crew needs to see but have not themselves spotted.

If the LOO pilot observes a potential safety of flight error, the corrective action should be positively announced. If, for example, the crew fails to level off at an assigned altitude, the LOO pilot should take increasingly direct measures to. "Aren't we supposed to level off at 12,000 feet?" "We are about to violate our altitude clearance." "We need to return to level immediately!"

Once the flight has been completed, the LOO pilot should take a moment to make note of the events, good and bad, and complete a written LOO form, shown below while considering assessment goals and criteria.

Before leaving the aircraft the LOO pilot should consider whether there are any questions that are best answered or issues that are best resolved in the cockpit. It may be prudent to indicate at this stage, for example, that an altimeter has been incorrectly set or a switch is in the wrong position rather than debate the issue later on in a briefing room.

Before debriefing, the examiner should consult his or her notes to decide the assessment for each section as well as the overall result. While the formal outcome will be presented with a written form that has room for two results, "Exceptional" and "Satisfactory," there is a third outcome that must also be considered.

Satisfactory / Meets Standards

We expect to see everything done just as the book says it should be and that is a truly satisfactory result. Even when there are momentary deviations that are positively corrected, or if the crew worked together as a team to overcome a lapse, that is satisfactory too. For example, "Joe you got a little distracted when looking for traffic and we deviated from our assigned altitude a little; but Ann caught it right away and you corrected. That is exactly the way a good crew is supposed to work!"

Having flown as a check airman, flight examiner, and LOO pilot over the decades I have come to the conclusion that the bottom line of a satisfactory performance is this: would I trust this crew to fly my family at night, in the weather? If so, they are satisfactory.

Exceptional / Exceeds Standards

An exceptional performance is indicated when an action taken by the crew sets them above what a fully qualified crew would be expected to do. For example, "Joe that was truly remarkable the way you anticipated the ceiling would drop just prior to our arrival and your decision to divert to the alternate meant we avoided the risk of a low altitude missed approach. And we got into our alternate ahead of all the other airplanes who didn't have that kind of situational awareness. Well done!"

Unsatisfactory

A crew that consistently exceeds relevant tolerances or fails to take prompt corrective action when tolerances are exceeded is indicative of unsatisfactory performance. An LOO, however, is not an evaluation so there is a range of possible outcomes.

• Performance does not reflect best practices and would have resulted in a 14 CFR 61.58 failure — If the crew appears to embrace an attitude that is clearly ill suited for a professional flight department but does not rise to the level of compromising safety, the LOO pilot could mark the LOO form result as "Satisfactory" with a remark that says "Recommend ___." That should be followed by a verbal critique. For example, I once flew an LOO where the crew was clearly unaware of the presence of Class B airspace over their approach path and violated the 14 CFR 91.117(c) prohibition against exceeding 200 knots in the airspace underlying Class B airspace. They got away with it. I marked the every item in the LOO form as "Satisfactory" but added the following to the overall remarks. "The crew should spend more attention to airspace rules, including 14 CFR 91.117(c), to preclude what could have been a violation, had the airspace been more crowded today." I think the embarrassment they felt at this moment motivated them to do better, so my job was done.
• Performance was unsafe and only luck or the LOO pilot's actions prevented damage to the aircraft or possible injury to persons — Herein lies an LOO's pilot biggest challenge. If you observe something unsafe that could have resulted in damage to the aircraft or other property, or could have resulted in injuries or death, what should you do? I can provide an example, but it is just an anecdotal example. Every situation will be unique. I was in the jump seat of a Falcon 900 when the crew was cleared to a new altitude while being vectored for a visual approach. The PM dialed in the new altitude but the PF neglected to select an autopilot vertical mode and the airplane remained at our previously cleared altitude. I waited 60 seconds and then tapped the PF on the shoulder and said, "you need a vertical mode." At that moment he spotted the runway, keyed his microphone and canceled with approach control, rolled into 45° of bank, extended the landing gear, changed the radio to tower frequency and got clearance to land. The PM looked up and said, "What?" In the eyes of ATC they got away with more infractions than I had time to count. I called their management company and recommended they both be fired. Our company attempted to do that but the pilots convinced the aircraft owners to leave the management company. Three months later their new management company saw what I saw and fired them both.

The LOO pilot should conduct a fair and unbiased debriefing based on identifiable factual items. A balance between friendliness and firmness should be maintained. If everything went well you should cover the flights chronologically and can use the LOO Form as a guide.

If you think there will be a lot of discussion you should try a facilitative approach to flush out the details of procedures and techniques to be learned. Pilots tend to learn best when participating in the learning process and will "buy in" to a change in behavior better this way. One effective facilitation method is to:

1. Start with an overview of the LOO, covering the positive points only.
2. Cover other points and ask a few opening questions per issue.
3. Get the applicant to do the thinking and talking.
4. Summarize at the end (it can be useful to get the applicant to summarize), steering the conversation in the direction you think best.

Try to cover good as well as bad points. With the good points, emphasize that you will profit from having seen them in action. For example, "I am going to add that technique to my bag of tricks!" With the bad points, try to interject procedures and techniques that will help them avoid them in the future.

Documentation

You may not see a need for documentation but it will serve you well in future audits and if you ever want to do trend analysis. The following LOO form is based on one we used at TAG Aviation US and is organized by special emphasis items. Each can be graded as “Exceeds Standards” or “Meets Standards.” A remarks section follows each item for quick notes to aid the debrief.

LOO Form Example