Photo: Line Observation, from Mark Eisner (with permission).
How well do you do your job as a professional pilot? How sure are you about that? There is a problem with the way we answer that question and it really doesn't matter if your are flying under 14 CFR 91, 125, or 135.
The essence of the issue here is nobody but you really knows how you perform on a real, live mission with your every day passengers on board, the normal pressures of weather, ATC, and other variables mucking everything up. Pilots flying for major airlines get their annual 121.440 line check which, on paper, is just as anemic as the 135.299. But in a large pool of pilots with a professional cadre of check airmen things can be tougher. The check airman is indoctrinated into the idea that the solvency of the company relies on them ensuring the line pilots are doing their jobs. We business aviation pilots don't have that.
You can fix all this by simply inviting an outsider to observe you on a trip with passengers. This outsider should have an airline transport rating flying a similar mission, but need not be type rated in the same aircraft. He or she is simply there to see how you do your job and if you comply with industry best practices. You can formalize this process, in fact you should. But even if you don't, the experience will make you a better pilot. When I was a check airman at TAG Aviation US, we called this the Line Operation Observation (LOO) program. Here's how it is done.
TAG Aviation US was a believer in a strong LOO program and insisted all of its pilots experience it annually. I could tell which crews were new to the process because many seemed to be reluctant participants. But once it was done, with very few exceptions, they all agreed it was the best learning exercise they had at this level of aviation. If done correctly, an LOO will make you a better and safer pilot. That is the bottom line, after all.
The purpose of the Line Operation Observation is to:
The LOO is not a checkride and the observed pilots' careers are not at risk. The LOO is an opportunity to see how well the pilots are doing in the eyes of another pilot. The better the qualifications of the observing pilot, the better the feedback is likely to be. The primary purpose of the LOO is to catch complacency before it becomes a problem.
The LOO also provides an excellent means to upgrade pilots from SIC to PIC status, from domestic to international captain, and for any other step along the way in the company pilot hierarchy.
The outcome of the LOO should be documented with a form (an example is provided below) that gives the observed pilot a tangible "pat on the back" or reminder of areas of possible improvement.
If you are in a large organization you can select highly qualified and respected pilots for the role of line observation pilots. These aren't necessarily the most experienced pilots, but those with the right "mind-set" for the job, more about that below. They should be trained, more about that below.
If you are in a smaller organization you can still institute an LOO program using existing pilots, but you should also look for an opportunity for an outsider's opinion as well. In either case, select a pilot you respect, someone who satisfies the requirements of a line observation pilot, given below. Invite their participation, something along these lines:
Our flight department is always looking for ways to ensure we are at the top of our game and would welcome an outside look from someone we respect. We are flying _____ (date) and would be pleased if you could join us in the jump seat for a look at how we do our jobs on a typical line operation. This isn't a checkride for us; it is simply a way for us to ensure we are observing best practices, flying safely, and operating to the level we aspire. We would welcome your honest feedback and know we can profit from your professional observations. If this is a new role for you, we have included a few guidelines about how these Line Operation Observations (LOOs) are normally conducted. Thank you very much for your assistance.
Even before you select your line observation pilots, you need to prepare your line pilots for the experience. Most pilots approach any evaluation as a checkride. Even after being told this is a "non punitive observation," they cannot help but think their jobs are at risk. You need to set their minds at ease.
If you are selecting an in-house pilot or inviting an outsider's opinion, select your line observation pilots with care.
If you are selecting an in-house LOO pilot, it is critically important this pilot have a reputation for technical expertise, integrity, and operational capability.
If you are inviting an outside pilot to perform a line observation, perhaps all you really need to do is ensure the pilot meets the qualifications given above and ask them to review the material on this page.
If, on the other hand, you are starting an in-house standards organization for your flight department, something more formal may be called for. You may find a good "check airman" or "flight examiner" program at many major airlines, the military, and in EASA countries. FlightSafety International used to have a very good course, but it is no longer of the same caliber. What follows is a track of self study. (If you have something better, please "Contact Eddie" below.
Line Observation Pilots should have a firm grasp of the following fundamental concepts of flight instruction. (Each link contains an in-depth discussion of the topic.)
The performance of any pilot under even these "relaxed" conditions will often be adversely affected by some degree of nervous tension. The LOO pilot can do much to alleviate this by adopting a friendly and sympathetic attitude. Any suggestion of haste during briefing should be avoided and the applicant should be encouraged to ask as many questions as he or she wishes. Clear and unhurried instructions at this stage will not only serve to put the applicant at his ease, but will ensure when airborne that the flight proceeds smoothly and without unnecessary delay.
Here is the pre-brief I used at TAG Aviation. You would obviously tailor this to your organization.
Before meeting the crew the LOO pilot must be properly prepared for the flight, reviewing the schedule requirements, the departure and arrival airports, the route of flight, weather, and possible contingency procedures.
The LOO pilot should encourage a friendly and relaxed atmosphere both before and during an LOO. A negative or hostile approach should never be used. Avoid negative comments or criticisms; all assessments should be reserved for the de-briefing.
The LOO pilot will follow the pilots from the initial crew briefing and preflight activities, through all flight activities, shutdown, and post-flight briefings.
The LOO pilot should never occupy a pilot's seat and must take care not to distract the crew. The LOO pilot should help in the effort to clear for other traffic and should politely point out traffic that the crew needs to see but have not themselves spotted.
If the LOO pilot observes a potential safety of flight error, the corrective action should be positively announced. If, for example, the crew fails to level off at an assigned altitude, the LOO pilot should take increasingly direct measures to. "Aren't we supposed to level off at 12,000 feet?" "We are about to violate our altitude clearance." "We need to return to level immediately!"
Once the flight has been completed, the LOO pilot should take a moment to make note of the events, good and bad, and complete a written LOO form, shown below while considering assessment goals and criteria.
Before leaving the aircraft the LOO pilot should consider whether there are any questions that are best answered or issues that are best resolved in the cockpit. It may be prudent to indicate at this stage, for example, that an altimeter has been incorrectly set or a switch is in the wrong position rather than debate the issue later on in a briefing room.
Before debriefing, the examiner should consult his or her notes to decide the assessment for each section as well as the overall result. While the formal outcome will be presented with a written form that has room for two results, "Exceptional" and "Satisfactory," there is a third outcome that must also be considered.
We expect to see everything done just as the book says it should be and that is a truly satisfactory result. Even when there are momentary deviations that are positively corrected, or if the crew worked together as a team to overcome a lapse, that is satisfactory too. For example, "Joe you got a little distracted when looking for traffic and we deviated from our assigned altitude a little; but Ann caught it right away and you corrected. That is exactly the way a good crew is supposed to work!"
Having flown as a check airman, flight examiner, and LOO pilot over the decades I have come to the conclusion that the bottom line of a satisfactory performance is this: would I trust this crew to fly my family at night, in the weather? If so, they are satisfactory.
An exceptional performance is indicated when an action taken by the crew sets them above what a fully qualified crew would be expected to do. For example, "Joe that was truly remarkable the way you anticipated the ceiling would drop just prior to our arrival and your decision to divert to the alternate meant we avoided the risk of a low altitude missed approach. And we got into our alternate ahead of all the other airplanes who didn't have that kind of situational awareness. Well done!"
A crew that consistently exceeds relevant tolerances or fails to take prompt corrective action when tolerances are exceeded is indicative of unsatisfactory performance. An LOO, however, is not an evaluation so there is a range of possible outcomes.
The LOO pilot should conduct a fair and unbiased debriefing based on identifiable factual items. A balance between friendliness and firmness should be maintained. If everything went well you should cover the flights chronologically and can use the LOO Form as a guide.
If you think there will be a lot of discussion you should try a facilitative approach to flush out the details of procedures and techniques to be learned. Pilots tend to learn best when participating in the learning process and will "buy in" to a change in behavior better this way. One effective facilitation method is to:
Try to cover good as well as bad points. With the good points, emphasize that you will profit from having seen them in action. For example, "I am going to add that technique to my bag of tricks!" With the bad points, try to interject procedures and techniques that will help them avoid them in the future.
You may not see a need for documentation but it will serve you well in future audits and if you ever want to do trend analysis. The following LOO form is based on one we used at TAG Aviation US and is organized by special emphasis items. Each can be graded as “Exceeds Standards” or “Meets Standards.” A remarks section follows each item for quick notes to aid the debrief.
Figure: Blank LOO Form (Click to download)