Pack-Out Changed After a Shift Change? Check Whether Your Team Grades Fruit the Same Way

By Coraline Liao, Brand Director

A change in pack-out after a shift change may reflect the fruit, the grading decisions or both. Before changing crop management or challenging a quality trend, check whether assessors apply the same written criterion consistently.

The useful question is specific: when different people inspect the same stable specimen under comparable conditions, do they assign the same category for one defined quality trait? A team can investigate that without claiming an official grade or judging an employee’s character.

Define one decision before selecting the fruit

Choose a single individual-fruit criterion and identify the buyer specification or applicable standard, version and product scope. Define the allowed categories before anyone sees the results. For a simple internal check these might be “meets this criterion,” “does not meet” and “uncertain/review.” These are not whole-lot grade labels.

For greenhouse tomatoes, USDA provides a separate greenhouse-tomato standard. The currently linked document, effective March 19, 2007, distinguishes defect definitions from tolerances, size and pack provisions. Its definitions matter only within the appropriate specification and market context.

Do not borrow sampling numbers from USDA’s general field-tomato inspection handbook: it explicitly excludes greenhouse tomatoes. Nor should an individual-fruit agreement check be presented as official U.S. grading, market access or food-safety clearance.

Separate a challenge set from a production sample

The quality lead can assemble a training set containing straightforward examples and cases close to the chosen decision boundary, using the operation’s safe handling procedure. Give each specimen a stable item ID. Record why it was selected and which criterion it represents.

A deliberately difficult set helps reveal ambiguous interpretation. It cannot estimate the commercial reject rate. If the business also needs a production audit, identify that separately selected sample and its sampling basis. Do not combine its results with the training set under one pack-out percentage.

Preserve independent judgments

Give assessors the agreed criterion and comparable observation conditions. Have them record their own classification before discussion, without seeing another person’s answers. Use role codes rather than unnecessary personal employee details.

For a repeat round, change the order and conceal prior ratings. Keep the interval and conditions documented. A fruit can ripen or acquire handling damage between rounds; if the relevant condition changes, invalidate that comparison and record why. Do not count a changed specimen as proof of inconsistent judgment.

Photographs can support review of visible features when the image method is appropriate. They cannot establish firmness, internal condition or every trait that a physical assessment requires. Use only authorized images, and retain the limitations of any image-based exercise.

Build three separate views of agreement

NIST’s measurement terminology distinguishes agreement under the same conditions from agreement when conditions such as the observer change. It also separates these ideas from closeness to a reference. That distinction is useful here, although NIST is not prescribing this categorical produce-check method.

1. Same assessor across rounds: pair that person’s valid ratings of the same unchanged items. This shows where the decision changes on repetition.
2. Different assessors: pair independent ratings of the same valid items under the recorded conditions. This shows differences between people applying the rule.
3. Each assessor against a reference: compare with a competent, documented reference decision where one exists. If none exists, report agreement and disagreement, not accuracy.

Make a separate matrix for each comparison and criterion. Name exactly what the rows and columns represent. Do not merge all rounds, assessors and references into one score.

The off-diagonal cells show the direction of disagreement. For example, repeated “meets” versus “does not meet” differences raise a different question from repeated uncertainty. Keep uncertain ratings in their named category; never silently recode them as acceptance.

If reporting a percentage, state the numerator and denominator. A descriptive all-category agreement percentage is matching-category pairs divided by valid paired records, multiplied by 100. It includes uncertain–uncertain matches, so report that count separately. It must not be called the percentage of correct commercial decisions.

Correct the boundary that caused the disagreement

Review the original independent records before discussing disputed specimens. The cause may be unclear wording, an unsuitable observation condition or a particular boundary that needs better examples. A high overall agreement rate can hide a recurring commercially important disagreement.

USDA’s Terminal Market Inspection Training Manual includes practical instruction and review of independently graded work. This supports the value of structured learning; it does not certify this internal kit or establish a universal passing percentage.

After clarifying the cause, run a fresh, comparable check. Repeating the group’s now-familiar answers is not sufficient evidence of improvement. Use the result to strengthen the specification and assessment process. It does not establish that a worker was careless, authorize personnel discipline or decide whether potentially unsafe produce may be sold.