One of the most important parts of winemaking is managing risk. Every vintage, at every stage of production, the winemakers only have one shot at making the best possible wine from the fruit they received. A lot of effort is expended on preventing things from going wrong and preserving the quality of the wine for the weeks, months, or even years that it takes to develop.
It is then unsurprising that when a prudent winemaker is presented with a new product, process, or method of production, they will rarely jump on it based on advertised benefits alone. Instead, they will generally choose to trial the novelty on a small scale first. This trial experience is an opportunity to ensure that the promise of the improvement is real, that the actual improvement is worth the cost of adopting the innovation, and that there are no unexpected side effects which would detract from the overall benefit to the winery.
Of course, one obvious downside to trials is that they take time, money, and other resources to run – all costs that do not immediately generate offsetting operational gains. Nevertheless, trials are an invaluable tool for learning about what is possible, what is best, and where the risks may lie. And, like with most things, there are the right and the wrong ways to conduct them.
Below, we present five steps to a right way, which maximizes the educational value of a trial, while reducing its costs:
1) Assign an accountable Trial Leader
Every trial should be overseen by a single winery employee – typically a winemaker. This person must be involved from the very start, lead all planning and preparation, be the main contact point for any 3rd parties (product vendors, cooperage reps, consultants, etc.) that support the trial. They also need to be accountable to the winery’s decision makers (Owner, General Manage, VP of Winemaking, etc.) when it comes to reporting results.
Why it matters: Imagine running a new oak trial where different winemakers conduct tastings at different times. Or missing a trial activity because two people thought that the other person will do it. A single Trial Leader owning all activities and information can help prevent these things from happening, while making reporting, course corrections, and final evaluation of the trial results that much easier.
2) Have a written Trial Execution Plan
In winemaking, as in life, it helps to know what to expect. Trials generally require time, space, money, and other resources. Without a solid Trial Execution Plan it is easy to miss key information, then have to scramble last minute to catch up.
Before starting a trial, the Trial Leader should make sure that they have confirmed:
- Trial timeline: how long will it take? Are there multiple stages? Where could there be delays?
- Human resources: who needs to be involved? What is expected from them? Are they available when needed, according to the timeline?
- Material resources: what equipment, if any, is needed? What about consumables? Is there enough in stock? Where to procure more? What are the cost, and how will these be paid?
- Risk mitigation: what can go wrong? What is the probability of it going wrong? What is the magnitude of the potential damage? How do we mitigate this?
- Success criteria and next steps: what makes a trial a success? More on this later…
Why it matters: winemaking is a slow process, and as such, trials can take a long time to complete. People get busy, team members change, things get forgotten and missed. A solid, written Trial Execution Plan serves as a manual, and reference guide for anyone involved in the trial. Most importantly, it helps the Trial Leader to plan their own time, and the time of anyone else needed for the trial.
3) Define Trial Success Criteria
This is one of the toughest, but also one the most important sections of the Trial Execution Plan. Here, the Trial Leader should involve the ultimate decision-makers to determine, together with the supporting 3rd parties, what trial outcomes the winery is looking for, and how to measure them. The measurement part is key – this needs to be objective and precise. For example, if trialing an automated SO2 dosing machine in the barrel room, a desirable outcome might be a reduction in labour costs. But what does this mean? Is it time saving, so that set amount of work is done faster? How much faster? Is a 10% saving enough? 25%? If the machine costs $5,000, what % saving hits the ROI target? Perhaps side effects should also be considered, like impact on wine quality, or health and safety risks? Good success criteria must be defined and measurable. So the (unhelpful) desired outcome “if it helps us add SO₂ faster we might buy it” changes to much more useful success criteria “we will buy the machine if it saves us at least 25% of the time currently spent on adding SO₂, reduces incidence dosing errors by at least 90%, has no negative effect on wine quality, and does not increase health and safety risks for the cellar workers”. With this, at the end of the trial the decision on whether to purchase the machine is clear – it either hit the success criteria (buy) or it has not (do not buy).
Why it matters: The purpose of any trial is for the winery to decide whether they should make a change. In other words, the needs to be a learning outcome. A trial that does not generate a clear go/no go outcome is a waste of time, money, and effort. It is important to define these outcomes with the relevant decision-makers. Different stakeholders will have different priorities. Building these priorities into the Trial Success Criteria increases the chances of a productive, informative trial that yields and actionable outcome.
4) Agree Post-Trial Actions
In the SO₂ doser example above, the next steps were straight forward: the winery either buys or does not buy the machine. But when trialing more complex changes, such as new fermentation methods, complex equipment, software systems, etc., it is important to consider in advance what happens if the trial is a success. Where will the changes be rolled out and when? What support is required, and what is available? Who needs to be involved in implementing the change? What about training? Are there other changes not directly related to the process or product trialled, that need to take place to support this? By working through the steps that would follow a successful trial the Trial Leader will likely also refine the Trial Execution Plan and the Trial Success Criteria, closing the trial preparation loop.
Why it matters: many wineries have experienced trying new things without really considering what they would do with the results of these trials. The trial may be well planned, may generate new information, and may even be a ‘success’ according to the set criteria, but still never lead to actionable insights and practical improvements in the winery because the Next Steps have not been properly considered.
5) Continue the Post-Trial Evaluation
Trial or not, successful wineries constantly review and evaluate their processes, looking for cost savings, quality improvements, and risk reductions. As such, even after a successful trial leads to a well-planned implementation of a new process or product, someone should continue studying its effect on operations. Scheduling ongoing check-ins with the relevant stakeholders to evaluate long term performance of implemented changes is a great way to make sure that nothing is missed.
Why it matters: in some instances, small scale trial outcomes may not be representative of large-scale changes. Employees who were not involved in the trial, or those in other departments, may bring a fresh perspective to the situation, and uncover new information. New problems might arise through repeated use of equipment. Certain effects may take longer to develop, and only present themselves after the trial has completed. Constantly asking whether something is working as well as it could, and how one might make it better, is a sure way for any winery to continue improving as a business, and achieving its goals.
Trial planning templates
These steps above have been used by BarrelWise to support planning and execution of trials, as well subsequent adoption of our products at scale.
Trials are a crucial part of life for the BarrelWise team. Every new product design is extensively tested in multiple winery cellars to ensure that it works as intended, with user feedback guiding changes, improvement, and development of new features. For the wineries, BarrelWise trials represent a low-risk way to evaluate the capabilities of a new tool and conduct a cost/benefit analysis on its implementation.
We would be happy to share our Trial Execution Plan template (which includes the Timeline and Success Criteria) with your winery, to help you prepare for any other upcoming trials. Just shoot us an email on hello@barrelwise.ca.
.svg)
.png)