When we began working on the next version of our chatbot, built on Argilla, we had a clear goal in mind: to reduce the manual load, speed up training, and improve model clarity through better labels. It wasn't about chasing buzzwords or bolting on extra layers. We just needed smarter labeling. That's where Distilabel came into the picture.

Argilla, on its own, gives you strong annotation workflows and visibility into data labeling quality. But as datasets grew and we introduced more nuanced feedback from annotators, we found ourselves stuck in the same loop. So, we looked at Distilabel not just as a tool but as a new backbone for managing label consolidation in a way that made sense for iterative model training.

Why Distilabel Fit Our Workflow

Let’s begin with what made Distilabel stand out. At its core, it’s a library built to help you train decision-making models for label aggregation. Instead of relying on fixed majority votes or hand-crafted heuristics, you can teach it how to resolve conflicts in labels. And when your annotations come from a mix of models, humans, and templates — that matters.

We weren’t interested in building a perfect pipeline from day one. What we needed was flexibility. Distilabel gave us that by letting us build custom distillation strategies. For our case, we used it to:

• Combine model predictions with human annotations
• Prioritize agreement over the number of labels
• Weigh annotator trust levels dynamically
• Fine-tune the output based on evolving definitions

There’s a difference between just tallying up votes and trying to reflect actual intent in data. With Distilabel, we could inject judgment without adding more manual review.

Step-by-Step: How We Built the Argilla 2.0 Chatbot Using Distilabel

Step 1: Start with Multi-Annotator Feedback

First, we collected responses for prompts using a mix of human reviewers and LLM-generated replies. Each prompt had at least three annotations — sometimes more, depending on ambiguity. The goal here wasn’t to chase volume but to get a broad enough view of intent and appropriateness.

All annotations were logged into Argilla, where we could track annotator ID, timestamps, explanations, and even confidence levels when available. That history turned out to be more useful than we originally thought.

Step 2: Design a Distillation Strategy

Next, we created a strategy to combine all that feedback into a single, meaningful label. Here’s how it looked:

• Each annotation was treated as a partial signal — never the full picture.
• We assigned weights based on annotator history (e.g., consistency with others, number of reviewed samples).
• LLMs were used as secondary raters, not primary decision-makers.
• When agreement was low, the strategy favored cautious labels rather than forced conclusions.

Distilabel let us train this strategy like a model. Over time, it improved at resolving common conflicts — especially in borderline helpfulness and hallucination cases.

Step 3: Create Distilled Datasets

Once we had confidence in the strategy, we applied it to the entire set of annotations to create a clean, distilled dataset. This wasn't just “one label per example.” In cases of nuanced feedback, we included metadata about uncertainty, prior conflicts, and even links to original annotations.

We versioned the datasets in Argilla and tagged them by the strategy used — so that later, we could compare the performance of models trained on different distillation approaches. This was key to our internal validation process.

Step 4: Fine-Tune with Iterative Feedback

The chatbot wasn’t built in one shot. After every round of training, we fed a subset of model outputs back into Argilla for fresh annotation. That meant every training loop gave us a new chance to test how well the distillation held up.

Distilabel didn’t just work on the first pass. We re-ran the strategy each time using updated signals, so the dataset evolved with the chatbot. In a way, it became the model’s memory — a smarter one that grew more consistent over time.

Where This Helped Most

Better Feedback Filtering

Before, we had to go through pages of disagreements to figure out why a certain prompt response was labeled “unhelpful” or “incorrect.” With Distilabel, we could surface explanations tied to disagreements and re-rank feedback samples for review.

This helped during the critical evaluation stage. Instead of reviewing random samples, we focused on high-uncertainty cases. And that made human reviews count more.

Faster Onboarding for New Annotators

New reviewers didn’t have to guess what makes a good label. We shared examples from Distilabel where label decisions were weighted heavily in one direction — and explained why. It made onboarding smoother, and over time, it reduced the range of disagreements.

Reduced Model Drift

Because we trained the model on outputs from a controlled distillation strategy, we had a much tighter grip on where and how things were changing. We didn’t need to overhaul everything after each fine-tuning. When hallucinations or tone issues cropped up, we tracked them down to specific rounds of annotations and retrained from just those segments.

Small Details That Made a Big Difference

• Keeping Annotator Notes: We asked annotators to leave short freeform notes. During distillation, these notes helped resolve edge cases that otherwise looked like random disagreements.
• Feedback Tiers: Not all labels carry the same weight. We tagged feedback as a high, medium, or low signal based on confidence and agreement. Distilabel could read this and adjust accordingly.
• Version Control via Argilla: Every major distillation run was versioned and stored with tags. This made it easy to trace model behavior back to a specific round of training data.

Wrapping Up

Creating the Argilla 2.0 chatbot wasn’t about scaling up for the sake of it. It was about improving the way we treat training data — with more care, more context, and less repetition. Distilabel made it possible to go beyond “pick the most common label” and toward something closer to understanding what makes a response useful.

In the end, what we got was a chatbot that reflects that same balance — clear, steady, and better at adapting to the kind of feedback that matters.