Label in the Loop
An efficient system for effective training of machine learning models through a labeling process.
Label in the Loop is an enabling technology for data driven organizations that need collaboration with or supervision of domain experts over an unlabeled body of data.
LITL’s core value proposition is smart selection of data samples and smart designation of those samples to domain experts.
LITL will help you with:
Reduction of time and cost of machine learning model training
Training machine learning models in no data or small data scenarios – allowing for gradual increase of model quality and its adaptation to changing data or requirements
Experts analytics (i.e. experts assignment, experts consensus and experts performance estimation)
How does it work?
LITL is an active learning system to effectively train ML models in the setup of unlabeled data. Specifically, LITL selects samples from an unlabeled pool that will have the most influence on the model’s performance and assigns them to experts according to their estimated performance on similar samples. Experts’ assignment process itself balances between exploration and exploitation of experts’ knowledge to estimate their performance more effectively. The whole process works iteratively by retraining the model with consecutive batches of samples. As the model learns on an increasing labeled dataset, it changes its uncertainty towards unlabeled samples, on which the sampling process is based.