Multi-label text tag prediction with LLM fine-tuning

To speed up the labeling process, we have built a system for automatic label suggestions based on a fine-tuned Large Language Model. Firstly, to reflect the Client's business success criteria, we have proposed suitable model accuracy metrics and chose an appropriate open source LLM model for learning and predicting class labels. Fine-tuning of the LLM using Tensorflow on cloud GPU servers was performed next, optimizing chosen accuracy metrics. The training process involved optimizing a chosen training loss function and performing a grid search for selecting optimal training meta-parameters for training models for a multi-label classification problem.

After training the models and obtaining train & test quantification accuracy metrics, we have deployed, served and incorporated real-time model predictions into Client's labeling process.

Based on collected Live usage metrics and using Client's feedback, we have then iteratively repeated the LLM fine-tuning process, re-trained the models and successively re-deployed new model versions to match Client's feedback and to achieve desired model accuracy thresholds measured during Live system usage. The final solution was developed using several cycles of train/launch/test/feedback iterations.