FlagAI is a quick, simple, and adaptable toolbox for training, fine-tuning, and deploying large-scale models. It is now focused on NLP models and tasks,but will soon support other modalities. FlagAI is compatible with a wide number of pre-trained models, including WuDao GLM, BERT, RoBERTa, GPT2, T5, and Huggingface Transformers models. It offers APIs for fast downloading and using these models, fine-tuning them on a variety of datasets, and sharing them with the community. In this article, we will go over the FlagAI and demonstrate how to install FlagAI with example.
FlagAI also has a prompt-learning toolbox for one-time assignments. These models can be used to perform tasks such as text classification, information extraction, question answering, summarization, and text generation on Chinese or English text.
Prerequisites
- Python version >= 3.8
- PyTorch version >= 1.8.0
- Installing CUDA and NCCL is also required for training/testing models on GPUs.
Why should I use FlagAI?
If you are looking for a powerful and easy-to-use toolkit for large-scale language models, then FlagAI is a great choice. Here some of the key features of FlagAI follows:
Download Models using API key
- FlagAI provides an API key that lets you to easily download pre-trained models and fine-tune them on a wide range of datasets collected from SuperGLUE and CLUE benchmarks for both Chinese and English text. You can save a lot of time and work by not having to train the models yourself.
- FlagAI now supports over 30 mainstream models, including the Language Model Aquila, the multilingual text and image representation model AltCLIP, the text-to-image generation model AltDiffusion Huggingface space, the WuDao GLM (with a maximum of 10 billion parameters), the EVA-CLIP, OPT, BERT, RoBERTa, GPT2, T5, ALM, and Huggingface Transformers models.
- This gives you a wide range of models to choose from, allowing you to discover the one that best suits your needs.
Parallel train with fewer than 10 lines of code
- FlagAI is supported by four of the most prominent data/model parallel libraries, including PyTorch, Deepspeed, Megatron-LM, and BMTrain.
- FlagAI enables easy integration, allowing users to parallelize their training/testing process with fewer than 10 lines of code. This can drastically accelerate the training process.
Conveniently use the few-shot learning toolkits
- FlagAI also provides a prompt-learning toolkit for few-shot jobs.
- This is useful if you just have a little amount of data to train on.
Particularly good at Chinese tasks
- FlagAI is particularly adept at Chinese assignments. This is due to the fact that it was trained on a big dataset of Chinese text.
- FlagAI is a wonderful solution if you are working on a Chinese language project.
Overall, FlagAI is a strong tool that can help you rapidly and easily design and deploy AI models. It is an excellent choice for both new and seasoned users.
How To Install FlagAI?
The command below is used to install FlagAI using pip,
pip install -U flagai
[Optional] To install FlagAI and develop locally, follow these steps
git clone https://github.com/BAAI-Open/FlagAI.git python setup.py install
[Optional] Install NVIDIA’s apex software for faster training
git clone https://github.com/NVIDIA/apex cd apex pip install -v -disable-pip-version-check -no-cache-dir -global-option=”-cpp_ext” -global-option=”-cuda_ext” ./
[Optional] DEEPSPEED should be installed for ZeRO optimizers
git clone https://github.com/microsoft/DeepSpeed cd DeepSpeed DS_BUILD_CPU_ADAM=1 DS_BUILD_AIO=1 DS_BUILD_UTILS=1 pip install -e . ds_report # check the deespeed status
[Optional] Install BMTrain (>= 0.2.2) for BMTrain training
git clone https://github.com/OpenBMB/BMTrain cd BMTrain python setup.py install
[Optional] Install BMInf for low-resource inference
pip install bminf
[Optional] For Flash Attention, install Flash-attention (>=1.0.2)
pip install flash-attn
How to use AutoLoad Class?
They offer a variety of models that have been trained to execute a different tasks. You can use AutoLoader to load these models and generate predictions.
Example of FlagAI Toolkit
The example code shows how to utilize the FlagAI toolkit’s AutoLoad class to fast load models and tokenizers.
Model and Tokenizer
The FlagAI toolkit’s AutoLoad class provides for the quick loading of models and tokenizers. This class can be used to simplify the process of loading the desired model and tokenizer. The following is an example of how to utilize the AutoLoad class:
from flagai.auto_model.auto_loader import AutoLoader # Specify the model and tokenizer names and Load the model and tokenizer auto_loader = AutoLoader( task_name=”title-generation”, model_name=”BERT-base-en” ) model = auto_loader.get_model() tokenizer = auto_loader.get_tokenizer()
This is an example of the title_generation task, but you may model additional tasks by changing the task_name. The model and tokenizer can then be used to fine-tune or test the model.
Predictor
The Predictor class is used to predict for various tasks, as shown in the following example.
from flagai.model.predictor.predictor import Predictor predictor = Predictor(model, tokenizer) test_data = [ “Four minutes after the red card, Emerson Royal nodded a corner into the path of the unmarked Kane at the far post, who nudged the ball in for his 12th goal in 17 North London derby appearances. Arteta’s misery was compounded two minutes after half-time when Kane held the ball up in front of goal and teed up Son to smash a shot beyond a crowd of defenders to make it 3-0.The goal moved the South Korea talisman a goal behind Premier League top scorer Mohamed Salah on 21 for the season, and he looked perturbed when he was hauled off with 18 minutes remaining, receiving words of consolation from Pierre-Emile Hojbjerg.Once his frustrations have eased, Son and Spurs will look ahead to two final games in which they only need a point more than Arsenal to finish fourth.”, ] for text in test_data: print( predictor.predict_generate_beamsearch(text, out_max_length=50, beam_size=3))
The seq2seq task is a type of natural language processing task where a model is trained to translate a sequence of words from one language to another. The predict_generate_beamsearch function can be used to generate a beam of possible translations for a given input sequence.
NER(Named Entity Recognition)
- Named entity recognition (NER) is the task of identifying named entities in a text, such as persons, organizations, and locations.
- Title generation is the process of creating a title for a given piece of text.
- Question answering: In this activity, you must answer questions concerning a specific piece of text.
from flagai.auto_model.auto_loader import AutoLoader from flagai.model.predictor.predictor import Predictor task_name = “ner” model_name = “RoBERTa-base-ch” target = [“O”, “B-LOC”, “I-LOC”, “B-ORG”, “I-ORG”, “B-PER”, “I-PER”] maxlen = 256 auto_loader = AutoLoader(task_name, model_name=model_name, load_pretrain_params=True, class_num=len(target)) model = auto_loader.get_model() tokenizer = auto_loader.get_tokenizer() predictor = Predictor(model, tokenizer) test_data = [ “On June 15, the Caocao Gaoling Cultural Relics Team of the Henan Provincial Institute of Cultural Relics and Archaeology publicly issued a statement admitting: “It has never been said that the unearthed beads belonged to the owner of the tomb”, “On April 8, the Beijing Winter Olympics and Winter Paralympics Summary and Commendation Conference was grandly held in the Great Hall of the People. General Secretary Xi Jinping attended the conference and delivered an important speech. In his speech, the General Secretary fully affirmed the Beijing Winter Olympics, Winter Olympics The outstanding achievements of the Paralympic Games comprehensively reviewed the extraordinary process of preparations for the seven years, in-depth summed up the valuable experience in preparing for the Beijing Winter Olympics and the Winter Paralympics, and profoundly explained the spirit of the Beijing Winter Olympics. Austrian heritage to promote high-quality development and put forward clear requirements.”, “On the 8th local time, the European Commission stated that the governments of EU member states have frozen a total of about 30 billion euros in assets related to Russian oligarchs and other sanctioned Russian personnel.”, “Under this state of handicap, Betfair’s Asian trading data shows that Bologna is hot. From the perspective of European betting, it is also hot for the home team. Palermo has lost two games in a row,” ] for t in test_data: entities = predictor.predict_ner(t, target, maxlen=maxlen) result = {} for e in entities: if e[2] not in result: result[e[2]] = [t[e[0]:e[1] + 1]] else: result[e[2]].append(t[e[0]:e[1] + 1]) print(f”result is {result}”)
Benefits of Using FlagAI
- FlagAI is designed to be simple and quick to use, even for beginners. The API is straightforward and easy to use, and the documentation is extensive.FlagAI is an open-source project, thus it is simple to extend and customize. New features, models, and datasets can be added.
- FlagAI is built to scale to huge datasets and models. It is capable of training and deploying models on a wide range of hardware platforms, including CPUs, GPUs, and TPUs.
Conclusion
In summary, FlagAI is a fast, simple, and flexible model training, fine-tuning, and deployment toolset. It may do a variety of downstream tasks, such as seq2seq, NER, title generating, and question answering. FlagAI is a useful tool for accelerating the development of large-scale AI applications. Please feel free to share your thoughts and feedback in the comment section below.
FlagAI: Train and Deploy Large-Scale Models Faster and Easier