Tasks¶
Tasks are classes that act as a “middleman” between you and a model. They know how to communicate with models running locally and in our API.
Supported Tasks¶
Tasks can be imported directly from backprop.
See the full reference in backprop.tasks.
To see which models are supported for each task, check out the Backprop Model Hub.
Q&A¶
Q&A answers a question based on a paragraph of text. It also supports previous questions and answers for a conversational setting.
Inference¶
from backprop import QA
qa = QA()
qa("Where does Sally live?", "Sally lives in London.")
"London"
Finetuning¶
The params dictionary for Q&A training consists of contexts, questions, answers, and optionally a list of previous Q/A pairs to be used in context.
The prev_qa parameter is used to make a Q&A system conversational – that is to say, it can infer the meaning of words in a new question, based on previous questions.
Finetuning Q&A also accepts keyword arguments max_input_length and max_output_length, which specify the maximum token length for inputs and outputs.
import backprop
# Initialise task
qa = backprop.QA()
"""
Set up training data for QA. Note that repeated contexts are needed,
along with empty prev_qas to match.
Input must be completely 1:1 each question has an associated
answer, context, and prev_qa (if prev_qa is to be used).
"""
questions = ["What's Backprop?",
"What language is it in?",
"When was the Moog synthesizer invented?"]
answers = ["A library that trains models", "Python", "1964"]
contexts = ["Backprop is a Python library that makes training and using models easier.",
"Backprop is a Python library that makes training and using models easier.",
"Bob Moog was a physicist. He invented the Moog synthesizer in 1964."]
prev_qas = [[],
[("What's Backprop?", "A library that trains models")],
[]]
params = {"questions": questions,
"answers": answers,
"contexts": contexts,
"prev_qas": prev_qas}
# Finetune
qa.finetune(params=params)
Consider the second example in the training data above. The question “What language is it in?” does not mean much on its own. But because the model can access the
matching prev_qa example, it gets additional information: as the first question reads, “What’s Backprop?”, it can infer that the “it” in the next question refers to Backprop as well.
If you use prev_qa, you must ensure it is 1:1 with the rest of your input data: even if a row does not have any previous question/answer pairs, an empty list is still required at that
index in prev_qa.
See the Q&A Notebook examples with code.
See the Q&A Task Reference.
Text Classification¶
Text Classification looks at input and assigns probabilities to a set of labels.
It is supported in 100+ languages: Afrikaans, Albanian, Amharic, Arabic, Armenian, Assamese, Azerbaijani, Basque, Belarusian, Bengali, Bengali Romanized, Bosnian, Breton, Bulgarian, Burmese, Burmese, Catalan, Chinese (Simplified), Chinese (Traditional), Croatian, Czech, Danish, Dutch, English, Esperanto, Estonian, Filipino, Finnish, French, Galician, Georgian, German, Greek, Gujarati, Hausa, Hebrew, Hindi, Hindi Romanized, Hungarian, Icelandic, Indonesian, Irish, Italian, Japanese, Javanese, Kannada, Kazakh, Khmer, Korean, Kurdish (Kurmanji), Kyrgyz, Lao, Latin, Latvian, Lithuanian, Macedonian, Malagasy, Malay, Malayalam, Marathi, Mongolian, Nepali, Norwegian, Oriya, Oromo, Pashto, Persian, Polish, Portuguese, Punjabi, Romanian, Russian, Sanskri, Scottish, Gaelic, Serbian, Sindhi, Sinhala, Slovak, Slovenian, Somali, Spanish, Sundanese, Swahili, Swedish, Tamil, Tamil Romanized, Telugu, Telugu Romanized, Thai, Turkish, Ukrainian, Urdu, Urdu Romanized, Uyghur, Uzbek, Vietnamese, Welsh, Western, Frisian, Xhosa, Yiddish.
Inference¶
import backprop
tc = backprop.TextClassification()
tc("I am mad because my product broke.", ["product issue", "nature"])
{"product issue": 0.98, "nature": 0.05}
Finetuning¶
Supplying parameters for text classification is straightforward: the params dict contains the keys “texts” and “labels”. The values of these keys are lists of input texts and the labels to which they are assigned. When you finetune, Backprop will automatically set up a model with the correct number of outputs (based on the unique labels passed in).
Finetuning text classification also accepts the keyword argument max_length, which specifoes the maximum token length for inputs.
import backprop
tc = backprop.TextClassification()
"""
Set up input data. Labels will automatically be used to set up
model with number of classes for classification.
"""
inp = ["This is a political news article",
"This is a computer science research paper",
"This is a movie review"]
out = ["Politics", "Science", "Entertainment"]
params = {"texts": inp, "labels": out}
# Finetune
tc.finetune(params)
Check the example Text Classification Notebook with code.
See the Text Classification Task Reference.
Sentiment/Emotion Detection¶
This is exactly what it says on the tin: analyzes emotional sentiment of some provided text input.
Inference¶
Use is simple: just pass in a string of text, and get back an emotion or list of emotions.
import backprop
emotion = backprop.Emotion()
emotion("I really like what you did there")
"approval"
Finetuning¶
Sentiment detection finetuning is currently a generative task. This will likely be converted to a wrapper around Text Classification in the future.
The schema will remain the same, however: the emotion task params dict contains the keys “input_text” and “output_text”. The inputs are the strings to be analysed, and the outputs are the emotions corresponding to those inputs.
Finetuning this task also accepts keyword arguments max_input_length and max_output_length, which specify the maximum token length for inputs and outputs.
import backprop
emote = backprop.Emotion()
# Provide sentiment data for training
inp = ["I really liked the service I received!",
"Meh, it was not impressive."]
out = ["positive", "negative"]
params = {"input_text": inp, "output_text": out}
# Finetune
emote.finetune(params)
See Sentiment Detection Notebook with code.
See the Emotion Task Reference.
Text Summarisation¶
Also self-explanatory: takes a chunk of input text, and gives a summary of key information.
Inference¶
import backprop
summarisation = backprop.Summarisation()
summarisation("This is a long document that contains plenty of words")
"short summary of document"
Finetuning¶
The summarisation input schema is a params dict with “input_text” and “output_text” keys. Inputs would be longer pieces of text, and the corresponding outputs are summarised versions of the same text.
Finetuning sumamrisation also accepts keyword arguments max_input_length and max_output_length, which specify the maximum token length for inputs and outputs.
import backprop
summary = backprop.Summarisation()
# Provide training data for task
inp = ["This is a long news article about recent political happenings.",
"This is an article about some recent scientific research."]
out = ["Short political summary.", "Short scientific summary."]
params = {"input_text": inp, "output_text": out}
# Finetune
summary.finetune(params)
See the example for Text Summarisation Notebook with code.
See the Text Summarisation Task Reference.
Image Classification¶
Image classification functions exactly like text classification but for images. It takes an image and a set of labels to calculate the probabilities for each label.
Inference¶
import backprop
ic = backprop.ImageClassification()
ic("/home/Documents/dog.png", ["cat", "dog"])
{"cat": 0.01, "dog": 0.99}
Finetuning¶
The params dict for image classification consists of “images” (input images) and “labels” (image labels). This task also includes variants for single-label and multi-label classification.
import backprop
ic = backprop.ImageClassification()
"""
Prep training images/labels. Labels are automatically used to set up
model with number of classes for classification.
"""
images = ["images/beagle/photo.jpg", "images/dachsund/photo.jpg", "images/malamute/photo.jpg"]
labels = ["beagle", "dachsund", "malamute"]
params = {"images": images, "labels": labels}
# Finetune
ic.finetune(params, variant="single_label")
Check the example Image Classification Notebook with code.
See the Image Classification Task Reference.
Image Vectorisation¶
Image Vectorisation takes an image and turns it into a vector.
This makes it possible to compare different images numerically.
Inference¶
import backprop
iv = backprop.ImageVectorisation()
iv("/home/Documents/dog.png")
[0.92949192, 0.23123010, ...]
Finetuning¶
When finetuning image vectorisation, the task input determines on the loss variant you plan to use. This comes in two flavors: triplet, or cosine similarity.
The default is triplet. This schema requires keys “images” (input images), and “groups” (group in which each image falls). This variant uses a distinct sampling strategy, based on group numbers. A given “anchor” image is compared to a positive match (same group number) and a negative match (different group number). The goal is to minimise the distance between the anchor vector and the positive match vector, while also maximising the distance between the anchor vector and negative match vector.
For cosine similarity, the schema is different. It requires keys “imgs1”, “imgs2”, and “similarity_scores”. When training on row x, this variant
vectorises imgs1[x] and imgs2[x], with the target cosine similarity being the value at similarity_scores[x].
import backprop
iv = backprop.ImageVectorisation()
# Set up training data & finetune (triplet variant)
images = ["images/beagle/photo.jpg", "images/shiba_inu/photo.jpg",
"images/beagle/photo1.jpg", "images/malamute/photo.jpg"]
groups = [0, 1, 0, 2]
params = {"images": images, "groups": groups}
iv.finetune(params, variant="triplet")
# Set up training data & finetune (cosine_similarity variant)
imgs1 = ["images/beagle/photo.jpg", "images/shiba_inu/photo.jpg"]
imgs2 = ["images/beagle/photo1.jpg", "images/malamute/photo.jpg"]
similarity_scores = [1.0, 0.0]
params = {"imgs1": imgs1, "imgs2": imgs2, "similarity_scores": similarity_scores}
iv.finetune(params, variant="cosine_similarity")
Check the example Image Vectorisation Notebook with code.
See the Image Vectorisation Task Reference.
Text Generation¶
Text Generation takes some text as input and generates more text based on it.
This is useful for story/idea generation or solving a broad range of tasks.
Inference¶
import backprop
tg = backprop.TextGeneration()
tg("I like to go to")
" the beach because I love the sun."
Finetuning¶
Text generation requires a params dict with keys “input_text” and “output_text”. The values here are simply lists of strings.
When trained, the model will learn expected outputs for a given context – this is how tasks such as generative sentiment detection or text summary can be trained.
Finetuning text generation also accepts keyword arguments max_input_length and max_output_length, which specify the maximum token length for inputs and outputs.
import backprop
tg = backprop.TextGeneration()
# Any text works as training data
inp = ["I really liked the service I received!",
"Meh, it was not impressive."]
out = ["positive", "negative"]
params = {"input_text": inp, "output_text": out}
# Finetune
tg.finetune(params)
Check the example Text Generation Notebook with code.
See the Text Generation Task Reference.
Text Vectorisation¶
Text Vectorisation takes some text and turns it into a vector.
This makes it possible to compare different texts numerically. You could see how similar the vectors of two different paragraphs are, to group text automatically or build a semantic search engine.
Inference¶
import backprop
tv = backprop.TextVectorisation()
tv("iPhone 12 128GB")
[0.92949192, 0.23123010, ...]
Finetuning¶
When finetuning text vectorisation, the task input determines on the loss variant you plan to use. Like with image vectorisation, this can be either “triplet” or “cosine_similarity”.
The default is cosine_similarity. It requires keys “texts1”, “texts2”, and “similarity_scores”. When training on row x, this variant
vectorises texts1[x] and texts2[x], with the target cosine similarity being the value at similarity_scores[x].
Triplet is different. This schema requires keys “texts” (input texts), and “groups” (group in which each piece of text falls). This variant uses a distinct sampling strategy, based on group numbers. A given “anchor” text is compared to a positive match (same group number) and a negative match (different group number). The goal is to minimise the distance between the anchor vector and the positive match vector, while also maximising the distance between the anchor vector and negative match vector.
Finetuning text vectorisation also accepts the keyword argument max_length which specifies the maximum token length for encoded text.
import backprop
tv = backprop.TextVectorisation()
# Set up training data & finetune (cosine_similarity variant)
texts1 = ["I went to the store and bought some bread",
"I am getting a cat soon"]
texts2 = ["I bought bread from the store",
"I took my dog for a walk"]
similarity_scores = [1.0, 0.0]
params = {"texts1": texts1, "texts2": texts2, "similarity_scores": similarity_scores}
tv.finetune(params, variant="cosine_similarity")
# Set up training data & finetune (triplet variant)
texts = ["I went to the store and bought some bread",
"I bought bread from the store",
"I'm going to go walk my dog"]
groups = [0, 0, 1]
params = {"texts": texts, "groups": groups}
tv.finetune(params, variant="triplet")
Check the example Text Vectorisation Notebook with code.
See the Text Vectorisation Task Reference.
Image-Text Vectorisation¶
Image-Text Vectorisation takes an associated text/image pair, and returns a normalized vector output.
This task could be used for making a robust image search system, that takes into account both input text and similar images.
Inference¶
import backprop
itv = backprop.ImageTextVectorisation()
image = "images/iphone/iphone-12-128GB.jpg"
text = "iPhone 12 128GB"
tv(image=image, text=text)
[0.82514237, 0.35281924, ...]
Finetuning¶
Similar to the other vectorisation tasks (text & image separately), this task has both triplet and cosine similarity loss variants. The variant determines the input data schema.
The default is triplet. This params dict requires keys “images” (input images), “texts” (input texts) and “groups” (group in which each image/text pair falls). This variant uses a distinct sampling strategy, based on group numbers. A given “anchor” image/text pair is compared to a positive match (same group number) and a negative match (different group number). The goal is to minimise the distance between the anchor vector and the positive match vector, while also maximising the distance between the anchor vector and negative match vector.
For cosine similarity, a few things are needed. It requires keys “imgs1”, “imgs2”, “texts1”, “texts2”, and “similarity_scores”. When training on row x, this variant
gets a normalized vector for imgs1[x] and texts[x], as well as one for and imgs2[x] and texts2[x].
The target cosine similarity between both normalized vectors is the value at similarity_scores[x].
import backprop
itv = backprop.ImageTextVectorisation()
# Prep training data & finetune (triplet variant)
images = ["product_images/crowbars/photo.jpg",
"product_images/crowbars/photo1.jpg",
"product_images/mugs/photo.jpg"]
texts = ["Steel crowbar with angled beak, 300mm",
"Crowbar tempered steel 300m angled",
"Sturdy ceramic mug, microwave-safe"]
groups = [0, 0, 1]
params = {"images": images, "texts": texts, "groups": groups}
itv.finetune(params, variant="triplet")
# Prep training data & finetune (cosine_similarity variant)
imgs1 = ["product_images/crowbars/photo.jpg", "product_images/mugs/photo.jpg"]
texts1 = ["Steel crowbar with angled beak, 300mm", "Sturdy ceramic mug, microwave-safe"]
imgs2 = ["product_images/crowbars/photo1.jpg", "product_images/hats/photo.jpg]
texts2 = ["Crowbar tempered steel 300m angled", "Dad hat with funny ghost picture on the front"]
similarity_scores = [1.0, 0.0]
params = {"imgs1": imgs1,
"imgs2": imgs2,
"texts1": texts1,
"texts2": texts2,
"similarity_scores": similarity_scores}
itv.finetune(params, variant="cosine_similarity")