Welcome to your guide on creating a multi-task learning model with dual prediction heads! In this tutorial, we will walk you through the setup and execution of a model designed to differentiate between keyword sentences versus statements/questions, as well as classify statements versus questions.

Concept Overview

Imagine teaching a talented student to tackle two subjects simultaneously—language and logic. This is what a multi-task learning model does by efficiently sharing knowledge across different tasks. In our case, we harness the power of transformer architecture to enable our model to learn how to predict keyword sentences and classify various statements at the same time!

Performance Metrics

Before diving into implementation, let’s consider the model’s effectiveness:

• Spaadia SQuaD Test accuracy: 0.9891
• Quora Keyword Pairs Test accuracy: 0.98048

Required Datasets

For our model, we will utilize the following datasets:

• Quora Keyword Pairs: Dataset Link
• Spaadia SQuaD pairs: Dataset Link

Implementation Steps

1. Clone the Model Repository

Start by cloning the model repository from Hugging Face using the following command:

bash git clone https://huggingface.co/shahrukhx01/bert-multitask-query-classifiers
cd bert-multitask-query-classifiers

2. Load the Model

To load the model, utilize the following Python code:

python
from multitask_model import BertForSequenceClassification
from transformers import AutoTokenizer
import torch

model = BertForSequenceClassification.from_pretrained(
    "shahrukhx01/bert-multitask-query-classifiers",
    task_labels_map={"quora_keyword_pairs": 2, "spaadia_squad_pairs": 2},
)

tokenizer = AutoTokenizer.from_pretrained("shahrukhx01/bert-multitask-query-classifiers")

3. Run Inference on Both Tasks

Now that the model is ready, let’s run inference for both the keyword versus statement/question classifier and the statement versus question classifier.

Keyword vs Statement/Question Classifier

Input your queries and classify:

python
input = ["keyword query", "is this a keyword query?"]
task_name = "quora_keyword_pairs"
sequence = tokenizer(input, padding=True, return_tensors="pt")["input_ids"]
logits = model(sequence, task_name=task_name)[0]
predictions = torch.argmax(torch.softmax(logits, dim=1).detach().cpu(), axis=1)

for input, prediction in zip(input, predictions):
    print(f"Task: {task_name}, Input: {input}, Prediction: {prediction.item()}")

Statement vs Question Classifier

Similarly, input your statements for classification:

python
input = ["where is berlin?", "is this a keyword query?", "Berlin is in Germany."]
task_name = "spaadia_squad_pairs"
sequence = tokenizer(input, padding=True, return_tensors="pt")["input_ids"]
logits = model(sequence, task_name=task_name)[0]
predictions = torch.argmax(torch.softmax(logits, dim=1).detach().cpu(), axis=1)

for input, prediction in zip(input, predictions):
    print(f"Task: {task_name}, Input: {input}, Prediction: {prediction.item()}")

Troubleshooting Tips

If you encounter any issues during implementation, consider the following troubleshooting ideas:

• Ensure you have all necessary dependencies installed, especially the transformers library.
• Double-check the dataset links to ensure you’ve downloaded the correct files.
• If the model fails to initialize, verify that you cloned the correct repository and maintained the structure.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

Now you have set up a powerful multi-task learning model capable of handling multiple predictions efficiently. Customize your model further as per your project needs to achieve exceptional results!

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.