In this article, we will guide you through the process of utilizing the BiomedTra-Small model for extractive Question Answering (QA) in the biomedical field, focusing on the Spanish language. This project emerged from the Extractive QA Biomedicine initiative during a hackathon organized by SOMOS NLP.

Motivation Behind the Model

Recent advancements have led to the development of Spanish Language Models trained on biomedical corpus, paving the way for this project to explore their utility in generating effective extractive QA models. The goal is to evaluate their performance against general masked language models.

Hackathon Models

The following models were trained during the hackathon:

• hackathon-pln-esroberta-base-bne-squad2-es
• hackathon-pln-esroberta-base-biomedical-clinical-es-squad2-es
• hackathon-pln-esroberta-base-biomedical-es-squad2-es
• hackathon-pln-esbiomedtra-small-es-squad2-es

Description of the Model

The BiomedTra-Small model is essentially a fine-tuned version of the mrm8488biomedtra-small-es model, specifically tailored to work with the squad_es (v2) training dataset.

Setting Hyperparameters

To ensure optimal performance, we adopted hyperparameters similar to those used in the deepset electra-base-squad2 model, which is designed for comparable tasks. Here are the key hyperparameters to set:

• Number of Training Epochs: 10
• Learning Rate: 1e-4
• Max Sequence Length: 384
• Document Stride: 128

Performance Evaluation

The models were evaluated on the hackathon-pln-esbiomed_squad_es_v2 development set. Here’s how they performed:

Model                                                              | Base Model       | Domain       | exact   | f1     | HasAns_exact | HasAns_f1 | NoAns_exact | NoAns_f1       
-------------------------------------------------------------------------------------------------------------------------   
hackathon-pln-esroberta-base-bne-squad2-es                       | General          |             | 67.63   | 75.69  | 53.73       | 70.05     | 81.21      | 81.21        
hackathon-pln-esroberta-base-biomedical-clinical-es-squad2-es    | Biomedical       |             | 66.84   | 75.23  | 53.02       | 70.00     | 80.34      | 80.34        
hackathon-pln-esroberta-base-biomedical-es-squad2-es             | Biomedical       |             | 67.63   | 74.56  | 47.68       | 61.70     | 87.13      | 87.13        
hackathon-pln-esbiomedtra-small-es-squad2-es                     | Biomedical       |             | 34.47   | 44.32  | 45.37       | 65.30     | 23.82      | 23.82        

Troubleshooting Tips

If you encounter any issues while working with the BiomedTra-Small model, here are a few troubleshooting strategies:

• Ensure that you have the correct versions of the libraries installed, specifically those required for Hugging Face models.
• Check your hyperparameter configurations to ensure that they align with the recommendations provided.
• Validate the format of your input data to make sure it meets the specifications of the model.
• If you’re still experiencing difficulties, consider reaching out to the community for support or refer to documentation resources.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.