Medical SAM Adapter

The Medical SAM Adapter, or MSA, is a groundbreaking project designed to fine-tune the Segment Anything Model (SAM) through adaptation specifically for Medical Imaging. For a deeper understanding of this innovative approach, check the paper titled Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation.

A Quick Overview

News

• Join our Discord to ask questions and discuss with others.
• 24-03-02: We have released our pre-trained Adapters in Medical-Adapter-Zoo. Try it without painful training. Credit: @shinning0821
• 23-05-10: This project is constantly updated; check the TODO list to see what’s next.
• 23-05-11: GitHub Discussion opened for more interactive engagement.
• 24-01-14: Launched v0.1.0-alpha, including support for MobileSAM.

Medical Adapter Zoo

We’ve released many pre-trained Adapters for various organs and lesions in the Medical-Adapter-Zoo. Select an adapter tailored to your needs. If you don’t find what you need, please suggest it through any contact method available to us (GitHub issue, HuggingFace community, or Discord). We’ll do our best to include it.

Requirement

To install the environment for Medical SAM Adapter, use the following command:

conda env create -f environment.yml
conda activate sam_adapt

Next, download the SAM checkpoint and put it in .checkpointsam. You can run:

wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth
mv sam_vit_b_01ec64.pth .checkpointsam

Make sure to create the folder if it does not exist.

Example Cases

Melanoma Segmentation from Skin Images (2D)

1. Download ISIC dataset part 1 from here and place the CSV files in .dataisic under your data path.
2. Begin adapting! Run the command:

python train.py -net sam -mod sam_adpt -exp_name msa_test_isic -sam_ckpt .checkpointsam/sam_vit_b_01ec64.pth -image_size 1024 -b 32 -dataset isic -data_path ..data

3. Evaluation: The code can automatically evaluate the model on the test set during training.
4. Result Visualization: Use the --vis parameter to control how many epochs you want to visualize.

REFUGE: Optic-disc Segmentation from Fundus Images (2D)

The REFUGE dataset contains 1200 fundus images with optic disc/cup segmentations. Follow similar steps as above for training the adapter.

Abdominal Multiple Organs Segmentation (3D)

This segment showcases how MSA can adapt SAM for 3D multi-organ segmentation using the BTCV challenge dataset:

1. Prepare the BTCV dataset according to MONAI instructions.
2. For adaptation, run:

python train.py -net sam -mod sam_adpt -exp_name msa-3d-sam-btcv -sam_ckpt .checkpointsam/sam_vit_b_01ec64.pth -image_size 1024 -b 8 -dataset decathlon -thd True -chunk 96 -data_path ..data -num_sample 4

Running on Your Own Dataset

It’s straightforward to run MSA on other datasets. Just create a new dataset class that returns:

• image: A tensor saving images (2D or 3D).
• label: The target masks.
• p_label: The prompt label.
• pt: The prompt (similar to SAM).

Troubleshooting

If you encounter any hurdles while setting up or running Medical SAM Adapter, here are some tips:

• Double-check that all directories and paths are correctly set up; missing directories can lead to errors.
• If you run out of memory, try reducing the batch size (-b) or the image size (-image_size).
• Consult the community on Discord or view issues on GitHub for similar problems encountered by others.
• Refer to the documentation in the project’s repository for guidance on additional parameters and their impacts.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.