Search-TTA-RL-VLN
Collection
Test-Time Adaptation Framework for Multimodal Visual Navigation and Search (https://search-tta.github.io/) • 7 items • Updated
How to use derektan95/LISA-AVS with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("text-generation", model="derektan95/LISA-AVS") # Load model directly
from transformers import AutoProcessor, AutoModelForCausalLM
processor = AutoProcessor.from_pretrained("derektan95/LISA-AVS")
model = AutoModelForCausalLM.from_pretrained("derektan95/LISA-AVS")How to use derektan95/LISA-AVS with vLLM:
# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "derektan95/LISA-AVS"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "derektan95/LISA-AVS",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'docker model run hf.co/derektan95/LISA-AVS
How to use derektan95/LISA-AVS with SGLang:
# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
--model-path "derektan95/LISA-AVS" \
--host 0.0.0.0 \
--port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "derektan95/LISA-AVS",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "derektan95/LISA-AVS" \
--host 0.0.0.0 \
--port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "derektan95/LISA-AVS",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'How to use derektan95/LISA-AVS with Docker Model Runner:
docker model run hf.co/derektan95/LISA-AVS
We fine-tune LISA reasoning segmentation model with dataset from AVS training dataset from AVS-Bench.
For more information on usage, please refer to the LISA-AVS Github repository here.
@inproceedings{tan2025searchtta,
title = {Search-TTA: A Multi-Modal Test-Time Adaptation Framework for Visual Search in the Wild},
author = {Tan, Derek Ming Siang and Shailesh, Shailesh and Liu, Boyang and Raj, Alok and Ang, Qi Xuan and Dai, Weiheng and Duhan, Tanishq and Chiun, Jimmy and Cao, Yuhong and Shkurti, Florian and Sartoretti, Guillaume Adrien},
booktitle = {Proceedings of The 9th Conference on Robot Learning},
pages = {2093--2120},
year = {2025},
volume = {305},
publisher = {PMLR}
}
@article{lai2023lisa,
title={LISA: Reasoning Segmentation via Large Language Model},
author={Lai, Xin and Tian, Zhuotao and Chen, Yukang and Li, Yanwei and Yuan, Yuhui and Liu, Shu and Jia, Jiaya},
journal={arXiv preprint arXiv:2308.00692},
year={2023}
}