SGLang Docker Image
Pre-built Docker image that runs TheBloke/Mistral-7B-Instruct-v0.2-AWQ on SGLang
Installation
The image is published to https://hub.docker.com/r/degroote22/lmscript-sglang
Docker-Compose
There is an example of a Docker-compose file in the repository.
Clone the LmScript repository and:
cd docker/sglangdocker-compose up