LLM Chat Interface

LLM Chat Interface#

The LLM Chat Interface is an interactive Open OnDemand application that provides a browser-based chat experience to support a variety of use cases. It is powered by Chainlit and Ollama. When you launch a session, the application starts an Ollama server on a GPU-equipped compute node and connects it to CURC-hosted large language models (LLMs). You can ask questions, draft and debug code, summarize documents, and analyze images (with vision-capable models), all from your web browser.

Important

The LLM Chat Interface is currently offered as a beta service. Functionality, available models, and resource allocations may change as we gather feedback and refine the service. Please report issues or make suggestions through the CURC support form.

Launching the LLM Chat Interface#

  1. Log in to Open OnDemand using your CURC credentials.

  2. Navigate to either the Interactive Apps drop-down menu or the My Interactive Sessions tab and select LLM Chat Interface.

  3. Review the launch form fields:

    • Ollama model path: Select which model library Ollama should load. The default CURC LLM Models uses CURC-hosted Ollama models. You may also provide the absolute path to your own Ollama model directory, if you have downloaded or fine-tuned models there. See Ollama documentation for more details.

    • Configuration type:

      • If you selected Preset configuration (recommended for most users), choose 10 cores, 1 GPU, 1 hour. This submits your job to the a100 testing partition with one GPU, which is required to run the LLM backend

      • If you selected Custom configuration, you must request at least one GPU in the gres field (for example, gpu:1:a100_3g.20gb). See the Limitations section for guidance on GPU memory (VRAM) and model size. For details on custom configuration options, see Configuring Open OnDemand interactive applications.

  4. Click Launch and wait for your session to start. When the job is ready, click Connect to LLM Chat Interface to open the chat in a new browser tab.

Important

When a session is launched with Preset configuration, the job is submitted to Alpine testing hardware (aa100 partition and gpu-testing QoS). Testing resources are shared and limited in capacity, so sessions may queue during high demand. For longer runtimes or heavier workloads, use Custom configuration and request appropriate resources (jobs will be subject to queue waits and may not start immediately).

Getting started#

Once you click Connect to LLM Chat Interface, you land on the welcome screen.

Chat history#

Chat history is saved per user under /projects directory, so you can resume past conversations from the sidebar. Conversation data is stored at:

/projects/<your_username>/.chainlit_data

Model selection#

Available models are loaded dynamically from the Ollama server when your session starts. To switch models:

  1. Open the model selector in the application header (labeled with the current model name).

  2. Browse the list of Chat Profiles. Each entry shows:

    • The model name

    • Capability tags such as Chat or Vision

    • Parameter size and quantization level

  3. Select a model. New messages in the current thread use the newly selected model. When you resume an older thread from the sidebar, the model saved with that thread is used.

Tip

Choose a smaller, chat-focused model for quick questions and code snippets. Switch to a vision model only when you need to analyze images, since vision models typically require more GPU memory and may respond more slowly.

Message actions#

After each assistant reply, action buttons may appear:

Action

Description

Regenerate

Ask the model to produce a new reply to your last message.

Copy code block

Copy the first fenced code block from the reply to your clipboard.

New chat

Clear the current conversation and return to the welcome screen.

Switch to vision model

Switch to a vision-capable model (shown when the current model does not support images).

Attaching files, text, and images#

Browser-based file upload is disabled in this application. Files must already exist on CURC filesystems. To attach a file, you must use the /file command followed by the file’s absolute path. This keeps data on cluster storage and avoids uploading large files through the web interface.

How to attach files#

Every attachment must start with /file. When your prompt includes a file, structure your message as follows:

  1. First line: /file followed by one or more absolute file paths (space-separated for multiple files).

  2. Following lines: Your question or instruction to the model.

Example 1#
/file /projects/$USER/data/results.csv
What trends do you see in this file?
Example 2: Multiple files upload#
/file /projects/$USER/my_report.pdf /projects/$USER/script.py
Summarize the report and check whether the script implements the methods described.

If you send /file with paths but no follow-up question, the assistant is asked to analyze the attached file(s) by default.

Important

Do not paste a bare filesystem path without the /file prefix, as the assistant will not attach the file. When files are attached successfully, the assistant displays a confirmation as such File attached from Alpine filesystem.

Allowed locations#

Attachments must be absolute paths under one of these:

  • /home/$USER/document.txt

  • /projects/$USER/myfile.pdf

  • /scratch/alpine/$USER/output.log

  • /pl/active/<allocation_name>/data.csv

Supported file types#

Type

Extensions / formats

Notes

Text and code

.py, .js, .ts, .html, .css, .json, .yaml, .md, .sh, .r, .sql, .csv, and many others

Full file contents are injected into the prompt.

Plain text

.txt, .md, .rst

Same as above.

PDF

.pdf

Text is extracted automatically. Scanned or image-only PDFs may not yield text (see Limitations).

Images

.png, .jpg, .jpeg, .gif, .webp, .bmp, .tif, .tiff

Requires a Vision model.

Attachment limits#

Limit

Value

Maximum files per message

20

Maximum file size

500 MB per file

Maximum PDF text extracted

120,000 characters per PDF (longer documents are truncated)

Limitations#

GPU memory (VRAM) and model size#

Every LLM Chat Interface session runs on GPU hardware. The preset configuration requests one GPU. Custom configurations also require at least one GPU in gres.

Large language models load into GPU video memory (VRAM). Important constraints:

  • Larger models use more VRAM. A model’s parameter size (shown in the model selector) is a rough guide: multi-billion-parameter models need substantially more memory than smaller ones.

  • Only one GPU is allocated by default in custom configuration. Very large models may fail to load, run slowly, or return errors if they exceed available VRAM on the assigned node.

  • Vision models and long contexts increase memory pressure. Attaching images, long PDFs, or maintaining a long conversation history all consume context window space and can contribute to out-of-memory failures or empty responses.

Important

This assistant was not trained on CU Research Computing (CURC) documentation. It does not have reliable, up-to-date knowledge of CURC-specific systems, policies, or procedures. The model may produce plausible-sounding but incorrect answers for system specific topics. Always verify CURC-specific information against official documentation.

Other limitations#

  • Context window. The backend uses a large but finite context window (32,768 tokens). Extremely long files, many attachments, or very long threads may be truncated or cause degraded responses.

  • Not for sensitive or regulated data. Do not paste export-controlled, HIPAA, or other restricted data into the chat. Treat prompts and attachments as you would any shared compute resource.

  • Resuming old threads reloads conversation text but does not re-inject large files from previous conversations; re-attach the files if you need the model to see them again.

Use cases#

  • Ask general coding questions

Write a short Python function that reads a CSV and computes column means.

  • Summarizing and questioning documents

/file /projects/$USER/papers/methods_supplement.pdf
List the experimental parameters in a table and note anything ambiguous.

  • Image and plot reviews

/file /projects/$USER/figures/confusion_matrix.png
Is this figure publication-ready? Suggest axis labels and caption text.

  • Deploying custom-trained or fine-tuned models that can answer questions about your lab’s protocols, instrumentation, research methods, and internal documentation.

Contents