Since we are using Docker to deploy vLLM, the host machine
*does not need to install CUDA Toolkit*. It
only requires the NVIDIA Driver and NVIDIA Container Toolkit.
Ensure your host machine has the NVIDIA driver installed. You can
verify it by running:
1
nvidia-smi
(If it fails, please install the driver via
sudo apt install nvidia-driver-550 or the official
.run file, then reboot.)
# Add current user to docker group (to run docker without sudo) sudo usermod -aG docker $USER # Note: You need to log out and log back in, or run `newgrp docker` to apply this change.
docker run --rm --gpus all nvidia/cuda:12.4.0-base-ubuntu22.04 nvidia-smi
If this command successfully prints your GPU information, your
host environment is fully ready!
2. Create a Virtual
Environment for SWE
* Note on CUDA Version:* The system’s
default CUDA version (e.g., CUDA 13.0) is often too new and causes
flashinfer JIT compilation errors (like
has no member "FlagHeads"). It is highly recommend to use
Docker for vLLM deployment later, but if you prefer Conda, ensure you
pin the CUDA version.
SWE-bench requires the model to output patches in a specific JSONL
format. Create a file named run_inference.py, the path is
/data/disk2/wye/run_inference.py :
import json from datasets import load_dataset from openai import OpenAI
# Connect to the vLLM server running in Docker client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
# Load the dataset dataset = load_dataset("princeton-nlp/SWE-bench_Lite", split="test")
predictions = [] # For testing purposes, we only run the first 3 instances. # Change to len(dataset) for a full run. for i, item inenumerate(dataset): if i >= 3: break instance_id = item["instance_id"] problem_statement = item["problem_statement"] prompt = f"""You are an expert software engineer. Please fix the following GitHub issue. Output ONLY the unified diff patch. Do not include any explanations or markdown formatting. Issue: {problem_statement} """ response = client.chat.completions.create( model="/models/Qwen2.5-Coder-7B-Instruct", messages=[{"role": "user", "content": prompt}], temperature=0.0, max_tokens=4096 ) patch = response.choices[0].message.content # ⚠️ CRITICAL: SWE-bench requires 'model_name_or_path' field in the JSONL predictions.append({ "instance_id": instance_id, "model_patch": patch + "\n", "model_name_or_path": "Qwen2.5-Coder-7B-Instruct" })
# Save as jsonl withopen("predictions.jsonl", "w", encoding="utf-8") as f: for pred in predictions: f.write(json.dumps(pred) + "\n") print("Inference completed. Saved to predictions.jsonl")
Run the script to get the path:
1
python run_inference.py
8. Run SWE-bench Evaluation
The evaluation phase strictly requires Docker to
restore the original repository environments and run unit tests. Ensure
your current user has Docker permissions
(sudo usermod -aG docker $USER).
After the evaluation finishes, you can check the detailed logs and
results in the ./eval_logs directory.
Use the SWE-agent to Test
Using LLM to solve practical problem is very difficult because of
lacking of context information and other problems.
Agent is good at solving problem which use LLM as “brain” and use
terminal/edit tools as “limbs”.It create a docker to read code and try
to solve problems.
1. Environment Preparation
First, we need the brain: SWE-agent
Then, we need a LLM to provide core think-function.
Third, we need a docker to test the real code.
End, we need a benchmark to judge the LLM’s performance.
1.1 Docker
Docker is essential for the agent to test the code.
1.2 vLLM
We can use vLLM to load a LLM directly. How ever it is probably to
have a lot of tools-conflicts problem. For example, I have a server with
CUDA version 13.0, while the vLLM only support version 12.4. It will
cause many compile problems.
So it is recommended to use Docker to deploy vLLM and run a LLM.
SWE-agent version 1.x uses a strict config validation.
parse_function expects to receive a
Dictionary or a specific object, rather than a plain
string. It requires a format similar to {“type”: “thought_action”}.
Due to the difficulty of the 7B model in consistently producing JSON
tool_call that meet OpenAI’s specifications, extracting Bash code blocks
through regular expressions can significantly enhance the compatibility
of smaller models.
So we need to amend the yaml file, change the parse_function from
function_calling into thought_action:
for item in dataset: if item["instance_id"] in target_ids: print(f"\n{'='*60}") print(f"📌 任务ID: {item['instance_id']}") print(f"📝 问题描述 (Issue 原文):\n{item['problem_statement']}") print(f"✅ 必须修复通过的测试 (FAIL_TO_PASS):\n{item['FAIL_TO_PASS']}") print(f"{'='*60}")
正在从本地缓存加载 SWE-bench Lite 数据集... Using the latest cached version of the dataset since princeton-nlp/SWE-bench_Lite couldn't be found on the Hugging Face Hub (offline mode is enabled). Found the latest cached dataset configuration 'default' at /home/ubuntu/.cache/huggingface/datasets/princeton-nlp___swe-bench_lite/default/0.0.0/6ec7bb89b9342f664a54a6e0a6ea6501d3437cc2 (last modified on Wed Jul 1 12:06:05 2026). 加载成功！共 300 个任务。
============================================================ 📌 任务ID: astropy__astropy-12907 📝 问题描述 (Issue 原文): Modeling's `separability_matrix` does not compute separability correctly for nested CompoundModels Consider the following model:
```python from astropy.modeling import models as m from astropy.modeling.separable import separability_matrix
cm = m.Linear1D(10) & m.Linear1D(5) ```
It's separability matrix as you might expect is a diagonal:
If I make the model more complex: ```python >>> separability_matrix(m.Pix2Sky_TAN() & m.Linear1D(10) & m.Linear1D(5)) array([[ True, True, False, False], [ True, True, False, False], [False, False, True, False], [False, False, False, True]]) ```
The output matrix is again, as expected, the outputs and inputs to the linear models are separable and independent of each other.
If however, I nest these compound models: ```python >>> separability_matrix(m.Pix2Sky_TAN() & cm) array([[ True, True, False, False], [ True, True, False, False], [False, False, True, True], [False, False, True, True]]) ``` Suddenly the inputs and outputs are no longer separable?
This feels like a bug to me, but I might be missing something?
============================================================ 📌 任务ID: astropy__astropy-14365 📝 问题描述 (Issue 原文): ascii.qdp Table format assumes QDP commands are upper case ### Description
ascii.qdp assumes that commands in a QDP file are upper case, for example, for errors they must be "READ SERR 1 2" whereas QDP itself is not case sensitive and case use "read serr 1 2".
As many QDP files are created by hand, the expectation that all commands be all-caps should be removed.
### Expected behavior
The following qdp file should read into a `Table` with errors, rather than crashing. ``` read serr 1 2 1 0.5 1 0.5 ```
> python Python 3.10.9 (main, Dec 7 2022, 02:03:23) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from astropy.table import Table >>> Table.read('test.qdp',format='ascii.qdp') WARNING: table_id not specified. Reading the first available table [astropy.io.ascii.qdp] Traceback (most recent call last): ... raise ValueError(f'Unrecognized QDP line: {line}') ValueError: Unrecognized QDP line: read serr 1 2 ```