1
Connect
Connect GitHub
Not connected
Connect Models
Optional · not configured
1
ConnectGitHub + model
2
SelectRepositories
3
ScreeningCriteria
4
WorkbenchRefine tasks
5
BenchmarkSave collection
2
Select Repositories
↗
No repositories loadedConnect GitHub or use the public demo.3
Screening Criteria
Starting scanPreparing repository request
0%
4
Task Workbench
Unsaved draft
Static screening complete.
0 tasks selectedSelected eligible candidates below will be packaged into this benchmark.
Package & validate on
Merged pull requestsOne entry per merged pull request.
0 merged PRs
PACKAGED OUTPUT
Task artifacts
DeepSWE-compatible candidate package, ready for execution calibration.
Calibration pending
Download task .tar.gz
Select a file to inspect it. TEST REMEDIATION
Remediation queue
Local test and harness changes awaiting calibration. Original merged pull requests remain unchanged.
1
ConnectModel access
2
SelectBenchmark
3
ConfigureModel × harness × effort
4
ExecuteLocal Docker
Model Setup
Run Configuration
Run on
Run targets
0 jobsChoose a benchmark to calculate the run.
Execution Logs
TASK INVENTORY
Tests
Choose a benchmark to inspect its packaged tasks.
| Task | Repository / PR | Category | Complexity | Tests | Environment | Date added | Last tested | Status |
|---|
PACKAGED TASK
Task
Task instruction
Select a task to inspect it.
Original merged change
Implementation and test changes packaged from the source pull request.
Select a task to inspect its original change.
EXECUTION DATA
Task results
Select a run log to inspect its results.
| Harness | Model | Effort | Cost per task | Steps / tokens | Did it pass? | Benchmark / task |
|---|
INFERENCE PROFILE
Quality vs. cost
Each point represents one routed model target on the selected benchmark.
Higher quality
Lower cost → Higher cost
No scored executionsQueue this benchmark in the Execution Suite to populate its inference profile.
BENCHMARK SNAPSHOT
Collection health
Immutable task composition at evaluation time.