72일차 정답 추가 (실습 정답본 · answers · VIBE-A · tbench)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
baf72561f7
commit
d88c5122d5
105 files changed
+24165
No files matched your search
@@ -0,0 +1,4 @@
|
||||
.venv/
|
||||
__pycache__/
|
||||
*.egg-info/
|
||||
.pytest_cache/
|
||||
@@ -0,0 +1,16 @@
|
||||
# tbench IMP-PLAN (강사가 직접 만든 기록)
|
||||
|
||||
## 엔진 1단계 · 부르고 재기 (R1 · R6 · R7 · R8)
|
||||
- [x] runner.py: call_once → {status, seconds, error_kind, timings}, run() 제너레이터
|
||||
- [x] stats.py: nearest-rank (1~10 → p50 5 · p95 10), 실패는 세고 시간 목록에서 뺌, p50 excl. first
|
||||
- [x] tests/conftest.py 가짜 서버, --plain · --warmup · --header (가리기)
|
||||
## TUI 1단계 (R2)
|
||||
- [x] app.py: Sparkline · 요약 · DataTable, worker(thread) + call_from_thread. 테스트: 지연 0.5초 × 6, 1.3초 뒤 1~5줄 → 끝에 6줄
|
||||
## 설치 (R14)
|
||||
- [x] [project.scripts] tbench = "tbench.cli:main", uv tool install --editable ., /tmp 에서 tbench --help
|
||||
## 엔진 2단계 (R3 · R13 · R11)
|
||||
- [x] --save (기본 ~/.tbench/results/), compare (url · method · body_sha256 · n · concurrency · warmup 다르면 WARNING), history (최신 위), --log
|
||||
## 엔진 3단계 (R4 · R10 · R9 · R12)
|
||||
- [x] --concurrency (0.3초 × 6: k=1 1.8초대, k=3 1초 안), --max-p95 · --max-failures 종료 코드 0 · 1, timings 평균, profile save/list · run
|
||||
## TUI 2단계 (R5)
|
||||
- [x] --pick DIR: OptionList, Enter 로 그 본문 실행
|
||||
@@ -0,0 +1,38 @@
|
||||
# tbench · 강사가 수업 준비 때 직접 만든 본 (2026-10-26)
|
||||
|
||||
> **정답 자료** · 학생 저장소에는 수업 뒤 [VIBE-A-TBENCH.md](../VIBE-A-TBENCH.md)와 함께 올립니다.
|
||||
> 이 본은 강사가 CLI 에이전트에 단계 요청문을 보내 만든 것이 아니라, Claude Code가 VIBE-A의 SPEC · IMP-PLAN을 보고 직접 썼습니다. 학생이 만든 것과 파일 구성이 달라도 됩니다. 요구 R1~R14의 **확인 방법**이 같으면 됩니다.
|
||||
|
||||
## 설치 · 실행
|
||||
|
||||
```bash
|
||||
cd ~/aio-01d4-stu/Last3Mon/72/tbench
|
||||
uv sync && uv run pytest -q # 가짜 서버 테스트 12개 (진짜 헬프데스크를 부르지 않음)
|
||||
uv tool install --editable . # 어느 폴더에서나 tbench (R14)
|
||||
cd /tmp && tbench --help
|
||||
```
|
||||
|
||||
| 하고 싶은 것 | 명령 |
|
||||
| --- | --- |
|
||||
| 화면(TUI)으로 재기 | `tbench http://localhost:5678/webhook/my-helpdesk --body-file requests/faq-vpn-nocache.json -n 20` |
|
||||
| 글자로 · 저장 · 기준 | `tbench <url> --body-file x.json -n 10 --plain --save --max-p95 2.5 --max-failures 0` |
|
||||
| 요청 고르기 | `tbench <url> --pick requests` |
|
||||
| 동시에 · 미리 부르기 | `--concurrency 3` · `--warmup 1` |
|
||||
| 키 머리글 | `--header 'Authorization: Bearer $MY_KEY'` (저장 · 기록에는 `***`) |
|
||||
| 비교 · 목록 | `tbench compare a.json b.json` · `tbench history` |
|
||||
| 설정 저장 · 부르기 | `tbench profile save vpn <url> --body-file requests/faq-vpn-nocache.json -n 10` · `tbench profile list` · `tbench run vpn --plain` |
|
||||
| 한 번마다 한 줄 | `--log runs.jsonl` |
|
||||
|
||||
저장 위치는 `~/.tbench/` (`results/`, `profiles.toml`). 테스트는 `TBENCH_HOME`을 임시 폴더로 바꿔 씁니다.
|
||||
|
||||
## 파일
|
||||
|
||||
| 파일 | 하는 일 |
|
||||
| --- | --- |
|
||||
| `src/tbench/runner.py` | 엔진: 한 번 부르기(시간 · 상태 · 실패 종류 · `timings`), `run()` 제너레이터 (warmup · 동시) |
|
||||
| `src/tbench/stats.py` | nearest-rank 백분위, 요약, 기준 검사(PASS · FAIL) |
|
||||
| `src/tbench/store.py` | 저장(조건 · 본문 해시 · 키 가리기), compare, history, 기록 줄 |
|
||||
| `src/tbench/profiles.py` | `~/.tbench/profiles.toml` |
|
||||
| `src/tbench/cli.py` | 명령줄, `--plain` 출력, 종료 코드 |
|
||||
| `src/tbench/app.py` | Textual 화면 (worker + `call_from_thread`), `--pick` 고르기 |
|
||||
| `tests/conftest.py` | pytest 안의 가짜 서버 (지연 · N번째마다 500 · 첫 응답만 느림 · 상태 코드 · `timings`) |
|
||||
@@ -0,0 +1,22 @@
|
||||
[project]
|
||||
name = "tbench"
|
||||
version = "0.1.0"
|
||||
description = "Call a webhook many times and show how long each call took"
|
||||
requires-python = ">=3.11"
|
||||
dependencies = ["httpx>=0.27", "textual>=0.80"]
|
||||
|
||||
[project.scripts]
|
||||
tbench = "tbench.cli:main"
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
packages = ["src/tbench"]
|
||||
|
||||
[dependency-groups]
|
||||
dev = ["pytest>=8", "pytest-asyncio>=0.23"]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
asyncio_mode = "auto"
|
||||
@@ -0,0 +1 @@
|
||||
"""tbench: call a webhook many times and show how long each call took."""
|
||||
@@ -0,0 +1,75 @@
|
||||
"""Textual screen: sparkline and summary on top, one table row per call as it arrives."""
|
||||
from pathlib import Path
|
||||
|
||||
from textual import work
|
||||
from textual.app import App, ComposeResult
|
||||
from textual.widgets import DataTable, Footer, Header, OptionList, Sparkline, Static
|
||||
from textual.widgets.option_list import Option
|
||||
|
||||
from .runner import read_body, run
|
||||
from .stats import fmt, summarize, summary_lines
|
||||
|
||||
|
||||
class BenchApp(App):
|
||||
TITLE = "tbench"
|
||||
BINDINGS = [("q", "quit", "Quit")]
|
||||
CSS = """
|
||||
Sparkline { height: 4; margin: 0 1; }
|
||||
#summary { height: auto; margin: 0 1; }
|
||||
OptionList { height: 1fr; }
|
||||
"""
|
||||
|
||||
def __init__(self, settings, headers, pick=None):
|
||||
super().__init__()
|
||||
self.settings, self.headers, self.pick = settings, headers, pick
|
||||
self.results, self.body = [], None
|
||||
|
||||
def compose(self) -> ComposeResult:
|
||||
yield Header()
|
||||
if self.pick:
|
||||
files = sorted(Path(self.pick).glob("*.json"))
|
||||
yield OptionList(*[Option(f.name, id=str(f)) for f in files], id="pick")
|
||||
yield Sparkline([], id="spark")
|
||||
yield Static("starting...", id="summary")
|
||||
yield DataTable(id="calls")
|
||||
yield Footer()
|
||||
|
||||
def on_mount(self):
|
||||
table = self.query_one("#calls", DataTable)
|
||||
table.add_columns("#", "status", "seconds", "note")
|
||||
if self.pick:
|
||||
self.sub_title = f"choose a body from {self.pick}"
|
||||
self.query_one("#summary", Static).update("Enter: run with this body")
|
||||
else:
|
||||
self.start(self.settings.get("body_file"))
|
||||
|
||||
def on_option_list_option_selected(self, event):
|
||||
event.option_list.remove()
|
||||
self.settings["body_file"] = event.option.id
|
||||
self.start(event.option.id)
|
||||
|
||||
def start(self, body_file):
|
||||
s = self.settings
|
||||
self.body = read_body(body_file)
|
||||
self.sub_title = f"{s['method']} {s['url']} body={Path(body_file).name if body_file else '-'}"
|
||||
self.measure()
|
||||
|
||||
@work(thread=True, exclusive=True)
|
||||
def measure(self):
|
||||
s = self.settings
|
||||
for r in run(s["url"], self.body, s["n"], s["method"], self.headers, s["timeout"], s["warmup"], s["concurrency"]):
|
||||
self.call_from_thread(self.add_result, r)
|
||||
self.call_from_thread(self.finish)
|
||||
|
||||
def add_result(self, r):
|
||||
self.results.append(r)
|
||||
label = f"w{r['i']}" if r["warmup"] else str(r["i"])
|
||||
status = str(r["status"]) if r["status"] is not None else r["error_kind"]
|
||||
note = " ".join(x for x in (r["note"], r["error_kind"] if r["status"] else "") if x)
|
||||
self.query_one("#calls", DataTable).add_row(label, status, fmt(r["seconds"]), note)
|
||||
measured = [x for x in self.results if not x["warmup"]]
|
||||
self.query_one("#spark", Sparkline).data = [x["seconds"] for x in measured if x["error_kind"] is None]
|
||||
self.query_one("#summary", Static).update("\n".join(summary_lines(summarize(measured, self.settings["n"]))))
|
||||
|
||||
def finish(self):
|
||||
self.sub_title += " (done, q to quit)"
|
||||
@@ -0,0 +1,166 @@
|
||||
"""Command line: `tbench <url> ...` (TUI, or --plain), compare, history, profile, run."""
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
from . import profiles, store
|
||||
from .runner import read_body, run
|
||||
from .stats import check, fmt, summarize, summary_lines
|
||||
|
||||
COMMANDS = ("compare", "history", "profile", "run")
|
||||
|
||||
|
||||
def parse_headers(items):
|
||||
headers = {}
|
||||
for item in items or []:
|
||||
if ":" not in item:
|
||||
raise SystemExit(f"bad --header {item!r}, use 'Name: value'")
|
||||
k, v = item.split(":", 1)
|
||||
headers[k.strip()] = os.path.expandvars(v.strip())
|
||||
return headers
|
||||
|
||||
|
||||
def add_run_options(p, url=True):
|
||||
if url:
|
||||
p.add_argument("url")
|
||||
p.add_argument("--body-file", help="JSON file to send as the body")
|
||||
p.add_argument("-n", type=int, default=None, help="measured calls (default 10)")
|
||||
p.add_argument("--method", default=None, help="HTTP method (default POST)")
|
||||
p.add_argument("--timeout", type=float, default=None, help="seconds before a call counts as timeout (default 30)")
|
||||
p.add_argument("--header", action="append", default=[], help="'Name: value', may repeat; $VARS are expanded")
|
||||
p.add_argument("--warmup", type=int, default=None, help="calls to make and throw away first")
|
||||
p.add_argument("--concurrency", type=int, default=None, help="calls in flight at once (default 1)")
|
||||
p.add_argument("--plain", action="store_true", help="print lines instead of the TUI")
|
||||
p.add_argument("--save", nargs="?", const="", default=None, help="save result JSON (default ~/.tbench/results/)")
|
||||
p.add_argument("--log", help="append one JSON line per call to this file")
|
||||
p.add_argument("--max-p95", type=float, default=None, help="FAIL (exit 1) if p95 seconds is above this")
|
||||
p.add_argument("--max-failures", type=int, default=None, help="FAIL (exit 1) if failures are above this")
|
||||
p.add_argument("--pick", metavar="DIR", help="choose the body from *.json in DIR (TUI)")
|
||||
|
||||
|
||||
def settings_from(args, base=None):
|
||||
s = dict(base or {})
|
||||
for key in ("url", "body_file", "n", "method", "timeout", "warmup", "concurrency", "max_p95", "max_failures"):
|
||||
value = getattr(args, key, None)
|
||||
if value is not None:
|
||||
s[key] = value
|
||||
if getattr(args, "header", None):
|
||||
s["headers"] = args.header
|
||||
s.setdefault("n", 10)
|
||||
s.setdefault("method", "POST")
|
||||
s.setdefault("timeout", 30.0)
|
||||
s.setdefault("warmup", 0)
|
||||
s.setdefault("concurrency", 1)
|
||||
return s
|
||||
|
||||
|
||||
def execute(s, plain, save_to, log_path, pick=None):
|
||||
headers = parse_headers(s.get("headers"))
|
||||
started = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
if not plain:
|
||||
from .app import BenchApp
|
||||
app = BenchApp(s, headers, pick=pick)
|
||||
app.run()
|
||||
results, body = app.results, app.body
|
||||
if not results:
|
||||
return 0
|
||||
else:
|
||||
body = read_body(s.get("body_file"))
|
||||
results = []
|
||||
log = open(log_path, "a", encoding="utf-8") if log_path else None
|
||||
print(f"{s['method']} {s['url']} n={s['n']} concurrency={s['concurrency']} warmup={s['warmup']}")
|
||||
try:
|
||||
for r in run(s["url"], body, s["n"], s["method"], headers, s["timeout"], s["warmup"], s["concurrency"]):
|
||||
results.append(r)
|
||||
if log:
|
||||
store.log_line(log, r)
|
||||
label = f"w{r['i']}" if r["warmup"] else f"#{r['i']}"
|
||||
status = r["status"] if r["status"] is not None else r["error_kind"]
|
||||
note = " ".join(x for x in (r["note"], r["error_kind"] if r["status"] else "") if x)
|
||||
print(f"{label:>4} {status!s:>8} {fmt(r['seconds']):>8}s {note}".rstrip())
|
||||
finally:
|
||||
if log:
|
||||
log.close()
|
||||
if log_path and not plain:
|
||||
with open(log_path, "a", encoding="utf-8") as log:
|
||||
for r in results:
|
||||
store.log_line(log, r)
|
||||
measured = [r for r in results if not r["warmup"]]
|
||||
summary = summarize(measured, s["n"])
|
||||
for line in summary_lines(summary):
|
||||
print(line)
|
||||
if save_to is not None:
|
||||
record = store.build_record(s["url"], s["method"], body, headers, s["n"], s["concurrency"],
|
||||
s["warmup"], started, results)
|
||||
print(f"saved {store.save(record, save_to or None)}")
|
||||
if s.get("max_p95") is not None or s.get("max_failures") is not None:
|
||||
passed, line = check(summary, s.get("max_p95"), s.get("max_failures"))
|
||||
print(line)
|
||||
return 0 if passed else 1
|
||||
return 0
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
argv = sys.argv[1:] if argv is None else argv
|
||||
if argv and argv[0] in COMMANDS:
|
||||
return sys.exit(sub_main(argv))
|
||||
p = argparse.ArgumentParser(prog="tbench", description="Call a URL many times and show how long each call took. "
|
||||
"Also: tbench compare A B | history | profile save/list | run NAME")
|
||||
add_run_options(p)
|
||||
args = p.parse_args(argv)
|
||||
sys.exit(execute(settings_from(args), args.plain, args.save, args.log, args.pick))
|
||||
|
||||
|
||||
def sub_main(argv):
|
||||
p = argparse.ArgumentParser(prog="tbench")
|
||||
sub = p.add_subparsers(dest="cmd", required=True)
|
||||
c = sub.add_parser("compare", help="compare two saved results")
|
||||
c.add_argument("a")
|
||||
c.add_argument("b")
|
||||
sub.add_parser("history", help="list saved results, newest first")
|
||||
prof = sub.add_parser("profile", help="save or list named settings")
|
||||
psub = prof.add_subparsers(dest="action", required=True)
|
||||
ps = psub.add_parser("save")
|
||||
ps.add_argument("name")
|
||||
add_run_options(ps)
|
||||
psub.add_parser("list")
|
||||
r = sub.add_parser("run", help="run a saved profile")
|
||||
r.add_argument("name")
|
||||
add_run_options(r, url=False)
|
||||
args = p.parse_args(argv)
|
||||
|
||||
if args.cmd == "compare":
|
||||
for line in store.compare(store.load(args.a), store.load(args.b)):
|
||||
print(line)
|
||||
return 0
|
||||
if args.cmd == "history":
|
||||
rows = store.history()
|
||||
print(f"{'started':19} {'n':>3} {'p50':>7} {'p95':>7} {'fail':>4} url")
|
||||
for started, url, n, p50, p95, failures, _ in rows:
|
||||
print(f"{started:19} {n:>3} {p50:>7} {p95:>7} {failures:>4} {url}")
|
||||
return 0
|
||||
if args.cmd == "profile" and args.action == "list":
|
||||
for name, values in profiles.load_all().items():
|
||||
print(f"{name:12} {values.get('url')} body={values.get('body_file', '-')} n={values.get('n', 10)}")
|
||||
return 0
|
||||
if args.cmd == "profile":
|
||||
s = {k: v for k, v in vars(args).items() if k in profiles.FIELDS and v is not None}
|
||||
if s.get("body_file"):
|
||||
s["body_file"] = str(Path(s["body_file"]).resolve())
|
||||
safe = []
|
||||
for h in args.header:
|
||||
name, _, value = h.partition(":")
|
||||
if store.mask_headers({name: 1})[name] == "***" and "$" not in value:
|
||||
print(f"note: {name.strip()} is kept as *** — pass it as an env var, e.g. 'Authorization: Bearer $MY_KEY'")
|
||||
value = " ***"
|
||||
safe.append(f"{name}:{value}")
|
||||
s["headers"] = safe
|
||||
print(f"saved {args.name} to {profiles.save(args.name, s)}")
|
||||
return 0
|
||||
base = profiles.load_all().get(args.name)
|
||||
if base is None:
|
||||
print(f"no profile {args.name!r} in {profiles.path()}")
|
||||
return 2
|
||||
return execute(settings_from(args, base), args.plain, args.save, args.log, args.pick)
|
||||
@@ -0,0 +1,37 @@
|
||||
"""Named settings in ~/.tbench/profiles.toml, so `tbench run <name>` works from any folder."""
|
||||
import tomllib
|
||||
|
||||
from .store import home
|
||||
|
||||
FIELDS = ("url", "body_file", "n", "method", "timeout", "warmup", "concurrency", "headers", "max_p95", "max_failures")
|
||||
|
||||
|
||||
def path():
|
||||
return home() / "profiles.toml"
|
||||
|
||||
|
||||
def load_all():
|
||||
p = path()
|
||||
return tomllib.loads(p.read_text(encoding="utf-8")) if p.exists() else {}
|
||||
|
||||
|
||||
def _value(v):
|
||||
if isinstance(v, str):
|
||||
return '"' + v.replace("\\", "\\\\").replace('"', '\\"') + '"'
|
||||
if isinstance(v, list):
|
||||
return "[" + ", ".join(_value(x) for x in v) + "]"
|
||||
return str(v).lower() if isinstance(v, bool) else str(v)
|
||||
|
||||
|
||||
def save(name, settings):
|
||||
profiles = load_all()
|
||||
profiles[name] = {k: v for k, v in settings.items() if k in FIELDS and v not in (None, [])}
|
||||
out = []
|
||||
for pname, values in profiles.items():
|
||||
out.append(f'["{pname}"]')
|
||||
out += [f"{k} = {_value(v)}" for k, v in values.items()]
|
||||
out.append("")
|
||||
p = path()
|
||||
p.parent.mkdir(parents=True, exist_ok=True)
|
||||
p.write_text("\n".join(out), encoding="utf-8")
|
||||
return p
|
||||
@@ -0,0 +1,67 @@
|
||||
"""Call a URL many times. run() yields one result dict per call, as it finishes."""
|
||||
import json
|
||||
import time
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
|
||||
import httpx
|
||||
|
||||
|
||||
def call_once(client, url, method, body, headers, timeout):
|
||||
"""One call, timed from just before sending until the whole body is read."""
|
||||
start = time.perf_counter()
|
||||
status, error_kind, timings = None, None, None
|
||||
try:
|
||||
response = client.request(method, url, content=body, headers=headers, timeout=timeout)
|
||||
response.read()
|
||||
status = response.status_code
|
||||
if status >= 500:
|
||||
error_kind = "http_5xx"
|
||||
elif status >= 400:
|
||||
error_kind = "http_4xx"
|
||||
else:
|
||||
try:
|
||||
data = response.json()
|
||||
if isinstance(data, dict) and isinstance(data.get("timings"), dict):
|
||||
timings = data["timings"]
|
||||
except ValueError:
|
||||
pass
|
||||
except httpx.TimeoutException:
|
||||
error_kind = "timeout"
|
||||
except httpx.TransportError:
|
||||
error_kind = "connect"
|
||||
return {"status": status, "seconds": round(time.perf_counter() - start, 4),
|
||||
"error_kind": error_kind, "timings": timings}
|
||||
|
||||
|
||||
def run(url, body=None, n=10, method="POST", headers=None, timeout=30.0, warmup=0, concurrency=1):
|
||||
"""Yield warmup calls (warmup=True) first, then n measured calls numbered 1..n."""
|
||||
headers = dict(headers or {})
|
||||
if body is not None and not any(k.lower() == "content-type" for k in headers):
|
||||
headers["Content-Type"] = "application/json"
|
||||
content = body.encode() if isinstance(body, str) else body
|
||||
with httpx.Client() as client:
|
||||
for w in range(warmup):
|
||||
r = call_once(client, url, method, content, headers, timeout)
|
||||
yield {"i": w + 1, "warmup": True, "note": "warmup", **r}
|
||||
first_note = "first call" if warmup == 0 else ""
|
||||
if concurrency <= 1:
|
||||
for i in range(1, n + 1):
|
||||
r = call_once(client, url, method, content, headers, timeout)
|
||||
yield {"i": i, "warmup": False, "note": first_note if i == 1 else "", **r}
|
||||
return
|
||||
with ThreadPoolExecutor(max_workers=concurrency) as pool:
|
||||
futures = {pool.submit(call_once, client, url, method, content, headers, timeout): i
|
||||
for i in range(1, n + 1)}
|
||||
for future in as_completed(futures):
|
||||
i = futures[future]
|
||||
yield {"i": i, "warmup": False, "note": first_note if i == 1 else "", **future.result()}
|
||||
|
||||
|
||||
def read_body(path):
|
||||
"""Read a JSON body file; keep the exact text so its hash matches what was sent."""
|
||||
if path is None:
|
||||
return None
|
||||
with open(path, encoding="utf-8") as f:
|
||||
text = f.read()
|
||||
json.loads(text)
|
||||
return text
|
||||
@@ -0,0 +1,69 @@
|
||||
"""Percentiles and summaries. Nearest-rank: sort, take the ceil(p * n)-th value."""
|
||||
import math
|
||||
|
||||
KINDS = ("connect", "timeout", "http_5xx", "http_4xx")
|
||||
|
||||
|
||||
def percentile(values, p):
|
||||
if not values:
|
||||
return None
|
||||
ordered = sorted(values)
|
||||
rank = max(1, math.ceil(p * len(ordered) - 1e-9))
|
||||
return ordered[rank - 1]
|
||||
|
||||
|
||||
def average_timings(results):
|
||||
"""Average every numeric field of response `timings` over successful calls."""
|
||||
sums, counts = {}, {}
|
||||
for r in results:
|
||||
for key, value in (r.get("timings") or {}).items():
|
||||
if isinstance(value, (int, float)) and not isinstance(value, bool):
|
||||
sums[key] = sums.get(key, 0) + value
|
||||
counts[key] = counts.get(key, 0) + 1
|
||||
return {k: round(sums[k] / counts[k], 1) for k in sums}
|
||||
|
||||
|
||||
def summarize(results, total=None):
|
||||
"""results: measured calls (no warmup). Failures are counted, not timed."""
|
||||
ok = [r for r in results if r["error_kind"] is None]
|
||||
times = [r["seconds"] for r in ok]
|
||||
rest = [r["seconds"] for r in ok if r.get("note") != "first call"]
|
||||
by_kind = {k: sum(1 for r in results if r["error_kind"] == k) for k in KINDS}
|
||||
return {
|
||||
"done": len(results),
|
||||
"total": total if total is not None else len(results),
|
||||
"ok": len(ok),
|
||||
"failures": len(results) - len(ok),
|
||||
"by_kind": by_kind,
|
||||
"p50": percentile(times, 0.5),
|
||||
"p95": percentile(times, 0.95),
|
||||
"max": max(times) if times else None,
|
||||
"p50_excl_first": percentile(rest, 0.5) if len(rest) < len(times) else None,
|
||||
"timings": average_timings(ok),
|
||||
}
|
||||
|
||||
|
||||
def fmt(seconds):
|
||||
return "-" if seconds is None else f"{seconds:.3f}"
|
||||
|
||||
|
||||
def summary_lines(s):
|
||||
kinds = " ".join(f"{k}={v}" for k, v in s["by_kind"].items() if v)
|
||||
lines = [
|
||||
f"done {s['done']}/{s['total']} failures {s['failures']}" + (f" ({kinds})" if kinds else ""),
|
||||
f"p50 {fmt(s['p50'])}s p95 {fmt(s['p95'])}s max {fmt(s['max'])}s"
|
||||
+ (f" p50 excl. first {fmt(s['p50_excl_first'])}s" if s["p50_excl_first"] is not None else ""),
|
||||
]
|
||||
if s["timings"]:
|
||||
lines.append("timings avg ms " + " ".join(f"{k} {v:g}" for k, v in s["timings"].items()))
|
||||
return lines
|
||||
|
||||
|
||||
def check(s, max_p95=None, max_failures=None):
|
||||
"""Return (passed, line) for the given limits."""
|
||||
problems = []
|
||||
if max_p95 is not None and (s["p95"] is None or s["p95"] > max_p95):
|
||||
problems.append(f"p95 {fmt(s['p95'])} > {max_p95}")
|
||||
if max_failures is not None and s["failures"] > max_failures:
|
||||
problems.append(f"failures {s['failures']} > {max_failures}")
|
||||
return (not problems, "FAIL " + ", ".join(problems) if problems else "PASS")
|
||||
@@ -0,0 +1,101 @@
|
||||
"""Save, compare and list results; per-call log lines; header masking. Everything lives under ~/.tbench."""
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
from .stats import fmt, summarize
|
||||
|
||||
SECRET_WORDS = ("authorization", "key", "token", "secret", "cookie", "password")
|
||||
|
||||
|
||||
def home():
|
||||
return Path(os.environ.get("TBENCH_HOME", Path.home() / ".tbench"))
|
||||
|
||||
|
||||
def mask_headers(headers):
|
||||
return {k: ("***" if any(w in k.lower() for w in SECRET_WORDS) else v) for k, v in (headers or {}).items()}
|
||||
|
||||
|
||||
def body_sha256(body):
|
||||
return hashlib.sha256((body or "").encode()).hexdigest()
|
||||
|
||||
|
||||
def build_record(url, method, body, headers, n, concurrency, warmup, started, results):
|
||||
measured = sorted((r for r in results if not r["warmup"]), key=lambda r: r["i"])
|
||||
return {
|
||||
"url": url, "method": method, "headers": mask_headers(headers),
|
||||
"body_sha256": body_sha256(body), "n": n, "concurrency": concurrency, "warmup": warmup,
|
||||
"started": started,
|
||||
"seconds": [r["seconds"] for r in measured if r["error_kind"] is None],
|
||||
"statuses": [r["status"] for r in measured],
|
||||
"errors": [r["error_kind"] for r in measured],
|
||||
"calls": [{k: r[k] for k in ("i", "status", "seconds", "error_kind", "note")} for r in measured],
|
||||
"summary": summarize(measured, n),
|
||||
}
|
||||
|
||||
|
||||
def save(record, path=None):
|
||||
if path is None:
|
||||
folder = home() / "results"
|
||||
folder.mkdir(parents=True, exist_ok=True)
|
||||
stamp = datetime.fromisoformat(record["started"]).strftime("%Y%m%d-%H%M%S")
|
||||
path = folder / f"{stamp}.json"
|
||||
k = 1
|
||||
while path.exists():
|
||||
k += 1
|
||||
path = folder / f"{stamp}-{k}.json"
|
||||
path = Path(path)
|
||||
path.write_text(json.dumps(record, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
return path
|
||||
|
||||
|
||||
def load(path):
|
||||
return json.loads(Path(path).read_text(encoding="utf-8"))
|
||||
|
||||
|
||||
def log_line(fh, result):
|
||||
row = {"t": time.time(), "i": result["i"], "status": result["status"],
|
||||
"seconds": result["seconds"], "error_kind": result["error_kind"]}
|
||||
if result["warmup"]:
|
||||
row["warmup"] = True
|
||||
fh.write(json.dumps(row) + "\n")
|
||||
fh.flush()
|
||||
|
||||
|
||||
def _change(a, b):
|
||||
if a is None or b is None:
|
||||
return "-"
|
||||
if a == 0:
|
||||
return "-" if b == 0 else "new"
|
||||
return f"{(b - a) / a * 100:+.1f}%"
|
||||
|
||||
|
||||
def compare(a, b):
|
||||
"""Return lines: warnings first, then a table of p50 · p95 · max · failures."""
|
||||
lines = []
|
||||
for key in ("url", "method", "body_sha256", "n", "concurrency", "warmup"):
|
||||
if a.get(key) != b.get(key):
|
||||
lines.append(f"WARNING: different {key}: {a.get(key)} vs {b.get(key)}")
|
||||
sa, sb = a["summary"], b["summary"]
|
||||
lines.append(f"{'':10}{'A':>10}{'B':>10}{'change':>10}")
|
||||
for key in ("p50", "p95", "max"):
|
||||
lines.append(f"{key:10}{fmt(sa[key]):>10}{fmt(sb[key]):>10}{_change(sa[key], sb[key]):>10}")
|
||||
lines.append(f"{'failures':10}{sa['failures']:>10}{sb['failures']:>10}{_change(sa['failures'], sb['failures']):>10}")
|
||||
return lines
|
||||
|
||||
|
||||
def history():
|
||||
folder = home() / "results"
|
||||
rows = []
|
||||
for path in sorted(folder.glob("*.json"), reverse=True) if folder.exists() else []:
|
||||
try:
|
||||
r = load(path)
|
||||
except (ValueError, OSError):
|
||||
continue
|
||||
s = r["summary"]
|
||||
rows.append((r["started"][:19].replace("T", " "), r["url"], r["n"], fmt(s["p50"]), fmt(s["p95"]), s["failures"], path.name))
|
||||
rows.sort(key=lambda row: row[0], reverse=True)
|
||||
return rows
|
||||
@@ -0,0 +1,68 @@
|
||||
"""A fake server inside pytest, so tests never call the real helpdesk (or OpenAI).
|
||||
|
||||
Query options: delay=<s>, fail_every=<k> (every k-th call answers 500), first_delay=<s>,
|
||||
status=<code>, timings=1 (answer with a helpdesk-like `timings`). Counters reset per test.
|
||||
"""
|
||||
import json
|
||||
import threading
|
||||
import time
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from urllib.parse import parse_qs, urlparse
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def log_message(self, *args):
|
||||
pass
|
||||
|
||||
def do_POST(self):
|
||||
length = int(self.headers.get("Content-Length") or 0)
|
||||
self.server.bodies.append(self.rfile.read(length).decode())
|
||||
self.handle_call()
|
||||
|
||||
def do_GET(self):
|
||||
self.handle_call()
|
||||
|
||||
def handle_call(self):
|
||||
q = {k: v[0] for k, v in parse_qs(urlparse(self.path).query).items()}
|
||||
with self.server.lock:
|
||||
self.server.count += 1
|
||||
count = self.server.count
|
||||
delay = float(q.get("first_delay", 0)) if count == 1 and "first_delay" in q else float(q.get("delay", 0))
|
||||
time.sleep(delay)
|
||||
status = int(q.get("status", 200))
|
||||
if "fail_every" in q and count % int(q["fail_every"]) == 0:
|
||||
status = 500
|
||||
payload = {"count": count}
|
||||
if q.get("timings"):
|
||||
payload["timings"] = {"classify_ms": 100 * count, "answer_ms": 300, "total_ms": 400 + 100 * count}
|
||||
data = json.dumps(payload).encode()
|
||||
try:
|
||||
self.send_response(status)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.send_header("Content-Length", str(len(data)))
|
||||
self.end_headers()
|
||||
self.wfile.write(data)
|
||||
except (BrokenPipeError, ConnectionResetError):
|
||||
pass
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def server(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("TBENCH_HOME", str(tmp_path / "home"))
|
||||
httpd = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
|
||||
httpd.daemon_threads = True
|
||||
httpd.count, httpd.lock, httpd.bodies = 0, threading.Lock(), []
|
||||
threading.Thread(target=httpd.serve_forever, daemon=True).start()
|
||||
httpd.base = f"http://127.0.0.1:{httpd.server_address[1]}/hook"
|
||||
yield httpd
|
||||
httpd.shutdown()
|
||||
httpd.server_close()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def body_file(tmp_path):
|
||||
p = tmp_path / "faq-vpn.json"
|
||||
p.write_text(json.dumps({"question": "VPN 접속이 자꾸 끊겨요", "no_cache": True}, ensure_ascii=False))
|
||||
return p
|
||||
@@ -0,0 +1,34 @@
|
||||
"""R2 · R5: rows appear while calls are still running; picking a body file starts the run."""
|
||||
import asyncio
|
||||
|
||||
from textual.widgets import DataTable, OptionList
|
||||
|
||||
from tbench.app import BenchApp
|
||||
from tbench.cli import settings_from
|
||||
|
||||
|
||||
def settings(url, n, body_file=None):
|
||||
return settings_from(type("A", (), {"url": url, "n": n, "body_file": body_file})())
|
||||
|
||||
|
||||
async def test_rows_arrive_one_by_one(server, body_file):
|
||||
app = BenchApp(settings(server.base + "?delay=0.5", 6, str(body_file)), {})
|
||||
async with app.run_test() as pilot:
|
||||
await asyncio.sleep(1.3)
|
||||
partial = app.query_one("#calls", DataTable).row_count
|
||||
assert 1 <= partial < 6
|
||||
await asyncio.sleep(2.5)
|
||||
await pilot.pause()
|
||||
assert app.query_one("#calls", DataTable).row_count == 6
|
||||
assert "done 6/6" in str(app.query_one("#summary").render())
|
||||
|
||||
|
||||
async def test_pick_a_body(server, body_file):
|
||||
app = BenchApp(settings(server.base, 2), {}, pick=str(body_file.parent))
|
||||
async with app.run_test() as pilot:
|
||||
assert app.query_one(OptionList).option_count == 1
|
||||
await pilot.press("enter")
|
||||
await asyncio.sleep(0.5)
|
||||
await pilot.pause()
|
||||
assert app.query_one("#calls", DataTable).row_count == 2
|
||||
assert server.bodies
|
||||
@@ -0,0 +1,129 @@
|
||||
import json
|
||||
import socket
|
||||
import time
|
||||
|
||||
import pytest
|
||||
|
||||
from tbench import cli, profiles, store
|
||||
from tbench.runner import run
|
||||
from tbench.stats import check, percentile, summarize
|
||||
|
||||
|
||||
def free_port():
|
||||
s = socket.socket()
|
||||
s.bind(("127.0.0.1", 0))
|
||||
port = s.getsockname()[1]
|
||||
s.close()
|
||||
return port
|
||||
|
||||
|
||||
# R1 · nearest-rank fixed by a test
|
||||
def test_percentile_nearest_rank():
|
||||
values = list(range(1, 11))
|
||||
assert percentile(values, 0.5) == 5
|
||||
assert percentile(values, 0.95) == 10
|
||||
assert percentile([3.0], 0.95) == 3.0
|
||||
assert percentile([], 0.5) is None
|
||||
|
||||
|
||||
def test_failures_are_counted_not_timed(server, body_file):
|
||||
results = list(run(server.base + "?delay=0.1&fail_every=3", body_file.read_text(), n=6))
|
||||
s = summarize(results, 6)
|
||||
assert s["failures"] == 2 and s["by_kind"]["http_5xx"] == 2
|
||||
assert s["ok"] == 4 and len(results) == 6
|
||||
|
||||
|
||||
def test_body_is_sent(server, body_file):
|
||||
list(run(server.base, body_file.read_text(), n=1))
|
||||
assert json.loads(server.bodies[0])["question"].startswith("VPN")
|
||||
|
||||
|
||||
# R6
|
||||
def test_first_call_and_warmup(server):
|
||||
results = list(run(server.base + "?first_delay=1&delay=0.05", None, n=4, method="GET"))
|
||||
assert results[0]["note"] == "first call"
|
||||
s = summarize(results, 4)
|
||||
assert s["max"] >= 1 and s["p50_excl_first"] < 0.5
|
||||
server.count = 0
|
||||
results = list(run(server.base + "?first_delay=1&delay=0.05", None, n=4, method="GET", warmup=1))
|
||||
measured = [r for r in results if not r["warmup"]]
|
||||
assert len(measured) == 4 and max(r["seconds"] for r in measured) < 0.5
|
||||
assert all(r["note"] != "first call" for r in measured)
|
||||
|
||||
|
||||
# R7
|
||||
def test_error_kinds(server):
|
||||
assert list(run(f"http://127.0.0.1:{free_port()}/x", None, n=1))[0]["error_kind"] == "connect"
|
||||
assert list(run(server.base + "?delay=3", None, n=1, timeout=1))[0]["error_kind"] == "timeout"
|
||||
assert list(run(server.base + "?status=500", None, n=1))[0]["error_kind"] == "http_5xx"
|
||||
assert list(run(server.base + "?status=404", None, n=1))[0]["error_kind"] == "http_4xx"
|
||||
|
||||
|
||||
# R8 · R3 · R11 · R13
|
||||
def test_save_masks_keys_compare_warns_history_and_log(server, body_file, tmp_path, capsys):
|
||||
out_a, out_b, log = tmp_path / "a.json", tmp_path / "b.json", tmp_path / "runs.jsonl"
|
||||
url = server.base + "?delay=0.05"
|
||||
cli.execute(cli.settings_from(type("A", (), {"url": url, "body_file": str(body_file), "n": 3,
|
||||
"header": ["Authorization: Bearer sk-test", "X-Api-Key: sk-test2"]})()),
|
||||
True, str(out_a), str(log))
|
||||
other = tmp_path / "other.json"
|
||||
other.write_text('{"question": "점심 메뉴"}')
|
||||
cli.execute(cli.settings_from(type("A", (), {"url": url, "body_file": str(other), "n": 3})()), True, str(out_b), None)
|
||||
assert "sk-test" not in out_a.read_text() and "sk-test" not in log.read_text()
|
||||
assert json.loads(out_a.read_text())["headers"]["Authorization"] == "***"
|
||||
assert len(log.read_text().splitlines()) == 3
|
||||
lines = store.compare(store.load(out_a), store.load(out_b))
|
||||
assert any(line.startswith("WARNING: different body_sha256") for line in lines)
|
||||
assert not any("WARNING" in line for line in store.compare(store.load(out_a), store.load(out_a)))
|
||||
|
||||
cli.execute(cli.settings_from(type("A", (), {"url": url, "n": 2})()), True, "", None)
|
||||
time.sleep(1.1)
|
||||
cli.execute(cli.settings_from(type("A", (), {"url": url, "n": 4})()), True, "", None)
|
||||
rows = store.history()
|
||||
assert [r[2] for r in rows] == [4, 2]
|
||||
|
||||
|
||||
# R4
|
||||
def test_concurrency_is_faster(server):
|
||||
t = time.perf_counter()
|
||||
list(run(server.base + "?delay=0.3", None, n=6, concurrency=1))
|
||||
one = time.perf_counter() - t
|
||||
t = time.perf_counter()
|
||||
results = list(run(server.base + "?delay=0.3", None, n=6, concurrency=3))
|
||||
three = time.perf_counter() - t
|
||||
assert one > 1.7 and three < 1.0
|
||||
assert sorted(r["i"] for r in results) == [1, 2, 3, 4, 5, 6]
|
||||
|
||||
|
||||
# R10
|
||||
def test_limits_and_exit_code(server):
|
||||
s = summarize(list(run(server.base + "?delay=0.2", None, n=3)), 3)
|
||||
assert check(s, max_p95=0.1) == (False, f"FAIL p95 {s['p95']:.3f} > 0.1")
|
||||
assert check(s, max_p95=5, max_failures=0) == (True, "PASS")
|
||||
with pytest.raises(SystemExit) as e:
|
||||
cli.main([server.base + "?status=500", "-n", "2", "--plain", "--max-failures", "0"])
|
||||
assert e.value.code == 1
|
||||
with pytest.raises(SystemExit) as e:
|
||||
cli.main([server.base, "-n", "2", "--plain", "--max-p95", "5"])
|
||||
assert e.value.code == 0
|
||||
|
||||
|
||||
# R9
|
||||
def test_timings_average(server):
|
||||
s = summarize(list(run(server.base + "?timings=1", None, n=2)), 2)
|
||||
assert s["timings"] == {"classify_ms": 150.0, "answer_ms": 300.0, "total_ms": 550.0}
|
||||
assert summarize(list(run(server.base, None, n=1)), 1)["timings"] == {}
|
||||
|
||||
|
||||
# R12
|
||||
def test_profile_save_and_run_from_another_folder(server, body_file, tmp_path, monkeypatch, capsys):
|
||||
with pytest.raises(SystemExit):
|
||||
cli.main(["profile", "save", "vpn", server.base, "--body-file", str(body_file), "-n", "3",
|
||||
"--header", "Authorization: Bearer sk-live"])
|
||||
assert "sk-live" not in profiles.path().read_text()
|
||||
monkeypatch.chdir(tmp_path)
|
||||
with pytest.raises(SystemExit) as e:
|
||||
cli.main(["run", "vpn", "--plain"])
|
||||
assert e.value.code == 0
|
||||
out = capsys.readouterr().out
|
||||
assert "done 3/3" in out and server.bodies
|
||||
Generated
+267
@@ -0,0 +1,267 @@
|
||||
version = 1
|
||||
revision = 3
|
||||
requires-python = ">=3.11"
|
||||
|
||||
[[package]]
|
||||
name = "anyio"
|
||||
version = "4.15.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "idna" },
|
||||
{ name = "typing-extensions", marker = "python_full_version < '3.15'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/a9/d2/f4d173e22df740bc37b1db102b386ba719b66e95b0f0d751f556b387e6d2/anyio-4.15.1.tar.gz", hash = "sha256:9f28306018cbd6d329e64a36d58256edff76dd996fe423bc957326e578b82a94", size = 276966, upload-time = "2026-09-05T10:42:39.44Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/12/b8/4bd346e22b28902df4d651910f5242c28d84e4a5c2435ca5c3f797ed7e2e/anyio-4.15.1-py3-none-any.whl", hash = "sha256:6152fdbbf9a77fdec97731721bebf7c4c44f7c29b424b0065826173efc7ed101", size = 132079, upload-time = "2026-09-05T10:42:37.923Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "certifi"
|
||||
version = "2026.10.23"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/fd/11/b524cbd9bbdaca58608e4cf4053fd463417549e1959fa8d3aa18319f9555/certifi-2026.10.23.tar.gz", hash = "sha256:a9895e6e2758b371e0545e321a3a8fd1fcfd9829ada8824a936dfb16aaaf36cd", size = 136225, upload-time = "2026-10-23T11:36:16.478Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/32/d1/a3652fe2c19134d0d15922177821d863cbd7e842d53a21bbe7e8e3a1b5ac/certifi-2026.10.23-py3-none-any.whl", hash = "sha256:e9b54de76e530bb6cf0ccd036787103acb58fd42ba930ea8bf948ddbaa2f61d1", size = 135082, upload-time = "2026-10-23T11:36:14.763Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "colorama"
|
||||
version = "0.4.6"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "h11"
|
||||
version = "0.16.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/01/ee/02a2c011bdab74c6fb3c75474d40b3052059d95df7e73351460c8588d963/h11-0.16.0.tar.gz", hash = "sha256:4e35b956cf45792e4caa5885e69fba00bdbc6ffafbfa020300e549b208ee5ff1", size = 101250, upload-time = "2025-04-24T03:35:25.427Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/04/4b/29cac41a4d98d144bf5f6d33995617b185d14b22401f75ca86f384e87ff1/h11-0.16.0-py3-none-any.whl", hash = "sha256:63cf8bbe7522de3bf65932fda1d9c2772064ffb3dae62d55932da54b31cb6c86", size = 37515, upload-time = "2025-04-24T03:35:24.344Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "httpcore"
|
||||
version = "1.0.9"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "certifi" },
|
||||
{ name = "h11" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/06/94/82699a10bca87a5556c9c59b5963f2d039dbd239f25bc2a63907a05a14cb/httpcore-1.0.9.tar.gz", hash = "sha256:6e34463af53fd2ab5d807f399a9b45ea31c3dfa2276f15a2c3f00afff6e176e8", size = 85484, upload-time = "2025-04-24T22:06:22.219Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/7e/f5/f66802a942d491edb555dd61e3a9961140fd64c90bce1eafd741609d334d/httpcore-1.0.9-py3-none-any.whl", hash = "sha256:2d400746a40668fc9dec9810239072b40b4484b640a8c38fd654a024c7a1bf55", size = 78784, upload-time = "2025-04-24T22:06:20.566Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "httpx"
|
||||
version = "0.28.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "anyio" },
|
||||
{ name = "certifi" },
|
||||
{ name = "httpcore" },
|
||||
{ name = "idna" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/b1/df/48c586a5fe32a0f01324ee087459e112ebb7224f646c0b5023f5e79e9956/httpx-0.28.1.tar.gz", hash = "sha256:75e98c5f16b0f35b567856f597f06ff2270a374470a5c2392242528e3e3e42fc", size = 141406, upload-time = "2024-12-06T15:37:23.222Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/2a/39/e50c7c3a983047577ee07d2a9e53faf5a69493943ec3f6a384bdc792deb2/httpx-0.28.1-py3-none-any.whl", hash = "sha256:d909fcccc110f8c7faf814ca82a9a4d816bc5a6dbfea25d6591d6985b8ba59ad", size = 73517, upload-time = "2024-12-06T15:37:21.509Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "idna"
|
||||
version = "3.20"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/f5/08/8eea9d4b8302028f3abb2c0813953f7aec26d33b7a8960ed760e65ff29fa/idna-3.20.tar.gz", hash = "sha256:a7db850025b95ded1eae8a46181a1a6c56c92c96f0e2b005d9ff8dc0210cab44", size = 216463, upload-time = "2026-09-17T14:11:04.752Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/58/a2/bb081bab032533a855d44de1d56f8e8426114ff1ba5d1f07a438a0a654f8/idna-3.20-py3-none-any.whl", hash = "sha256:ab7ae7122974553370f0bdb919e1a960b2cd1bc1ef0276416d896db81c14582c", size = 69583, upload-time = "2026-09-17T14:11:03.168Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "iniconfig"
|
||||
version = "2.3.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/01/e1/2069291243c926a2ff1cd706c7f3eeb9b62144bf60f77c9fb9ff2fb26bd3/iniconfig-2.3.1.tar.gz", hash = "sha256:67f4b9c50da0dedf52af349e7749a80a9057a5031199791b906c3bb3ae878960", size = 21209, upload-time = "2026-10-06T22:48:38.076Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/56/43/4ca9e49d27a1fcf6bece6f6aec0ea46bb9112489b93d4b688fb415457bdb/iniconfig-2.3.1-py3-none-any.whl", hash = "sha256:9121e2c1fdb355232495be3194c8dfe87ccc2d5dee45947b78e68f499790d7a7", size = 7552, upload-time = "2026-10-06T22:48:36.959Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "linkify-it-py"
|
||||
version = "2.2.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/45/98/7a1a5f31fd5c7ba93e963b168e244b8e3dd705b3d2a718e3c3307583bf57/linkify_it_py-2.2.0.tar.gz", hash = "sha256:907acd2d17ac1fbb9ddb62c8957ccbd6158cac602231a15c3b0cd1e215f03cee", size = 32939, upload-time = "2026-08-29T07:07:08.305Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/13/d4/1152d1c7ab42d8b908be64fd200ddc870dc9d4925e951198702084aa1a7d/linkify_it_py-2.2.0-py3-none-any.whl", hash = "sha256:3adc40eb5af300b2605fcfdb968c24e1d780a90f1f2221af7c15e5111e94d443", size = 21971, upload-time = "2026-08-29T07:07:07.164Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "markdown-it-py"
|
||||
version = "4.2.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "mdurl" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/06/ff/7841249c247aa650a76b9ee4bbaeae59370dc8bfd2f6c01f3630c35eb134/markdown_it_py-4.2.0.tar.gz", hash = "sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49", size = 82454, upload-time = "2026-05-07T12:08:28.36Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/b3/81/4da04ced5a082363ecfa159c010d200ecbd959ae410c10c0264a38cac0f5/markdown_it_py-4.2.0-py3-none-any.whl", hash = "sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a", size = 91687, upload-time = "2026-05-07T12:08:27.182Z" },
|
||||
]
|
||||
|
||||
[package.optional-dependencies]
|
||||
linkify = [
|
||||
{ name = "linkify-it-py" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "mdit-py-plugins"
|
||||
version = "0.6.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "markdown-it-py" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/59/fc/f8d0863f8862f25602c0404d75568e89fb6b4109804645e5cdfb1be5cf56/mdit_py_plugins-0.6.1.tar.gz", hash = "sha256:a2bca0f039f39dbd35fb74ae1b5f998608c437463371f0ff7f49a19a17a114d0", size = 56114, upload-time = "2026-05-13T09:03:38.91Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/a5/69/6da5581c6a7fede7dc261bf4e67d6adca4196f176b43288b55b3db395b6e/mdit_py_plugins-0.6.1-py3-none-any.whl", hash = "sha256:214c82fb2ac524472ab6a5bcab1de80f73b50443e187f401bfd77efbc7c6481d", size = 66663, upload-time = "2026-05-13T09:03:37.76Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "mdurl"
|
||||
version = "0.1.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/d6/54/cfe61301667036ec958cb99bd3efefba235e65cdeb9c84d24a8293ba1d90/mdurl-0.1.2.tar.gz", hash = "sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba", size = 8729, upload-time = "2022-08-14T12:40:10.846Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl", hash = "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", size = 9979, upload-time = "2022-08-14T12:40:09.779Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "packaging"
|
||||
version = "26.3"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/7d/fa/3944b40b07da9ce895c0e6303a5ab7d53da063554f534556b134a54d6093/packaging-26.3.tar.gz", hash = "sha256:94edc256424af38762eb31306eed28beb9f0efc50a8837492c9d6fd6004aed79", size = 313412, upload-time = "2026-08-04T18:15:28.737Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/63/34/ba1c580383c9eada3711951fef0795c80b829a078d72188184bcab9dd527/packaging-26.3-py3-none-any.whl", hash = "sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c", size = 129956, upload-time = "2026-08-04T18:15:27.159Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "platformdirs"
|
||||
version = "4.13.5"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/58/e6/d0abbb256b44e6a05748c6942511ff67db8dbb068de0c35b24bf1b395b09/platformdirs-4.13.5.tar.gz", hash = "sha256:05831adf87d392760f4b5c3a0a95579ab5a10b18b9b9bd43680dcb47544788a0", size = 62362, upload-time = "2026-10-19T01:02:11.778Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/6c/8a/1d8cb3842909dc193a73835adedf412e32e18f8ffd59f43983949622fe6f/platformdirs-4.13.5-py3-none-any.whl", hash = "sha256:53d3ec4280b815c1e0b02edad7f9ee8fda1ba9c0865dd9c2e6e63849ce82fc1e", size = 32779, upload-time = "2026-10-19T01:02:10.17Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pluggy"
|
||||
version = "1.7.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/bf/db/7fc19e6f2dc92a966727031389fc2e08b558f0f25eb7403c1119ad4713cd/pluggy-1.7.0.tar.gz", hash = "sha256:d1eaa46ebb595891b860ab086b4d09c8588af65ebd4361b8e8f4bb8920b90ba8", size = 123304, upload-time = "2026-10-15T09:50:58.343Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/40/9e/2b38731e0fc536806f16490e1a12d7f0dc2a1235aa8cc07bcc75416a7daa/pluggy-1.7.0-py3-none-any.whl", hash = "sha256:7dd7b0d8832ba3cb632c306926ded123429211b83641b35dc5c41ad2d34f9bec", size = 27082, upload-time = "2026-10-15T09:50:56.808Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pygments"
|
||||
version = "2.21.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/49/2e/ced460408999b33da6b31b0021b0f37d329e202d4169aeb164493778f25b/pygments-2.21.0.tar.gz", hash = "sha256:610ca751c9bc2492b38eb9a38a7fbc93edbbb2d7182edaf34e66ae493dee5c8c", size = 5005329, upload-time = "2026-08-17T08:02:48.824Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/71/46/17f022dd3e953bf20a04a028a21ec746d942f8d2af30fa0f124fa0e6a684/pygments-2.21.0-py3-none-any.whl", hash = "sha256:2363c69b61c4a97c838da3b130dcd6468f4848992b21a82f2a63ec34377137d9", size = 1250147, upload-time = "2026-08-17T08:02:44.912Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pytest"
|
||||
version = "9.1.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "colorama", marker = "sys_platform == 'win32'" },
|
||||
{ name = "iniconfig" },
|
||||
{ name = "packaging" },
|
||||
{ name = "pluggy" },
|
||||
{ name = "pygments" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/e4/47/b9efed96c114afcfa3c9d3fe98a76a1d14c74a9e266d397cf6eb64be5e01/pytest-9.1.1.tar.gz", hash = "sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313", size = 1636369, upload-time = "2026-06-19T10:58:32.857Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/24/25/1de2678b631f5a49215c6c96fff41ba892b0a34df68d6d80292b1b48aa7f/pytest-9.1.1-py3-none-any.whl", hash = "sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c", size = 386536, upload-time = "2026-06-19T10:58:31.347Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pytest-asyncio"
|
||||
version = "1.4.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "pytest" },
|
||||
{ name = "typing-extensions", marker = "python_full_version < '3.13'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/f8/9a/887535e477847f5159377ab35303923644840b13c0ca03c0a467f84d4680/pytest_asyncio-1.4.1.tar.gz", hash = "sha256:e8266eb094e5ee9ffdb2068c9836f33e209ea77a970eb846933d416b5602669b", size = 145237, upload-time = "2026-10-25T21:05:37.135Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/46/d7/cd9e2708afdbf059658adaf08bdb0792ca52489871051d6ce7d498b51277/pytest_asyncio-1.4.1-py3-none-any.whl", hash = "sha256:507afa78ec77c6661164ef1a2368bc7820383fd1a14b2a7f813de8b5a92949cc", size = 17061, upload-time = "2026-10-25T21:05:35.473Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rich"
|
||||
version = "15.0.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "markdown-it-py" },
|
||||
{ name = "pygments" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/c0/8f/0722ca900cc807c13a6a0c696dacf35430f72e0ec571c4275d2371fca3e9/rich-15.0.0.tar.gz", hash = "sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36", size = 230680, upload-time = "2026-04-12T08:24:00.75Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/82/3b/64d4899d73f91ba49a8c18a8ff3f0ea8f1c1d75481760df8c68ef5235bf5/rich-15.0.0-py3-none-any.whl", hash = "sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb", size = 310654, upload-time = "2026-04-12T08:24:02.83Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tbench"
|
||||
version = "0.1.0"
|
||||
source = { editable = "." }
|
||||
dependencies = [
|
||||
{ name = "httpx" },
|
||||
{ name = "textual" },
|
||||
]
|
||||
|
||||
[package.dev-dependencies]
|
||||
dev = [
|
||||
{ name = "pytest" },
|
||||
{ name = "pytest-asyncio" },
|
||||
]
|
||||
|
||||
[package.metadata]
|
||||
requires-dist = [
|
||||
{ name = "httpx", specifier = ">=0.27" },
|
||||
{ name = "textual", specifier = ">=0.80" },
|
||||
]
|
||||
|
||||
[package.metadata.requires-dev]
|
||||
dev = [
|
||||
{ name = "pytest", specifier = ">=8" },
|
||||
{ name = "pytest-asyncio", specifier = ">=0.23" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "textual"
|
||||
version = "8.2.8"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "markdown-it-py", extra = ["linkify"] },
|
||||
{ name = "mdit-py-plugins" },
|
||||
{ name = "platformdirs" },
|
||||
{ name = "pygments" },
|
||||
{ name = "rich" },
|
||||
{ name = "typing-extensions" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/00/21/39a76b01bd5eea82a04baaca7580e105d8c59450df03998345bb2cfb307b/textual-8.2.8.tar.gz", hash = "sha256:3f106a9fbc73e39dd266c9712432087de78a6d644084c7c241d6a25c3169115b", size = 1860502, upload-time = "2026-06-30T06:51:24.495Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/fb/be/35261223d9416a0751cdff1c7b4a6f881387218a12d439fe22fefebc8c04/textual-8.2.8-py3-none-any.whl", hash = "sha256:267375fd402dc8d981457212efa71f0e3365fd17bba144ba9bb3ed7563cb374a", size = 731418, upload-time = "2026-06-30T06:51:26.364Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "typing-extensions"
|
||||
version = "4.16.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/f6/cc/6253133b5bb138fc3306cebfbda2c520f545d36b5be2c7255cc528bb45d6/typing_extensions-4.16.0.tar.gz", hash = "sha256:dc983d19a509c94dba722ee6abd33940f7c05a89e243c47e907eb4db6f1a43e5", size = 113555, upload-time = "2026-07-02T08:40:05.92Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/49/d3/b8441a820a491ddfc024b0b0cf0393375b75ea13866d9c66727e54c2fc80/typing_extensions-4.16.0-py3-none-any.whl", hash = "sha256:481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8", size = 45571, upload-time = "2026-07-02T08:40:04.659Z" },
|
||||
]
|
||||
Reference in new issue
Block a user